KR20210023690A - Apparatus for measuring a contribution of content and method thereof - Google Patents

Abstract

The present invention relates to a method for measuring contribution of content to identify a bad user and an apparatus thereof. According to one embodiment of the present invention, the method executed by a computing device comprises the following steps: determining, by a banned content identification module, whether the content is banned from a bulletin board, wherein the prohibited content identification module is a module for determining whether the content is banned on the bulletin board by using data extracted from the content; determining, by a bad user identification module, whether a user who created the content is a bad user when it is determined that the content is banned on the bulletin board, and determining, by a contribution measurement module, determining contribution of the content when it is determined that the content is allowed on the bulletin board, wherein the bad user identification module is a module for determining whether the user is a bad user by using content creation user log data and the contribution measurement module is a module for measuring the contribution of the content by the content page log data; and giving a penalty given to the user when the user is determined as a bad user by the bad user identification module, and measuring the contribution of the content by the contribution measurement module when the user is determined as a normal user by the bad user identification module.

Description

Method and device for measuring content contribution {APPARATUS FOR MEASURING A CONTRIBUTION OF CONTENT AND METHOD THEREOF}

The present invention relates to a method and an apparatus for measuring the contribution of content. More specifically, it relates to a method and an apparatus for deriving a quantitative evaluation of log data on content using machine learning, and measuring a contribution degree based on this.

An online community is a space where people with common interests or environments communicate online. Users who use the online community upload content to the bulletin board provided in the community or browse the uploaded content.

In such an online community, there is a need for a method in which the online community manager distributes the profits generated by managing the online community to users who upload the content. The conventional profit distribution method is limited to a method based on simply number of views or using a fixed fee.

The profit distribution method using views cannot exclude errors due to fraudulent clicks, macros, and traffic attacks, and the method using a fixed fee requires a separate personal evaluation procedure to convert the fee for each user, resulting in considerable labor costs. Is consumed.

The technical problem to be solved by the present invention is to provide a method of measuring the contribution of content and an apparatus to which the method is applied.

Another technical problem to be solved by the present invention is to provide a method for identifying prohibited content and an apparatus to which the method is applied.

Another technical problem to be solved by the present invention is to provide a method for identifying a bad user and an apparatus to which the method is applied.

Another technical problem to be solved by the present invention is to provide a method and apparatus for generating an artificial intelligence-based model for measuring the contribution of content.

Another technical problem to be solved by the present invention is to provide a method and apparatus for generating an artificial intelligence-based model for identifying prohibited content.

Another technical problem to be solved by the present invention is to provide a method of generating an artificial intelligence-based model for identifying a bad user and an apparatus to which the method is applied.

Another technical problem to be solved by the present invention is to provide a fair distribution method for profits generated from a web service and an apparatus to which the method is applied, using a contribution degree determined based on log data related to contents.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

In the method of measuring contribution according to an embodiment of the present invention for solving the technical problem, in a method performed by a computing device, determining whether or not content is prohibited on a bulletin board by a prohibited content identification module, the prohibited content The identification module is a module that determines whether the content is prohibited on the bulletin board by using the data extracted from the content. When it is determined that the content is prohibited on the bulletin board, the content is created by a bad user identification module. Determines whether the user is a bad user, and when it is determined that the content is allowed in the bulletin board, the contribution level of the content is determined by a contribution measurement module, and the bad user identification module uses the content creation user log data A module that determines whether the user is a bad user, and the contribution measurement module is a module that measures the contribution of the content using content page log data, and when the user is determined as a bad user by the bad user identification module , When the user is determined to be a normal user by giving a penalty to the user and by the bad user identification module, determining a contribution degree of the content by the contribution measurement module.

In one embodiment, the forbidden content identification module inputs the image included in the content to a Convolutional Neural Network (CNN)-based Not Safe For Work (NSFW) detector, and the content is prohibited in the bulletin board by the output data. It may be determined whether or not. Here, the output data includes a probability that the image contains a prohibited item, and the step of determining whether the content is prohibited on the bulletin board is that when the probability is greater than or equal to a reference value, the content is prohibited on the bulletin board. It may include the step of determining. In this case, in the case of content created by a user who is determined to be a bad user by the bad user identification module, the reference value may be formed to be lower than the content written by a normal user.

In an embodiment, the content includes text and an image, and the prohibited content identification module determines whether or not the content is prohibited on the bulletin board by text included in the content. It may include a second module that determines whether the content is prohibited on the bulletin board by the included image. Here, the prohibited content identification module may determine whether the content is prohibited on the bulletin board by the second module after determining whether the content is prohibited on the bulletin board by the first module.

In one embodiment, the bad user identification module may be generated by learning the artificial neural network by obtaining a plurality of features extracted from the user log data and inputting a training dataset composed of the plurality of features into an artificial neural network. have. Here, the plurality of features are the user's subscription, number of days of connection, number of posts, number of comments, number of content views, number of likes, number of likes, number of dislikes, number of reports, number of likes, number of likes received, number of reports received, type of report received. , Number of deleted posts, number of deleted comments, IP, total stay time, average stay time, total click count and device type. In addition, the artificial neural network may be based on ensemble learning.

In one embodiment, the user log data includes the user's membership date, access days, number of posts, number of comments, number of content views, number of likes, number of dislikes, number of reports given, number of likes received, number of dislikes received, number of reports received, It can include the type of report received, the number of deleted posts, the number of deleted comments, IP, total stay time, average stay time, total click count, and device type.

In one embodiment, in a method performed by a computing device having an artificial neural network, obtaining a plurality of features extracted from content page log data, inputting the plurality of features into the artificial neural network, and outputting from the artificial neural network Determining a content contribution by using data, wherein the plurality of features include a click position of a user who clicks a specific part of the content page, a click count of the click position, a previous page of the content page, and the content A page after the page, a type of a viewing device for the content page, user information for writing comments on the content page, and user information for viewing the content page may be included. Here, the artificial neural network may be performed by ensemble learning based on supervised learning by inputting a learning dataset composed of the plurality of features. In addition, the plurality of features include a text length included in the content, whether an image is included in the content, the number of images included in the content, an image file type included in the content, an image capacity included in the content, and The number of included image downloads, the creation time of the content, the number of views of the content page, the number of likes of the content page, the number of dislikes of the content page, the number of reports of the content page, the report type of the content page, the content page The number of comments and the residence time of the content page may be further included.

In the method of identifying prohibited content according to an embodiment of the present invention, in a method performed by a computing device having an artificial neural network, obtaining a feature extracted from a text included in the content and inputting the feature into the artificial neural network And determining whether the content is prohibited on the bulletin board by using the data output from the artificial neural network, wherein the obtaining of the characteristic includes a vector in which Korean alphabets, numbers, and special characters are similar according to the appearance similarity of the text. It may include the step of extracting. Here, the output data includes a probability that the text contains a prohibited item, and the step of determining whether the content is prohibited on the bulletin board is determined that the content is prohibited on the bulletin board when the probability is greater than or equal to a reference value. Including the step of or including a probability that the text includes a required item, and determining whether the content is prohibited on the bulletin board, when the probability is less than a reference value, determining that the content is prohibited in the bulletin board It may include steps.

In one embodiment, the prohibited item may belong to any one of abusive material group, pornographic material group, violent material group, and copyright violation group. In addition, it further comprises the step of adding the first item as a prohibited item, wherein the first item is located within a reference distance based on the first prohibited item, and the probability of including the first prohibited item is greater than or equal to a reference value. The content may include the first item more than a reference number of times.

In an embodiment, the required item may be determined by a category of the bulletin board.

1 is a diagram illustrating a system for measuring a contribution degree according to an exemplary embodiment of the present invention.
2 is a flowchart illustrating a method of measuring a contribution degree according to another embodiment of the present invention.
3 is a flow chart for specifically explaining some operations of the method described with reference to FIG. 2.
4 is a flow chart for specifically explaining some operations of the method described with reference to FIG. 3.
5 to 6 are flow charts for specifically explaining some operations of the method described with reference to FIG. 2.
7 is a flow chart for specifically explaining some operations of the method described with reference to FIG. 6.
8 is a diagram for describing a computing device capable of implementing an apparatus for measuring a contribution degree according to another exemplary embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the technical idea of the present invention is not limited to the following embodiments, but may be implemented in various different forms, and only the following embodiments complete the technical idea of the present invention, and in the technical field to which the present invention pertains. It is provided to completely inform the scope of the present invention to those of ordinary skill in the art, and the technical idea of the present invention is only defined by the scope of the claims.

In adding reference numerals to elements of each drawing, it should be noted that the same elements are assigned the same numerals as possible, even if they are indicated on different drawings. In addition, in describing the present invention, when it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used with meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically. The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase.

In addition, in describing the constituent elements of the present invention, terms such as first, second, A, B, (a) and (b) may be used. These terms are for distinguishing the constituent element from other constituent elements, and the nature, order, or order of the constituent element is not limited by the term. When a component is described as being "connected", "coupled" or "connected" to another component, the component may be directly connected or connected to that other component, but another component between each component It will be understood that elements may be “connected”, “coupled” or “connected”.

As used in the specification, "comprises" and/or "comprising" refers to the presence of one or more other components, steps, actions and/or elements in which the recited component, step, operation and/or element is Or does not preclude additions.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Hereinafter, a configuration and operation of a target object detection system according to an embodiment of the present invention will be described with reference to FIG. 1. 1 is a diagram illustrating a system for measuring a contribution degree according to an exemplary embodiment of the present invention.

The contribution measurement system according to the present embodiment may be understood as a system for measuring the contribution of content created by a user. Here, the content is uploaded to the online community and may be understood as including text or images. Here, the degree of contribution may be a criterion of the revenue distributed by the administrator to the users who created the content.

Each of the constituent elements of the contribution measurement system shown in FIG. 1 represents functional elements that are functionally divided, and any one or more constituent elements may be integrated and implemented with each other in an actual physical environment.

Referring to FIG. 1, the contribution measurement system includes a database 100, a user terminal 200, a prohibited content identification device 300, a bad user identification device 400, a contribution measurement device 500, and a manager terminal 600. It may include. Hereinafter, each component will be described in detail.

The database 100 may store content, content page log data, and user log data.

The database 100 may transmit content to the user terminal 200. In addition, the database 100 may transmit content, content page log data, and user log data to the manager terminal 600. In addition, the database 100 may transmit content to the prohibited content identification device 300. In addition, the database 100 may transmit user log data to the device 400 for identifying a bad user. Furthermore, the database 100 may transmit content page log data to the apparatus 500 for measuring the degree of contribution.

The database 100 may receive content, user log data, and content page log data created by a user from the user terminal 200.

The database 100 may be an external storage such as a database server or network file storage. In addition, it may be an internal storage medium such as an HDD or SSD. However, it is not limited thereto, and may be a cloud service. Cloud service refers to an on-demand outsourcing service that provides various IT resources such as servers, storage, and software to users. For example, a cloud service that provides storage is a service that provides users with various data to be stored in storage of a service providing server connected to a network and downloaded through a network when necessary. The cloud service may be IaaS (Infrastructure as a Service) that can provide a virtualized infrastructure environment to users or SaaS (Software as a Service) that provides application software through the Internet, but is not limited thereto.

Next, the user terminal 200 may receive and display content from the database 100. More specifically, the user terminal 200 transmits a content browsing request to the database 100 according to a user's input. Then, the user terminal 200 receives content from the database 100. Then, the user terminal 200 outputs the received content on the screen. The user terminal 200 may have a web browser or a dedicated application installed in order to output content.

The user terminal 200 that can be referred to in an embodiment of the present invention may be any device as long as it is a device capable of outputting the content received from the database 100 through a network. For example, the user device 200 that can be referred to in an embodiment of the present invention is a desktop, a workstation, a server, a laptop, a tablet, a smartphone ( Smart Phone) or a phablet (Phablet), but is not limited thereto, and a portable multimedia player (PMP), personal digital assistants (PDA), or an e-book terminal (E- It can also be a type of device such as Book Reader).

In addition, the user terminal 200 may transmit content, user log data, and content page log data created by the user to the database 100. The user terminal 200 that can be referred to in an embodiment of the present invention may be any device as long as it is a device capable of receiving an input of content created by a user.

Next, the prohibited content identification device 300 may be a device that analyzes the content and determines whether the content is prohibited on the bulletin board of the online community. The prohibited content identification device 300 may analyze the content created by the user and transmit the analyzed result to the database 100. In addition, the prohibited content identification device 300 may transmit the analyzed result to the bad user identification device 400. In addition, the prohibited content identification device 300 may transmit the analyzed result to the contribution measurement device 500. Furthermore, the prohibited content identification device 300 may transmit the analyzed result to the manager terminal 600.

In one embodiment, the prohibited content identification device 300 may transmit a request to delete the content stored in the database 100 when it is determined that the content is prohibited in the bulletin board as a result of analyzing the content created by the user.

In an embodiment, the prohibited content identification device 300 may transmit a request to delete the uploaded content to the manager terminal 600 when it is determined that the content is prohibited in the bulletin board as a result of analyzing the content created by the user. Such a prohibited content identification device 300 according to an embodiment of the present invention will be described in detail later with reference to FIGS. 2 to 8.

Next, the bad user identification device 400 may be a device that determines whether the user is a bad user by analyzing user log data. The bad user identification device 400 may analyze user log data and transmit the analyzed result to the database 100. In addition, the bad user identification device 400 may transmit the analyzed result to the prohibited content identification device 300. In addition, the bad user identification device 400 may transmit the analyzed result to the contribution measurement device 500. Furthermore, the bad user identification device 400 may transmit the analyzed result to the manager terminal 600.

In an embodiment, when a user log data analysis result determines that a bad user is a bad user, the bad user identification device 400 may store bad user data by matching an attribute capable of specifying the user in the database 100. Such, a bad user identification device 400 according to an embodiment of the present invention will be described in detail later with reference to FIGS. 2 to 8.

Next, the contribution measurement device 500 may be a device that determines a content contribution by analyzing content page log data. The contribution measurement device 500 may analyze content page log data and transmit the analyzed result to the database 100. In addition, the contribution measurement device 500 may transmit the analyzed result to the user terminal 200. Further, the contribution measurement device 500 may transmit the analyzed result to the manager terminal 600.

In an embodiment, the contribution measurement device 500 may match the determined contribution to the content as a result of analyzing the content page log data with the content and store it in the database 100.

In an embodiment, the contribution measurement device 500 may transmit the determined contribution to the content as a result of analyzing the content page log data to the user terminal 200, and the user who created the content may check the determined contribution. As described above, an apparatus 500 for measuring a contribution degree according to an embodiment of the present invention will be described in detail later with reference to FIGS. 2 to 8.

Next, the manager terminal 600 may receive content, user log data, and content page log data from the database 100. In addition, the manager terminal 600 may receive an input of a user requesting a profit distribution from the user terminal 200. In addition, the manager terminal 600 may receive a result of analyzing the content created by the user from the prohibited content identification device. In addition, the manager terminal 600 may receive a result of analyzing user log data from the bad user identification device 400. Furthermore, the manager terminal 600 may receive a result of analyzing the content page log data from the contribution measurement device 500.

The manager terminal 600 may manage the contribution measurement system according to an embodiment of the present invention. An appropriate request for managing the contribution measurement system may be transmitted to individual components of the contribution measurement system.

The manager terminal 600 that can be referred to in an embodiment of the present invention may be any device as long as it is a device capable of performing the above-described operation.

Hereinafter, a method of measuring a contribution degree according to another embodiment of the present invention will be described with reference to FIG. 2. The method of measuring the degree of contribution according to the present embodiment may be performed by a computing device. Also, the method according to the present embodiment may be performed by being divided by the first computing device and the second computing device. Hereinafter, in performing each operation of the method according to the present embodiment, if the description of the subject is omitted, the subject may be interpreted as being the computing device.

Referring to FIG. 2, it is determined whether or not the content is prohibited on the bulletin board by the prohibited content identification module (S100). Here, the prohibited content identification module is a module that determines whether or not the content is prohibited on the bulletin board by using data extracted from the content. Next, if the content created by the prohibited content identification module is prohibited (S200), it is determined whether the content creation user is a bad user by the bad user identification module (S300). Here, the bad user identification module is a module that determines whether the user is a bad user by using the content creation user log data. Next, if the content created by the prohibited content identification module is allowed (S200), the content contribution is determined by the contribution measurement module (S500). Here, the contribution measurement module is a module that measures the contribution of the content by using log data of the content page. Next, when the user who created the content by the bad user identification module is determined as a bad user (S400), a penalty is given to the user determined as a bad user (S600). When the user who created the content by the bad user identification module is determined as a normal user (S500), the content contribution is determined by the contribution measurement module.

According to the present embodiment, in order to measure the contribution of the content, the appropriateness of the content and the appropriateness of the user who created the content may be determined in advance. As the appropriateness of the content content and the appropriateness of the user who created the content are determined in advance, prohibited content or bad users can be automatically detected.

According to the present embodiment, various attributes related to the content are considered, so that the contribution of the content can be measured fairly. Based on this contribution, the online community manager can manage the online community and distribute the profits generated fairly. As the generated profits are distributed fairly, various users can flow into the online community, so it can be a win-win for both administrators and users.

In an embodiment, if the user is determined to be a bad user, a penalty may be given. Here, penalty is a kind of penalty applied to bad users. For example, it is possible to confiscate user experience from online communities. For another example, content creation may be prohibited by setting a period. As another example, you can limit the entry of the online community by setting a period of time. However, the present invention is not limited to this example, and various restrictions related to the use of the online community by bad users may be applied. Accordingly, penalties including all of these limitations may be included in the present invention.

Hereinafter, a detailed description of the prohibited content identification module will be described with reference to FIGS. 3 to 4. 3 is a flow chart for specifically explaining some operations of the method described with reference to FIG. 2.

Referring to FIG. 3, step S100 described with reference to FIG. 2 is embodied.

In step S110, it is determined whether the content is prohibited on the bulletin board by the text included in the content by the first module. In step S120, it is determined whether or not the content is prohibited on the bulletin board by the image included in the content by the second module.

In one embodiment, the first module and the second module may run in parallel. As the first module and the second module proceed in parallel, a time for identifying prohibited content may be shortened.

In an embodiment, after being determined by the first module, whether or not the content is prohibited may be determined by the second module. By determining whether the content including the text is prohibited content based on the text, the efficiency of image analysis by the second module may be increased. For a more specific example, when the text includes pornographic material, a model for detecting the pornographic material may be determined during image analysis by the second module. For another example, when the text includes a violent material, a model for detecting the violent material may be determined when an image is analyzed by the second module.

Referring to FIG. 4 for a more detailed description of the prohibited content identification module described with reference to FIGS. 2 to 3. 4 is a flow chart for specifically explaining some operations of the method described with reference to FIG. 3.

Referring to FIG. 4, steps S110 and S120 described with reference to FIG. 3 are embodied.

In step S111, the feature extracted from the text included in the content is obtained, and in step S112, the feature extracted from the text is input to the artificial neural network, and whether the content is prohibited on the bulletin board using the data output by the artificial neural network. Is judged. In step S121, the feature extracted from the image included in the content is obtained, and in step S122, the feature extracted from the image is input to the artificial neural network, and whether the content is prohibited on the bulletin board using the data output by the artificial neural network. Is judged.

In one embodiment, whether or not the content is prohibited on the bulletin board may be determined by data output by inputting an image included in the content to a Convolutional Neural Network (CNN)-based Not Safe For Work (NSFW) detector. Here, the output data may include a probability that the image includes a prohibited item. In this case, if the probability is greater than or equal to the reference value, it may be determined that the content is prohibited on the bulletin board. According to the present embodiment, a specific method of identifying prohibited content may be provided.

For a specific example, after analyzing an image included in the content using a tensor flow, it may be determined whether or not pornography, which is a prohibited item, is included by an NSFW model in which pornography is learned. Here, if Neutral among the NSFW scores is 30% or less, it may be determined as prohibited content. However, the present invention is not limited to this example, and all known methods for analyzing an image using a model generated by machine learning and identifying whether the analyzed image contains prohibited items are included in the present invention. I can. In this example, the prohibited item is limited to pornography, but the prohibited item is not limited to pornography. Prohibited items that may be referenced in some embodiments will be specified later.

In an embodiment, whether or not the content is prohibited on the bulletin board may be determined by obtaining features extracted from text included in the content, inputting it into an artificial neural network, and using data output from the artificial neural network. Here, the output data includes a probability that the text included in the content includes a prohibited item, and when the probability is greater than or equal to a reference value, it may be determined that the content is prohibited on the bulletin board. In addition, the output data includes a probability that the text included in the content includes an essential item, and when the probability is less than or equal to the reference value, it may be determined that the content is prohibited on the bulletin board.

According to the present embodiment, it may be determined that content including the prohibited item or not including the required item is prohibited on the bulletin board. In the identification of prohibited content, appropriate content may be uploaded to the online community bulletin board by determining whether a prohibited item or a required item is included. As the appropriateness of the content content is determined in advance, it can be helpful in calculating the fair contribution.

In one embodiment, the prohibited item may belong to any one of an abusive group, a violent group, and a copyright violation group. For example, when the prohibited item is included in the profanity group, the text related to the profanity may be defined in advance as a plurality of prohibited items. For another example, when the prohibited item is included in the violence group, text related to the violence may be defined in advance as a plurality of prohibited items. However, the present invention is not limited thereto, and all methods of determining the appropriateness of text by prohibited items may be included in the present invention.

In an embodiment, the required item may be determined by a category of the bulletin board. According to the present embodiment, articles that are not related to the subject of the bulletin board may be determined as prohibited content. For example, when content irrelevant to'bicycle' is uploaded to a bulletin board related to'bicycle', it may be determined as prohibited content. Here, text related to'bicycle' may be defined in advance as a plurality of essential items.

A method for identifying prohibited content that can be referred to in an embodiment of the present invention in which it is determined whether or not the content is prohibited on a bulletin board is determined by text included in the content using an artificial neural network, using a model generated by machine learning. All known methods of analyzing text and identifying whether the analyzed text contains prohibited items may be included in the present invention. The artificial neural network may output a probability of including a prohibited item after noise such as a typo is minimized through a CNN-based layer. For the CNN-based text classification method, various documents such as "Character-level Convolutional Networks for Text Classification", Xiang Zhang et al., 2015, etc. may be referenced. Here, 40 Korean alphabets, 26 English alphabets, and spaces Including 34 special characters and 10 numbers can be used to be quantized to suit Korean text.

A method for identifying prohibited content that can be referred to in an embodiment of the present invention in which it is determined whether or not the content is prohibited on a bulletin board by text included in the content using an artificial neural network is, for example, a profanity among prohibited items. In the case of group analysis, the prohibited items may include sweet words, profanity, vulgar sounds, harsh sounds, etc., and learning may be performed through an ensemble model based on supervised learning by labeling them.

In an embodiment, the extracted features to be input to the artificial neural network may be extracted as a vector in which Korean alphabets, numbers, and special characters are similar according to the appearance similarity of the text. For specific examples of the characteristics of Hangul, the alphabet'O', the number '0', and the Korean consonant'ㅇ' have similar appearances even though their meanings are different. For another example, the alphabet'l', the number '1', and the Korean vowel'ㅣ' have similar appearances even though they have different meanings. Contents created including prohibited items by partially changing Hangul consonants or vowels can be identified by extracting and learning individual characters having an appearance similarity greater than or equal to the reference value as illustrated in the example.

In one embodiment, report data of an online community user may be used for the addition of prohibited items. By using the report data, inappropriate text can be added to prohibited items. In one embodiment, using the report type data, a group of prohibited items may be determined and added. For example, if there is a text whose report data is more than a standard value, and the type of report of the text is pornography, the text may be added to the pornography group.

In one embodiment, when the first item is text located within a reference distance based on the first prohibited item, and a plurality of contents whose probability of including the first prohibited item is greater than or equal to the reference value includes the first item more than the reference number of times, the first item 1 item can be included in prohibited items.

According to the present embodiment, the first prohibited item included in the prohibited item and the first item used within the reference distance together with the reference number or more may be included as the prohibited item. Since the added first item is the first prohibited item and the text used more than the reference number of times, the probability of corresponding to the prohibited item may be high. In addition, even when the user's report data is not accumulated, the prohibited item may be automatically added while executing the prohibited content identification module according to whether the first prohibited item is included.

In an embodiment, the data output by the artificial neural network includes a probability of including a prohibited item, and when the probability is greater than or equal to a reference value, it may be determined that the content is prohibited on the bulletin board. Here, in the case of content created by a bad user, the reference value may be lower than that of content created by a normal user. According to the present embodiment, since the reference value is formed to be low in the case of writing by a bad user, it is possible to restrict writing by a bad user and to easily prohibit contents with a high probability of including prohibited contents. The bad user identification referenced in this embodiment may be embodied by a bad user identification module to be described later.

Hereinafter, a defective user identification module will be described in detail with reference to FIG. 5. 5 is a flow chart for specifically explaining some operations of the method described with reference to FIG. 2.

5, step S300 is embodied.

In step S310, a feature extracted from the user log data is obtained, and in step S320, the feature is input to an artificial neural network, and whether or not the user is a bad user is determined using the output data.

In an embodiment, the bad user identification module may be generated by obtaining a plurality of features extracted from user log data, and inputting a training dataset composed of the plurality of features into an artificial neural network to train the artificial neural network. Here, the artificial neural network may be based on ensemble learning. All known methods for solving the binary classification problem of determining whether the artificial neural network used in the bad user identification module is a bad user or a normal user can be used in the present invention.

In one embodiment, the user log data is the user's membership date, number of days accessed, number of posts, number of comments, number of content views, number of likes, number of dislikes, number of reports, number of likes, number of likes received, number of reports received, received Report type, number of deleted posts, number of deleted comments, IP, total stay time, average stay time, total number of clicks and device type can be included.

Here, the user's IP can be used as a criterion for whether it corresponds to a bad user through the user's location information, and a suspected device such as a device of a type that is too old according to the user's device type is used as a criterion for determining a bad user I can take it.

In an embodiment, at least two or more of the collected log data may be selected, and data determined by a ratio of the data may be used as a criterion for determining a bad user. For example, through a value obtained by dividing the user's number of posts by the user's subscription, a user having an excessively large number of posts compared to the subscription period may be determined as a bad user. For another example, through a value obtained by dividing the number of likes by the number of content views, a user having an excessively large number of likes relative to the number of views may be determined as a bad user. However, the present invention is not limited to this example, and all methods that can be a criterion for determining a bad user by linking previously collected log data may be included in the present invention.

According to the present embodiment, various user attributes that may be extracted from user log data may be reflected to determine whether the user is a bad user.

Hereinafter, with reference to FIGS. 6 to 7, the contribution measurement module will be described in detail. 6 is a flow chart for specifically explaining some operations of the method described with reference to FIG. 2.

Referring to FIG. 6, step S400 is embodied.

In step S410, a feature extracted from the content page log data is obtained, and in step S420, the feature is input to an artificial neural network, and a contribution degree is determined using the output data.

In one embodiment, the content page log data includes a click position of a user who clicks a specific part of the content page, the number of clicks at the click position, a previous page of the content page, a page after the content page, and It may include a browsing device type, comment writing user information of the content page, and browsing user information of the content page.

Here, the click position of a user who clicks on a specific part of the content page may use an operation of a user browsing the content to click or enlarge a specific part of the content as a criterion for measuring the degree of contribution. For example, in the case of content in which the click location is evenly spread throughout the content, it may be a basis for a high contribution rating. The number of clicks can be similarly applied.

Here, the page before or after the content page may be used as a criterion for measuring the degree of contribution by determining how the content is accessed. For example, content with a large number of direct access through a URL may be evaluated as an illegal number of views by a macro, which may serve as a basis for a low contribution rate.

Here, the comment writing user information and the browsing user information may be used as a criterion for measuring the degree of contribution of the content based on the information of the user who wrote or viewed the comment. For example, if there are many comments from users who have written a large number of posts with high contribution, or if there are many comments from users who are judged as normal users, it may be a basis for the high contribution rating.

In one embodiment, the content page log data includes: a text length included in the content, whether an image is included in the content, the number of images included in the content, an image file type included in the content, and an image capacity included in the content , The number of image downloads included in the content, the creation time of the content, the number of views of the content page, the number of likes of the content page, the number of dislikes of the content page, the number of reports of the content page, the report type of the content page, It may further include the number of comments on the content page and a residence time of the content page.

In an embodiment, at least two or more of the collected log data are selected, and data determined by the ratio of the data may be used as a criterion for measuring the degree of contribution. For example, through a value obtained by dividing the number of likes by the number of content views, it is possible to evaluate the contribution of content that has an excessively large number of likes compared to the number of views. However, the present invention is not limited to this example, and all methods that can be used as a criterion for determining a content contribution level by linking previously collected log data may be included in the present invention.

According to the present embodiment, various attributes that can be extracted from the content page log data may be reflected to measure the contribution.

Hereinafter, with reference to FIG. 7, it will be described in more detail. 7 is a flow chart for specifically explaining some operations of the method described with reference to FIG. 6.

Referring to FIG. 7, step S420 is embodied.

In step S421, the contribution is calculated by a plurality of models included in the ensemble model, the average value of the contribution calculated in step S422 is obtained, and in step S423, if the determination coefficient falls within the reference range, the content is determined in step S424. Contribution is determined.

In an embodiment, a feature extracted using content page log data may be extracted as a vector, and the feature vector may be used as training and verification data. The ratio of training and verification data can be selected as 8:2 or 7:3, as in a general machine learning process. However, the present invention is not limited thereto.

In an embodiment, when learning the contribution measurement model, neurons constituting the neural network may be expressed as follows.

,

,

y is an activation function, and various functions such as sigmoid, tanh, and ReLU may be applied. β is the weight for each feature and is optimized through the artificial intelligence learning process. ε may be a bias that is initialized and set for learning. ε is noise, and may include, for example, noise such as a data storage error and server rack. x is a feature extracted from the log data of the content page. n means the number of users. During the supervised learning process, parameters for each model can be tuned, and model performance can be verified and evaluated through engineering work to generate an object type classification model.

In one embodiment, the following equation may be used to initially set an individual y value. For example, the initial value of y1 of user 1 is as follows. However, it is not limited thereto.

In an embodiment, it may be verified whether the contribution of the model learned by the artificial neural network is a significant value. For this verification, the verification data described above may be used. The difference between the contribution obtained by inputting the verification data into the model and the actual value can be obtained as a coefficient of determination. Here, when the determination coefficient falls within the reference range, it is determined to be significant, and the contribution degree may be determined. For example, the reference range of the determination coefficient may be set to a range of 0.5 or more and 1 or less. However, it is not limited thereto.

In an embodiment, an ensemble model combining several models may be used to improve the performance of measuring contribution. Here, an ensemble model based on supervised learning may be used to perform learning to measure the degree of contribution or measure the degree of contribution. Here, the ensemble model may include Deep Learning, AdaBoost, Neural Network, and Ridge Penalized Regression model. However, it is not limited thereto.

In an embodiment, the average of the contributions measured by several models may be obtained, so that the performance of the contribution measurement may be improved.

A plurality of modules that may be referred to in an embodiment of the present invention may be a module generated by an artificial neural network. Known machine learning techniques for generating modules can be applied. That is, the present invention is not limited to the machine learning technology specifically illustrated to explain the generation of a specific module.

In the method of measuring contribution according to an embodiment of the present invention, a fair contribution may be determined by considering various attributes in measuring the contribution of content. Through the determined contribution, the manager can fairly distribute the profits generated by managing the online community to users. The inflow of users increases through fair distribution of profits, so both users and managers can win-win.

So far, with reference to FIGS. 1 to 7, a method of measuring a contribution degree according to an embodiment of the present invention and its application field have been described. Hereinafter, an exemplary computing device 1500 capable of implementing an apparatus for measuring a contribution degree according to an embodiment of the present invention will be described.

8 is a diagram illustrating a computing device 1500 capable of implementing an apparatus for measuring contribution according to an embodiment of the present invention.

As shown in FIG. 8, the computing device 1500 loads one or more processors 1510, a bus 1550, a communication interface 1570, and a computer program 1591 executed by the processor 1510. The memory 1530 may include a storage 1590 for storing the computer program 1591. However, only the constituent elements related to the embodiment of the present invention are shown in FIG. 8. Accordingly, those of ordinary skill in the art to which the present invention pertains can recognize that other general-purpose components may be further included in addition to the components illustrated in FIG. 8.

The processor 1510 controls the overall operation of each component of the computing device 1500. The processor 1510 includes a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), or any type of processor well known in the technical field of the present invention. Can be. Also, the processor 1510 may perform an operation on at least one application or program for executing the method according to the embodiments of the present invention. The computing device 1500 may include one or more processors.

The memory 1530 stores various types of data, commands, and/or information. The memory 1530 may load one or more programs 1591 from the storage 1590 to execute the method according to embodiments of the present invention. The memory 1530 may be implemented as a volatile memory such as RAM, but the technical scope of the present invention is not limited thereto.

The bus 1550 provides communication functions between components of the computing device 1500. The bus 1550 may be implemented as various types of buses such as an address bus, a data bus, and a control bus.

The communication interface 1570 supports wired/wireless Internet communication of the computing device 1500. In addition, the communication interface 1570 may support various communication methods other than Internet communication. To this end, the communication interface 1570 may be configured to include a communication module well known in the art.

According to some embodiments, the communication interface 1570 may be omitted.

The storage 1590 may non-temporarily store the one or more programs 1591 and various data.

The storage 1590 is a nonvolatile memory such as a ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, etc., a hard disk, a removable disk, or well in the technical field to which the present invention pertains. It may be configured to include any known computer-readable recording medium.

The computer program 1591 may include one or more instructions that when loaded into the memory 1530 cause the processor 1510 to perform a method/operation according to various embodiments of the present invention. That is, the processor 1510 may perform methods/operations according to various embodiments of the present disclosure by executing the one or more instructions.

In the above case, the apparatus for measuring a contribution degree according to an embodiment of the present invention may be implemented through the computing device 1500.

So far, various embodiments of the present invention and effects according to the embodiments have been mentioned with reference to FIGS. 1 to 8. The effects according to the technical idea of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

The technical idea of the present invention described with reference to FIGS. 1 to 8 so far may be implemented as a computer-readable code on a computer-readable medium. The computer-readable recording medium is, for example, a removable recording medium (CD, DVD, Blu-ray disk, USB storage device, removable hard disk) or a fixed recording medium (ROM, RAM, computer-equipped hard disk). I can. The computer program recorded on the computer-readable recording medium may be transmitted to another computing device through a network such as the Internet and installed in the other computing device, thereby being used in the other computing device.

In the above, even if all the constituent elements constituting an embodiment of the present invention have been described as being combined into one or operating in combination, the technical idea of the present invention is not necessarily limited to this embodiment. That is, as long as it is within the scope of the object of the present invention, one or more of the components may be selectively combined and operated.

Although the operations are illustrated in a specific order in the drawings, it should not be understood that the operations must be executed in the specific order shown or in a sequential order, or all illustrated operations must be executed to obtain a desired result. In certain situations, multitasking and parallel processing may be advantageous. Moreover, the separation of the various components in the above-described embodiments should not be understood as necessarily requiring such separation, and the described program components and systems may generally be integrated together into a single software product or packaged into multiple software products. It should be understood that there is.

Although the embodiments of the present invention have been described with reference to the accompanying drawings above, those of ordinary skill in the art to which the present invention pertains can also implement the present invention in other specific forms without changing the technical idea or essential features. I can understand that there is. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the technical ideas defined by the present invention.

Claims (19)

In the method performed by the computing device,
Determining whether the content is prohibited on the bulletin board by the prohibited content identification module, wherein the prohibited content identification module is a module for determining whether the content is prohibited on the bulletin board using data extracted from the content;
When it is determined that the content is prohibited in the bulletin board, it is determined by the bad user identification module whether the user who created the content is a bad user, and when it is determined that the content is allowed in the bulletin board, the contribution measurement module Determining the degree of contribution of the content, wherein the bad user identification module is a module that determines whether the user is a bad user by using the user log data who created the content, and the contribution measurement module uses the log data of the content page A module for measuring the degree of contribution, the step; And
When the user is determined to be a bad user by the bad user identification module, a penalty is given to the user, and when the user is determined to be a normal user by the bad user identification module, the contribution of the content is determined by the contribution measurement module. Comprising the step of determining,
How to measure content contribution. The method of claim 1,
The prohibited content identification module,
Whether the content is prohibited on the bulletin board is determined by the data output by inputting the image included in the content into a Convolutional Neural Network (CNN)-based NSFW (Not Safe For Work) detector,
How to measure content contribution. The method of claim 2,
The output data is,
Contains the probability that the image contains prohibited items,
Determining whether the content is prohibited on the bulletin board,
In case the probability is greater than or equal to a reference value, determining that the content is prohibited in the bulletin board,
How to measure content contribution. The method of claim 3,
The reference value,
In the case of content created by a user determined as a bad user by the bad user identification module, it is formed lower than the content created by a normal user,
How to measure content contribution. The method of claim 1,
The content is,
Contains text and images,
The prohibited content identification module,
A first module in which it is determined whether the content is prohibited on the bulletin board by text included in the content, and a second module in which it is determined whether or not the content is prohibited in the bulletin board by an image included in the content. Including modules,
How to measure content contribution. The method of claim 5,
The prohibited content identification module,
After determining whether the content is prohibited on the bulletin board by the first module, it is determined whether or not the content is prohibited on the bulletin board by the second module,
How to measure content contribution. The method of claim 1,
The bad user identification module,
It is generated by obtaining a plurality of features extracted from the user log data, inputting a training dataset composed of the plurality of features into an artificial neural network, and training the artificial neural network,
How to measure content contribution. The method of claim 7,
The plurality of features,
Member since, access days, number of posts, number of comments, number of content views, number of likes, number of dislikes, number of reports, number of likes received, number of dislikes received, number of reports received, type of report received, number of deleted posts, number of comments deleted , IP, total dwell time, average dwell time, total clicks and device type,
How to measure content contribution. The method of claim 7,
The artificial neural network,
By ensemble learning,
How to measure content contribution. The method of claim 1,
The user log data,
Member since, access days, number of posts, number of comments, number of content views, number of likes, number of dislikes, number of reports, number of likes received, number of dislikes received, number of reports received, type of report received, number of deleted posts, number of comments deleted , IP, total dwell time, average dwell time, total clicks and device type,
How to measure content contribution. In the method performed by a computing device having an artificial neural network,
Obtaining a plurality of features extracted from the content page log data; And
Including the step of inputting the plurality of features to the artificial neural network, and determining a content contribution degree using data output from the artificial neural network,
The plurality of features,
The click position of a user who clicks a specific part of the content page, the number of clicks of the click position, the previous page of the content page, the page after the content page, the type of viewing device of the content page, the user who writes a comment on the content page Including information and browsing user information of the content page,
How to measure content contribution. The method of claim 11,
The artificial neural network,
By inputting a learning dataset composed of the plurality of features, supervised learning-based ensemble learning,
How to measure content contribution. The method of claim 11,
The plurality of features,
The length of the text included in the content, whether the content contains an image, the number of images included in the content, the image file type included in the content, the image size included in the content, the number of images downloaded in the content, the Content creation time, number of views of the content page, number of likes of the content page, number of dislikes of the content page, number of reports of the content page, report type of the content page, number of comments on the content page, and number of comments on the content page Further comprising a residence time,
How to measure content contribution. In the method performed by a computing device having an artificial neural network,
Obtaining features extracted from text included in the content; And
Inputting the feature into the artificial neural network, and determining whether the content is prohibited on the bulletin board using data output from the artificial neural network,
The step of obtaining the feature,
Including the step of extracting the Korean alphabet, alphabet, number, and special characters as similar vectors according to the appearance similarity of the text,
How to identify prohibited content. The method of claim 14,
The output data is,
Contains the probability that the text contains a prohibited item,
Determining whether the content is prohibited on the bulletin board,
In case the probability is greater than or equal to a reference value, determining that the content is prohibited in the bulletin board,
How to identify prohibited content. The method of claim 15,
The prohibited items above are:
Belonging to any one of abusive group, pornographic group, violent group, and copyright violation group,
How to identify prohibited content. The method of claim 15,
Further comprising the step of adding the first item as a prohibited item,
The first item is located within a reference distance based on the first prohibited item, and a plurality of contents having a probability of including the first prohibited item being equal to or greater than a reference value include the first item at least a reference number of times,
How to identify prohibited content. The method of claim 14,
The output data is,
Contains the probability that the text contains a required item,
Determining whether the content is prohibited on the bulletin board,
If the probability is less than the reference value, comprising the step of determining that the content is prohibited in the bulletin board,
How to identify prohibited content. The method of claim 18,
The required items above are:
That is determined by the category of the bulletin board,
How to identify prohibited content.

Images