KR102477565B1 - Method and system for predicting reporting information for posts - Google Patents

Abstract

The present disclosure relates to a post reporting prediction method. The post reporting prediction method includes the steps of: training a first machine learning model using information for a first learning post; training the second machine learning model using information for a second learning post and reporting information for the second learning post; and predicting reporting information for a post based on the information for the post using the trained second machine learning model. The second machine learning model is a model obtained by adding one or more layers to the trained first machine learning model.

Description

Post reporting prediction method and system{METHOD AND SYSTEM FOR PREDICTING REPORTING INFORMATION FOR POSTS}

The present disclosure relates to a post report prediction method and system, and more specifically, for an overall understanding of posts posted on an online platform, a first machine learning model is pre-trained, and one of the pre-trained first machine learning models is used. The present invention relates to a method and system for predicting report information about a post using the second machine learning model that has been learned after transfer learning of the second machine learning model to which the above layers are added.

Due to the rapid growth of information and communication technology and the widespread use of information and communication devices such as the Internet, PCs, and smartphones, the act of posting posts on online platforms, viewing posts posted on online platforms, or commenting on posts, etc. Interacting with other people has become a daily routine. The moment posts are posted on online platforms, all Internet users around the world can read them. For this reason, it is necessary for administrators of online platforms to quickly take appropriate action against inappropriate posts posted on online platforms.

However, it is inefficient for online platform administrators to directly monitor all of the numerous posts posted on the online platform, and in the case of popular online platforms where numerous posts are posted in one second, this is close to impossible. Using the report prediction model to monitor only posts for which reports are predicted and then taking appropriate measures, there is an advantage in that limited resources can be used efficiently. In order to efficiently operate an online platform using this report prediction model, it is necessary to improve the prediction accuracy of the report prediction model as much as possible. In addition, the reporting prediction model may output inaccurate prediction results for postings with unlearned patterns, so there is a problem in that prompt action may not be taken for inappropriate postings.

The present disclosure provides a posting reporting prediction method, a computer program stored in a recording medium, and a device (system) to solve the above problems.

The present disclosure may be implemented in a variety of ways, including a method, apparatus (system) or computer program stored on a readable storage medium.

According to an embodiment of the present disclosure, a post report prediction method, performed by at least one processor, includes training a first machine learning model using information on a first learning post; Training a second machine learning model using the information about the post and reporting information about the second learning post, and predicting the reporting information about the post based on the information about the post using the learned second machine learning model. And, the second machine learning model is a model in which one or more layers are added to the learned first machine learning model.

According to an embodiment of the present disclosure, the information on the first learning post and the information on the second learning post may include an image included in the learning post, a category of the learning post, price information associated with the learning post, or a region associated with the learning post. contains at least one of the information

According to an embodiment of the present disclosure, the information about the first learning post includes a plurality of information items about the first learning post, and the step of training the first machine learning model includes a plurality of information items about the first learning post. Masking some of the information items, and receiving information about a first learning post in which some of the information items are masked, and training a first machine learning model to predict some of the masked information items.

According to an embodiment of the present disclosure, some of the information items to be masked are selected from among an image included in the first learning posting, a category of the first learning posting, price information related to the first learning posting, or region information related to the first learning posting. contains at least one

According to an embodiment of the present disclosure, the step of learning the second machine learning model includes receiving information about the second learning post and training the second machine learning model to predict reporting information about the second learning post. Include steps.

According to an embodiment of the present disclosure, the step of predicting report information about a post may include a probability of reporting a post and a plurality of posts about a report, based on information about a post, using a second machine learning model that has been learned. Predicting the probability of each of the reported items is included.

According to an embodiment of the present disclosure, the step of predicting report information for a post may include determining a final report probability of each of a plurality of report items for a post based on a report probability for a post and a probability of each of a plurality of report items. A step of calculating is further included.

According to an embodiment of the present disclosure, the method further includes detecting an abnormal post based on the predicted reporting information, wherein the abnormal posting is determined by a first predefined threshold value when the sum of two or more final reporting probabilities among a plurality of reporting items is set. or a post in which the variance of two or more final report probabilities among a plurality of reported items is less than the second predefined threshold, or the sum of the final reported probabilities of two or more among the plurality of reported items is greater than the first predefined threshold and a plurality of reported items A variance of two or more of the final report probabilities is less than or equal to a second predefined threshold.

A computer program stored in a computer readable recording medium is provided to execute the method according to an embodiment of the present disclosure on a computer.

An information processing system according to an embodiment of the present disclosure includes a memory and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program includes: 1 The first machine learning model is trained using the information on the learning post, the second machine learning model is trained using the information on the second learning post and the report information on the second learning post, and the learned Using a second machine learning model, the second machine learning model includes instructions for predicting report information about a post based on information about the post, wherein one or more layers are added to the first machine learning model that has been trained. is a model that has been

According to some embodiments of the present disclosure, a pre-trained machine learning model using information about training posts may have general characteristics about posts posted on an online platform (e.g., features hidden in information contained in posts, features hidden in posts). correlation among a plurality of included information items, etc.) can be learned.

According to some embodiments of the present disclosure, not only text content included in the learning post among information about the learning post, but also images, category information, price information, region information, etc. included in the learning post are tokenized and at least some of them are masked By training the machine learning model by doing so, the machine learning model can be trained to understand the characteristics of posts in a more diversified way.

According to some embodiments of the present disclosure, by constructing a report information prediction model using a machine learning model that has completed learning about general characteristics of posts, the accuracy of forecasting report information can be improved, and based on the predicted report information. can take more prompt and appropriate action on inappropriate postings.

According to some embodiments of the present disclosure, by detecting even abnormal posts that are difficult to detect by a general report prediction model, prompt and appropriate measures may be taken for abnormal posts.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned are clear to those skilled in the art (referred to as "ordinary technicians") from the description of the claims. will be understandable.

Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein like reference numbers indicate like elements, but are not limited thereto.
1 is a diagram illustrating an example of a posting report prediction method according to an embodiment of the present disclosure.
2 is a block diagram showing the internal configuration of an information processing system according to an embodiment of the present disclosure.
3 is a functional block diagram showing the internal configuration of a processor according to an embodiment of the present disclosure.
4 to 6 are diagrams illustrating examples of a method of learning a first machine learning model according to an embodiment of the present disclosure.
7 is a diagram illustrating an example of predicting report information using a second machine learning model according to an embodiment of the present disclosure.
8 is a diagram illustrating an example of predicting report information using a second machine learning model according to an embodiment of the present disclosure.
9 is a diagram illustrating an example of calculating a final report probability of each of a plurality of report items for a posting according to an embodiment of the present disclosure.
10 is a diagram illustrating an example of a final reporting probability of each of a plurality of reporting items calculated for postings according to an embodiment of the present disclosure.
11 is a flowchart illustrating an example of a posting report prediction method according to an embodiment of the present disclosure.

Hereinafter, specific details for the implementation of the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, if there is a risk of unnecessarily obscuring the gist of the present disclosure, detailed descriptions of well-known functions or configurations will be omitted.

In the accompanying drawings, identical or corresponding elements are given the same reference numerals. In addition, in the description of the following embodiments, overlapping descriptions of the same or corresponding components may be omitted. However, omission of a description of a component does not intend that such a component is not included in an embodiment.

Advantages and features of the disclosed embodiments, and methods of achieving them, will become apparent with reference to the following embodiments in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms, but only the present embodiments make the present disclosure complete, and the present disclosure does not extend the scope of the invention to those skilled in the art. It is provided only for complete information.

Terms used in this specification will be briefly described, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected from general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but they may vary according to the intention of a person skilled in the related field, a precedent, or the emergence of new technology. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the terms and the general content of the present disclosure, not simply the names of the terms.

Expressions in the singular number in this specification include plural expressions unless the context clearly dictates that they are singular. Also, plural expressions include singular expressions unless the context clearly specifies that they are plural. When it is said that a certain part includes a certain component in the entire specification, this means that it may further include other components without excluding other components unless otherwise stated.

Also, the term 'module' or 'unit' used in the specification means a software or hardware component, and the 'module' or 'unit' performs certain roles. However, 'module' or 'unit' is not meant to be limited to software or hardware. A 'module' or 'unit' may be configured to reside in an addressable storage medium and may be configured to reproduce one or more processors. Thus, as an example, a 'module' or 'unit' includes components such as software components, object-oriented software components, class components, and task components, processes, functions, and attributes. , procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, or variables. Functions provided within components and 'modules' or 'units' may be combined into a smaller number of components and 'modules' or 'units', or further components and 'modules' or 'units'. can be further separated.

According to one embodiment of the present disclosure, a 'module' or 'unit' may be implemented with a processor and a memory. 'Processor' should be interpreted broadly to include general-purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, and the like. In some circumstances, 'processor' may refer to an application specific integrated circuit (ASIC), programmable logic device (PLD), field programmable gate array (FPGA), or the like. 'Processor' refers to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or a combination of any other such configurations. You may. Also, 'memory' should be interpreted broadly to include any electronic component capable of storing electronic information. 'Memory' includes random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable-programmable read-only memory (EPROM), It may also refer to various types of processor-readable media, such as electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, and the like. A memory is said to be in electronic communication with the processor if the processor can read information from and/or write information to the memory. Memory integrated with the processor is in electronic communication with the processor.

In the present disclosure, a 'system' may include at least one of a server device and a cloud device, but is not limited thereto. For example, a system may consist of one or more server devices. As another example, a system may consist of one or more cloud devices. As another example, the system may be operated by configuring a server device and a cloud device together.

In the present disclosure, a 'machine learning model' may include any model used to infer an answer to a given input. According to one embodiment, the machine learning model may include an artificial neural network model including an input layer (layer), a plurality of hidden layers, and an output layer. Here, each layer may include a plurality of nodes. In some embodiments of the present disclosure, at least some of the plurality of machine learning models described as separate machine learning models may be implemented as one machine learning model. Similarly, in some embodiments of the present disclosure, a model described as one machine learning model may be implemented as a plurality of machine learning models.

In the present disclosure, 'each of a plurality of A' or 'each of a plurality of A' may refer to each of all components included in a plurality of A's, or each of some components included in a plurality of A's. .

1 is a diagram illustrating an example of a posting report prediction method according to an embodiment of the present disclosure. According to one embodiment, the information processing system may predict report information for postings posted on the online platform. Here, the online platform may include any online platform or a specific online platform accessible through an application installed on a user terminal or a web browser. For example, as shown in FIG. 1 , the information processing system may predict whether or not to report information and/or reported item information of each used trading post posted on a used trading platform. As a specific example, the information processing system can predict that normal (non-inappropriate) postings posted on the used trading platform will not be reported (shown as 'unreported' in FIG. 1), and a handgun that is prohibited for sale can be sold. It can be predicted that the postings 110 that do so will be reported as a report item of 'prohibited sales'.

The information processing system may use a machine learning model to predict report information on posts. According to an embodiment, the information processing system may pre-train a first machine learning model for overall understanding of posts posted on an online platform. Then, the information processing system may fine-tune a second machine learning model to predict report information about posts, where the second machine learning model has completed learning about general characteristics of posts. It may be configured using a first machine learning model. In this way, by constructing a reporting information prediction model using the first machine learning model that has been trained on the general characteristics of posts, it is possible to improve the accuracy of reporting information prediction, and for inappropriate posts based on the predicted reporting information More prompt and appropriate action can be taken.

On the other hand, even if it is an inappropriate post, it is a post that does not belong to any of the regulated reporting items, a post for which it is ambiguous to define the reason for reporting, or a new type of post for which similar posts were not included in the learning data (e.g. quarantine pass ( vaccine pass), posts selling newborn babies, etc.) that are inappropriate, but for which it is difficult to predict whether or not to report and/or what to report by the report prediction model (hereinafter referred to as abnormal posts), action will be taken promptly. may not support For example, a general report prediction model can relatively accurately predict whether or not a report is reported for a post 110 selling a gun that is clearly prohibited for sale, but whether a post 120 selling a newborn baby is reported. Inaccurate prediction results for (122) or reporting item (124) may be output. According to some embodiments of the present disclosure, the information processing system can detect such abnormal posts, so that prompt and appropriate action can be taken for the abnormal posts.

Hereinafter, for a smooth understanding of the present disclosure, a used trading post posted on a used trading platform will be described as a specific example of a 'post', but the scope of the present disclosure is not limited thereto. Various types of posts posted on any online platform may correspond to 'posts' in the present disclosure.

2 is a block diagram showing an internal configuration of an information processing system 200 according to an embodiment of the present disclosure. The information processing system 200 may include a memory 210, a processor 220, a communication module 230 and an input/output interface 240. As shown in FIG. 2 , the information processing system 200 may be configured to communicate information and/or data through a network using the communication module 230 .

Memory 210 may include any non-transitory computer readable recording medium. According to one embodiment, the memory 210 may include a permanent mass storage device such as read only memory (ROM), disk drive, solid state drive (SSD), flash memory, and the like. can As another example, a non-perishable mass storage device such as a ROM, SSD, flash memory, disk drive, etc. may be included in the information processing system 200 as a separate permanent storage device separate from memory. In addition, the memory 210 may store an operating system and at least one program code (eg, a code for learning a machine learning model installed and driven in the information processing system 200, predicting reported information, etc.).

These software components may be loaded from a computer-readable recording medium separate from the memory 210 . A recording medium readable by such a separate computer may include a recording medium directly connectable to the information processing system 200, for example, a floppy drive, a disk, a tape, a DVD/CD-ROM drive, a memory card, etc. It may include a computer-readable recording medium. As another example, software components may be loaded into the memory 210 through the communication module 230 rather than a computer-readable recording medium. For example, at least one program is a computer program (eg, machine learning model learning, It may be loaded into the memory 210 based on a program for predicting report information, etc.).

The processor 220 may be configured to process commands of a computer program by performing basic arithmetic, logic, and input/output operations. Commands may be provided to a user terminal (not shown) or other external system by the memory 210 or the communication module 230 . For example, the processor 220 trains a first machine learning model using information on a first training post, and uses information on a second training post and report information on a second learning post to train a second machine learning model. The learning model can be trained. In addition, the processor 220 may predict report information about a post based on the information about the post by using the learned second machine learning model.

The communication module 230 may provide a configuration or function for a user terminal (not shown) and the information processing system 200 to communicate with each other through a network, and the information processing system 200 may provide an external system (for example, a separate configuration or function to communicate with a cloud system, etc.). For example, control signals, commands, data, etc. provided under the control of the processor 220 of the information processing system 200 are transmitted through the communication module 230 and the network to the user terminal and/or the communication module of the external system. It may be transmitted to a terminal and/or an external system. For example, a user terminal (eg, an online platform manager's terminal) and/or an external system may receive information about a posting predicted to be reported, information about an abnormal posting, and the like from the information processing system 200 .

In addition, the input/output interface 240 of the information processing system 200 is connected to the information processing system 200 or means for interface with a device (not shown) for input or output that the information processing system 200 may include. can be In FIG. 2 , the input/output interface 240 is shown as an element configured separately from the processor 220 , but is not limited thereto, and the input/output interface 240 may be included in the processor 220 . The information processing system 200 may include more components than those of FIG. 2 . However, there is no need to clearly show most of the prior art components.

3 is a functional block diagram showing an internal configuration of a processor 220 according to an embodiment of the present disclosure. As shown, the processor 220 may include a pre-learning unit 310, a transfer learning unit 320, a report prediction unit 330, and the like. According to an embodiment, the pre-learning unit 310 may train a first machine learning model to understand overall characteristics of posts. For example, the pre-learning unit 310 may train a first machine learning model by using information on a first learning posting stored in the learning data DB 340 . As a specific example, the pre-learning unit 310 masks some of the information about the first learning post, inputs the partially masked information about the first learning post to the first machine learning model, and predicts the information before the masking A first machine learning model may be trained to do so. A specific example of learning the first machine learning model by the pre-learning unit 310 will be described later in detail with reference to FIGS. 4 to 6 . The first machine learning model is pre-trained on general characteristics of posts posted on online platforms (eg, specific online platforms, specific online platform groups, arbitrary online platforms, etc.) (eg, information included in posts). hidden characteristics, correlations between multiple information items included in posts, etc.) can be learned.

According to an embodiment, the transfer learning unit 320 may train a second machine learning model for predicting report information on posts. Here, the second machine learning model may be a model obtained by adding one or more layers to the first machine learning model learned by the prior learning unit 310 . For example, the second machine learning model may be a model obtained by adding a classifier for prediction of report information (whether not reported and/or reported items, etc.) to the first machine learning model. According to an embodiment, in the course of learning the second machine learning model by the transfer learning unit 320, weights or parameters of the first machine learning model included in the second machine learning model may be newly adjusted.

According to an embodiment, the transfer learning unit 320 uses information about the second learning post and report information (eg, whether or not it has been reported and/or reported items) about the second learning post, and information about the post. Based on , a second machine learning model may be trained to predict reporting information for a post. A method for the transfer learning unit 320 to learn the second machine learning model will be described later in detail with reference to FIGS. 7 to 8 .

According to an embodiment, the report prediction unit 330 may predict report information about a post by using the learned second machine learning model. For example, the report prediction unit 330 uses the second machine learning model, based on the information about the post, the probability of reporting (or whether to report) and / or each of a plurality of reporting items about the post. can predict the probability of (or at least one reported item). An example in which the report prediction unit 330 predicts report information for a post using the second machine learning model will be described later in detail with reference to FIGS. 7 and 8 .

In addition, the report prediction unit 330 reports the final report of each of the plurality of report items for the post based on the probability of reporting the post predicted by the second machine learning model and the probability of each of the plurality of report items for the post. probabilities can be calculated. In this regard, it will be described later in detail with reference to FIG. 9 .

Additionally, the report prediction unit 330 may detect normal posts, expected reports, and/or abnormal posts based on the report information predicted by the second machine learning model. For example, the report prediction unit 330 reports the post when the probability of reporting the post is greater than or equal to the first threshold, or the probability of at least one of the probabilities of each of the plurality of reported items for the post is greater than or equal to the second threshold. It can be detected as an expected post. As another example, the reporting prediction unit 330 may select a post when the sum of at least two or more of the final reporting probabilities of each of the plurality of reporting items for the post is greater than or equal to the third threshold and/or the variance of at least two or more is less than or equal to the fourth threshold. It can be detected as an abnormal post. In this regard, it will be described later in detail with reference to FIG. 10 .

In case of detecting posts expected to report and/or abnormal posts, the report prediction unit 330 connects to the information processing system or communicates with internal/external systems, user terminals (terminals of administrators of online platforms), etc. Alternatively, information indicating that an abnormal posting has been detected, information on the detected posting, and the like may be provided. Based on this information, the administrator of the online platform can take prompt and appropriate action against expected posts and/or abnormal posts.

According to an embodiment, the learning data DB 340 may store learning data used for learning the machine learning model (the first machine learning model and the second machine learning model). For example, the learning data DB 340 includes information on a first learning post for learning a first machine learning model, information about a second learning post for learning a second machine learning model, and information on a second learning post for learning a second machine learning model. Report information can be stored. In this disclosure, in order to distinguish and refer to learning posts for learning the first machine learning model and the second machine learning model, the first learning post and the second learning post are referred to as separate learning posts, but by this reference The scope of the present disclosure is not limited. That is, the first learning posts and the second learning posts may include the same learning posts or may be composed of different non-overlapping learning posts. Also, each of the first learning post and the second learning post may refer to a plurality of learning postings.

According to an embodiment, the information on the learning post may include a plurality of information items on the learning post. For example, the information on the first learning post may include the title of the first learning post, contents of the first learning post, images included in the first learning posting, category of the first learning posting, and price information associated with the first learning posting. or at least one of local information associated with the first learning post. Similarly, information about the second learning post may also include the title of the second learning post, the content of the second learning post, the image included in the second learning post, the category of the second learning post, price information associated with the second learning post, or At least one of local information associated with the second learning post may be included. In addition, the report information on the second learning post may include information on whether or not the second learning post has been reported, information on reported items, and the like.

In FIG. 3, the configuration of the processor 220 has been described separately for each function, but this does not necessarily mean that they are physically separated. For example, although the pre-learning unit 310 and the transfer learning unit 320 have been separately described, this is only to aid understanding of the invention, and one arithmetic unit installed in the processor 220 may perform two or more functions. may be

4 to 6 are diagrams illustrating an example of a method of learning a first machine learning model 410 according to an embodiment of the present disclosure.

4 illustrates an example of a first machine learning model 410 and information 400 on a first learning post for training the first machine learning model according to an embodiment of the present disclosure. The first machine learning model 410 may be trained to understand general characteristics of posts posted on the online platform using the information 400 on the first learning posts. In one embodiment, the information 400 about the first learning post includes the title of the first learning post, content of the first learning post, images included in the first learning post, category of the first learning post, first learning post It may include price information associated with, region information associated with the first learning post, and the like.

As an example of learning the first machine learning model 410, the information processing system may self-supervised learn the first machine learning model 410 using the information 400 on the first learning post. there is. Specifically, the information processing system may train the first machine learning model 410 to infer a prediction result 430 by receiving information 420 about a post partially masked. Here, the prediction result 430 inferred by the first machine learning model 410 may include information before masking of the masked portion of the input.

5 illustrates an example for intuitively understanding a method of self-supervised learning for first machine learning according to an embodiment of the present disclosure. According to an embodiment, the information processing system may mask some of the information about the training post and train the first machine learning model to infer the masked information. For example, the information processing system masks information about images included in 'A' training posts among information 512 on 'A' training posts, and performs machine learning to infer images included in 'A' training posts. model can be trained. According to an embodiment, inferring the image included in the 'A' learning post may be selecting an image included in the 'A' learning post from among the image list 516 including a plurality of images, where , The image list 516 may be composed of at least some of a plurality of images included in a plurality of learning posts.

As another example, the information processing system may mask 524 some of the text information of the information 520 about 'me' learning post, and train the first machine learning model to infer the masked information. According to an embodiment, inferring each of the text information masked from the 'me' learning post may be selecting at least one of a text list including a plurality of text information, where the text list is a plurality of learning posts. It may consist of at least a part of a plurality of (tokenized) text information included in the post.

As another example, the information processing system may mask 534 price information associated with 'many' learning postings among information on 'many' learning postings, and train the first machine learning model to infer the masked price information. . According to an embodiment, inferring the price information associated with the 'many' learning posts may be selecting a price range to which the price information associated with the 'many' learning posts belongs from the list of price ranges 536 .

In the examples described above, information about a learning post (or multiple information items about a learning post) may be tokenized and/or converted into an embedding vector and used for learning. This will be described later in detail with reference to FIG. 6 .

In this way, among the information about learning posts, not only text contents included in learning posts, but also images, category information, price information, and regional information included in learning posts are tokenized, and at least some of them are masked to learn a machine learning model. By doing so, a machine learning model can be trained to understand the characteristics of posts in a more diverse way.

6 shows a specific example of learning the first machine learning model 410 according to an embodiment of the present disclosure. According to one embodiment, the information processing system may tokenize information about the learning post and convert the tokenized information into an embedding vector. In some embodiments of the present disclosure, information about (learning) posts input to a machine learning model in a learning process of a machine learning model or an inference/prediction process using the machine learning model may refer to tokenized information. Similarly, in some embodiments of this disclosure, token/tokenized information may refer to an embedding vector for token/tokenized information.

According to an embodiment, the information processing system may train the first machine learning model 410 to receive a plurality of tokens 610 and infer a prediction result for each token. A plurality of tokens 610 input to the first machine learning model 410 may be tokenized information about learning posts, and some of the tokens 612 are mask tokens (shown as [MASK] in FIG. 6). ) can be replaced with That is, some of the information about the training post input to the first machine learning model 410 may be masked. The first machine learning model 410 may be trained to output tokens before masking as prediction results 622 for some masked tokens.

For example, information about 'D' learning post may be tokenized and input to the first machine learning model 410, and among them, information about an image included in 'D' learning post may be at least one mask token. It may be replaced with and input to the first machine learning model 410. Here, the first machine learning model 410 may be trained to output information obtained by tokenizing an image included in the 'Ra' training post as a prediction result of at least one mask token.

According to some embodiments of the present disclosure, not only text content included in the learning post among information about the learning post, but also images, category information, price information, region information, etc. included in the learning post are tokenized and at least some of them are masked By training the machine learning model by doing so, the machine learning model can be trained to understand the characteristics of posts in a more diversified way.

7 is a diagram illustrating an example of predicting report information using a second machine learning model 710 according to an embodiment of the present disclosure. According to an embodiment, the information processing system may predict report information about a post by using the second machine learning model 710 . For example, the information processing system may predict whether to report a post (report probability) 730 based on the information 720 about a post using the second machine learning model 710 . Here, the second machine learning model 710 may be a model in which a classifier 714 for predicting whether or not to report has been added to the first machine learning model 712 for which general characteristics of posts have been pre-learned.

As a specific example of predicting report information, the information processing system provides information about 'A' post (title of 'A' post, contents of 'A' post, images included in 'A' post, category of 'A' post, Price information related to 'A' post, local information related to 'A' post, etc.) are input to the second machine learning model 710 to report whether the output 'A' post is reported (eg, reported or unreported). ) or report probability (eg, 0 or more and 1 or less real numbers), it is possible to predict whether the 'A' post is a post that is predicted to be reported.

According to an embodiment, the second machine learning model 710 may be supervised learning using information about the second learning post and reporting information (eg, whether or not it has been reported) 700 about the second learning post. there is. In some embodiments, as the second machine learning model 710 undergoes transfer learning, weights or parameters of the first machine learning model 712 included in the second machine learning model 710 may be newly adjusted.

Information (700, 720) may be tokenized information or an embedding vector for tokenized information.

8 is a diagram illustrating an example of predicting report information using a second machine learning model 810 according to an embodiment of the present disclosure. In FIG. 8, contents overlapping with the example described above with reference to FIG. 7 are omitted or briefly described, and descriptions are focused on added or changed parts. According to one embodiment, the information processing system can predict whether to report and report items as report information. For example, the information processing system uses the second machine learning model 810, based on the information 820 on the post, to determine the probability 832 of reporting the post and each of a plurality of reporting items about the post. Probability 834 can be predicted. Here, the second machine learning model 810 may be a model in which a classifier 814 is added to the first machine learning model 812 for which general characteristics of posts have been pre-learned, and to predict whether or not to report and to predict reported items. .

Additionally, the information processing system may calculate a final report probability of each of a plurality of report items for a post, based on the report probability 832 for the post and the probability 834 for each of the plurality of report items. In this regard, it will be described later in detail with reference to FIG. 9 .

As a specific example of predicting report information, the information processing system inputs information on post 'A' into the second machine learning model 810, and the reporting probability (for example, 0.75) for output 'A' post and Based on the probability of each of the multiple reported items (e.g., the probability of 'prohibited item' is 0.78, the probability of 'non-trade' is 0.02, …, the probability of 'other' is 0.03), 'A' post is reported It is possible to predict whether a post is a predicted post and with which reporting item the report is predicted.

According to an embodiment, the second machine learning model 810 performs supervised learning using information on the second learning post and report information on the second learning post (eg, whether or not it has been reported and reported items) 800. It can be. In some embodiments, as the second machine learning model 810 undergoes transfer learning, weights or parameters of the first machine learning model 812 included in the second machine learning model 810 may be newly adjusted.

9 is a diagram illustrating an example of calculating a final report probability 910 of each of a plurality of report items for a posting according to an embodiment of the present disclosure. According to an embodiment, the information processing system calculates a final report probability 910 of each of a plurality of report items for a post based on a report probability 832 for a post and a probability 834 for each of a plurality of report items. can do. For example, the information processing system multiplies the report probability 832 for a post and the probability 834 of each of a plurality of report items output by the second machine learning model, so that each of a plurality of report items for a post A final report probability 910 may be calculated.

As a specific example, by multiplying the reporting probability of 0.75 for posts by the probability of 0.78 for 'prohibited items for sale', the final probability of 'prohibited items for sale' can be calculated. By multiplying the probability 0.02 of the reported item 'not a transactional post', the final probability of the reported item 'not a transactional post' can be calculated as 0.015. In this way, it is possible to calculate the final report probability 910 of each of the plurality of report items for the post.

10 is a diagram illustrating examples of final reporting probabilities 1014, 1024, and 1034 of each of a plurality of reporting items calculated for postings according to an embodiment of the present disclosure. The final reporting probabilities (1014, 1024, 1034) of each of the plurality of reporting items calculated for the first posting, second posting, and third posting are information on the first posting (1012) and information on the second posting ( 1022), and information 1032 on the third post is input into the second machine learning model 1000 according to an embodiment of the present disclosure.

The first post is an example of a normal post (for example, a post selling clothes) for which reporting is not expected. Looking at the final reporting probability 1014 of each of the plurality of reporting items calculated for the first post, it can be seen that each probability (0.13, 0.02, ..., 0.07) is low, and the sum of each probability is also low. In this way, a post having a low sum of the final reporting probabilities 1014 of each of the plurality of reporting items may be predicted as a normal posting for which reporting is not expected.

The second post is an example of a typical post where reporting is predicted (eg, a post selling a handgun). Looking at the final reporting probability (1024) of each of the plurality of reporting items calculated for the second posting, it can be seen that the final reporting probability is concentrated on the 'prohibited item for sale'. In this way, posts in which the final reporting probability is concentrated in some reporting items can be predicted as typical postings for which reporting is predicted.

The third post is an example of an abnormal post (eg, a post selling newborn babies, a post selling quarantine passes, etc.). Abnormal posts are posts that are inappropriate or require appropriate action by the manager/administrator, such as posts that do not fall under any of the regulated reporting items, posts for which the reason for reporting is ambiguous, or posts of a new type that did not include similar posts in the learning data. and may refer to posts for which accurate reporting and/or reporting items are difficult to predict by a report prediction model. Looking at the final report probability 1034 of each of the plurality of report items calculated for the third posting, it can be seen that the final report probability is not concentrated on a specific report item, but the probability is evenly distributed over several report items.

According to the general report prediction model, abnormal posts such as the third post are often not distinguished/classified separately from typical posts that are expected to be reported, such as the second post, because the sum of the final report probabilities of each reported item is high. According to some embodiments of the present disclosure, the sum of the final reporting probabilities of two or more of the plurality of reporting items is equal to or greater than a first predefined threshold, and the variance of the two or more final reporting probabilities of the plurality of reporting items is a second predefined threshold. Abnormal posts can be detected by detecting posts that are below the set threshold. Accordingly, after the administrator/operator reviews the posts detected as abnormal, the posts are private/deleted, sanctions are imposed on users who have posted abnormal posts, new reporting items are created appropriately, or existing reporting items are reorganized. Appropriate and prompt action may be taken against abnormal posts, such as

11 is a flowchart illustrating an example of a posting report prediction method 1100 according to an embodiment of the present disclosure. According to one embodiment, method 1100 may begin with a processor (eg, at least one processor of an information processing system) training a first machine learning model using information about a first training post. (S1110). In one embodiment, information on the first training post included in training data for training the first machine learning model may include a plurality of information items on the learning post. For example, the information on the first learning post may include the title of the first learning post, content, an image included in the first learning posting, a category of the first learning posting, price information associated with the first learning posting, or the first learning posting. It may include at least one of local information associated with. Here, the (first) learning posts may refer to a plurality of learning posts.

As a specific example of learning the first machine learning model, first, the processor may mask some information items among a plurality of information items for the first learning post. Then, the processor may train the first machine learning model to predict the masked partial information items by receiving information about the first training post in which the partial information items are masked. According to an embodiment, some information items masked herein include: a title of the first learning post, a content of the first learning post, an image included in the first learning post, a category of the first learning post, and a category associated with the first learning post. It may include at least one of price information and region information associated with the first learning post.

According to an embodiment, the first machine learning model pre-trained using information about training posts is a general characteristic (eg For example, characteristics hidden in information included in posts, correlations between a plurality of information items included in posts, etc.) may be learned.

Then, the processor may train a second machine learning model by using the information on the second learning post and the report information on the second learning post (S1120). Here, the second machine learning model may be a model in which one or more layers are added to the learned first machine learning model. For example, the second machine learning model may be a model in which a classifier for predicting report information is added to the first machine learning model that has been learned. As a specific example of learning the second machine learning model, the processor may train the second machine learning model to receive information about the second learning post and predict reporting information about the second learning post.

According to an embodiment, information on the second learning post may also include a plurality of information items on the learning post, similarly to the information on the first learning post. For example, the information about the second learning post may include the title of the second learning post, content, images included in the second learning posting, category of the second learning posting, price information associated with the second learning posting, or second learning posting. It may include at least one of local information associated with. Here, the (second) learning posts may refer to a plurality of learning posts. The information on the second learning post used for learning the second machine learning model may include information on the same learning post as the first learning post used for learning the first machine learning model, and It can also be information about other non-overlapping learning posts. Also, in an embodiment, the report information on the second learning post may include information on whether to report the second learning post and/or information about reporting items on the second learning post. As described above, by constructing and training a second machine learning model for predicting reporting information for posts using the first machine learning model that has learned general characteristics of posts, a high-quality product with improved accuracy in predicting reporting information is provided. Report prediction models can be created.

Thereafter, the processor may predict report information about a post based on the information about the post by using the learned second machine learning model (S1130). For example, the processor may predict a report probability of a post and a probability of each of a plurality of reporting items for a post, based on the information on the post, using the second machine learning model learned. Additionally, the processor may calculate a final report probability of each of a plurality of report items for a post, based on the report probability of the post and the probability of each of the plurality of report items. In this way, by predicting the report information of posts using a high-quality report prediction model with improved accuracy of report information prediction, it is possible to take prompt and appropriate measures for posts that are expected to be reported.

According to an embodiment, the processor may detect an abnormal posting based on predicted reporting information. For example, a post in which the sum of two or more final reporting probabilities of a plurality of reporting items is greater than or equal to a first predefined threshold and/or the variance of two or more final reporting probabilities of a plurality of reporting items is less than or equal to a second predefined threshold It can be detected as an abnormal post. In this way, by detecting even abnormal posts that are not well detected by the general report prediction model, prompt and appropriate measures can be taken.

The flowchart shown in FIG. 11 and the above description are only examples, and may be implemented in various ways. For example, the order of some steps may be changed or one or more steps may be added/changed/deleted to the extent that the technical meaning of the present disclosure is not changed.

The above method may be provided as a computer program stored in a computer readable recording medium to be executed on a computer. The medium may continuously store programs executable by a computer or temporarily store them for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or combined hardware, but is not limited to a medium directly connected to a certain computer system, and may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, and ROM, RAM, flash memory, etc. configured to store program instructions. In addition, examples of other media include recording media or storage media managed by an app store that distributes applications, a site that supplies or distributes various other software, and a server.

The methods, acts or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or combinations thereof. Those skilled in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchange of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques may include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs) ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, and other electronic units designed to perform the functions described in this disclosure. , a computer, or a combination thereof.

Accordingly, the various illustrative logical blocks, modules, and circuits described in connection with this disclosure may be incorporated into a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or may be implemented or performed in any combination of those designed to perform the functions described in A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

In firmware and/or software implementation, the techniques include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), PROM ( on a computer readable medium, such as programmable read-only memory (EPROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage device, or the like. It can also be implemented as stored instructions. Instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

If implemented in software, the techniques may be stored on or transmitted over as one or more instructions or code on a computer readable medium. Computer readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of non-limiting example, such computer readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or desired program code in the form of instructions or data structures. It can be used for transport or storage to and can include any other medium that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium.

For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable , fiber optic cable, twisted pair, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of medium. Disk and disc as used herein include CD, laser disc, optical disc, digital versatile disc (DVD), floppy disk, and blu-ray disc, where disks are usually magnetic data is reproduced optically, whereas discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer readable media.

A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium can be coupled to the processor such that the processor can read information from or write information to the storage medium. Alternatively, the storage medium may be integral to the processor. The processor and storage medium may reside within an ASIC. An ASIC may exist within a user terminal. Alternatively, the processor and storage medium may exist as separate components in a user terminal.

Although the embodiments described above have been described as utilizing aspects of the presently-disclosed subject matter in one or more stand-alone computer systems, the disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. . Further, aspects of the subject matter in this disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across multiple devices. These devices may include PCs, network servers, and portable devices.

Although the present disclosure has been described in relation to some embodiments in this specification, various modifications and changes may be made without departing from the scope of the present disclosure that can be understood by those skilled in the art. Moreover, such modifications and variations are intended to fall within the scope of the claims appended hereto.

110: Posts selling handguns
120: Posts selling newborns
122: Report or not
124: Declaration item

Claims (10)

In the post report prediction method, performed by at least one processor,
learning a first machine learning model by using information on a first learning post;
learning a second machine learning model by using the information on the second learning post and the reporting information on the second learning post; and
Predicting report information about the post based on the information about the post, using the learned second machine learning model.
including,
The second machine learning model is a model in which one or more layers are added to the learned first machine learning model,
The information on the first learning posting includes a plurality of information items on the first learning posting;
Learning the first machine learning model,
masking some information items among a plurality of information items for the first learning post; and
Learning the first machine learning model to predict the masked partial information items by receiving information about a first training post in which the partial information items are masked;
Including, post report prediction method.
According to claim 1,
The information on the first learning post and the information on the second learning post,
A post report prediction method comprising at least one of an image included in the learning post, a category of the learning post, price information associated with the learning post, or local information associated with the learning post.
delete According to claim 1,
Some information items to be masked include:
and at least one of an image included in the first learning posting, a category of the first learning posting, price information associated with the first learning posting, or local information associated with the first learning posting.
According to claim 1,
Learning the second machine learning model,
And receiving information on the second learning post and learning the second machine learning model to predict report information on the second learning post.
According to claim 1,
The step of predicting the reporting information for the post is,
Using the learned second machine learning model, based on the information about the posting, predicting a probability of reporting the post and a probability of each of a plurality of reporting items for the posting, including reporting a posting. prediction method.
According to claim 6,
The step of predicting the reporting information for the post is,
Calculating a final reporting probability of each of the plurality of reporting items for the posting based on the reporting probability of the posting and the probability of each of the plurality of reporting items.
In the post report prediction method, performed by at least one processor,
learning a first machine learning model by using information on a first learning post;
learning a second machine learning model by using the information on the second learning post and the reporting information on the second learning post;
predicting report information about the post based on the information about the post by using the learned second machine learning model; and
Detecting abnormal posts based on the predicted report information
including,
The second machine learning model is a model in which one or more layers are added to the learned first machine learning model,
The step of predicting the reporting information for the post is,
predicting a report probability for the post and a probability of each of a plurality of report items for the post, based on the information on the post, using the learned second machine learning model; and
Calculating a final report probability of each of the plurality of report items for the post based on the report probability of the post and the probability of each of the plurality of report items
including,
The above abnormal post,
A post in which the sum of the final reporting probabilities of two or more of the plurality of reporting items is equal to or greater than a first predefined threshold;
A post in which the variance of the final reporting probability of two or more of the plurality of reporting items is less than or equal to a second predefined threshold, or
A post report prediction method in which the sum of the final report probabilities of two or more of the plurality of report items is equal to or greater than a first predefined threshold and the variance of the final report probabilities of two or more of the plurality of report items is equal to or less than the second predefined threshold.
A computer program stored in a computer-readable recording medium to execute the method according to any one of claims 1, 2, and 4 to 8 on a computer.
As an information processing system,
Memory; and
at least one processor connected to the memory and configured to execute at least one computer readable program contained in the memory
including,
The at least one program,
Training a first machine learning model using the information on the first training post;
Using the information on the second learning post and the reporting information on the second learning post, a second machine learning model is trained,
Including instructions for predicting report information on the post based on the information on the post using the learned second machine learning model;
The second machine learning model is a model in which one or more layers are added to the learned first machine learning model,
The information on the first learning posting includes a plurality of information items on the first learning posting;
Learning the first machine learning model,
Masking some information items among a plurality of information items for the first learning post;
And learning the first machine learning model to predict the masked partial information item by receiving information about a first learning post in which the partial information item is masked.

Images