WO2023132054A1 - 学習システム、学習方法、及びプログラム - Google Patents

学習システム、学習方法、及びプログラム Download PDF

Info

Publication number
WO2023132054A1
WO2023132054A1 PCT/JP2022/000352 JP2022000352W WO2023132054A1 WO 2023132054 A1 WO2023132054 A1 WO 2023132054A1 JP 2022000352 W JP2022000352 W JP 2022000352W WO 2023132054 A1 WO2023132054 A1 WO 2023132054A1
Authority
WO
WIPO (PCT)
Prior art keywords
group
learning model
labeling
learning
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/000352
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
恭輔 友田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rakuten Group Inc
Original Assignee
Rakuten Group Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rakuten Group Inc filed Critical Rakuten Group Inc
Priority to PCT/JP2022/000352 priority Critical patent/WO2023132054A1/ja
Priority to US18/018,269 priority patent/US20240256941A1/en
Priority to JP2022574615A priority patent/JP7302107B1/ja
Priority to TW112100374A priority patent/TWI836840B/zh
Publication of WO2023132054A1 publication Critical patent/WO2023132054A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • the present disclosure relates to learning systems, learning methods, and programs.
  • Non-Patent Document 1 describes a technique of using transfer learning to make a learning model learn a small amount of labeled training data, and labeling a large amount of unlabeled data. ing.
  • labeling using a learning model is performed after transforming unlabeled data so as to resemble the distribution of labeled training data.
  • Non-Patent Document 1 is a technique for automatically labeling unlabeled data, but it simply labels a small amount of randomly selected data. It is nothing more than something to do. Randomly selected data is not data that does not satisfy the labeling conditions, so the technique of Non-Patent Document 1 cannot label data that does not satisfy the labeling conditions without much effort. rice field.
  • One of the purposes of the present disclosure is to carry out labeling of data that does not satisfy labeling conditions without much effort.
  • a learning system includes: a first determination unit that determines whether each of a plurality of first data satisfies a first condition regarding labeling; a first learning model creation unit that creates a first learning model capable of being labeled based on a first group that is a group of the given first data; A second group conversion unit that converts the second group so that the distribution of the second group, which is the group of the first data to which is not assigned, approaches the distribution of the first group, and the first learning model and the second group converted by the second group conversion unit.
  • labeling of data that does not satisfy labeling conditions can be performed without much effort.
  • FIG. 2 is a functional block diagram showing an example of functions realized by the learning system;
  • FIG. It is a figure which shows an example of a target database. It is a figure which shows an example of a 1st group database. It is a figure which shows an example of a 2nd group database. It is a figure which shows an example of the process which converts a 2nd group.
  • FIG. 4 is a flow chart showing an example of processing executed by the learning system; It is a figure which shows an example of the functional block in a modification.
  • FIG. 10 is a diagram showing an example of distributions of first to fourth groups;
  • FIG. 1 is a diagram showing an example of the overall configuration of a learning system.
  • the learning system S includes a server 10 , a user terminal 20 and an administrator terminal 30 .
  • Network N is any network such as the Internet or a LAN.
  • the learning system S only needs to include at least one computer, and is not limited to the example in FIG.
  • the server 10 is a server computer.
  • Control unit 11 includes at least one processor.
  • the storage unit 12 includes a volatile memory such as RAM and a nonvolatile memory such as a hard disk.
  • the communication unit 13 includes at least one of a communication interface for wired communication and a communication interface for wireless communication.
  • the user terminal 20 is the user's computer.
  • the user terminal 20 is a personal computer, smart phone, tablet terminal, or wearable terminal.
  • Physical configurations of the control unit 21, the storage unit 22, and the communication unit 23 are the same as those of the control unit 11, the storage unit 12, and the communication unit 13, respectively.
  • the operation unit 24 is an input device such as a mouse or touch panel.
  • the display unit 25 is a liquid crystal display or an organic EL display.
  • the administrator terminal 30 is the administrator's computer.
  • the administrator terminal 30 is a personal computer, smart phone, tablet terminal, or wearable terminal.
  • the physical configurations of the control unit 31, the storage unit 32, the communication unit 33, the operation unit 34, and the display unit 35 are the same as those of the control unit 11, the storage unit 12, the communication unit 13, the operation unit 24, and the display unit 25, respectively. be.
  • each computer has a reading unit (for example, a memory card slot) for reading a computer-readable information storage medium, and an input/output unit (for example, a USB port) for inputting/outputting data with an external device. At least one may be included.
  • a program stored in an information storage medium may be supplied via at least one of the reading section and the input/output section.
  • the learning system S is applied to fraud detection in an SNS (Social Networking Service) will be taken as an example.
  • the service targeted for fraud detection may be of any type and is not limited to SNS. Examples of other services will be described in variations below.
  • the learning system S can be used for any purpose other than fraud detection. Usage examples for other purposes will also be described in modified examples described later.
  • This embodiment is characterized by a configuration related to SNS fraud detection.
  • the structure itself which provides SNS utilizes various well-known structures.
  • Fraud detection is the detection of fraud. Cheating is any act that deviates from legitimate use of the service. For example, fraudulent activity is activity that violates the terms of service, violates the law, or is otherwise nuisance. For example, on SNS, posting that slanders others, posting that encourages the trading of illegal products, posting inconceivably large amounts of posts, or unauthorized login by impersonating another person. constitutes misconduct. A user who has registered to use the SNS may act fraudulently, and a third party who has not registered to use the SNS may act fraudulently.
  • FIG. 2 is a diagram showing an example of fraud detection performed in an SNS.
  • the server 10 performs both SNS provision and fraud detection in this embodiment, the SNS provision and fraud detection may be performed by different computers.
  • the user terminal 20 displays the top screen G of the SNS. From the top screen G, the user can use various services provided by SNS.
  • fraud detection is performed when a user posts something on an SNS.
  • the execution timing of fraud detection may be arbitrary timing, and is not limited to the time of posting.
  • fraud detection may be performed when a user logs in, or fraud detection may be performed when a user comments on another user's post.
  • fraud detection may be performed when a user accesses a specific page on an SNS.
  • the server 10 executes fraud detection in SNS based on target data used in fraud detection and the current learning model M0.
  • the target data is the data that is the target of labeling in fraud detection.
  • Labeling is the process of classifying target data.
  • the process of estimating whether or not there is fraud corresponds to labeling.
  • the target data is given either a first label indicating that it is fraudulent or a second label that indicates that it is legitimate (not fraudulent).
  • the target data is data relating to features of users or third parties using SNS.
  • the target data includes at least one of static items and dynamic items.
  • a static item is an item that does not change in principle as long as the user ID is the same.
  • Static items are user information pre-registered in the SNS.
  • User information may be any information about the user, such as name, gender, email address, age, date of birth, occupation, nationality, area of residence, or address.
  • Information called demographic information that indicates user attributes is an example of user information.
  • Dynamic items are items that can change each time, even if the user ID is the same.
  • a dynamic item is information that is generated or obtained on the fly rather than pre-registered information.
  • the content of uploaded posts, posted posts viewed, other operation details, location of use, time of use, number of times of use, frequency of use, or type of user terminal 20 can be dynamically changed. Corresponds to item.
  • the meaning of the word "learning model” in the learning model M0 is the same for the first learning model M1 to the fourth learning model M4 described later.
  • the learning model M0 and the first learning model M1 to the fourth learning model M4 in FIG. When distinguishing between them, any numerical value from “0" to "4" is written at the end of the symbol "M".
  • Each learning model M has the same meaning of the word "learning model", but differs in the method of creating training data.
  • the learning model M is a model that uses machine learning.
  • the learning model M is sometimes called AI (Artificial Intelligence).
  • Machine learning itself can use various known methods.
  • Machine learning in this embodiment includes deep learning and reinforcement learning.
  • the learning model M may be supervised machine learning, semi-supervised machine learning, or unsupervised machine learning.
  • learning model M may be a neural network.
  • various models used in known fraud detection can be used.
  • the learning model M calculates the feature amount of the target data, and labels the target data based on the feature amount.
  • the feature amount is represented by a multidimensional vector is taken as an example, but the feature amount can be represented in any format and is not limited to the multidimensional vector.
  • the feature quantity may be represented by an array or a single numerical value.
  • the learning model M outputs either a first value indicating fraud or a second value indicating non-fraud. Instead of outputting value-based information, a score having an intermediate value such as a fraud probability of 30% may be output. Scores indicate the probability of belonging to each label.
  • target data is generated immediately when some kind of posting is made on the SNS.
  • the target data may be input to the current learning model M0 immediately, or may be input to the learning model M0 after a certain amount of time (for example, several minutes to several months) has passed. That is, when something is posted on the SNS, fraud detection may be performed in real time, or fraud detection may be performed after a certain amount of time has passed.
  • a malicious third party illegally obtains a user ID and password, pretends to be a legitimate user, and commits fraudulent acts on an SNS.
  • the place where the legitimate user usually uses the SNS and the place where the third party impersonates the legitimate user and uses the SNS are different. often different.
  • the time during which an authorized user normally uses the SNS may differ from the time during which a third party impersonates the authorized user and uses the SNS. For this reason, in order to detect fraudulent activity by a third party, items such as the place of use or the time of use may be effective in the target data.
  • a user's fraudulent act using his or her own user ID and password will be referred to as a user's fraudulent act.
  • a user's fraudulent act may occur at a place where SNS is usually used.
  • a user's fraudulent act may occur during the time when the SNS is normally used. For this reason, in order to detect a user's fraudulent activity, items such as the place of use or the time of use of the target data may not be very effective. In other words, the items effective for detecting the user's fraudulent behavior and the effective items for detecting the third party's fraudulent behavior may differ from each other.
  • the legitimate user who is the victim will notice the fraudulent activity and report it to the administrator. often do.
  • the administrator receives a report from a legitimate user, analyzes the target data when a third party's fraudulent act occurs, and creates training data for the learning model M0.
  • the administrator causes the learning model M0 to learn the created training data so that it can be detected immediately when a similar fraudulent act occurs. Therefore, training data for detecting third-party fraud can be relatively easy to create.
  • a user's misconduct occurs, it is more difficult to report to the administrator than a third party's misconduct because the misconduct is being performed with the user's own user ID and password.
  • the victim may report it, but in the case of other fraudulent acts such as mass posting that interferes with the operation of the SNS, the only victim is the administrator, so no one can report it. may not be reported.
  • the administrator may be late in noticing the occurrence of the fraudulent act, or may not notice the fraudulent act in the first place. Therefore, training data for detecting user fraud can be relatively difficult to create.
  • target data that do not satisfy the rules will not be monitored at all, so they cannot be used as training data. Since the administrator's monitoring does nothing more than checking whether the rules are valid or not, the accuracy of fraud detection by the learning model M0 may not differ much from the accuracy of the rules. Therefore, in the present embodiment, labeling is automatically performed on target data that does not meet the rules and is not monitored.
  • FIG. 3 is a diagram showing an overview of the learning system S.
  • the server 10 stores a target database DB1 in which a large amount of target data is stored.
  • the server 10 acquires n (n is an integer equal to or greater than 2. For example, several tens to several thousand or more) pieces of target data from the target database DB1.
  • the server 10 determines whether each of the n target data satisfies the current rule.
  • the current rule will be referred to as the first rule.
  • the first rule includes multiple rules such as rules a, b, and so on.
  • a rule is a condition that can be determined based on items included in target data. For example, if there is a tendency for posts with 500 characters or more as a tendency of user misconduct, the administrator sets a rule a such that "posts with 500 characters or more are subject to monitoring.” Define For example, if there is a tendency that the number of specific keywords in one post is 5 or more as a tendency of user misconduct, the administrator sets the rule b as "the number of keywords contained in the target data is 5 or more Define a rule such as "If it is, it will be monitored.” Similarly for other rules, the administrator identifies the user's fraudulent tendencies through past monitoring and defines a first rule.
  • Each rule included in the first rule includes the value of the item included in the target data and whether or not to be monitored (whether to be set as the first group or whether it is fraudulent). Show relationship.
  • the rule determines the values of the items included in the target data one after another like conditional branching in a flow chart.
  • rules may be in a form called a decision tree.
  • a machine learning method that creates a decision tree from data is sometimes called decision tree learning, so the rule may correspond to the machine learning method.
  • various rules used in known fraud detection can be used.
  • the target data satisfies any one of the plurality of rules included in the first rule, it may be determined that the target data satisfies the first rule, or if the target data satisfies a predetermined number of rules or more. Alternatively, it may be determined that the target data satisfies the first rule. Alternatively, for example, a score may be associated with each rule, and it may be determined that the target data satisfies the first rule when the total value of the scores of the rules satisfied by the target data is equal to or greater than a threshold.
  • a single rule may correspond to the first rule instead of including a plurality of rules as in FIG.
  • the number of target data satisfying the first rule be k (k is an integer equal to or smaller than n) out of n target data.
  • the number of target data that do not satisfy the first rule is nk.
  • a group of k target data satisfying the first rule will be referred to as a first group.
  • a group of nk target data that does not satisfy the first rule is called a second group. Since the first group is subject to monitoring, it is assigned a label by the administrator.
  • the administrator causes the administrator terminal 30 to display the contents of the k target data belonging to the first group.
  • the administrator confirms the contents of the k target data and assigns a label indicating whether or not it is illegal.
  • the second group is not subject to monitoring, so no label is assigned by the administrator.
  • the server 10 creates the first learning model M1 based on the first group labeled by the administrator. As described above, the accuracy of fraud detection by the first learning model M1 may not differ much from that of the first rule.
  • One of the purposes of this embodiment is to automatically assign a label to the second group that is not subject to monitoring.
  • the contents of the first learning model M1 may not differ much from the first rule, even if the nk pieces of target data belonging to the second group are input to the first learning model M1, almost all The target data may be given a label indicating that it is not fraudulent. That is, the same result as the first rule may be obtained.
  • the server 10 transforms the second group so that the distribution of the second group approaches the distribution of the first group.
  • This conversion itself can use the method of Non-Patent Document 1 described in the prior art.
  • This conversion enables the first learning model M1 to specify the features of items other than the items of the target data that are emphasized in the current labeling. That is, by converting the distribution of the second group so that it approaches the distribution of the first group, the first learning model M1 focuses on features other than the features emphasized in the current labeling, Labeling for the second group is performed.
  • rule a included in the first rule is "to be monitored when the number of characters in a post is 500 characters or more". Furthermore, it is assumed that, through monitoring, the administrator assigns fraud confirmation labels indicating fraud to most of the target data of posts of 500 characters or more. In this case, the first learning model M1 emphasizes the number of characters among the features of the target data. Even if items other than the number of characters are important as features indicating the user's fraudulent behavior, the first learning model M1 focuses only on the number of characters, and there is a possibility that the features of the other items cannot be noticed. be.
  • the second group contains a lot of target data for posts of less than 500 characters. Since the first learning model M1 performs labeling with emphasis on the number of characters, even if the target data belonging to the second group is directly input to the first learning model M1, the first learning model M1 will strongly focus on the number of characters. Labeling is executed, and almost all target data are given a label indicating that they are not illegal. By bringing the distribution of the second group closer to the distribution of the first group, the first learning model M1 performs labeling by paying attention to features other than the number of characters. For example, when the number of times of use by users who commit fraud tends to be large, the first learning model M1 focuses on not only the number of characters but also the number of times of use in the target data. In other words, the first learning model M1 can identify features of the target data that do not distinguish between the first group and the second group (that is, features that cannot be distinguished by the current first rule).
  • the server 10 creates the second learning model M2 based on the first group labeled by monitoring and the second group labeled by the first learning model M1.
  • the second learning model M2 has more training data than the first learning model M1, and another feature (for example, the number of times of use) that cannot be captured by the first rule is learned by the second group.
  • the accuracy of fraud detection is higher than with the learning model M1.
  • a second learning model M2 may be created based only on the second group, but since the features of the first group are also important in fraud detection, the second learning model M2 is based on both the first and second groups. Assume that M2 is created.
  • the second learning model M2 can be used for various purposes. An example of utilization of the second learning model M2 will be described later in a modified example.
  • FIG. 4 is a functional block diagram showing an example of functions realized by the learning system S. As shown in FIG. In this embodiment, a case where the main functions are realized by the server 10 will be described.
  • the data storage unit 100 is realized mainly by the storage unit 12 .
  • Other functions are realized mainly by the control unit 11 .
  • the data storage unit 100 stores data necessary for fraud detection.
  • the data storage unit 100 stores a target database DB1, a first group database DB2, and a second group database DB3.
  • FIG. 5 is a diagram showing an example of the target database DB1.
  • the target database DB1 is a database in which target data is stored.
  • the target data includes user ID, user name, gender, age, number of followers, number of followers, number of characters in the post, number of keywords included in the post, number of punctuation marks included in the post, place of use, time of use, number of times of use, and Includes items such as frequency of use.
  • target data is generated each time a post is received on an SNS
  • target data can be generated at any timing and is not limited to the example of this embodiment.
  • the target data may be generated when a certain amount of time has passed since the post on the SNS was received.
  • the target data may be generated when the administrator performs a predetermined operation from the administrator terminal 30.
  • Target data can include any item that can be used in fraud detection, and is not limited to the example of FIG.
  • there are other items such as the number of line breaks included in the post, the number of pictograms included in the post, the number of spaces included in the post, the elapsed time since the user ID was issued, or the trajectory of the mouse pointer at the time of posting. good too.
  • the items to be included in the target data shall be specified by the administrator.
  • FIG. 6 is a diagram showing an example of the first group database DB2.
  • the first group database DB2 is a database that stores target data belonging to the first group.
  • the first group database DB2 stores pairs of target data belonging to the first group and labels assigned by monitoring by the administrator. Assuming that there are k pieces of target data belonging to the first group, k pairs are stored in the first group database DB2.
  • the target data and label pairs stored in the first group database DB2 correspond to the training data of the first learning model M1.
  • this pair also corresponds to the training data of the second learning model M2.
  • the first group database DB2 can be said to be a database storing training data for the first learning model M1, or a database storing training data for the second learning model M2.
  • both the target data determined to be fraudulent and the target data determined not to be fraudulent are the first learning model M1 and the second learning model M1.
  • the target data for which fraud has been confirmed may be used as the training data for the first learning model M1 and the second learning model M2.
  • FIG. 7 is a diagram showing an example of the second group database DB3.
  • the second group database DB3 is a database that stores target data belonging to the second group.
  • the second group database DB3 stores target data belonging to the second group and labels given by the first learning model M1. Assuming that there are nk target data belonging to the second group, nk pairs are stored in the second group database DB3.
  • the target data and label pairs stored in the second group database DB3 correspond to the training data of the second learning model M2. Therefore, the second group database DB3 can also be said to be a database in which training data for the second learning model M2 is stored.
  • the target data belonging to the second group are not monitored by the administrator, but some target data may be monitored.
  • the target data estimated to be fraudulent by the first learning model M1 may be monitored.
  • both the target data determined to be fraudulent and the target data determined not to be fraudulent are the first learning model M1 and the second learning model M1.
  • the target data for which fraud has been confirmed may be used as the training data for the first learning model M1 and the second learning model M2.
  • the data storage unit 100 stores a first learning model M1 and a second learning model M2.
  • the first learning model M2 and the second learning model M2 include a program portion for calculating the feature amount of target data and a parameter portion referred to in calculating the feature amount. Pairs of target data and labels stored in the first group database DB2 have been learned as training data in the first learning model M1. Pairs of target data and labels stored in the second group database DB3 have been learned as training data in the second learning model M2.
  • the data stored in the data storage unit 100 is not limited to the above example.
  • the data storage unit 100 can store arbitrary data necessary for labeling target data.
  • the data storage unit 100 may store a user database that stores basic information about users who have registered to use the SNS.
  • the user database stores basic information such as user IDs, passwords, and names.
  • the data storage unit 100 may store the current learning model M0.
  • the data storage unit 100 may store data regarding the first rule.
  • a first determination unit 101 determines whether each of a plurality of target data satisfies a first rule. The first determination unit 101 determines whether the target data satisfies the first rule for each target data. In the example of FIG. 3, all of the n pieces of target data stored in the target database DB1 are to be judged by the first judging unit 101, but only some of the n pieces of object data are , may be subject to determination by the first determination unit 101 . For example, among the n pieces of target data, only target data generated in the most recent fixed period or a predetermined number of randomly selected target data may be subject to determination by the first determination unit 101 .
  • the first determination unit 101 determines whether each of the n target data satisfies each of a plurality of rules a, b, etc. included in the first rule. do. Whether or not each rule is satisfied may be determined by comparison with a threshold value, character string matching, or the like. In this embodiment, the first determination unit 101 determines that the target data satisfies the first rule when the target data satisfies any one of a plurality of rules included in the first rule. The first determination unit 101 may determine that the target data satisfies the first rule when the target data satisfies a predetermined number or more of rules included in the first rule.
  • the first determination unit 101 calculates the score of the target data based on the determination result of whether or not the target data satisfies each of the plurality of rules included in the first rule, and the calculated score is equal to or greater than the threshold. case, it may be determined that the target data satisfies the first rule.
  • Each of the n target data stored in the target database DB1 is an example of the first data. Therefore, the part describing this target data can be read as the first data.
  • the first data is data to be determined by the first determination unit 101 .
  • the first data can also be said to be data to be labeled.
  • the learning system S is used for fraud detection
  • the first data is data targeted for fraud detection.
  • the first rule is an example of the first condition. Therefore, the description of the first rule can be read as the first condition.
  • the first condition is a condition regarding labeling.
  • the first condition is a criterion for determination by the first determination unit 101 .
  • the target data is labeled based on the first condition. For example, when the target data satisfying the first condition is to be monitored as in the present embodiment, the first condition can also be said to be a condition indicating whether or not to be monitored. If it becomes a monitoring target, labeling is performed by the administrator, so the first condition corresponds to a condition regarding labeling.
  • the first condition may be any condition and is not limited to the first rule.
  • the first condition may be the current learning model M0, or may be a conditional branch not called a rule.
  • the target data indicates the behavior of users who use SNS.
  • SNS is an example of a predetermined service. Therefore, where SNS is described, it can be read as a predetermined service.
  • the predetermined service may be any other service.
  • a predetermined service is provided based on user information about the user. User information is information registered by a user. The aforementioned static items correspond to user information.
  • the labeling of this embodiment is a process of determining whether or not the behavior of a user who has valid user information is fraudulent.
  • a user having valid user information is a user who has logged in with his/her own user ID and password.
  • the label in this embodiment is a confirmed fraud label indicating that fraud has been confirmed.
  • the providing unit 102 provides target data that satisfies the first rule to an administrator who executes labeling. Providing the target data to the administrator means transmitting the target data to the administrator terminal 30 .
  • the providing unit 102 provides the administrator with k pieces of target data belonging to the first group stored in the first group database DB2. For example, when the server 10 receives a predetermined request from the administrator terminal 30, the providing unit 102 transmits k target data belonging to the first group to the administrator terminal 30, thereby obtaining k Provide subject data to administrators.
  • the designation receiving unit 103 receives designation of a label by an administrator.
  • the designation reception unit 103 is realized by the server 10, so that the designation reception unit 103 receives data indicating the result of designation by the administrator from the administrator terminal 30, thereby allowing the designation of the label by the administrator. accept.
  • the administrator manually specifies the labels of all target data provided to the administrator.
  • Temporary labels are assigned to the target data in advance, and checks are performed by the administrator. may be broken. Since the target data provided to the administrator satisfies the first rule, the provisional label indicates fraudulent. An administrator may correct the error if the temporary label is incorrect.
  • the first group labeling unit 104 performs first group labeling.
  • monitoring is performed by the administrator, so the first group labeling unit 104 performs labeling of the first group based on designation by the administrator.
  • the administrator manually specifies the labels of all target data provided to the administrator. Perform the labeling of the first group by associating the label with
  • the first group labeling unit 104 associates the target data provided to the administrator with the check result by the administrator, thereby confirming the labeling of the first group. Execute. The first group labeling unit 104 performs the labeling of the first group so that the temporary label is assigned as the real label to the target data whose temporary label has not been corrected by the administrator. The first group labeling unit 104 executes labeling of the first group so that the label corrected by the administrator is added to the target data whose temporary label has been corrected by the administrator.
  • the first group labeling section 104 may label the first group based on the determination result of the first determination section 101 . For example, if it is determined in advance to assign a label indicating that it is illegal when the first rule is satisfied, the first group labeling unit 104 labels target data belonging to the first group as illegal. A first group of labeling may be performed by assigning a label indicating that .
  • each rule included in the first rule may be associated with a label. For example, if the target data satisfies rule a, the target data is labeled as fraudulent, and if the target data satisfies rule b, the target data is not fraudulent.
  • a label may be associated with each individual rule, such as a label indicating that there is no rule.
  • the first group labeling unit 104 may label the first group by assigning to the target data a label associated with the rule satisfied by the target data.
  • the first learning model creation unit 105 creates a labelable first learning model M1 based on a first group, which is a group of labeled target data that satisfies the first rule.
  • Creating the first learning model M1 means executing the learning process of the first learning model M1. That is, making the first learning model M1 learn the training data corresponds to creating the first learning model M1.
  • various techniques used in machine learning can be used. For example, the learning process may utilize backpropagation or gradient descent.
  • the first learning model creation unit 105 creates the first learning model M1 using pairs of target data belonging to the first group and labels given to the target data as training data.
  • the first learning model creation unit 105 performs Adjust the parameters of the first learning model M1.
  • the first learning model creation unit 105 may use all the target data stored in the first group database DB2 as training data, or may use only a part of the target data as training data.
  • the second group conversion unit 106 converts the second group so that the distribution of the second group, which is the group of the first data that does not satisfy the first rule and is not labeled, approaches the distribution of the first group. Convert. Converting the second group means changing the feature amount of the target data belonging to the second group.
  • a second group conversion unit 106 converts the second group based on a predetermined conversion function. Various known functions can be used for this conversion function itself, and for example, the function described in Non-Patent Document 1 may be used.
  • the second group conversion unit 106 converts the second group based on a method of matching the source domain and the target domain.
  • Various known methods can be used as this method, and for example, the method described as the related art in Non-Patent Document 1 may be used.
  • the second group transform unit 106 repeats the process of selecting samples from the source domain and the process of determining the weighting coefficients of the transform function (Borgwardt, Karsten M., Gretton, Arthur, Rasch, Malte J., Kriegel, Hans-Peter, Scholkopf, Bernhard, and Smola, Alexander J. Integrating structured biological data by kernel maximum mean discrepancy. In ISMB, pp. 49-57, 2006). may
  • the second group transformation unit 106 uses a method (Pan, Sinno Jialin, Tsang, Ivor W., Kwok, James T., and Yang, Qiang. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks, 22(2): 199-210, 2011).
  • the second group conversion unit 106 uses a technique called kernel-reproducing Hilbert space (Gong, Boqing, Shi, Yuan, Sha, Fei, and Grauman, Kristen. Geodesic flow kernel for unsupervised domain adaptation. In CVPR, pp. 2066 -2073, 2012), the second group may be transformed.
  • FIG. 8 is a diagram showing an example of processing for converting the second group.
  • the distribution D1 of the first group is shaded, and the distribution D2 of the second group is not shaded.
  • Black circles or white circles indicate how the feature amount of the target data belonging to the first group and the feature amount of the target data belonging to the second group are plotted in the multidimensional space.
  • the black circles in FIG. 8 indicate features that have not been transformed, and the white circles indicate features that have been transformed.
  • the target data belonging to the first group satisfies the first rule, so the feature amount distribution is concentrated within a certain range.
  • the target data belonging to the second group does not satisfy the first rule, so the distribution of the feature quantity is different from the distribution of the first group.
  • the second group conversion unit 106 converts the target data belonging to the second group such that the distribution of the feature amount of the second group approaches the distribution of the feature amount of the first group. For example, the distribution D2 of the second group after transformation approaches the distribution of the first group D1.
  • the second group conversion unit 106 calculates the average value of the feature values of the k pieces of target data belonging to the first group as the representative value of the distribution D1 of the first group.
  • the second group conversion unit 106 calculates the average value of the feature amounts of the nk pieces of target data belonging to the second group as the representative value of the distribution D2 of the second group.
  • the second group conversion unit 106 converts the second group such that the representative value of the distribution D2 of the second group approaches the representative value of the distribution D1 of the first group.
  • the first learning model M1 emphasizes the number of characters as in the example described above, this conversion causes the portion corresponding to the number of characters in the feature amount of the target data belonging to the second group to approach the first group.
  • the number of characters of the target data belonging to the second group is originally less than 500 characters, but is converted as if it were 500 characters or more).
  • the distribution D2 of the second group generally approaches the distribution D1 of the first group.
  • the representative value of the distributions D1 and D2 in the above example may not be the average value of the feature amounts of the entire first group or second group.
  • the representative value may be an average value of feature amounts of randomly selected target data, or may be a feature amount at the mode value in a probability distribution.
  • the second group labeling unit 107 performs labeling of the second group based on the first learning model M1 and the second group converted by the second group conversion unit 106.
  • FIG. The second group labeling unit 107 inputs the converted target data belonging to the second group to the first learning model M1, and associates the output from the first learning model M1 with the target data, thereby Perform labeling.
  • the first learning model M1 outputs a score
  • the second group labeling unit 107 performs labeling by associating the target data belonging to the second group with the score output from the first learning model M1. You may
  • the second learning model creation unit 108 is different from the first learning model M1 based on the first group and the second group labeled by the second group labeling unit 107, and is capable of labeling.
  • Creating the second learning model M2 means executing the learning process of the second learning model M2. That is, making the second learning model M2 learn the training data corresponds to creating the second learning model M2.
  • various techniques used in machine learning can be used. For example, the learning process may utilize backpropagation or gradient descent.
  • the second learning model creation unit 108 creates the second learning model M2 using pairs of target data belonging to the first group and labels given to the target data as training data.
  • the second learning model creation unit 108 when target data belonging to the first group is input to the second learning model M2, performs Adjust the parameters of the second learning model M2.
  • the second learning model creation unit 108 may use all the target data stored in the first group database DB2 as training data, or may use only a part of the target data as training data.
  • the second learning model creation unit 108 creates the second learning model M2 using pairs of target data belonging to the second group and labels given to the target data as training data.
  • the second learning model creation unit 108 is configured so that when target data belonging to the second group is input to the second learning model M2, the label associated with the target data is output from the second learning model M2. Adjust the parameters of the second learning model M2.
  • the second learning model creating unit 108 may use all the target data stored in the second group database DB3 as training data, or may use only a part of the target data as training data.
  • the second learning model creation unit 108 generates the second learning model M2 based on the second groups labeled by the second group labeling unit 107 and before conversion by the second group conversion unit 106.
  • the second learning model creating unit 108 may create the second learning model M2 based on the second group converted by the second group converting unit 106 .
  • the second learning model creation unit 108 creates a second group based on the second group before conversion by the second group conversion unit 106 and the second group after conversion by the second group conversion unit 106 A learning model M2 may be created.
  • FIG. 9 is a flow chart showing an example of processing executed by the learning system S. As shown in FIG. The processing of FIG. 9 is executed by the server 10, the user terminal 20, and the administrator terminal 30. FIG. The processing in FIG. 9 is executed by control units 11, 21 and 31 operating according to programs stored in storage units 12, 22 and 32, respectively.
  • the user terminal 20 accesses the server 10 and executes login processing for logging into the SNS with the server 10 (S1).
  • the user terminal 20 uploads the post to the server 10 (S2).
  • the server 10 When receiving the post, the server 10 generates target data (S3), and executes fraud detection based on the learning model M0, which is the current fraud detection model (S4).
  • the generation of the target data in S3 may be executed based on the data received from the user terminal 20 and the user database stored in the server 10. FIG. If fraud is detected at the time of S4, the post will not be accepted. If no fraud is detected, the submission is accepted.
  • the server 10 stores the target data generated in S3 in the target database DB1 (S5).
  • the server 10 determines whether or not to change the learning model M0, which is the current fraud detection model (S6). If it is determined not to change the current fraud detection model (S6; N), this process ends. If it is determined to change the current fraud detection model (S6; Y), the server 10 refers to the target database DB1 and determines whether each of the n target data satisfies the first rule (S7 ).
  • the server 10 stores the k target data satisfying the first rule as the first group in the first group database DB2 (S8).
  • the server 10 stores nk pieces of target data that do not satisfy the first rule as a second group in the second group database DB3 (S9).
  • the server 10 provides the administrator with the first group based on the first group database DB2 (S10).
  • the administrator terminal 30 When the administrator terminal 30 receives the first group, it accepts label designation by the administrator (S11). In S11, monitoring by the administrator is executed. The administrator terminal 30 transmits the monitoring result by the administrator to the server 10 (S12). When the server 10 receives the monitoring results from the administrator, it updates the first group database DB2 (S13).
  • the server 10 creates the first learning model M1 based on the labeled first group (S14).
  • the server 10 executes the learning process of the first learning model M1 using pairs of target data and labels stored in the first group database DB2 as training data.
  • the server 10 adjusts the parameters of the first learning model M1 so that when target data belonging to the first group is input, a label corresponding to this target data is output.
  • the server 10 converts the second group based on the first group database DB2 and the second group database DB3 so that the distribution of the second group approaches the distribution of the first group (S15).
  • the server 10 performs labeling of the second group based on the second group converted in S15 and the first learning model M1 created in S14 (S16).
  • the server 10 inputs the target data belonging to the second group to the first learning model M1 and acquires the output from the first learning model M1.
  • the server 10 updates the second group database DB3 so that the target data input to the first learning model M1 and the label output from the first learning model M1 are paired.
  • the server 10 creates the second learning model M2 based on the labeled first group and the labeled second group (S17), and the process ends.
  • the server 10 uses both the target data and label pairs stored in the first group database DB2 and the target data and label pairs stored in the second group database DB3 as training data, and performs a second The learning process of the learning model M2 is executed.
  • the server 10 adjusts the parameters of the second learning model M2 so that when target data belonging to the first group is input, a label corresponding to this target data is output.
  • the server 10 adjusts the parameters of the second learning model M2 so that when target data belonging to the second group is input, a label corresponding to this target data is output.
  • the learning system S of this embodiment creates the first learning model M1 based on the first group.
  • the learning system S transforms the second group such that the distribution of the second group approaches the distribution of the first group.
  • the learning system S performs labeling of the second group based on the first learning model M1 and the transformed second group.
  • labeling of target data that does not satisfy the first rule can be performed without much effort.
  • the second group is not monitored by the administrator, the labeling of the second group can be executed with high accuracy. Since the administrator does not have to monitor the second group, the burden on the administrator can be reduced. Since the monitoring of the second group is not performed, the time required for labeling the second group can be shortened. As a result, fraud can be quickly detected from the target data belonging to the second group.
  • Security in the SNS is enhanced when a user's fraudulent activity that lowers the security in the SNS is detected.
  • the learning system S also creates a second learning model M2 based on the first group and the labeled second group.
  • the second learning model M2 which can detect user fraud more accurately than the first learning model M1 can be created without much effort. Since the monitoring of the second group is not executed, the time required to create the second learning model M2 can be shortened. As a result, it is possible to quickly create the second learning model M2 capable of detecting the user's fraudulent behavior, which makes it easier to detect the user's fraudulent behavior.
  • Security in the SNS is enhanced when a user's fraudulent activity that lowers the security in the SNS is detected.
  • the learning system S creates a second learning model M2 based on the second group before conversion. Thereby, it is possible to create the second learning model M2 in which the characteristics of the user's fraudulent behavior are learned more accurately. As a result, it becomes easier to detect fraudulent actions by the user. Security in the SNS is enhanced when a user's fraudulent activity that lowers the security in the SNS is detected.
  • the learning system S provides the administrator with the first data belonging to the first group, and accepts label designation by the administrator.
  • the learning system S performs the labeling of the first group based on the designation by the administrator.
  • the target data to be monitored by the administrator can be minimized, thereby reducing the burden on the administrator.
  • the accuracy of the first learning model M1 is enhanced.
  • using the highly accurate first learning model M1 also increases the accuracy of the labeling of the second group.
  • the target data indicates the behavior of the user using the SNS
  • the SNS is provided based on the user information.
  • the labeling of target data is a process of determining whether or not the behavior of a user who has valid user information is fraudulent, and the label is an fraudulence confirmation label indicating that fraudulence has been confirmed.
  • labeling for detecting user's fraudulent behavior in SNS can be executed without much effort. It becomes easy to detect user's fraudulent activity in SNS.
  • FIG. 10 is a diagram showing an example of functional blocks in the modification.
  • a second rule creation unit 109, a second determination unit 110, a third learning model creation unit 111, a fourth group conversion unit 112, a fourth group labeling unit 113, a second target data labeling unit 114, a third 4 A learning model creation unit 115, a first usage determination unit 116, a second usage determination unit 117, and an additional learning unit 118 are realized.
  • Each of these functions is realized mainly by the control unit 11 .
  • the target database DB1 described in the embodiment is referred to as a first target database DB1.
  • the learning system S can be applied to fraud detection in services other than SNS.
  • Other services may be of any kind, for example, payment services, e-commerce services, travel booking services, financial services, or communication services.
  • Modification 1 takes fraud detection in payment services as an example.
  • Modifications 2 to 10 also take fraud detection in payment services as an example, but modifications 2 to 10 are also applicable to any service.
  • the payment service is a service related to electronic payment. Electronic payment is sometimes called cashless payment.
  • electronic payment using a credit card is taken as an example, but the payment means that can be used in the payment service may be of any type and is not limited to credit cards.
  • electronic money, points, bank accounts, debit cards, or crypto assets may correspond to payment means.
  • codes such as barcodes or two-dimensional codes may also be used in electronic payment, so the code may correspond to payment means.
  • the settlement service can be used for various purposes such as remittance to other users or charge, other than payment at the store.
  • the user can use not only a physical credit card but also a credit card registered in the payment application installed on the user terminal 20.
  • credit cards registered in other services such as e-commerce services or travel reservation services may be used. For example, even if a malicious third party does not steal a physical credit card, there is a possibility that a third party may illegally obtain a user ID and password, impersonate a legitimate user, and use the credit card.
  • the characteristics of fraudulent behavior by third parties and the characteristics of fraudulent behavior by users may differ.
  • one example of fraudulent behavior by a user is fraudulent behavior by a staff member of a member store.
  • Member store clerks are assumed to have registered for use of the settlement service. Therefore, the store clerk of the member store is also the user.
  • a store clerk at a member store uses his/her own credit card at the POS terminal of his/her own store to disguise that a product that is not actually on sale has been purchased, and attempts to convert the credit card into cash. You may purchase items such as cash vouchers that cannot be purchased with a card.
  • the fraudulent act by the clerk of the affiliated store is called the fraudulent act of the affiliated store.
  • the target data of Modification 1 is data relating to user characteristics in the payment service.
  • the target data includes items such as credit card number, brand, amount used, place of use, time of use, number of times of use, and frequency of use. If the information on the purchased product can be acquired, the target data may include the information on the purchased product.
  • the first rule of Modification 1 indicates the characteristics of fraudulent behavior by member stores.
  • the first determination unit 101 determines whether or not the target data satisfies a first rule that indicates the characteristics of fraudulent behavior by a member store.
  • the first group is a group of target data that satisfies a first rule that characterizes the fraudulent behavior of a member store, is monitored by an administrator, and is labeled. Based on this first group, the first learning model creation unit 105 creates a first learning model M1 capable of detecting fraudulent activity by member stores.
  • the second group is a group of target data that does not satisfy the first rule that indicates the characteristics of fraudulent behavior by member stores, is not subject to monitoring by the administrator, and is not labeled.
  • the second group conversion unit 106 converts the second group so that the distribution of the second group approaches the distribution of the first group in the same manner as the method described in the embodiment.
  • a second group labeling unit 107 performs labeling of the second group based on the converted second group.
  • the second learning model creating unit 108 creates a second learning model M2. In the second learning model M2, features of fraudulent behavior of member stores that are not defined in the first rule are learned.
  • the first rule is a rule regarding the amount of money used.
  • the first learning model M1 is a model that emphasizes the usage amount.
  • the second group consists of target data with relatively low usage amounts, but by transforming the second group so as to approach the distribution of the first group, the first learning model M1 has other features than the usage amounts. (for example, the number of times of use).
  • the second learning model M2 performs labeling by focusing not only on the usage amount but also on other features, so it is possible to perform labeling focusing on features that are not defined in the first rule.
  • the learning system S of Modification 1 can carry out labeling of target data in the payment service without much effort. Also, for the same reason as the learning system S described in the embodiment, it is necessary to accurately label the second group in the payment service, reduce the burden of monitoring by the administrator in the payment service, and It is possible to shorten the time required for group labeling, quickly detect fraudulent activity from the target data in the payment service, and detect fraudulent activity by merchants in the payment service to enhance security.
  • the second learning model M2 may be used to create new rules that are different from the first rules.
  • a new rule is called 2nd rule.
  • the second rule is a rule applied instead of the first rule.
  • the first rule is no longer used.
  • the purpose of use of the second rule is the same as that of the first rule.
  • Modification 2 will exemplify a case where the second rule is used to detect fraud in payment services.
  • a second rule is an example of a second condition. Therefore, the part described as the second rule can be read as the second condition.
  • the second condition is different from the first condition and is a condition related to labeling.
  • the second condition is a criterion for determination by the second determination unit 110, which will be described later.
  • the target data is labeled based on the second condition.
  • the second condition can also be said to be a condition indicating whether or not to be monitored.
  • the second condition may be any condition and is not limited to the second rule.
  • the second condition may be a condition conforming to the second learning model M2, and may be a conditional branch not called a rule.
  • the learning system S of Modification 2 includes a second rule creation unit 109 and a second determination unit 110 .
  • a second rule creating unit 109 creates a second rule based on the second learning model M2.
  • a second rule creation unit 109 creates a second rule from the second learning model M2 using a predetermined rule creation method.
  • a known method can be used for the rule creation method itself.
  • the second rule creating unit 109 may use decision tree learning to create the second rule from the second learning model M2.
  • the second rule creation unit 109 may create the second rule based on the items of the target data that the second learning model M2 attaches importance to when performing labeling. This item may be judged based on an index called an impact value.
  • the impact value is the degree of importance in labeling. The higher the impact value, the more important it is in labeling.
  • the impact value itself can be obtained by various known methods. An impact value may be obtained by the method of measuring
  • the second rule creating unit 109 creates a second rule so as to include items with relatively high impact values as conditional branches.
  • the target data to be determined by the first determination unit 101 described in the embodiment is referred to as first target data.
  • the second determination unit 110 determines whether or not each of the plurality of second target data different from the plurality of first target data satisfies the second rule.
  • the second target data is target data generated after the first target data.
  • the second target data is data relating to actions after the first target data.
  • the second target data is data regarding recent actions.
  • the items themselves included in the second target data are the same as those in the first target data.
  • the second target data is an example of the second data. Therefore, the part described as the second target data can be read as the second data.
  • the second data is data to be determined by the second determination unit 110 .
  • the second data can also be said to be data to be labeled.
  • the second data is data targeted for fraud detection.
  • the learning system S of Modification 2 creates a second rule based on the second learning model M2.
  • the learning system S determines whether each of the plurality of second target data satisfies the second rule.
  • the learning system S when used for fraud detection in payment services, even if the trend of fraud changes over time, the second rule reflects the latest trend. to keep up with the latest fraud trends by creating As a result, fraudulent activities can be detected quickly, increasing security in payment services.
  • the second rule of modification 2 may be applied to fraud detection in the current payment service, but may also be used to create a new learning model.
  • Modified Example 3 a case will be described in which processing similar to that of the embodiment is executed based on the second rule to create a new learning model. That is, by repeatedly executing the processing described in the embodiment, creation of a new learning model is repeated.
  • the data storage unit 100 of Modification 3 stores a second target database DB4, a third group database DB5, and a fourth group database DB6.
  • the second target database DB4 is a database that stores a plurality of second target data.
  • n pieces of second target data which are the same as the first target data stored in the first target database DB1 are stored in the second target database DB4.
  • the number of second target data items to be stored may be any number.
  • the method itself for creating the second target data may be the same as that for the first target data.
  • the third group database DB5 is a database that stores second target data belonging to the third group.
  • the third group database DB5 stores pairs of second target data belonging to the third group and labels assigned by monitoring by the administrator. Assuming that there are k pieces of second target data belonging to the third group, k pairs are stored in the third group database DB5.
  • the pairs of second target data and labels stored in the third group database DB2 correspond to training data of the third learning model M3.
  • the fourth group database DB6 is a database that stores target data belonging to the fourth group.
  • the fourth group database DB6 stores the second target data belonging to the fourth group and the label given by the third learning model M3. Assuming that there are nk pieces of second target data belonging to the fourth group, nk pairs are stored in the fourth group database DB6. The pairs of second target data and labels stored in the fourth group database DB6 correspond to training data of the fourth learning model M4.
  • the data storage unit 100 stores the third learning model M3 and the fourth learning model M4.
  • the third model M3 and fourth model M4 include a program portion for calculating the feature amount of the second target data and a parameter portion referred to in the feature amount calculation. Pairs of the second target data and labels stored in the third group database DB5 have been learned as training data in the third learning model M3.
  • the fourth learning model M4 has learned pairs of the second target data and labels stored in the fourth group database DB6 as training data.
  • the learning system S of Modification 3 includes a third learning model creation unit 111, a fourth group conversion unit 112, and a fourth group labeling unit 113.
  • the third learning model creation unit 111 creates a third learning model M3 that can be labeled based on the third group, which is the group of the second target data that satisfies the second rule and is labeled.
  • the processing of the third learning model creating unit 111 differs from that of the first learning model creating unit 105 in that the third group is used, but the other points are the same as those of the first learning model creating unit 105 .
  • the third learning model creation unit 111 creates a third learning model M3 using pairs of second target data belonging to the third group and labels assigned to the second target data as training data.
  • the fourth group conversion unit 112 converts the distribution of the fourth group, which is the group of the second target data that does not satisfy the second rule and is not labeled, so that the distribution of the fourth group approaches the distribution of the third group. to convert The processing of the fourth group conversion unit 112 differs from that of the second group conversion unit 106 in that the third group and the fourth group are used, but the other points are the same as those of the second group conversion unit 106 .
  • a fourth group conversion unit 112 converts the fourth group based on a predetermined conversion function.
  • the fourth group labeling unit 113 performs labeling of the fourth group based on the third learning model M3 and the fourth group converted by the fourth group conversion unit 112.
  • the processing of the fourth group labeling unit 113 differs from that of the second group labeling unit 107 in that the third learning model M3 and the fourth group are used, but the other points are the same as those of the second group labeling unit 107.
  • the fourth group labeling unit 113 inputs the converted second target data belonging to the fourth group to the third learning model M3, and associates the second target data with the output from the third learning model M3. Perform the labeling of the fourth group.
  • the learning system S of Modification 3 creates a third learning model M3 based on the third group.
  • the learning system S transforms the fourth group such that the distribution of the fourth group approaches the distribution of the third group.
  • the learning system S performs fourth group labeling based on the third learning model M3 and the transformed fourth group.
  • the labeling of the second target data that does not satisfy the second rule can be performed without much effort.
  • the learning system S is applied to fraud detection in payment services, by repeating the process of Modification 3, it is possible to continuously update the rules to detect the latest trends in fraud.
  • the second learning model M2 is not used to create a new second rule as in modification 2, but instead of the learning model M0, which is the current fraud detection model, the second learning model M2 may be the current fraud detection model.
  • the learning system S includes a second target data labeling section 114 .
  • the second target data labeling unit 114 labels each of the plurality of second target data different from the plurality of first target data based on the second learning model M2.
  • the second target data labeling unit 114 inputs each of the plurality of second target data to the second learning model M2 and acquires the output from the second learning model M2, thereby Perform each labeling.
  • the learning system S of Modification 4 executes labeling of each of the plurality of second target data based on the second learning model M2. This increases the accuracy of the labeling of the second target data. For example, when the learning system S is applied to fraud detection in payment services, it is possible to accurately detect fraud in payment services using the second learning model M2 that reflects the latest fraud trends.
  • Modification 5 For example, modification 3 may be applied to modification 4, and the second learning model M2 may be used as the second condition.
  • the learning system S of Modification 5 includes a third learning model creation unit 111 , a fourth group conversion unit 112 , and a fourth group labeling unit 113 as in Modification 3 .
  • the processing of the third learning model creating unit 111 is different from the processing described in the modification 3.
  • the third learning model creation unit 111 of Modification 5 creates a third learning model M3 that can be labeled based on the third group that is the group of the second data labeled by the second learning model M2.
  • the processes of the fourth group conversion unit 112 and the fourth group labeling unit 113 are the same as those described in the third modification.
  • the learning system S of Modification 5 creates a third learning model M3 based on the third group.
  • the learning system S transforms the fourth group such that the distribution of the fourth group approaches the distribution of the third group.
  • the learning system S performs fourth group labeling based on the third learning model M3 and the transformed fourth group.
  • the labeling of the second target data whose fraud is not estimated by the second learning model M2 can be performed without much effort.
  • the learning system S is applied to fraud detection in payment services, by repeating the process of modification 5, it is possible to continuously update the model to detect the latest trends in fraud.
  • the fourth learning model M4 may be created based on the labeling result of the fourth group.
  • the first target data belonging to the first group may be used as training data.
  • the learning system S of Modification 6 includes a fourth learning model creation unit 115 .
  • the fourth learning model creation unit 115 creates the first learning model M1, the second learning model M2, and the A fourth learning model M4 that is different from any of the third learning models M3 and that can be labeled is created.
  • the processing of the fourth learning model creating unit 115 is different from that of the second learning model creating unit 108 in that the first group, the third group, and the fourth group are used as training data, but other points are the same. be.
  • the fourth learning model creation unit 115 creates a fourth learning model M4 using pairs of first target data belonging to the first group and labels given to the first target data as training data.
  • the fourth learning model creation unit 115 creates a fourth learning model M4 using pairs of second target data belonging to the third group and labels assigned to the second target data as training data.
  • the fourth learning model creation unit 115 creates a fourth learning model M4 using pairs of second target data belonging to the fourth group and labels assigned to the second target data as training data.
  • the learning system S of Modification 6 selects any one of the first learning model M1, the second learning model M2, and the third learning model M3 based on the first group, the third group, and the labeled fourth group.
  • a fourth learning model M4 that is different from the above and that can be labeled is created.
  • the fourth learning model M4 which can detect user fraud more accurately than the third learning model M3, can be created without much effort.
  • FIG. 11 is a diagram showing an example of distributions of the first to fourth groups.
  • the distribution of the third group is indicated by the symbol D3, and the distribution of the fourth group is indicated by the symbol D4.
  • the distribution D1 of the first group and the distribution D3 of the third group are far apart, there is a possibility that the latest fraudulent behavior trend has changed significantly. In this case, it may be better not to use the first group in the learning of the fourth learning model M4. Therefore, in Modified Example 7, when the distribution D1 of the first group and the distribution D3 of the third group are similar, the first group is used for learning of the fourth learning model M4.
  • the learning system S of Modification 7 includes a first usage determination unit 116 .
  • the first usage determining unit 116 determines whether or not to use the first group in creating the fourth learning model M based on the similarity between the distribution D1 of the first group and the distribution D3 of the third group. do.
  • Distribution similarity is how similar the distributions are. The smaller the deviation of the distributions, the more similar the distributions. Distribution similarity is expressed based on a predetermined index. Henceforth, this index is called similarity.
  • the first usage determination unit 116 calculates the degree of similarity based on the distribution D1 of the first group and the distribution D3 of the third group. For example, the first usage determining unit 116 calculates a first representative value, which is a representative value of the feature amount of the first target data, based on the first target data belonging to the first group. The first usage determination unit 116 calculates a second representative value, which is a representative value of the feature amount of the second target data, based on the second target data belonging to the third group. The meaning of the representative value is as described in the embodiment.
  • the first usage determining unit 116 calculates the reciprocal of the distance between the first representative value and the second representative value as the degree of similarity. Since the similarity is the reciprocal of the distance, the shorter the distance, the higher the similarity. The first usage determination unit 116 determines whether or not the degree of similarity is equal to or greater than a predetermined threshold. The first use determining unit 116 determines not to use the first group in creating the fourth learning model M4 if the similarity is less than the threshold, and if the similarity is greater than or equal to the threshold, the fourth learning model It is determined that the first group is used in creating M4.
  • the fourth learning model creation unit 115 creates the fourth learning model M4 without using the first group. In this case, the first target data belonging to the first group is not used as training data for the fourth learning model M4.
  • the fourth learning model M4 is created based on the first group. In this case, the first target data belonging to the first group is used as training data for the fourth learning model M4.
  • the learning system S of Modification 7 determines whether to use the first group in creating the fourth learning model M4. judge.
  • the learning system S creates a fourth learning model M4 not based on the first group if it is determined not to use the first group, and if it is determined to use the first group, the learning system S Based on, a fourth learning model M4 is created. This increases the accuracy of the fourth learning model M4.
  • the fourth learning model creation unit 115 creates the fourth learning model M4 further based on the second group labeled by the second group labeling unit 107. good.
  • the fourth learning model creation unit 115 creates a fourth learning model M4 by using a pair of the first target data belonging to the second group and the label given to the second target data as training data.
  • the learning process itself may be the same as Modification 6 or Modification 7.
  • the learning system S of Modification 8 creates a fourth learning model further based on the second group labeled by the second group labeling unit 107 .
  • the fourth learning model M4 which can detect user fraud more accurately than the third learning model M3, can be created without much effort.
  • the learning system S of Modification 9 includes a second usage determination unit 117 .
  • the second use determination unit 117 determines whether or not to use the second group in creating the fourth learning model based on the similarity between the distribution D2 of the second group and the distribution D4 of the fourth group. .
  • the meaning of similarity is the same as in Modification 7.
  • the second usage determination unit 117 calculates the degree of similarity based on the distribution D2 of the second group and the distribution D4 of the fourth group.
  • the second usage determining unit 117 calculates a third representative value, which is a representative value of the feature amount of the first target data, based on the first target data belonging to the second group.
  • the second usage determination unit 117 calculates a fourth representative value, which is a representative value of the feature amount of the second target data, based on the second target data belonging to the fourth group.
  • the second usage determination unit 117 calculates the reciprocal of the distance between the third representative value and the fourth representative value as the degree of similarity. Since the similarity is the reciprocal of the distance, the shorter the distance, the higher the similarity. The second usage determination unit 117 determines whether or not the degree of similarity is greater than or equal to a predetermined threshold. If the similarity is less than the threshold, the second use determination unit 117 determines not to use the second group in creating the fourth learning model M4. It is determined that the second group is used in creating M4.
  • the fourth learning model creation unit 115 creates the fourth learning model M4 without using the second group. In this case, the first target data belonging to the second group is not used as training data for the fourth learning model M4.
  • the fourth learning model creation unit 115 creates a fourth learning model based on the second group when the second use determination unit 117 determines to use the second group. In this case, the second target data belonging to the second group is used as training data for the fourth learning model M4.
  • the learning system S of Modification 9 determines whether or not to use the second group in creating the fourth learning model. do.
  • the learning system S creates a fourth learning model M4 not based on the second group if it is determined not to use the second group, and if it is determined to use the second group, the learning system S Based on, a fourth learning model M4 is created. This increases the accuracy of the fourth learning model M4.
  • Modification 10 For example, as the labeling result of the second group, the embodiment explained the case of creating a new second learning model M2, but the labeling result of the second group can be used for other purposes. Modification 10 describes a case where the labeling result of the second group is used in additional learning of the first learning model M1.
  • the learning system S of Modification 10 includes an additional learning unit 118 .
  • the additional learning unit 118 Based on the second group labeled by the second group labeling unit 107, the additional learning unit 118 performs additional learning of the first learning model already trained by the first group.
  • various techniques used in machine learning can be used.
  • the learning process may utilize backpropagation or gradient descent.
  • a process adopted in a technique called transfer learning or fine tuning may be used.
  • the additional learning unit 118 adjusts the parameters of the first learning model M1 using a pair of the first target data belonging to the second group and the label given to the first target data as training data.
  • the additional learning unit 118 is configured so that when the first target data belonging to the second group is input to the first learning model M1, the label associated with this first target data is output from the first learning model M1. , adjust the parameters of the first learning model M1.
  • the additional learning unit 118 may use all the first target data stored in the second group database DB3 as training data, or may use only a part of the first target data as training data.
  • the learning system S of Modification 10 performs additional learning of the first learning model M1 already trained by the first group, based on the labeled second group. This increases the accuracy of the first learning model M1.
  • the learning system S can be used for various purposes other than fraud detection.
  • the learning system S can be used for various types of labeling, for example, labeling of objects included in images, labeling of contents of documents, labeling of whether or not the user continues to use the service, or labeling of the user's preferences.
  • the learning system S can also be used for labeling.
  • the learning system S may perform the labeling of the second group without creating the second learning model M2. Labels assigned to target data belonging to the second group can be used for various purposes such as fraud detection and marketing.
  • the functions described as being realized by the server 10 may be realized by the administrator terminal 30, or may be realized by another computer.
  • the functions described as being implemented by the server 10 may be shared among multiple computers.
  • data to be stored in the data storage unit 100 may be stored in a database server different from the server 10 .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Analysis (AREA)
PCT/JP2022/000352 2022-01-07 2022-01-07 学習システム、学習方法、及びプログラム Ceased WO2023132054A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/JP2022/000352 WO2023132054A1 (ja) 2022-01-07 2022-01-07 学習システム、学習方法、及びプログラム
US18/018,269 US20240256941A1 (en) 2022-01-07 2022-01-07 Learning system, learning method, and program
JP2022574615A JP7302107B1 (ja) 2022-01-07 2022-01-07 学習システム、学習方法、及びプログラム
TW112100374A TWI836840B (zh) 2022-01-07 2023-01-05 學習系統、學習方法及程式產品

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/000352 WO2023132054A1 (ja) 2022-01-07 2022-01-07 学習システム、学習方法、及びプログラム

Publications (1)

Publication Number Publication Date
WO2023132054A1 true WO2023132054A1 (ja) 2023-07-13

Family

ID=86996675

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/000352 Ceased WO2023132054A1 (ja) 2022-01-07 2022-01-07 学習システム、学習方法、及びプログラム

Country Status (4)

Country Link
US (1) US20240256941A1 (https=)
JP (1) JP7302107B1 (https=)
TW (1) TWI836840B (https=)
WO (1) WO2023132054A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20250363133A1 (en) * 2024-05-22 2025-11-27 Mastercard International Incorporated Method of creating a data labeling software program

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190354801A1 (en) * 2018-05-16 2019-11-21 Nec Laboratories America, Inc. Unsupervised cross-domain distance metric adaptation with feature transfer network
WO2020202327A1 (ja) * 2019-03-29 2020-10-08 楽天株式会社 学習システム、学習方法、及びプログラム
JP2020198041A (ja) * 2019-06-05 2020-12-10 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ 訓練装置、訓練方法、推定装置及びプログラム

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109102023A (zh) * 2018-08-14 2018-12-28 阿里巴巴集团控股有限公司 一种分类模型生成方法及装置、一种数据识别方法及装置
US11664820B2 (en) * 2019-09-25 2023-05-30 Nokia Technologies Oy Domain adaptation
JP2023056056A (ja) * 2020-03-05 2023-04-19 ソニーセミコンダクタソリューションズ株式会社 データ生成方法、学習方法及び推定方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190354801A1 (en) * 2018-05-16 2019-11-21 Nec Laboratories America, Inc. Unsupervised cross-domain distance metric adaptation with feature transfer network
WO2020202327A1 (ja) * 2019-03-29 2020-10-08 楽天株式会社 学習システム、学習方法、及びプログラム
JP2020198041A (ja) * 2019-06-05 2020-12-10 株式会社Ｐｒｅｆｅｒｒｅｄ Ｎｅｔｗｏｒｋｓ 訓練装置、訓練方法、推定装置及びプログラム

Also Published As

Publication number Publication date
US20240256941A1 (en) 2024-08-01
TWI836840B (zh) 2024-03-21
JP7302107B1 (ja) 2023-07-03
TW202336646A (zh) 2023-09-16
JPWO2023132054A1 (https=) 2023-07-13

Similar Documents

Publication Publication Date Title
US12153693B2 (en) Sensitive data classification
US20240303664A1 (en) Document-based fraud detection
US12118552B2 (en) User profiling based on transaction data associated with a user
US20250061461A1 (en) Generating a fraud prediction utilizing a fraud-prediction machine-learning model
US20190205993A1 (en) Transaction data categorizer system and method
CN113706258B (zh) 基于组合模型的产品推荐方法、装置、设备及存储介质
CN111966886B (zh) 对象推荐方法、对象推荐装置、电子设备及存储介质
CN109584037A (zh) 贷款用户信用评分的计算方法、装置和计算机设备
CN114641811A (zh) 用于用户网络活动异常检测的系统、方法和计算机程序产品
US20230196453A1 (en) Deduplication of accounts using account data collision detected by machine learning models
US12112307B1 (en) Systems and methods for providing early payment recommendations
CN115375177A (zh) 用户价值评估方法、装置、电子设备及存储介质
CN115713248A (zh) 对用于交易所的数据打分和评价的方法
Bae et al. A personal credit rating prediction model using data mining in smart ubiquitous environments
JP6933780B1 (ja) 不正検知システム、不正検知方法、及びプログラム
JP7302107B1 (ja) 学習システム、学習方法、及びプログラム
US11989243B2 (en) Ranking similar users based on values and personal journeys
JP2003323601A (ja) 信頼性尺度付き予測装置
CN119168731A (zh) 一种服务推荐方法、装置、计算机设备及存储介质
Vaquero Literature review of credit card fraud detection with machine learning
CN117350461A (zh) 企业异常行为预警方法、系统、计算机设备及存储介质
CN116777639A (zh) 案件风险评级方法、装置、计算机设备及存储介质
Hu Risk management of accounting systems: Using artificial intelligence for real-time analysis and decision-making
Lu et al. How data-sharing nudges influence people's privacy preferences: A machine learning-based analysis
El-kaime et al. The data mining: a solution for credit card fraud detection in banking

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2022574615

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 18018269

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22918643

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22918643

Country of ref document: EP

Kind code of ref document: A1