WO2023276073A1 - 学習モデル評価システム、学習モデル評価方法、及びプログラム - Google Patents

学習モデル評価システム、学習モデル評価方法、及びプログラム Download PDF

Info

Publication number
WO2023276073A1
WO2023276073A1 PCT/JP2021/024841 JP2021024841W WO2023276073A1 WO 2023276073 A1 WO2023276073 A1 WO 2023276073A1 JP 2021024841 W JP2021024841 W JP 2021024841W WO 2023276073 A1 WO2023276073 A1 WO 2023276073A1
Authority
WO
WIPO (PCT)
Prior art keywords
learning model
information
card
user
authentication
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/024841
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
恭輔 友田
周平 伊藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rakuten Group Inc
Original Assignee
Rakuten Group Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rakuten Group Inc filed Critical Rakuten Group Inc
Priority to PCT/JP2021/024841 priority Critical patent/WO2023276073A1/ja
Priority to US17/911,407 priority patent/US20240202743A1/en
Priority to JP2022529394A priority patent/JP7176158B1/ja
Priority to TW111121017A priority patent/TWI827086B/zh
Publication of WO2023276073A1 publication Critical patent/WO2023276073A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/018—Certifying business or products
    • G06Q30/0185—Product, service or business identity fraud
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31—User authentication
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/55—Detecting local intrusion or implementing counter-measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/38—Payment protocols; Details thereof
    • G06Q20/40—Authorisation, e.g. identification of payer or payee, verification of customer or shop credentials; Review and approval of payers, e.g. check credit lines or negative lists
    • G06Q20/401—Transaction verification
    • G06Q20/4016—Transaction verification involving fraud or risk level assessment in transaction processing

Definitions

  • This disclosure relates to a learning model evaluation system, a learning model evaluation method, and a program.
  • the accuracy of the learning model as in Patent Document 1 may gradually decline if recent trends are not learned by the learning model. For example, if the fraud detection accuracy of the learning model declines, there is a possibility that even if the behavior is actually fraudulent, it may be inferred to be legitimate. Conversely, it can be presumed to be fraudulent even if it is actually a legitimate action. Therefore, it is important to accurately evaluate the fraud detection accuracy of the learning model.
  • One of the purposes of this disclosure is to accurately evaluate the accuracy of learning models for detecting fraud in services.
  • a learning model evaluation system includes authenticated information acquiring means for acquiring authenticated information regarding behavior of an authenticated user who has performed predetermined authentication from a user terminal capable of using a predetermined service; output acquisition means for acquiring an output from a learning model for detecting fraud in the service based on the authenticated information; and evaluation for evaluating the accuracy of the learning model based on the output corresponding to the authenticated information. and means.
  • FIG. 4 is a diagram showing an example of how an IC chip of a card is read by an NFC unit; It is a figure which shows an example of a learning model. It is a functional block diagram showing an example of the function realized by the fraud detection system of the first embodiment. It is a figure which shows the data storage example of a user database. It is a figure which shows the data storage example of a training database. 4 is a flow chart showing an example of processing executed in the first embodiment; FIG.
  • FIG. 9 is a functional block diagram showing an example of functions implemented by the fraud detection system of the second embodiment;
  • FIG. 10 is a flow chart showing an example of processing executed in the second embodiment;
  • FIG. 13 is a diagram showing an example of the overall configuration of the fraud detection system of modification 1-1;
  • FIG. 10 is a diagram showing an example of a screen displayed on the user terminal of modification 1-1; It is a figure which shows an example of the flow which increases the upper limit after registration of a card.
  • FIG. 4 is a diagram showing an example of how an IC chip of a card is read by an NFC unit; It is a functional block diagram in the modification concerning a 1st embodiment. It is a figure which shows the data storage example of a user database.
  • a first embodiment which is an example of embodiments of a learning model creation system according to the present disclosure, will be described.
  • a case where a learning model creation system is applied to a fraud detection system will be taken as an example.
  • the portion described as a fraud detection system in the first embodiment can be read as a learning model creation system.
  • the learning model creation system may create a learning model, and the fraud detection itself may be performed by another system. That is, the learning model creation system may not include the fraud detection function of the fraud detection system.
  • FIG. 1 is a diagram showing an example of the overall configuration of a fraud detection system.
  • the fraud detection system S includes a server 10 and user terminals 20 .
  • Each of the server 10 and the user terminal 20 can be connected to a network N such as the Internet.
  • the fraud detection system S only needs to include at least one computer, and is not limited to the example shown in FIG.
  • a plurality of servers 10 may exist.
  • the server 10 is a server computer.
  • the server 10 includes a control section 11 , a storage section 12 and a communication section 13 .
  • Control unit 11 includes at least one processor.
  • the storage unit 12 includes a volatile memory such as RAM and a nonvolatile memory such as a hard disk.
  • the communication unit 13 includes at least one of a communication interface for wired communication and a communication interface for wireless communication.
  • the user terminal 20 is the user's computer.
  • the user terminal 20 is a smartphone, tablet terminal, wearable terminal, or personal computer.
  • the user terminal 20 includes a control section 21 , a storage section 22 , a communication section 23 , an operation section 24 , a display section 25 , an imaging section 26 and an IC chip 27 .
  • the physical configurations of the control unit 21 and the storage unit 22 are the same as those of the control unit 11 and the storage unit 12, respectively.
  • the physical configuration of the communication unit 23 may be the same as that of the communication unit 13, but the communication unit 23 of the first embodiment further includes an NFC (Near field communication) unit 23A.
  • the NFC unit 23A includes a communication interface for NFC.
  • NFC itself can use various standards, for example, international standards such as ISO/IEC18092 or ISO/IEC21481.
  • the NFC unit 23A includes hardware such as an antenna complying with standards, and realizes, for example, a reader/writer function, a peer-to-peer function, a card emulation function, a wireless charging function, or a combination thereof.
  • the operation unit 24 is an input device such as a touch panel.
  • the display unit 25 is a liquid crystal display or an organic EL display.
  • the imaging unit 26 includes at least one camera.
  • the IC chip 27 is a chip compatible with NFC.
  • the IC chip 27 may be a chip of any standard, for example, a FeliCa (registered trademark) chip, or a so-called Type A or Type B chip in the contactless standard.
  • the IC chip 27 includes hardware such as an antenna conforming to the standard, and stores, for example, information necessary for services used by users.
  • At least one of the programs and data stored in the storage units 12 and 22 may be supplied via the network N.
  • at least one of the server 10 and the user terminal 20 has a reading unit (for example, an optical disk drive or a memory card slot) that reads a computer-readable information storage medium, and an input/output unit for inputting/outputting data with an external device. (eg, a USB port) and/or may be included.
  • a reading unit for example, an optical disk drive or a memory card slot
  • an input/output unit for inputting/outputting data with an external device. (eg, a USB port) and/or may be included.
  • at least one of the program and data stored in the information storage medium may be supplied via at least one of the reading section and the input/output section.
  • the fraud detection system S detects fraud in services provided to users. Fraud means illegal activity, violation of the Terms of Service, or other nuisance. In the present embodiment, an example is given in which the act of using a service by pretending to be someone else by logging in with another person's user ID and password is considered illegal. Therefore, the part describing this act can be read as fraudulent.
  • the fraud detection system S can detect various frauds. Other examples of fraud will be described in modified examples below.
  • Detecting fraud means estimating or judging the presence or absence of fraud. For example, outputting information indicating whether it is fraudulent or outputting a score indicating the degree of suspicion of fraud corresponds to detecting fraud. For example, if the score is expressed numerically, the higher the score, the higher the suspicion of fraud. Scores may be represented by characters such as S rank, A rank, and B rank in addition to numerical values. The score can also be referred to as the probability or likelihood of fraud.
  • an administrative service is only described as a service.
  • the server 10 performs both service provision and fraud detection, but a computer other than the server 10 may provide the service.
  • the user terminal 20 is installed with an application of a public institution (hereinafter simply called an application). When a user uses a service for the first time, the user registers for use of the service in order to issue a user ID necessary for logging in to the service.
  • FIG. 2 is a diagram showing an example of the flow of usage registration.
  • the display unit 25 displays a registration screen G1 for inputting information required for use registration.
  • the user inputs information such as a desired user ID, password, name, address, telephone number, and personal number of the user in the input form F10.
  • a user ID is information that can uniquely identify a user in a service.
  • a personal number is information that can identify an individual recorded on a personal number card issued by a public institution. In 1st Embodiment, a personal number card is only described as a card.
  • Possession authentication is authentication using a property possessed only by an authorized person.
  • Possessed items are not limited to cards, and may be arbitrary items.
  • the possession may be an information storage medium or paper.
  • Possessions are not limited to tangible items, and may be intangible items such as electronic data.
  • FIG. 3 is a diagram showing an example of the flow of possession authentication.
  • a start screen G5 for starting possession authentication is displayed on the display unit 25 as shown in FIG.
  • NFC authentication is possession authentication executed by reading information recorded in the IC chip of the card with the NFC unit 23A.
  • Image authentication is possession authentication executed by photographing the card with the photographing unit 26 .
  • NFC authentication and image authentication are not distinguished, they are simply referred to as possession authentication.
  • FIG. 4 is a diagram showing an example of how the IC chip of the card is read by the NFC section 23A.
  • the card C1 in FIG. 4 is a fictitious one prepared for explanation of the first embodiment.
  • the NFC section 23A reads information recorded on the IC chip cp.
  • the NFC unit 23A can read arbitrary information in the IC chip cp. In the first embodiment, the case where the NFC unit 23A reads the personal number recorded in the IC chip cp will be described.
  • the user terminal 20 transmits to the server 10 the personal number read from the IC chip cp. Since this personal number is input from the user terminal 20 to the server 10, this personal number is hereinafter referred to as an input personal number.
  • Input in the first embodiment means sending some data to the server 10 .
  • a correct personal number is registered in advance at the time of use registration.
  • this personal number will be referred to as a registered personal number.
  • the personal number When there is no particular distinction between the input personal number and the registered personal number, they may simply be referred to as the personal number.
  • the server 10 receives the input personal number from the user terminal 20. If the user is the valid owner of the card C1, the input personal number and the registered personal number of the logged-in user match. When the input personal number matches the registered personal number of the logged-in user, a success screen G7 indicating that possession authentication has succeeded is displayed on the display unit 25, as shown in FIG. As shown in the success screen G7, the number of services that can be used from the user terminal 20 for which possession authentication has succeeded increases.
  • the display unit 25 displays a failure screen G8 indicating that possession authentication has failed. In this case, the services available from the user terminal 20 remain restricted. The user returns to the reading screen G6 and reads the card C1 again or inquires of the call center. If a third party logs in illegally, the card C1 is not at hand and possession authentication cannot be successful, so services available from the third party's user terminal 20 are restricted.
  • the optical character recognition may be performed at the user terminal 20.
  • the method of acquiring the input personal number from the captured image is not limited to optical character recognition. As this method itself, various known methods can be used. For example, if a code (for example, a bar code or a two-dimensional code) containing the input personal number is formed on the card C1, the input personal number may be acquired using the code photographed in the photographed image. The process of acquiring the input personal number from the code may be executed by the server 10 or by the user terminal 20 .
  • the number of services available from user terminals 20 whose possession authentication has succeeded is greater than the services available from user terminals 20 whose possession authentication has not succeeded. Even if a third party illegally obtains the user ID and password and logs in illegally, the third party does not possess the card C1 and cannot succeed in possession authentication, so available services are limited. Therefore, unauthorized use of the service by a third party is suppressed, and the security of the service is enhanced.
  • the learning model is a model that uses machine learning.
  • Machine learning is sometimes called artificial intelligence.
  • Machine learning itself can use various known methods, for example, neural networks.
  • deep learning or reinforcement learning is also classified as machine learning, so the learning model may be a model created using deep learning or reinforcement learning.
  • the learning model may be a rule or decision tree model using machine learning.
  • supervised learning is taken as an example, but unsupervised learning or semi-supervised learning may also be used.
  • the learning model of the first embodiment can detect not only fraud by a third party who has logged in illegally with another person's user ID, but also fraud by a user who has logged in with his own user ID. For example, a user may log in to a service with his or her own user ID and request a large number of certificates for mischievous purposes, or may make a reservation at a window and cancel without notice. If there is a certain tendency in the behavior of such fraudulent users, the learning model can detect fraud by learning this tendency.
  • the training data is often created manually by the creator of the learning model M. To improve the accuracy of the learning model M, it is necessary to prepare a large amount of training data. It takes a lot of time and effort for an administrator to create all of them manually. For example, administrators must determine whether individual actions in the service are legitimate or fraudulent and create training data.
  • the user who has carried out the possession authentication possesses the physical card necessary for carrying out the possession authentication, so the probability of not committing fraud is extremely high. Even if a fraudulent user can illegally obtain a user ID and password through phishing, etc., there is a probability that the user cannot steal the physical card and use the service without carrying out possession authentication. very high. Even if a fraudulent user steals a physical card, if a user who has performed possession authentication commits fraud, the fraudulent user can be easily identified. In addition, the probability of using the service without carrying out possession authentication is very high. For example, a fraudulent user may enter a number that is not his or her personal number as the personal number to complete the user registration. 2 and 3, even if a number other than one's own personal number is entered, the service can be used under restrictions.
  • the behavior of the user who has performed possession authentication is considered valid, and training data is created.
  • a user who has performed possession authentication will be referred to as an authenticated user.
  • the training data of the first embodiment is created based on the behavior of authenticated users.
  • the training data includes an input portion that includes location information, date and time information, and usage information, and an output portion that indicates legitimacy.
  • the location information indicates the location of the user terminal 20.
  • the location may be indicated by any information, such as latitude and longitude, address, mobile base station information, wireless LAN access point information, or IP address.
  • the location information may be the distance from the center where the service is usually used.
  • the center may be an average value of locations used by a certain user ID, or may be an average value of locations used by a certain user terminal 20 .
  • the date and time information indicates the date and time when the service was used.
  • the usage information indicates how the service was used.
  • the usage information can also be said to be a service usage history. For example, the usage information indicates the type of service used, details of usage, user's operation, or a combination thereof.
  • the fraud detection system S of the first embodiment provides training data for the learning model M using supervised learning based on the authenticated information of the authenticated user who has a very high probability of not committing fraud. create.
  • the creator of the learning model M does not have to manually create training data, and the creation of the learning model M is simplified.
  • details of the first embodiment will be described.
  • FIG. 6 is a functional block diagram showing an example of functions realized by the fraud detection system S of the first embodiment. Here, functions realized by each of the server 10 and the user terminal 20 will be described.
  • the server 10 implements a data storage unit 100 , an authenticated information acquisition unit 101 , a creation unit 102 , and a fraud detection unit 103 .
  • the data storage unit 100 is realized mainly by the storage unit 12 .
  • Each of the authenticated information acquisition unit 101 , the creation unit 102 , and the fraud detection unit 103 is implemented mainly by the control unit 11 .
  • the data storage unit 100 stores data necessary for creating the learning model M.
  • FIG. the data storage unit 100 stores a user database DB1, a training database DB2, and a learning model M.
  • FIG. 7 is a diagram showing an example of data storage in the user database DB1.
  • the user database DB1 is a database that stores information about users who have completed usage registration.
  • the user database DB1 stores user IDs, passwords, names, addresses, telephone numbers, registered personal numbers, terminal IDs, possession authentication flags, service usage settings, location information, date and time information, and usage information.
  • the terminal ID is information that allows the user terminal 20 to be identified. 1st Embodiment demonstrates the case where the server 10 issues terminal ID. A terminal ID is issued based on a predetermined rule. The server 10 issues terminal IDs so as not to overlap with other terminal IDs. An expiration date may be set for the terminal ID. A terminal ID can be issued at any timing. For example, the terminal ID is issued at the timing when the application is started, the timing when the expiration date set in the terminal ID expires, or the timing when the operation for updating the terminal ID is performed.
  • the user terminal 20 can be identified by any information other than the terminal ID.
  • the user terminal 20 is identified by an IP address, information stored in a cookie, an ID stored in a SIM card, an ID stored in the IC chip 27, or individual identification information of the user terminal 20. may Information that can identify the user terminal 20 in some way may be stored in the user database DB1.
  • the terminal ID associated with the user ID is the terminal ID of the user terminal 20 that has logged in with this user ID. Therefore, when a user who is the legitimate owner of a certain user ID logs in from a new user terminal 20, the terminal ID of this user terminal 20 is associated with this user ID. Even if a third party illegally logs in using this user ID, the terminal ID of the third party's user terminal 20 is associated with this user ID.
  • a terminal ID is associated with a possession authentication flag, usage settings, time information, location information, date and time information, and usage information.
  • information such as a possession authentication flag is associated with each combination of user ID and terminal ID.
  • the user ID “taro.yamada123” has logged in from two user terminals 20 .
  • User ID “hanako.suzuki999” has been logged in from three user terminals 20 .
  • User ID “kimura9876” has been logged in from only one user terminal 20 .
  • the possession authentication flag is information indicating whether possession authentication has been executed. For example, a possession authentication flag of "1" indicates that NFC authentication has been performed. The fact that the possession authentication flag is “2" indicates that image authentication has been performed. A possession authentication flag of "0" indicates that possession authentication has not been executed.
  • the initial value of the possession authentication flag is "0" because possession authentication is not executed at the time of use registration.
  • the possession authentication flag changes to "1" or "2". In the case where possession authentication can be executed at the time of use registration, if the user executes possession authentication at the time of use registration, the initial value of the possession authentication flag becomes "1" or "2".
  • the usage settings indicate the types of services that can be used from the app.
  • the usage setting with the possession authentication flag “1” or “2” allows more services to be used than the usage setting with the possession authentication flag “0”. It is assumed that the relationship between the presence/absence of possession authentication execution and the usage setting (that is, the relationship between the possession authentication flag and the usage setting) is defined in advance in the data storage unit 100 .
  • the use setting of possession authentication flag "1” or “2” is a setting that allows all services to be used.
  • the use setting of possession authentication flag "0" is a setting that allows only some services to be used.
  • location information When a service is used while logged in with a certain user ID from a certain user terminal 20, location information, date and time information, and usage information associated with the combination of this user ID and this user terminal 20 are updated.
  • a known method using GPS, a mobile base station, or the like can be used as the method itself for acquiring the date and time information.
  • the usage information may store information corresponding to the service, and the details are as described above.
  • the learning model M is a model that uses machine learning.
  • Machine learning is sometimes called artificial intelligence.
  • Machine learning itself can use various known methods, for example, neural networks.
  • deep learning or reinforcement learning is also classified as machine learning, so the learning model M may be a model created using deep learning or reinforcement learning.
  • supervised learning is taken as an example, but unsupervised learning or semi-supervised learning may also be used.
  • the authenticated information acquisition unit 101 acquires authenticated information regarding behavior of an authenticated user who has performed predetermined authentication from a user terminal 20 that can use a predetermined service.
  • this authentication is possession authentication for confirming whether or not the user possesses a predetermined card C1 using the user terminal 20
  • the possession authentication it can be read as the predetermined authentication. That is, where NFC authentication or image authentication is described, it can be read as predetermined authentication.
  • an authenticated user is a user who has performed possession authentication from the user terminal 20, but the authenticated user may be a user who has performed predetermined authentication from the user terminal 20.
  • the predetermined authentication is authentication that can be executed from the user terminal 20.
  • the predetermined authentication may be the authentication at the time of login, but in the first embodiment, the predetermined authentication is different from the authentication at the time of login.
  • the predetermined authentication is not limited to possession authentication using the card C1.
  • Various authentication methods can be used for predetermined authentication.
  • the predetermined authentication may be possession authentication for confirming belongings other than the card C1.
  • the personal belongings may be arbitrary items that can be identified.
  • the possession may be an identification card other than a card such as a passport, an information storage medium on which some kind of authentication information is recorded, or a piece of paper on which some kind of authentication information is formed.
  • the possession may be an electronic object such as a code containing authentication information.
  • the prescribed authentication is not limited to possession authentication.
  • the predetermined authentication may be knowledge authentication such as password authentication, passcode authentication, PIN authentication, or password authentication. If the predetermined authentication is password authentication, it is assumed that a password different from that used at login is used.
  • the predetermined authentication may be biometric authentication such as face authentication, fingerprint authentication, or iris authentication. In the first embodiment, a case will be described where the predetermined authentication is more secure than the login authentication, but the login authentication may be more secure than the predetermined authentication. Authentication at the time of login is not limited to password authentication, and any authentication method may be used.
  • the card C1 used for possession authentication in the first embodiment includes an input personal number used for possession authentication.
  • the input personal number is electronically recorded in the IC chip cp of the card C1.
  • the input personal number is also formed on the surface of the card C1.
  • a registered personal number that is correct in possession authentication is registered in the user database DB1.
  • Each of the input personal number and the registered personal number is an example of authentication information used at the time of authentication.
  • authentication information corresponding to the authentication method may be used.
  • the authentication information may be a password, passcode, PIN, or password.
  • biometric authentication each piece of authentication information may be a facial photograph, facial features, fingerprint pattern, or iris pattern.
  • the server 10 acquires from the user terminal 20 the input personal number of the card C1 acquired using the NFC unit 23A.
  • the server 10 refers to the user database DB1 and determines whether or not the input personal number obtained from the user terminal 20 matches the registered personal number associated with the logged-in user ID. If they match, possession authentication succeeds. If they do not match, possession authentication fails.
  • the server 10 acquires a photographed image of the card C1 from the user terminal 20.
  • the server 10 uses optical character recognition to acquire the input personal number from the captured image.
  • the flow of possession authentication after the input personal number is acquired is the same as NFC authentication.
  • the input personal number is printed on the surface of the card C1, but the input personal number may be embossed on the surface of the card C1.
  • the input personal number may be formed on at least one of the front and back sides of the card C1.
  • the service of the first embodiment can be logged in from each of a plurality of user terminals 20 with the same user ID.
  • the authentication unit 101 can perform possession authentication for each user terminal 20 while logging in to the service from the user terminal 20 with the user ID. For example, assume that the user with the user ID “taro.yamada123” in FIG. 7 uses two user terminals 20 . These two user terminals 20 are described as a first user terminal 20A and a second user terminal 20B.
  • Authenticated information is information about the actions of authenticated users. Actions are operation contents for the user terminal 20, information transmitted from the user terminal 20 to the server 10, or a combination thereof. In other words, behavior is information that indicates how the service was used.
  • a combination of location information, date and time information, and usage information corresponds to information about actions.
  • a combination of the authenticated user's location information, date and time information, and usage information is an example of authenticated information. Therefore, hereinafter, this combination is described as authenticated information.
  • the authenticated information is not limited to the example of the first embodiment, and may be any information related to some action of the authenticated user.
  • the authenticated information may be any characteristic that has some correlation with whether or not it is fraudulent.
  • the authenticated information includes the time from the user's login until reaching a predetermined screen, the number or types of screens displayed before reaching this screen, the number of operations on a certain screen, the number of pointers It may be a trajectory, or a combination thereof.
  • the authenticated information may be information corresponding to the service. Other examples of authenticated information will be described in modified examples below.
  • authenticated information is stored in the user database DB1.
  • the authenticated information acquisition unit 101 refers to the user database DB1 and acquires authenticated information.
  • the authenticated information acquiring unit 101 acquires a plurality of pieces of authenticated information, but the authenticated information acquiring unit 101 only needs to acquire at least one piece of authenticated information.
  • the authenticated information acquisition unit 101 acquires authenticated information for a predetermined period (for example, about one week to one month) that is the most recent date and time indicated by the date and time information. may retrieve all authenticated information stored in The authenticated information acquisition unit 101 does not have to acquire all the authenticated information within the predetermined period, and may randomly select and acquire part of the authenticated information within the predetermined period.
  • the authenticated information acquiring unit 101 may acquire a sufficient number of authenticated information for the learning model M to learn.
  • the creation unit 102 creates training data that includes an input portion that is authenticated information and an output portion that indicates legitimacy.
  • the input portion can be expressed in any form, such as vector form, array form, or as a single number. It is assumed that the input portion is a numerical representation of items included in location information, date and time information, and usage information included in the authenticated information. This quantification may be performed inside the learning model M.
  • the input part corresponds to the behavior feature amount.
  • the output part corresponds to the correct answer of the output of the learning model M.
  • the fraud detection unit 103 uses the created learning model M to detect fraud.
  • the fraud detection unit 103 acquires the target user's location information, date and time information, and usage information, and stores them in the user database DB1. A combination of these pieces of information is the target information shown in FIG.
  • the fraud detection unit 103 acquires the output of the learning model M based on the target information of the target user when a predetermined fraud detection timing comes.
  • the fraud detection unit 103 inputs target information to the learning model M and acquires output from the learning model M. After executing the conversion process, the target information on which the process has been executed may be input to the learning model M.
  • the fraud detection unit 103 restricts the provision of the service to the target user, that is, the target user's use of the service.
  • the fraud detection unit 103 does not restrict the use of the service by the target user if this output indicates legitimacy.
  • the timing of fraud detection may be any timing, for example, when the button B30 on the top screen G3 is selected, when information registered in the user database DB1 is changed, when logging in to a service, or when any may be executed.
  • the user terminal 20 implements a data storage unit 200 , a display control unit 201 and a reception unit 202 .
  • the data storage unit 200 is implemented mainly by the storage unit 22 .
  • Each of the display control unit 201 and the reception unit 202 is implemented mainly by the control unit 21 .
  • the data storage unit 200 stores data required for the processing described in the first embodiment.
  • the data storage unit 200 stores applications.
  • the display control unit 201 causes the display unit 25 to display each screen described with reference to FIGS. 2 and 3 based on the application.
  • the reception unit 202 receives a user's operation on each screen.
  • the user terminal 20 transmits the content of the user's operation to the server 10 .
  • the user terminal 20 transmits location information and the like necessary for acquiring authenticated information.
  • FIG. 9 is a flow chart showing an example of processing executed in the first embodiment.
  • the processing shown in FIG. 9 is executed by the control units 11 and 21 operating according to programs stored in the storage units 12 and 22, respectively.
  • This processing is an example of processing executed by the functional blocks shown in FIG. It is assumed that user registration has been completed before this process is executed. It is assumed that the user terminal 20 stores in advance the terminal ID issued by the server 10 .
  • the server 10 acquires the authenticated information of the authenticated user based on the user database DB1 (S100).
  • the server 10 acquires the authenticated information stored in the record whose date and time indicated by the date and time information is within the most recent predetermined period among the records with the possessed authentication flag of "1" or "2".
  • the server 10 creates training data based on the authenticated information acquired in S100 (S101). In S101, the server 10 creates training data including an input portion as authenticated information and an output portion indicating fraud, and stores the training data in the training database DB2. The server 10 determines whether or not the creation of training data has been completed (S102). In S102, the server 10 determines whether or not a predetermined number of training data have been created.
  • the process returns to S100, and new training data is created and stored in the training database DB2.
  • the server 10 creates a learning model M based on the training database DB2 (S103).
  • the server 10 transfers the individual training data to the learning model M so that the output part of the training data is output when the input part of the individual training data stored in the training database DB2 is input. let them learn
  • the user terminal 20 activates the application based on the operation of the target user, and displays the top screen G3 on the display unit 25 (S104).
  • a login may be performed between the server 10 and the user terminal 20 when the application is started. The login may require the user to enter a user ID and password, or the user terminal 20 may store information indicating that the user has logged in in the past, and this information may be used for the login. Thereafter, when the user terminal 20 somehow accesses the server 10, the location information, date and time information, and usage information associated with the terminal ID of the user terminal 20 are updated as appropriate.
  • the server 10 displays the top screen such that the button B30 of the unavailable service cannot be selected based on the usage setting associated with the terminal ID of the user terminal 20 before the login is successful and the top screen G3 is displayed.
  • G3 display data may be generated and transmitted to the user terminal 20 .
  • the user terminal 20 identifies the operation of the target user based on the detection signal from the operation unit 24 (S105). In S105, either the button B30 for using administrative services or the button B31 for carrying out possession authentication is selected. If the user terminal 20 has already executed possession authentication, the button B31 may not be selectable. Note that when the target user performs an operation for terminating the application or an operation for shifting the application to the background (S105; end), this processing ends.
  • the user terminal 20 requests the server 10 to provide the type of service selected by the target user from the button B30 (S106).
  • the server 10 inputs the target information of the target user to the learning model M and acquires the output from the learning model M (S107).
  • the target information is location information, date and time information, and usage information of the target user (that is, the logged-in user). If the target user has logged in from a plurality of user terminals 20, the output from the learning model M is acquired based on the target information associated with the terminal IDs of the user terminals 20 currently logged in.
  • the server 10 refers to the output from the learning model M (S108). If the output from the learning model M indicates fraud (S108; fraudulent), the server 10 restricts the provision of services (S109). At S109, the server 10 does not provide the type of service selected by the user. An error message is displayed on the user terminal 20 . If the output from the learning model M indicates valid (S108; valid), a service providing process for providing a service between the server 10 and the user terminal 20 is executed (S110), and this process ends. . In S ⁇ b>110 , the server 10 refers to the user database DB ⁇ b>1 and acquires usage settings associated with the user ID of the logged-in user and the terminal ID of the user terminal 20 . The server 10 provides services based on this usage setting. The server 10 receives user operation details from the user terminal 20 and executes processing according to the operation details.
  • the learning model M is created based on the authenticated information so that the behavior of the authenticated user is estimated to be valid.
  • the learning model M can be created without the creator of the learning model M manually creating training data. It can be simplified.
  • a series of processes from creation of training data to learning of the learning model M can be automated, and the learning model M can be created quickly.
  • a learning model M that has learned the latest trends can be quickly applied to the fraud detection system S, and fraud can be detected with high accuracy. As a result, unauthorized use of the service is prevented and security is enhanced.
  • the learning model M of the second embodiment may be created by a method different from that of the first embodiment.
  • the learning model M may be created based on training data manually created by the creator of the learning model M.
  • the learning model M may be created based on training data created using a known training data creation support tool. Therefore, the fraud detection system S of the second embodiment does not have to include the functions described in the first embodiment.
  • description of the same points as in the first embodiment is omitted.
  • the maximum amount that can be used from the application is set for each card.
  • This maximum amount may mean the maximum amount of the card itself (so-called usage limit or limit), but in modification 1-1, it is not the maximum amount of the card itself, but the maximum amount of the application. .
  • the maximum amount is the total amount that can be used from the application for a predetermined period (for example, one week or one month).
  • the upper limit amount may be the upper limit amount for one payment process.
  • the card's upper limit varies depending on the possession authentication method performed when the card was registered. The higher the security of the possession verification performed when the card was registered, the higher the maximum amount of this card. For example, the security code may be leaked by phishing or the like, so security code authentication is the lowest security. On the other hand, NFC authentication or image authentication, in principle, cannot be successful unless the user possesses a physical card, so security is assumed to be higher than that of security code authentication.
  • FIG. 15 is a diagram showing an example of the flow of increasing the maximum amount after card registration.
  • a selection screen G13 for selecting a card for carrying out possession authentication is displayed on the display unit 25 as shown in FIG.
  • a list L130 of registered cards is displayed on the selection screen G13. The user selects a card for possession authentication from the list L130.
  • the user terminal 20 transmits the electronic money ID read from the IC chip cp to the business server 30 . Since this electronic money ID is input from the user terminal 20 to the provider server 30, this electronic money ID is hereinafter referred to as an input electronic money ID.
  • the correct electronic money ID is registered in the issuer server 40 . Hereinafter, this electronic money ID will be referred to as a registered electronic money ID.
  • this electronic money ID When the input electronic money ID and the registered electronic money ID are not distinguished from each other, they may simply be referred to as electronic money ID.
  • the upper limit of the other card (card 1" in the example of FIG. 15) different from card C2 on which NFC authentication has been performed is also increased from 30,000 yen to 100,000 yen.
  • the limits on other cards do not need to be increased. Even if it is associated with the same user ID as the card C2 on which NFC authentication has been performed, if the holder is different, there is a possibility that a third party has registered without permission, so the maximum amount will not be increased. . If the same combination of registered card information and registered electronic money ID as the combination of input card information and input electronic money ID is not registered in the issuer server 40, possession authentication fails. In this case, a failure screen G16 similar to the failure screen G8 in FIG.
  • the information used for possession authentication is not limited to the input electronic money ID.
  • a point ID that can identify points may be used for possession authentication. It is assumed that the point ID is included in card C2.
  • the card number and expiration date of card C2 may be used for possession authentication.
  • modification 1-1 some information contained in the card C2 or information associated with this information may be used for possession authentication. good too.
  • the data storage unit 300 stores a user database DB1, a training database DB2, and a learning model M. These data are generally the same as those in the first embodiment, but the specific contents of the user database DB1 are different from those in the first embodiment.
  • FIG. 18 is a diagram showing an example of data storage in the user database DB1.
  • the user database DB1 is a database that stores information about users who have completed usage registration.
  • the user database DB1 stores user IDs, passwords, names, payment methods of payment sources, registered card information, electronic cash information, location information, date and time information, and usage information.
  • a user registers for use a user ID is issued and a new record is created in the user database DB1. This record stores registered card information and electronic cash information along with the password and name specified at the time of use registration.
  • Electronic cash information is information about electronic cash that can be used from the app.
  • the electronic cash information includes an electronic cash ID that can identify the electronic cash and the balance of the electronic cash.
  • the electronic cash may be chargeable with the card C2 registered by the user.
  • the setting of the upper limit amount of charge in this case may correspond to the usage setting.
  • Information stored in the user database DB1 is not limited to the example in FIG.
  • the authenticated information acquisition unit 301, creation unit 302, and fraud detection unit 303 are the same as the authenticated information acquisition unit 101, creation unit 102, and fraud detection unit 103, respectively.
  • the learning model M in modification 1-1 is a model for detecting fraudulent payment processing.
  • the creation unit 302 When the authenticated user inputs location information such as the store where the payment process was executed, date and time information when the payment process was executed, and usage information such as the payment amount, the creation unit 302 outputs information indicating that the payment is valid. Create a learning model M so that
  • the second card C3 is given the reference numeral C3 to distinguish it from the first card C2, but the second card C3 is not shown in the drawing.
  • the second card C3 associated with the first card C2 is the second card C3 associated with the same user ID as the first card C2.
  • the first card C2 and the second card C3 may be directly associated instead of using the user ID.
  • the second card C3 is a card for which possession authentication has not been performed.
  • the second card C3 may be a card for which possession authentication can be performed, but possession authentication has not been performed. If the second card C3 is a card capable of carrying out possession authentication, the second card C3 may correspond to the first card C2.
  • the second card C3 is a card that does not support NFC authentication or image authentication.
  • the second card C3 does not include an input electronic money ID used for NFC authentication or image authentication.
  • the creation unit 302 creates the learning model M based on the authenticated information corresponding to the first card C2.
  • the creating unit 302 does not have to use the location information, the date and time information, and the usage information corresponding to the second card C3 in creating the learning model M.
  • the method itself for creating the learning model M based on the authenticated information is as described in the first embodiment.
  • the data storage unit 300 stores a database that stores information on various cards. It is assumed that name information of various cards is stored in this database. The first name information and the second name information are obtained from this database. If the business server 30 does not manage this database, the business server 30 requests the issuer server 40 to compare the first name information and the second name information, and obtains only the comparison result from the issuer server 40. do it.
  • the comparison unit 304 compares the first holder and the second holder.
  • the comparison unit 304 refers to the user database DB1, acquires the first and second holders, and sends the comparison result to the authenticated information acquisition unit 101.
  • FIG. As described above, the first name information and the second name information may be other information.
  • the first holder of the first card C2 (card No. 2) with the user ID "taro.yamada123" and the second holder of the second card C3 (card No. 1) and are both "TARO YAMADA". Therefore, when possession authentication of the first card C2 is executed, the second card C3 is also used in the learning of the learning model M.
  • the second card C3 is also used in the learning of the learning model M.
  • the second holder of the second card C3 (No. 3 card) is "MIKI OKAMOTO", which is different from the first holder.
  • the other second card C3 may have been registered by a third party without permission, and the behavior using the other second card C3 may not be legitimate. Not used for learning.
  • the modification 1-3 when the result of comparison between the first name information about the name of the first card C2 and the second name information about the name of the second card C3 is a predetermined result, the first card By creating the learning model M based on the authenticated information corresponding to C2 and the authenticated information corresponding to the second card C3, more authenticated information is learned to increase the accuracy of the learning model M. higher. As a result, it is possible to effectively prevent unauthorized use of the service, improve security, and prevent deterioration of convenience.
  • the second card C3 described in modification 1-3 may be a card that does not support possession authentication.
  • the authenticated information corresponding to the second card C3 may be information related to the behavior of the authenticated user using the second card C3 for which possession authentication has not been performed.
  • a card that does not support possession authentication is a card for which possession authentication cannot be executed.
  • a card that does not contain an IC chip does not support NFC authentication.
  • a card that does not have an input electronic money ID formed on its face does not support image authentication.
  • a card that does not include an input electronic money ID used for possession authentication is a card that does not support possession authentication.
  • the learning model M may be learned using behavior of an unauthenticated user who has not performed possession authentication.
  • the fraud detection system S further includes an unauthenticated information acquisition unit 305 that acquires unauthenticated information regarding behavior of an unauthenticated user who has not been authenticated.
  • An unauthenticated user is a user whose possession authentication flag is not "1" or "2".
  • an unauthenticated user is a user whose possession authentication flag is at least partly "0".
  • the unauthenticated information acquiring unit 305 refers to the user database DB1 and acquires the unauthenticated information of the unauthenticated user.
  • the unauthenticated information is a combination of the unauthenticated user's location information, date and time information, and usage information.
  • the unauthenticated information may be arbitrary information and is not limited to a combination thereof.
  • the creating unit 302 creates training data indicating whether the behavior of the unauthenticated user is legitimate or illegal, and makes the learning model M learn based on this training data.
  • the training data created using the authenticated user will be referred to as first training data
  • the training data created using the unauthenticated user will be referred to as second training data.
  • the data structures themselves of the first training data and the second training data are the same as described in the first embodiment.
  • the creation unit 302 may acquire the output from the learned learning model M based on the unauthenticated information, and create the second training data based on the output. .
  • the creating unit 302 presents the creator of the learning model M with the output of the learning model M corresponding to the unauthenticated information. The creator of learning model M checks whether this output is correct. Authors modify this output as necessary.
  • the learning model M of modification 1-7 shall output a score related to fraud in the service.
  • Modification 1-7 describes the case where the score indicates the degree of legitimacy, but the score may indicate the degree of dishonesty.
  • the score indicates the probability of being classified as legitimacy.
  • the score indicates the degree of fraud, the score indicates the probability of being classified as fraudulent.
  • Various known methods can be used as the method itself for calculating the score by the learning model M.
  • the creating unit 302 acquires the score from the learning model M based on the unauthenticated behavior of the unauthenticated user.
  • the creating unit 302 changes this score based on the subsequent actions of the unauthenticated user. It is assumed that the score change method is determined in advance in the data storage unit 100 .
  • a relationship is defined between an action classified as fraudulent and the amount of change in the score (in this modified example, the amount of decrease because the score indicates the degree of legitimacy) when this action is performed.
  • the relationship between an action classified as legitimate and the amount of change in the score when this action is performed (in this modified example, the score indicates the degree of legitimacy, so the amount of increase) is defined. If the unauthenticated user behaves suspected of being fraudulent, the creation unit 302 changes the score based on the amount of change corresponding to the behavior so that the degree of fraud increases. When an unauthenticated user behaves in a way that is suspected of being legitimate, the creating unit 302 changes the score based on the amount of change corresponding to the behavior so that the degree of fraud is weakened.
  • the creation unit 302 may change this classification result. For example, suppose that the output of the learning model M is "1" indicating that it is illegal or "0" indicating that it is valid. If the output corresponding to the unauthenticated information is "1" and the unauthenticated user is classified as fraudulent, the creating unit 302 determines if the unauthenticated user continues to act with a high probability of being legitimate. , the second training data may be created after changing this output to "0".
  • the modification 1-7 based on the unauthenticated information after the output corresponding to the unauthenticated information is acquired, the content of the output is changed, and based on the changed output content, the second By creating training data, the accuracy of the learning model M is further enhanced.
  • an upper limit may be set such that the score corresponding to unauthenticated information indicates more fraud than the score corresponding to authenticated information.
  • the creating unit 302 determines the upper limit of the score corresponding to the unauthenticated information. For example, the creating unit 302 determines the average score of the authenticated information as the upper limit score of the unauthenticated information. In addition, for example, the creation unit 302 determines the lowest value or the predetermined lowest value among the scores of the authenticated information as the upper limit value of the score corresponding to the unauthenticated information.
  • the learning model M outputs a score corresponding to unauthenticated information based on the upper limit.
  • the learning model M outputs a score corresponding to unauthenticated information so as not to exceed the upper limit. For example, even if the internally calculated score of the learning model M exceeds the upper limit, the learning model M outputs the score so that the output score is equal to or less than the upper limit.
  • the upper limit value may be an average score obtained by inputting unauthenticated information into the learning model M, or the like.
  • the method itself for creating the learning model M using the score corresponding to the unauthenticated information is as described in Modification 1-7.
  • the accuracy of the learning model M is improved by outputting the score corresponding to the unauthenticated information based on the upper limit set to indicate fraud more than the score corresponding to the authenticated information. is higher.
  • the creating unit 302 creates a learning model M based on the authenticated information and the confirmed information.
  • the only difference from the first embodiment and the other modifications is that the definite information is used, and the method of creating the learning model M itself is the same as the first embodiment and the other modifications. That is, the creation unit 302 outputs a result of being valid when the authenticated information is input, and outputs a result associated with the confirmation information (unauthorized A learning model M is created so that the result of whether it is true or valid) is output.
  • modification 1-9 by creating a learning model M based on the authenticated information and the confirmed information of the confirmed user, learning is performed using more information, and the accuracy of the learning model M is higher.
  • the learning model M may be a model of unsupervised learning.
  • the creating unit 302 creates a learning model M based on the authenticated information so that fraudulent behavior in the service is an outlier.
  • the creating unit 302 creates a learning model M for unsupervised learning such that when a plurality of pieces of authenticated information are input, these pieces of authenticated information are clustered into the same cluster.
  • this learning model M when information about fraudulent behavior different from the characteristics indicated by the authenticated information is input, it is output as an outlier. That is, fraudulent actions are output as not belonging to the cluster of authenticated information.
  • Unsupervised learning itself can use various methods, for example, principal component analysis, vector quantization, non-negative matrix factorization, k-means method, or Gaussian mixture model, etc., in addition to the clustering method described above. method is available.
  • the fraud detection unit 303 acquires the output of the learning model M based on the target information of the target user, and determines that it is fraudulent if the output is an outlier. The fraud detection unit 303 determines that the output is legitimate if the output is not an outlier.
  • unsupervised learning is used by creating a learning model M using unsupervised learning such that fraudulent behavior in the service is an outlier based on authenticated information.
  • Creation of the learning model M can be simplified.
  • a series of processes for creating the learning model M can be automated, and the learning model M can be created quickly.
  • a learning model M that has learned the latest trends can be quickly applied to the fraud detection system S, and fraud can be detected with high accuracy.
  • unauthorized use of the service is prevented and security is enhanced. It is also possible to prevent a decrease in convenience, such as when the target user's behavior, which should be legitimate, is presumed to be fraudulent and the service cannot be used.
  • the fraud detection system S of the second embodiment can also be applied to electronic payment services as described in Modifications 1-1 to 1-10 of the first embodiment.
  • FIG. 19 is a functional block diagram of a modification according to the second embodiment.
  • FIG. 19 also shows functions in modifications 2-2 to 2-9 after modification 2-1.
  • the provider server 30 includes a data storage unit 300, an authenticated information acquisition unit 301, a creation unit 302, a fraud detection unit 303, a comparison unit 304, an unauthenticated information acquisition unit 305, a confirmed information acquisition unit 306, an output acquisition unit 307, an evaluation A unit 308 and a process execution unit 309 are included.
  • Each of the output acquisition unit 307 , the evaluation unit 308 , and the processing execution unit 309 is realized mainly by the control unit 31 .
  • modification 2-1 it is possible to accurately evaluate fraud detection accuracy of the learning model M for detecting fraud in electronic payment services.
  • the fraud detection system S includes a processing execution unit 309 that executes processing for creating a learning model M using recent behavior in the service when the accuracy of the learning model M becomes less than a predetermined accuracy. may contain.
  • This process may be a process of notifying the creator of the learning model M to recreate the learning model M, or a process of recreating the learning model M by the same method as in the first embodiment.
  • any means such as e-mail can be used for notification.
  • the process of re-creating the learning model M may be the process of creating the learning model M as in the first embodiment using the latest authenticated information, and particularly the learning model M as in the first embodiment. Methods other than model M creation may be utilized.
  • the learning model M may be created by a system other than the fraud detection system S.
  • the learning model M when the accuracy of the learning model M becomes less than a predetermined accuracy, the learning model M is generated by executing the process for creating the learning model M using recent behavior in the service. It is possible to deal with the case where the fraud detection accuracy of the model M is lowered. A learning model M that has learned the latest trends can be quickly applied to the fraud detection system S, and fraud can be detected with high accuracy. As a result, unauthorized use of the service is prevented and security is enhanced. It is also possible to prevent a decrease in convenience, such as when the target user's behavior, which should be legitimate, is presumed to be fraudulent and the service cannot be used.
  • the evaluation unit 308 may evaluate the accuracy of the learning model M based on the authenticated information and the confirmed information.
  • the fraud detection system S of Modification 2-3 includes the confirmed information acquisition unit 306 similar to Modification 1-9.
  • the evaluation method of the learning model M itself is as described in the second embodiment.
  • the evaluation unit 308 uses not only the authenticated information but also the confirmed information to calculate the accuracy rate.
  • the evaluation unit 308 determines that the output obtained by inputting the definite information to the learning model M is an output corresponding to the definite information (for example, the result of whether or not the creator of the learning model M is incorrect). It judges whether or not it shows, and calculates the accuracy rate.
  • any index other than the accuracy rate can be used.
  • Modification 2-4 For example, as in Modification 1-2, when each of the first card C2 and the second card C3 can be used, the output acquisition unit 307, based on the authenticated information corresponding to the first card C2, An output corresponding to the first card C2 may be obtained.
  • the evaluation unit 308 evaluates the accuracy of the learning model M based on the output corresponding to the first card C2.
  • the method itself for evaluating the accuracy of the learning model M based on the output of the learning model M is as described in the second embodiment.
  • the accuracy of the learning model M is evaluated based on the output corresponding to the first card C2.
  • the authenticated information corresponding to the first card C2 which has a very high probability of being valid, it is possible to accurately evaluate the learning model M described in the second embodiment, quickly respond to recent trends, and improve service quality. It is possible to effectively prevent unauthorized use, improve security, and prevent deterioration of convenience.
  • the output acquisition unit 307 identifies the second card C3 based on the authenticated information corresponding to the second card C3. You can get the output.
  • the evaluation unit 308 evaluates the accuracy of the learning model M based on the output corresponding to the first card C2 and the output corresponding to the second card C3.
  • the method itself for evaluating the accuracy of the learning model M based on the output of the learning model M is as described in the second embodiment.
  • the evaluation unit 308 uses not only the output corresponding to the first card C2 but also the output corresponding to the second card C3 to calculate the accuracy rate.
  • the evaluation unit 308 determines whether or not the output obtained by inputting the authenticated information corresponding to the second card C3 to the learning model M indicates legitimacy, and calculates the accuracy rate.
  • any index other than the accuracy rate can be used.
  • the first card when the result of comparison between the first name information about the name of the first card C2 and the second name information about the name of the second card C3 is a predetermined result, the first card By evaluating the accuracy of the learning model M based on the output corresponding to C2 and the output corresponding to the second card C3, the learning model M can be evaluated more accurately using more information. As a result, it is possible to effectively prevent unauthorized use of the service, improve security, and prevent deterioration of convenience.
  • the second card C3 of Modification 2-5 may be a card that does not support possession authentication. Only the second card C3 described in modification 2-5 does not support possession authentication, and the evaluation method itself of evaluation unit 308 is as described in modification 2-5.
  • the fraud detection system S may include a creation unit 302 as in the modification 1-1.
  • the creating unit 302 creates a learning model M for detecting fraud in the service, based on the authenticated information, so that the behavior of the authenticated user is estimated to be legitimate.
  • the fraud detection system S of Modification 2-7 may have the same configuration as Modification 1-1.
  • modification 2-7 the creation of the learning model M described in the first embodiment is simplified, the learning model M is created quickly, unauthorized use of the service is prevented, security is improved, and convenience is reduced. Prevention can be effectively realized.
  • second training data indicating whether the behavior of the unauthenticated user is legitimate or illegal is created, and the learning model M is generated based on the second training data.
  • the learning model M is generated based on the second training data.
  • Modification 2-9 For example, as in modification 1-6, the creation unit 302 acquires the output from the trained learning model M based on the unauthenticated information, and creates the second training data based on the output. good too.
  • the fraud detection system S of Modification 2-9 may have the same configuration as Modification 1-6.
  • the output from the trained learning model M is obtained, and based on the output, by creating the second training data, more information is obtained.
  • the accuracy of the learning model M is further improved by using it.
  • the possession authentication method may be changed according to the degree of fraud.
  • the degree of fraud is information indicating the degree of fraud or information indicating the degree of suspicion of fraud.
  • the degree of fraud may be represented by another index.
  • the degree of fraud may be represented by characters such as S rank, A rank, and B rank.
  • the degree of fraud may be calculated using the learning model M, or the degree of fraud may be calculated using rules.
  • the degree of fraud may be calculated such that the degree of fraud increases as the IP addresses vary.
  • the degree of fraud may be calculated such that the degree of fraud increases as URLs accessed by users vary.
  • the degree of fraud may be calculated such that the farther the access location is from the center of use or the more the access locations vary, the higher the fraud degree.
  • the NFC unit 23A and the photographing unit 26 may be determined depending on the degree of fraudulent use of the user. For example, it may be determined to use the NFC unit 23A when the degree of fraud is equal to or greater than a threshold, and to use the imaging unit 26 when the degree of fraud is less than the threshold. Conversely, it may be determined to use the imaging unit 26 when the degree of fraud is equal to or greater than the threshold, and to use the NFC unit 23A when the degree of fraud is less than the threshold.
  • the degree of fraud is equal to or greater than the threshold, it is determined to use both the NFC unit 23A and the imaging unit 26, and if the degree of fraud is less than the threshold, either the NFC unit 23A or the imaging unit 26 is used. may be determined to utilize. Information identifying which of the NFC unit 23A and the photographing unit 26 has been determined to be used for authentication may be transmitted to the provider server 30, and this information may be confirmed in possession authentication.
  • the authentication information used for authentication may be determined based on the degree of fraud of the user. For example, the authentication information used in authentication is determined so that the higher the degree of fraud, the more authentication information used in authentication. Further, for example, the authentication information used for authentication is determined so that the lower the degree of fraud, the less the authentication information used for authentication. Further, for example, if the degree of fraud is equal to or greater than the threshold, it is determined to use the first authentication information with a relatively large amount of information, and if the degree of fraud is less than the threshold, it is determined to use the second authentication information with a relatively small amount of information. It is determined.
  • the fraud detection system S can be applied to any service other than administrative services and electronic payment services.
  • the fraud detection system S can be applied to other services such as e-commerce services, travel reservation services, communication services, financial services, insurance services, auction services, or SNS.
  • the learning model M is created using the authenticated information of the authenticated user who has performed predetermined authentication such as possession authentication from the user terminal 20. You can do so.
  • the fraud detection system S of the second embodiment when applying the fraud detection system S of the second embodiment to other services, the accuracy of the learning model M is evaluated using the authenticated information of the authenticated user who has performed predetermined authentication such as possession authentication. do it.
  • the card used for possession authentication may be an insurance card, driver's license, membership card, or student ID card.
  • the card used for possession authentication may be an electronic card (virtual card) instead of a physical card.
  • the card used for possession authentication fails, manual determination by an administrator may be performed.
  • the possession authentication corresponding to a certain card number fails a predetermined number of times, the card number may be restricted so that no further possession authentication is performed. In this case, the card may be restricted from being registered in the application unless permitted by the administrator.
  • the possession authentication may be executed by reading the information storage medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Accounting & Taxation (AREA)
  • General Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Computer Hardware Design (AREA)
  • Finance (AREA)
  • Marketing (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
PCT/JP2021/024841 2021-06-30 2021-06-30 学習モデル評価システム、学習モデル評価方法、及びプログラム Ceased WO2023276073A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/JP2021/024841 WO2023276073A1 (ja) 2021-06-30 2021-06-30 学習モデル評価システム、学習モデル評価方法、及びプログラム
US17/911,407 US20240202743A1 (en) 2021-06-30 2021-06-30 Learning model evaluation system, learning model evaluation method, and program
JP2022529394A JP7176158B1 (ja) 2021-06-30 2021-06-30 学習モデル評価システム、学習モデル評価方法、及びプログラム
TW111121017A TWI827086B (zh) 2021-06-30 2022-06-07 學習模型評價系統、學習模型評價方法及程式產品

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/024841 WO2023276073A1 (ja) 2021-06-30 2021-06-30 学習モデル評価システム、学習モデル評価方法、及びプログラム

Publications (1)

Publication Number Publication Date
WO2023276073A1 true WO2023276073A1 (ja) 2023-01-05

Family

ID=84139567

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/024841 Ceased WO2023276073A1 (ja) 2021-06-30 2021-06-30 学習モデル評価システム、学習モデル評価方法、及びプログラム

Country Status (4)

Country Link
US (1) US20240202743A1 (https=)
JP (1) JP7176158B1 (https=)
TW (1) TWI827086B (https=)
WO (1) WO2023276073A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7483103B1 (ja) 2023-06-29 2024-05-14 ＰａｙＰａｙ株式会社 情報処理装置、情報処理方法および情報処理プログラム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014167680A (ja) * 2013-02-28 2014-09-11 Ricoh Co Ltd 画像処理システム、処理制御方法及び画像処理装置
JP2019008369A (ja) * 2017-06-20 2019-01-17 株式会社リコー 情報処理装置、認証システム、認証方法およびプログラム
JP2020115175A (ja) * 2019-01-17 2020-07-30 大日本印刷株式会社 情報処理装置、情報処理方法及びプログラム
WO2021038775A1 (ja) * 2019-08-28 2021-03-04 富士通株式会社 制御方法、制御プログラムおよび空調制御装置

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005258801A (ja) * 2004-03-11 2005-09-22 Matsushita Electric Ind Co Ltd 個人認証システム
US10298614B2 (en) * 2010-11-29 2019-05-21 Biocatch Ltd. System, device, and method of generating and managing behavioral biometric cookies
US20150073987A1 (en) * 2012-04-17 2015-03-12 Zighra Inc. Fraud detection system, method, and device
US20220116390A1 (en) * 2014-10-13 2022-04-14 Vivial Mobile Llc Secure two-way authentication using encoded mobile image
CN107203467A (zh) * 2016-03-18 2017-09-26 阿里巴巴集团控股有限公司 一种分布式环境下监督学习算法的基准测试方法和装置
JP6954082B2 (ja) * 2017-12-15 2021-10-27 富士通株式会社 学習プログラム、予測プログラム、学習方法、予測方法、学習装置および予測装置
US11789699B2 (en) * 2018-03-07 2023-10-17 Private Identity Llc Systems and methods for private authentication with helper networks
US11151232B2 (en) * 2019-01-28 2021-10-19 EMC IP Holding Company LLC User authentication by endpoint device using local policy engine and endpoint data
JP7419035B2 (ja) * 2019-11-22 2024-01-22 キヤノン株式会社 学習モデル管理システム、学習モデル管理方法、およびプログラム
US11290464B2 (en) * 2019-12-18 2022-03-29 Voya Services Company Systems and methods for adaptive step-up authentication
JP6933780B1 (ja) * 2019-12-26 2021-09-08 楽天グループ株式会社 不正検知システム、不正検知方法、及びプログラム
US12079812B2 (en) * 2020-06-09 2024-09-03 Capital One Services, Llc Utilizing machine learning and trusted transaction card locations to generate a geographical map of the trusted transaction cards

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014167680A (ja) * 2013-02-28 2014-09-11 Ricoh Co Ltd 画像処理システム、処理制御方法及び画像処理装置
JP2019008369A (ja) * 2017-06-20 2019-01-17 株式会社リコー 情報処理装置、認証システム、認証方法およびプログラム
JP2020115175A (ja) * 2019-01-17 2020-07-30 大日本印刷株式会社 情報処理装置、情報処理方法及びプログラム
WO2021038775A1 (ja) * 2019-08-28 2021-03-04 富士通株式会社 制御方法、制御プログラムおよび空調制御装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ISHIKAWA, TATSUO: "Realization of multi-factor authentication using my number card", MONTHLY AUTOMATIC RECOGNITION, vol. 32, no. 14, 10 December 2019 (2019-12-10), pages 24 - 26, XP009542378 *

Also Published As

Publication number Publication date
JPWO2023276073A1 (https=) 2023-01-05
JP7176158B1 (ja) 2022-11-21
TW202307758A (zh) 2023-02-16
TWI827086B (zh) 2023-12-21
US20240202743A1 (en) 2024-06-20

Similar Documents

Publication Publication Date Title
US11075942B2 (en) Identity verification and account information updating methods and apparatuses
JP7177303B1 (ja) サービス提供システム、サービス提供方法、及びプログラム
US20210224795A1 (en) Escrow non-face-to-face cryptocurrency transaction device and method using phone number
JP7230120B2 (ja) サービス提供システム、サービス提供方法、及びプログラム
TWI793885B (zh) 認證系統、認證方法、及程式產品
US20210264299A1 (en) Fraud estimation system, fraud estimation method and program
JP7195473B1 (ja) サービス提供装置、サービス提供方法、およびプログラム
JP7165840B1 (ja) 不正検知システム、不正検知方法、及びプログラム
JP7271778B2 (ja) サービス提供システム、サービス提供方法、及びプログラム
US11947643B2 (en) Fraud detection system, fraud detection method, and program
JP7176158B1 (ja) 学習モデル評価システム、学習モデル評価方法、及びプログラム
JP7176157B1 (ja) 学習モデル作成システム、学習モデル作成方法、及びプログラム
JP7238214B1 (ja) 不正検知システム、不正検知方法、及びプログラム
JP7165841B1 (ja) 不正検知システム、不正検知方法、及びプログラム
TWI807829B (zh) 認證系統、認證方法及程式產品
JP7104133B2 (ja) カード登録システム、カード登録方法、及びプログラム

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2022529394

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 17911407

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21948378

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21948378

Country of ref document: EP

Kind code of ref document: A1