RU2820329C2 - Method and system for detecting fraudulent calls and alerting subscribers thereabout - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to communication, namely, detects fraudulent calls in online mode during conversation of subscribers and notifies subscribers about them. Technical result is achieved due to continuous additional training of the model based on previously selected patterns in the form of embeddings using a plurality of embedders.

EFFECT: high accuracy of determining the fact of suspected fraud and confidentiality of data during a conversation between subscribers.

15 cl, 5 dwg

Description

TECHNICAL FIELD

[1] This technical solution, in general, relates to the field of communications, namely, it detects fraudulent calls online during a call between subscribers and alerts subscribers about them.

BACKGROUND OF THE ART

[2] Currently, the issue of protecting against fraudulent calls and timely informing subscribers about them is very urgent.

[3] “Statistics for the first half of 2021 showed us that more than 70% of calls from unknown numbers come from scammers. Every month they withdraw from the accounts of citizens who have trusted them from 3.5 to 5 billion rubles. The threat is not only the persistence of attackers - scammers use social engineering methods and regularly invent new pressure schemes. A static solution will not be suitable to counter them,” (c) (Dmitry Teplitsky, General Director of ABK).

[4] Known from the prior art is the “Method for recognizing and processing a spam call (RU 2765483, Limited Liability Company “INTERCONNECT” (RU), published: 01/31/2022), which consists in the fact that the cellular operator receives at the switching node message to establish the connection of an incoming voice call from the calling party to the called party, routes the connection of the incoming voice call to the network node of the intelligent periphery, characterized in that before routing the connection of the incoming voice call to the node of the intelligent periphery, the cellular operator checks the fact of establishment using the identifiers of the calling and called parties connection of the calling and called parties for period T, if the fact of connection of the calling and called parties for period T is not confirmed, the cellular operator connects the incoming call with the program logic of computer analysis and synthesis of texts in natural languages of the network node of the intelligent periphery, which has a voice audio interface, and the program logic of the computer analysis and synthesis of texts in natural languages of the intelligent periphery node, which has a voice audio interface, generates statements in affirmative, interrogative and incentive forms and searches for patterns in the caller’s statements to recognize a voice bot or person in the caller, as well as advertising or fraudulent content.

[5] Known from the prior art is the “Method for recognizing and processing a spam call (RU 2762389, Limited Liability Company “Algorithm” (RU), published: 12/20/2021), which consists in the fact that, in cellular networks of GSM standards, IN, IMS ensures the formation of the maximum possible array of data for applying analysis rules, a necessary and sufficient list of signs for recognizing a subscriber making unwanted calls, as well as the selection of subscribers making unwanted calls based on their behavior - for advertising or fraudulent purposes. This provides a cost-effective way for the cellular operator to handle an unwanted call by keeping the unwanted call on the connection for as long as possible without the callee being involved.

[6] The general disadvantages of existing solutions are the lack of analysis of a specific online conversation between subscribers for the presence of suspected fraud and the corresponding notification without reference to telephony data and the participation of the operator.

DISCLOSURE OF INVENTION

[7] This technical solution is aimed at eliminating the disadvantages inherent in existing solutions known from the prior art.

[8] The technical problem solved in this technical solution is the quick and accurate determination of the fact of alleged fraud during a call during a conversation between subscribers and notification of such a fact.

[9] The main technical result that appears when solving the above problem is to increase the accuracy and speed of determining the fact of alleged fraud during a conversation between subscribers.

[10] An additional technical result that appears when solving the above problem is ensuring the subscriber’s security during calls through timely notification of the detected fact of alleged fraud.

[11] The specified technical results are achieved by implementing a method for identifying fraudulent calls and notifying subscribers about them, which includes the following steps:

• during a conversation, audio data is decoded;

• then the audio data goes through the speech transcription procedure;

• the text obtained in the previous step is analyzed for the presence of suspected fraud using a pre-trained machine learning model;

• if suspected fraud is detected at the previous step, the subscriber is notified about this.

[12] In one of the particular examples of the method, the subscriber is notified during a conversation in which the fact of alleged fraud is revealed.

[13] In another particular example of the method, the subscriber is notified after a conversation in which the fact of alleged fraud is revealed.

[14] In another particular example of the method, the subscriber is notified by playing an audio message.

[15] In another particular example of the method, the subscriber is notified using a video message.

[16] In another particular example of the method, the subscriber is notified using a text message.

[17] In another particular example of the method, the subscriber is notified using an icon.

[18] In one of the particular examples of the method, audio data with human speech is decoded.

[19] In another particular example of the method, audio data with synthesized speech is decoded.

[20] In one of the particular examples of implementation of the method, analogue telephony services are used as telephony services.

[21] In another particular example of the method, IP telephony services are used as telephony services.

[22] In another particular example of the implementation of the method, software services for audio and video calls are used as telephony services.

[23] In one of the particular examples of the implementation of the method, in addition to analyzing the fact of alleged fraud, an analysis is carried out for the incorrectness of the dialogue and an appropriate notification is generated.

[24] In another particular example of the method, if suspected fraud is detected, the connection is disconnected.

[25] In one of the particular examples of the method, the machine learning model used is additionally trained and improved with each use for analysis.

[26] In addition, the stated technical result is achieved through a system for identifying fraudulent calls and alerting subscribers about them, containing:

- at least one data processing device;

- at least one data storage device;

- at least one program, where one or more programs are stored on one or more data storage devices and executed on one or more data processing devices, and one or more programs performs the following steps:

• during a conversation, audio data is decoded;

• then the audio data goes through the speech transcription procedure;

• the text obtained in the previous step is analyzed for the presence of suspected fraud using a pre-trained machine learning model;

• if suspected fraud is detected at the previous step, the subscriber is notified about this.

BRIEF DESCRIPTION OF THE DRAWINGS

[27] The features and advantages of the present technical solution will become apparent from the following detailed description and the accompanying drawings, in which:

[28] fig. 1 illustrates a block diagram of the claimed method;

[29] fig. 2 illustrates the operation diagram of a particular embodiment of the described technical solution;

[30] fig. 3 illustrates an audio data processing circuit;

[31] FIG. 4 illustrates the operation of a speech recognition system in real time under load;

[32] fig. 5 illustrates a system for implementing the claimed method.

IMPLEMENTATION OF THE INVENTION

[33] The terms and concepts necessary for the implementation of this technical solution will be described below.

[34] Transcriptions are the decoding of information from audio or video into text form. Initially, they separate manual transcription with human participation and automatic transcription using special services for automatic translation of audio data into text.

[35] The RTP protocol (Real-time Transport Protocol) operates at the application level (OSI - 7) and is used to transmit real-time traffic. The protocol was developed by the Audio-Video Transport Working Group at the IETF and first published in 1996 as RFC 1889 (RFC 1889 has been deprecated since RFC 3550 in 2003). The RTP protocol carries in its header the data necessary to restore audio data or video images at the receiving node, as well as data about the type of information encoding (JPEG, MPEG, etc.). In particular, the header of this protocol contains the timestamp and packet number. These parameters make it possible, with minimal delays, to determine the order and moment of decoding of each packet, as well as to interpolate lost packets. RTP does not have a standard reserved port number. The only limitation is that the connection is made using an even number, and the next odd number is used for RTCP communication. The fact that RTP uses dynamically assigned port addresses makes it difficult for it to traverse firewalls; a STUN server is typically used to get around this problem. Connection establishment and termination are not included in the list of RTP capabilities; such actions are performed by a signaling protocol (for example, RTSP or SIP protocol).

[36] Telephone fraud (occasionally fraud, from the English fraud “fraud”) is a type of fraud in the field of information technology, in particular, unauthorized actions and unlawful use of resources and services, theft of someone else’s property or the acquisition of rights to someone else’s property by entering, deleting , modification of information or other interference with the operation of data processing or transmission facilities of information and telecommunication networks. Vishing (from Voice phishing) should be highlighted - one of the methods of fraud using social engineering, which consists of attackers using telephone communication and playing a certain role (bank employee, buyer, etc.), under various pretexts lure confidential information from the payment card holder or encourage them to perform certain actions with their card account/payment card.

There are the following types:

• direct money swindling, when scammers call on behalf of a relative and ask for money;

• blackmail, when scammers call on behalf of a law enforcement officer;

• banking fraud, when a call is received on a mobile phone from scammers posing as a bank or security officer;

• Fraudsters can also use calls to force you to install a fraudulent application;

• obtaining personal data that allows an unwanted operation to be carried out, leading to financial or other losses.

[37] The claimed technical solution can be implemented, for example, by a system, a machine-readable medium, a server, etc. In this technical solution, a system means, including a computer system, a computer (electronic computer), CNC (computer numerical control), PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given, clearly defined sequence of operations (actions, instructions).

[38] A command processing device means an electronic unit or an integrated circuit (microprocessor) that executes machine instructions (programs).

[39] An instruction processing device reads and executes machine instructions (programs) from one or more data storage devices, such as devices such as random access memory (RAM) and/or read only memory (ROM). ROM can be, but is not limited to, hard drives (HDD), flash memory, solid-state drives (SSD), optical storage media (CD, DVD, BD, MD, etc.), etc.

[40] Program - a sequence of instructions intended for execution by a computer control device or a command processing device.

[41] The term "instructions" as used in this application may refer generally to software instructions or software commands that are written in a given programming language to perform a specific function, such as, for example, retrieving and processing data, generating a user profile , reception and transmission of signals, analysis of received data, user identification, etc. Instructions can be implemented in a variety of ways, including, for example, object-oriented methods. For example, instructions can be implemented using the programming language C++, Java, Python, various libraries (for example, "MFC"; Microsoft Foundation Classes), etc. Instructions that carry out the processes described in this solution can be transmitted over both wired and wireless data channels, for example, Wi-Fi, Bluetooth, USB, WLAN, LAN, etc.

[42] The presented method for identifying fraudulent calls and notifying subscribers about them (FIG. 1 shows a diagram of the method) solves the problem of quickly and accurately determining the fact of alleged fraud in a call during a conversation between subscribers and notifying about this fact by sequentially performing the following steps:

• during a conversation, audio data is decoded (101 in the method diagram);

• then the audio data goes through the speech transcription procedure (102 in the method diagram);

• the text obtained in the previous step is analyzed for the presence of alleged fraud using a pre-trained machine learning model (103 in the method diagram);

• if the fact of alleged fraud is detected at the previous step, the subscriber is notified about this (104 on the method diagram).

[43] In a particular embodiment of the above-described technical solution (the interaction diagram is shown in FIG. 2), the caller (initiator - 201 in the diagram) talks to the subscriber (abonent - 203 in the diagram). RTP traffic passes through the media server (media-proxy - 204 in the diagram). RTP traffic, depending on the codec used, is converted into wav pern using the recording service (207 in the diagram) and sent to a specialized dispatch service (206 in the diagram). Thus, the only data about the call is RTP (media) traffic. This eliminates the concept of phone calls as the solution analyzes audio data. The audio is sent in real time to the speech recognition service (205 in the diagram).

After receiving the transcription, the text is sent to the service for detecting fraudulent calls (208 in the diagram). If the service has detected fraudulent intentions, the dispatch service (206 in the diagram) receives a corresponding signal. Next, the dispatch service (206 in the diagram) sends a command to the media server (204 in the diagram) to play a warning record about possible fraud. It should be noted that fraud detection occurs during a conversation, and the notification is played as follows: the media stream in the channel in which fraud is detected is stopped for a while, a warning is played (one option could be for the voice assistant to work in the telephone channel), the media stream is resumed.

[44] In other implementations of the described technical solution, the subscriber notification can be implemented in the form of a video message, via a text message or using an icon on the screen of a user device (for example, PUSH notifications).

[45] Processing of audio data on the telephony side in the above embodiment of the described technical solution occurs as follows (the diagram is shown in FIG. 3): the Initiator subscriber (301 in the diagram) makes a call to the Abonent subscriber (303 in the diagram): an RTP connection is established between them, which passes through the media server (304 in the diagram). Recording of audio traffic is carried out by a recording service. It receives an RTP stream and, depending on the codec, converts it to PCM WAV. Standard codecs are supported: g711, g729, alaw, ulaw, amr-wb, amr-nb and others. The recording service, using TCP/SSL, sends audio to the dispatch service (Dispatcher - 305 in the diagram).

[46] Telephony may not be the only source of audio. Since in the above-described particular implementation the audio stream is processed without SIP events, it is possible to use this technology using:

stereo recordings of a conversation between two people (when diarization is used, it is possible to use a mono recording of a conversation);

online processing of a conversation with two participants in any online communication systems: Skype, Telegram, Discord and other software services for audio and video calls.

[47] The dispatch service in the above implementation of the described technical solution interacts between the system components:

• as media data enters the dispatch service, it is sent to the speech recognition service;

• the result of the speech recognition service (conversation transcription) is sent in real time to the fraud detection service FraudDetectionService;

• when fraud is detected, the dispatch service sends a command to the media proxy to play a pre-recorded signal file.

The components interact through the TCP network protocol. Messages are packed using protobuf technology.

[48] In other implementations of the described technical solution, the subscriber is notified after a conversation in which the fact of alleged fraud is revealed; in one of the particular examples of the implementation of the method, in addition to analyzing the fact of alleged fraud, an analysis is carried out for the incorrectness of the dialogue and an appropriate notification is generated; in another particular example of the method, if suspected fraud is detected, the connection is disconnected.

[49] Speech recognition (transcription) in the above embodiment of the described technical solution is performed using the ASR model: ASR is based on the kaldi framework, with its help Mel-Frequency Cepstral Coefficients (MFCCs) are extracted from telephone speech datasets for further training of the acoustic part of the model (based on hidden Markov models (HMM), Gaussian mixture model (GMM)) and a neural network). Afterwards, training continues on text datasets in the linguistic part, which is an n-gram model, and the RNNLM (Recurrent Neural Network Language Modeling Toolkit) neural network is used as rescoring. The speech recognition service interacts with the dispatch service using a network connection via tcp. As the dispatch service sends audio packets, they are collected into a single buffer. Once enough audio has appeared in the service, a spectrogram is generated. The spectrogram is fed to the input of the speech recognition model. The result of the model is a path in the graph of this model. It reconstructs the text spoken by the subscriber. In FIG. Figure 4 clearly shows how the system can work in real time under load and recognize speech in real time: the graph shows the x-axis - the number of simultaneous connections to the speech recognition service (i.e. the number of simultaneously recognized channels, the y-axis - recognition delay, in milliseconds. The average and median latency are dark blue and orange lines. Above are the percentile lines: the 90th percentile indicates that 90 percent of all measurements were less than a certain mark, respectively, 85% had a delay less than 120 ms, 90% less. 150 ms, 95% have less than 200 ms, meanwhile - the 95th percentile of phrase recognition time is less than 200 ms - the mark at which a person feels a delay. That is, on average, the service recognizes text faster than 100 ms, and 95% of all phrases. are recognized faster than 200 ms. Thus, the performance of the speech recognition system allows you to transcribe text online with minimal delay. It is possible to transcribe audio data from both human and synthesized speech. After receiving the transcription, the text is returned to the dispatch service.

[50] In the above implementation of the described technical solution, the fraud detection service (FraudDetectionService) receives a transcription of the conversation and, using machine learning models, determines whether the conversation is fraudulent.

[51] In the above implementation of the described technical solution, the fraud detection model is pre-trained on the following data:

731 dialogues on fraudulent scenarios (source Youtube);

134 simulations of fraudulent dialogues;

10,368 conversations about fraudulent activities (without fraud);

21 long thematic non-fraud dialogues;

40,000 contact center calls (no fraud);

sets of conversational data from social networks and open chats (vk, telegram).

[52] In the above embodiment of the described technical solution, the fraud detection model has the following operating logic:

online conversation-to-text (ASR) conversion;

converting text into embeddings (obtaining a numerical representation);

(at the stage of training/tuning the model) use of a series of embedders:

BertMulLangAllSidesChained: (OpenAI BERT) sequence of replicas (raw dialogue as-is, completely);

SbertAllSidesChained: (BERT from Sber) sequence of replicas (raw dialogue as-is, completely);

SbertAHSidesSep: (BERT from Sber) sequence of replicas (broken on the sides by subscriber A, subscriber B);

SbertByReplica (BERT from Sber) sequence of replicas in the order received;

BoW; fastText;

Universal Sentence Encoder;

Word2Vec;

T5;

(at the model training/tuning stage) text data and corresponding embeddings are cached to optimize execution speed using a unique call identifier;

due to the asymmetry of the data set and the goal of minimizing the number of False Positive errors and maximizing True Positive, a variant of assessing * the quality of the Weighted AUC model (in particular, Exponential AUC) was used;

Class separation (fraudulent call, legitimate call) is carried out using logistic regression with the parameters:

L-BFGS optimization algorithm;

L2-regularization;

the selection of the threshold was carried out according to the rule of minimizing the value of the linear combination of the prices of type I/II errors and the corresponding probabilities (the resulting threshold value for the model was 0.8);

if fraud attempts are detected, a signal is sent to the dispatch service.

[53] The accuracy of the above model as of January 2022 was:

- False Positive Rate: 0.002 (less than 1% of false positives of the model);

- True Positive Rate: 0.81 (that is, more than 80% of correct detection of fraud).

[54] In one of the particular examples of the implementation of the described technical solution, the machine learning model used is additionally trained and improved with each use of it for analysis, including using feedback from subscribers.

[55] In general (shown in Fig. 5), the system for identifying fraudulent calls and alerting subscribers about them (500) contains one or more processors (501), memory devices, such as RAM (502) and ROM, united by a common information exchange bus. (503) and input/output interfaces (504).

[56] The processor (501) (or multiple processors, multi-core processor, etc.) may be selected from a variety of devices commonly used today, for example, from manufacturers such as: Intel™, AMD™, Apple™, Samsung Exynos ™, MediaTEK™, Qualcomm Snapdragon™, etc. The processor or one of the usable processors in the system (500) must also include a graphics processor, for example an NVIDIA GPU with a CUDA-compatible programming model or Graphcore, the type of which is also suitable for carrying out the method in whole or in part, and can also be used for training and application of machine learning models in various information systems.

[57] RAM (502) is a random access memory and is designed to store machine-readable instructions executable by the processor (501) to perform the necessary logical data processing operations. RAM (502) typically contains executable operating system instructions and related software components (applications, program modules, etc.). In this case, the available memory capacity of the graphics card or graphics processor can act as RAM (502).

[58] A ROM (503) is one or more permanent storage devices, such as a hard disk drive (HDD), a solid state drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media ( CD-R/RW, DVD-R/RW, BlueRay Disc, MD), etc.

[59] To organize the operation of device components (500) and organize the operation of external connected devices, various types of I/O interfaces (504) are used. The choice of appropriate interfaces depends on the specific design of the computing device, which can be, but is not limited to: PCI, AGP, PS/2, IrDa, Fire Wire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro , mini, type C), TRS/Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[60] To ensure user interaction with the device (500), various means (505) of I/O information are used, for example, a keyboard, a display (monitor), a touch display, a touchpad, a joystick, a mouse, a light pen, a stylus, a touchpad, trackball, speakers, microphone, augmented reality tools, optical sensors, tablet, light indicators, projector, camera, biometric identification tools (retina scanner, fingerprint scanner, voice recognition module), etc.

[61] The networking facility (506) enables data transmission via an internal or external computer network, such as an Intranet, the Internet, a LAN, or the like. One or more means (506) may be used, but not limited to: Ethernet card, GSM modem, GPRS modem, LTE modem, 5G modem, satellite communication module, NFC module, Bluetooth and/or BLE module, Wi-Fi module and etc.

[62] The specific selection of device elements (500) to implement various software and hardware architectural solutions may vary while maintaining the required functionality provided. In particular, such an implementation can be performed using electronic components used to create digital integrated circuits. Not limited to, microcircuits can be used, the logic of which is determined during manufacture, or programmable logic integrated circuits (FPGAs), the logic of which is specified by programming. For programming, programmers and debugging environments are used to set the desired structure of a digital device in the form of a circuit diagram or program in special hardware description languages: Verilog, VHDL, AHDL, etc. An alternative to FPGAs are: programmable logic controllers (PLCs), basic matrix crystals ( BMK) requiring a factory production process for programming; ASICs are specialized custom large-scale integrated circuits (LSIs), which are significantly more expensive for small-scale and single-piece production. Thus, implementation can be achieved by standard means based on the classical principles of implementing the fundamentals of computer technology.

[63] The submitted application materials disclose preferred examples of implementation of a technical solution and should not be interpreted as limiting other, particular examples of its implementation that do not go beyond the scope of the requested legal protection, which are obvious to specialists in the relevant field of technology.

Claims (22)

1. A method for identifying fraudulent calls and notifying subscribers about them, characterized by the fact that: • during a conversation, audio data is decoded; • then the audio data goes through the speech transcription procedure; • patterns are highlighted in the form of embeddings using a variety of embedders; • the text obtained in the previous step is analyzed for the presence of the fact of alleged fraud using a pre-trained machine learning model, while the model is trained on the basis of previously identified patterns in the form of embeddings using a variety of embedders, while the machine learning model used is additionally trained and improved with each use of it for analysis; • if suspected fraud is detected at the previous step, the subscriber is notified about this. 2. The method according to claim 1, characterized in that the subscriber is notified during a conversation in which the fact of alleged fraud is revealed. 3. The method according to claim 1, characterized in that the subscriber is notified after a conversation in which the fact of alleged fraud is revealed. 4. The method according to claim 1, characterized in that the subscriber is notified by playing an audio message. 5. The method according to claim 1, characterized in that the subscriber is notified using a video message. 6. The method according to claim 1, characterized in that the subscriber is notified using a text message. 7. The method according to claim 1, characterized in that the subscriber is notified using an icon. 8. The method according to claim 1, characterized in that audio data with human speech is decoded. 9. The method according to claim 1, characterized in that audio data with synthesized speech is decoded. 10. The method according to claim 1, characterized in that analogue telephony services are used as telephony services. 11. The method according to claim 1, characterized in that IP telephony services are used as telephony services. 12. The method according to claim 1, characterized in that software services for audio and video calls are used as telephony services. 13. The method according to claim 1, characterized by the fact that in addition to analyzing the fact of alleged fraud, an analysis is carried out for the incorrectness of the dialogue and an appropriate notification is generated. 14. The method according to claim 1, characterized in that if suspected fraud is detected, the connection is disconnected. 15. A system for identifying fraudulent calls and alerting subscribers about them, containing: • at least one processor; • at least one memory coupled to the processor, which contains machine-readable instructions that, when executed by at least one processor, enable execution of the method according to any one of claims. 1-14.