EP2603905B1 - Method and device for detecting and verifying attempts to manipulate a self-service terminal - Google Patents

Description

The invention relates to a method for recognizing and verifying manipulation attempts on a self-service terminal according to the preamble of claim 1. In addition, the invention relates to a device operating according to the method, in particular a data processing and control unit, and a self-service terminal equipped therewith, in particular one designed as an ATM Self-service terminal.

At self-service terminals, hereinafter also referred to as self-service terminals for short, in particular at ATMs, criminal acts in the form of manipulations are often carried out with the aim of removing sensitive data, in particular PINs (Personal Identification Numbers) and / or card numbers, from self-service users - Spying on terminals. In particular, attempts at manipulation are known in which so-called skimming devices, such as keyboard superstructures and the like, are illegally installed in the control area or control panel. Such keyboard superstructures often have their own power supply, as well as a processor, a memory and an operating program, so that an unsuspecting user is spied on when entering his PIN or when inserting his bank card. The spied out data will then be over A transmitter integrated in the keyboard superstructure is transmitted to a remote receiver or is stored in a data memory located in the keyboard superstructure. Many of the skimming devices found today can only be distinguished with the human eye from original operating elements (keyboard, card reader, etc.) with great difficulty.

In order to thwart such attempts at manipulation, monitoring systems are often used which have one or more cameras which are mounted in the area of the location of the self-service terminal and which cover the entire control panel and often also the area where the user is located.

From the GB 2 351 585 A a method and a device for recognizing and verifying manipulation attempts on a self-service terminal are known, which has at least one control element ("keypad 21") provided for users of the self-service terminal, to which at least one camera ("30") is aligned Image data generated by the camera are fed to a first classifier ("micro-processor 32"), by means of which the image data is used to check whether an attempt to manipulate the control element can be recognized, the image data being based on first characteristics ("Differences in the two images") ) be evaluated. Furthermore, the image data are also fed to a second classifier ("micro-processor ... programmed to analyze 'traffic patterns"), by means of which the image data are used to check whether the detection of a manipulation attempt is plausible ("to determine if unusual activity is occuring "), where the Image data are evaluated on the basis of second characteristics ("traffic patterns").

From the JP 2007 280317 A a system for detecting suspicious objects in the operating area of an ATM is known. For this purpose, a surveillance camera records images from the control panel (front panel) of the ATM between the end of a transaction and the beginning of the next transaction, in order to use image comparison to identify whether suspicious objects have been attached or deposited. If so, the image acquisition will stop during the next transaction.

the US 2009/2013 72 A1 discloses a method and apparatus for integrated monitoring of an ATM or the like. Several cameras are installed in order to take pictures, in particular video recordings (video data), of the user, the card slot of the ATM and of the surrounding area. A time stamp and other relevant data can be linked to the video data and saved.

the WO 2005/109 315 A1 discloses a security system for monitoring an automated teller machine (ATM) including a camera that provides images of selected portions of the ATM. A controller automatically determines the difference between a reference image of the ATM and a subsequently captured image in order to recognize a change in the ATM.

Another solution is, for example, in the DE 201 02 477 U1 described. By means of the local camera surveillance Both the control panel itself and the area in front of the user can be recorded. A sensor is also provided to distinguish whether a person is in the occupied zone.

From the US 2009/0057395 A1 a method and a device for recognizing and verifying manipulation attempts at an ATM are known. There will be different. Sensors are used (see text section [0063]), such as cameras or proximity sensors, which emit signals or data to a microprocessor-controlled unit (“microprocessor 32”). There, data processing is then carried out on the basis of a statistical model which has a classifier (see text sections [0081] - [0085]) in order to infer a normal operating state or an abnormal operating state of the ATM, the latter could indicate an attempt at tampering. Accordingly, it is proposed there to recognize a manipulation attempt on the basis of a classifier which is fed with data from various sensors. The reliability of this principle, however, requires the use of several different sensors and the effort required to match the classifier to the various sensors.

There are basically devices and methods for recognizing manipulation attempts on a self-service terminal known, wherein a camera is directed to at least one control element, such as keyboard, cash dispenser, etc., and the image data generated by the camera are evaluated by means of a classifier to a Tampering attempt to be recognized. However, the known solutions require a high level of hardware and Software expenditure in order to achieve the most secure and error-free detection of manipulation.

A further, improved and inexpensive to implement method for recognizing manipulation attempts on a self-service terminal and a device operating on it are described in the earlier patent application filed by the applicant with the application number DE 2010055016 suggested. DE 201 02 477 U1 discloses a method according to the preamble of claim 1.

On the basis of the above statements, it is the object of the present invention to propose a reliable, yet cost-effective solution which overcomes the disadvantages of the known methods and devices. In particular, camera-supported detection of manipulation attempts with high reliability but low hardware expenditure and limited software expenditure should be made possible.

The object is achieved by a method with the features of claim 1 and by a device operating according to it and by a self-service terminal equipped with it.

Accordingly, it is proposed that the image data generated by the camera be fed to a first classifier, by means of which the image data is used to check whether an attempt to manipulate the control element can be recognized, and that the image data are also fed to a second classifier in parallel with the first classifier, by means of which the image data are used to check whether the detection of a manipulation attempt is plausible. On the image data fed to the first and second classifiers an edge detection is applied by creating at least one edge image and comparing its characteristic data with the pattern data of a reference edge image.

Thus, a parallel structure of two or more classifiers is proposed, which are applied to the same image data or to data derived therefrom, the first classifier being set up to detect an attempted manipulation, but the second classifier being set up based on the image data received from the / to check the situation recorded by the camera (s) for plausibility of a manipulation event. An alarm is only triggered when both classifiers show a positive result, i.e. affirm a manipulation attempt and its plausibility.

The invention is based on the knowledge that the conventional devices often work incorrectly and also display skimming when there is no attempt to manipulate the self-service terminal. The applicant has observed that users on the control panel or the control console of self-service terminals, such as Leave ATMs, personal items, especially purses, wallets, notes, etc. lying around and this can then lead to a false skimming alarm.

According to the invention, by using a further classifier that works in parallel, which checks the plausibility of the image data, it is ensured that an alarm is only generated if a real attempt at manipulation must reasonably be assumed based on the situation recorded by the camera. For example, in the case of a wallet left lying on the keyboard, the first classifier would recognize an abnormal situation and want to indicate an attempt at manipulation, but the second classifier would rather consider the wallet, in particular its contour and / or position, to be atypical for a keyboard superstructure (manipulation - or skimming superstructure) and thus prevent the triggering of a false alarm.

The image data are preferably processed independently of one another in the classifiers. The classifiers thus arrive at their results independently of one another, the first result (output value of the first classifier) being verified or not by the second result (output value of the second classifier). In this way, meaningful and non-meaningful events (manipulation attempts) can be reliably separated. According to the invention, a structure of two or more classifiers working in parallel is proposed for this purpose.

According to the invention, it is provided that the first classifier evaluates the image data on the basis of first features in order to obtain a first output value which indicates the probability of a change in the visual appearance of the Control element displays. These first features relate to the edge lengths of a keyboard whose photo is subjected to edge image detection.

The second classifier, on the other hand, evaluates the image data on the basis of second features in order to obtain a second output value which indicates the probability of the presence of a change in the visual appearance of the operating element that is typical for manipulation. These second features relate to the relative position or position of the edges to one another. In the case of a keyboard and a skimming superstructure, most of the edges are parallel or at right angles to one another; in the case of an object that is left lying around, such as a wallet, there are also edges that are neither parallel nor perpendicular to the other edges (the keyboard). A plausibility test therefore leads to the result that there is very likely no attempt at manipulation. The "ambient lighting" feature also leads to very reliable results.

The output values of the classifiers are preferably between 0 and 1. In order to obtain unambiguous yes / no statements, the first output value is preferably compared with a first threshold value or the second output value with a second threshold value, which then results in a first or second binary value can be obtained, which in turn can be processed logically. The binary values can, for example, be fed to an AND link in order to obtain a reliable statement as to whether the self-service terminal has been manipulated or not. If an attempt at manipulation is detected, the self-service terminal can also be provided in addition or as an alternative to triggering an alarm to block and / or trigger an additional camera (portrait camera) to take photos of suspicious people who may have carried out the manipulation. Provision can also be made for the manipulation detection or the camera (s) to be deactivated during maintenance of the self-service terminal in order to avoid false alarms.

These and other particularly advantageous refinements also emerge from the subclaims.

An alarm is preferably only triggered if an attempt to manipulate the operating element is recognized by means of the first classifier and if the recognition of the manipulation attempt is assessed as plausible by means of the second classifier.

Otherwise, a display, in particular notification, is generated for an operator and / or user of the self-service terminal, namely when an attempt to manipulate the operating element is recognized by means of the first classifier and when the manipulation attempt is recognized as implausible by means of the second classifier Is evaluated. The notification for the operator and / or user of the self-service terminal is preferably sent via a communication service, in particular email or SMS.

A device is also proposed for recognizing and verifying manipulation attempts on a self-service terminal, which has at least one operating element for users, to which at least one camera is aligned, the device being connected to the at least one camera and generated by the camera Receives image data, and wherein the device has a data processing unit with a first classifier, which uses the image data to check whether an attempt to manipulate the operating element can be recognized, the data processing unit of the device having a second classifier that processes the image data in parallel with the first classifier and checks whether the detection of a manipulation attempt is plausible.

In addition, a self-service terminal is proposed that has such a device. The device can preferably be implemented by means of a computer or PC present in the self-service terminal.

In a preferred application, the self-service terminal is designed as an ATM. The at least one control element represents an element suitable for manipulation, in particular a keyboard or a PIN pad, a cash dispenser and / or a card input funnel have edges that are delimited from homogeneous surfaces. As a result, the image data generated by the camera (s) can be processed by means of an edge detection, for example by creating at least one edge image and comparing its characteristic data with the sample data of a reference edge image. The use of edge detection not only results in a significant reduction in data, but also increases the speed and reliability of the image evaluation. The use of several parallel classifiers increases the reliability of the end result in particular.

The invention makes it possible, in particular, to significantly improve the recognition of overbuilt individual or multiple operating elements. This is particularly true with regard to the reliability of the generation of skimming alarms. The elements that are particularly suitable for manipulation and / or the elements that are arranged in areas of the control panel that are particularly suitable for manipulation are preferably recorded by the camera, such as, for example, cash dispenser, keyboard, card input funnel and / or screen. The elements are therefore preferably operating elements in the narrower sense, but can also be other elements, such as a storage area in the operating area or the like. In addition, the fact that objects are left lying around is reliably detected. It is also proposed here to notify the user and / or the operator of the self-service terminal if a foreign object was detected on the basis of the image recording on the control element (e.g. keyboard) that does not have the typical properties of a manipulation device (keyboard superstructure, dummy, etc.) and is therefore very likely an object that the last user of the self-service terminal left there. Accordingly, an automatic notification service can also be implemented, which in particular notifies users or customers that personal items have been left at the self-service terminal.

Fig. 1

zeigt in Form eines Blockschaltbildes den Aufbau eines erfindungsgemäßen SB-Terminals, das als Geldautomat ausgestaltet ist;

Fig. 2

zeigt ein schematisches Ablaufdiagramm für ein erfindungsgemäßes Verfahren;

Fig. 3

eine Entscheidungs-Matrix zur Veranschaulichung der Ergebnisbildung aus den Ausgabewerten der parallel arbeitenden Klassifikatoren.

The invention and the advantages resulting therefrom are described below on the basis of exemplary embodiments and with reference to the accompanying schematic drawings, which show the following:

Fig. 1

shows in the form of a block diagram the structure of a self-service terminal according to the invention, which is designed as an ATM;

Fig. 2

shows a schematic flow diagram for a method according to the invention;

Fig. 3

a decision matrix to illustrate the formation of results from the output values of the classifiers working in parallel.

That in the Fig. 1 Self-service terminal shown is designed as a cash machine ATM and has a control panel with several elements or control elements, of which a keyboard KBD, a cash dispensing slot SHT and a card input funnel CSL are shown in the form of individual function blocks. Each operating element is monitored by a camera CAM, CAM 'or CAM ", which in turn is connected to a control device CTR, which processes and evaluates the image signals or data generated by the camera.

The control device CTR forms a device for recognizing and verifying attempts to manipulate the said control elements KBD, SHT or CSL and is implemented by means of computer-aided hardware, here for example by means of the hardware of a PC integrated in the ATM. The image data coming from the cameras (see also IN in Fig. 2 ) are processed in an image processing unit IPRC, for example by subjecting them to edge detection. As a result, the amount of data can be reduced significantly, without essential information about the properties of the to lose the photographed object or control element. The edge images or edge image data obtained from the edge detection thus represent essential properties on the basis of which a change or manipulation of the object (eg keyboard KDB) can be recognized.

For the reliable detection of manipulation attempts, the device CTR has a manipulation detection module M100, M100 'or M100 ″ for each picture element, which essentially contains a data processing entity that carries out a parallel structured two-step classification of the corresponding image data. In each manipulation detection module M100, M100 'or M100 "two classifiers arranged in parallel are used (see 110 and 120 in Fig. 2 ). For example, the first manipulation detection module M100 is used for monitoring the keyboard KBD, the function of which is explained in more detail below with reference to FIG Fig. 2 and 3 is described. The device CTR also includes an alarm unit ALRT, which triggers an alarm in the event of a manipulation attempt that has been reliably recognized. If the presence of a foreign object is detected, but a manipulation attempt is classified as implausible, the ALRT unit sends a notification to the operator of the ATM and / or to the last user to inform him that an object has been left at the ATM . In order to notify the user, user data can be accessed, which can be queried by a control center (user administration) as part of the use authentication at the ATM (user ID), which is required anyway.

Based on Fig. 2 and 3 An example of a method for recognizing and verifying manipulation attempts is now described:
the Fig. 2 FIG. 10 shows a schematic flow diagram for the method 100, which relates to the mode of operation of the manipulation detection module M100 and essentially comprises the steps 101 to 130 described below.

In a first step 101, the data generated by the camera CAM and stored in the image processing unit IPRC (see Sect. Fig. 1 ) data preprocessed for edge image data is provided as input data IN for the subsequent classifications. Features or properties A, B, C, D are extracted from the input data or processed image data, hereinafter also referred to as image data IN for short, which represent characteristic recognition features for the monitored object (here: control element KDB). These are, for example, the following features: edge length (top, bottom, left, right), distance between the edges of reference points or reference lines, angle between edges, angle between edges in comparison with reference lines, histogram, lighting conditions and the like.

These features, of which four features A, B, C and D are given here by way of example, are then classified in a parallel structure in two different ways. For this purpose, a first subset of the features (e.g. features A and B) is fed to a first classifier CF in a step 110 and a, preferably different, second subset (e.g. features C and D) is fed to a second classifier SC in a parallel step 120.

The first classifier CF assumes the function of a main classifier or a manipulation evidence collector, which checks whether manipulation is likely or not. This is done by checking features A (edge lengths at different positions) and, for example, B (distances between different edges) by comparing them with corresponding reference values. The output value OUT1 specifies the number or frequency with which features (one or more of them) do not match the reference values. The output value is between 0 and 1. An output value of 0.7 and more indicates that (very) many deviations were detected, so that there is (very) likely a manipulation of the object (here KBD keyboard or PIN pad). However, it cannot be said with certainty whether the detected manipulation is real manipulation, e.g. a keyboard overlay.

The second (parallel working) classifier SC therefore assumes the function of a secondary classifier or a checker (verifier) and checks whether the image data IN reproduces a typical situation for a manipulation at all. This check is done by checking the features C (angles between edges) and, for example, D (ambient light conditions). The output value OUT2 indicates the number or frequency with which the recorded features (one or more of them) do not deviate from the typical reference values. An output value of 0.3 and less indicates that the plausibility for a manipulation is low. This means that (very) few typical angles and / or ambient light conditions are recognized so that it is (very) likely that there is a foreign object in the area of the monitored object (here KBD keyboard), which cannot, however, correspond to an object of manipulation (e.g. keyboard superstructure). This would result in an output value OUT2 of 0.3, for example, which indicates that there is (more likely) no manipulation.

In order to arrive at a meaningful, unambiguous end result, the output values OUT1 and OUT2 are weighted and / or compared with threshold values TH1 and TH2 in further parallel steps 111 and 112, so that logically linkable values OUT1 * and OUT2 * result which Display either a YES or NO. If the value OUT1 * corresponds to the logical value "1", this means that the classification CF has recognized manipulation. This is symbolized here by a "Y" (for English "Yes"). Otherwise the result is an "N" (for English "No"). If the value OUT2 * also corresponds to a "Y", this means that the manipulation is plausible. The threshold values TH1 and TH2 are set, for example, in the middle value range, i.e. at around 0.5, so that output values greater than 0.5 mean a clear "Y" (statement "Yes" or "Yes").

By a logical AND operation carried out in step 130, which is also based on the Fig. 3 illustrated, one obtains the final result. Like the one in the Fig. 3 The decision matrix shown shows, a manipulation attempt is clearly recognized and verified only when both classifiers each deliver a positive result "Y", ie when the first classifier CF detects manipulation and the second detects a plausible manipulation situation independently.

The second classifier SC ensures a sensible decision and could therefore also be referred to as a "sanity checker". Due to the strict separation of the two classifiers or their tasks (recognition of changes or checking for plausibility), the manipulation recognition proposed here is very robust against incorrect decisions.

As described above, the camera signals (raw image data), for example, are first subjected to image processing (edge detection) (see block IPRC in Fig. 1 ). Features are then extracted from this (A to D in Fig. 2 ), the edge lengths, and z. Relate to distances, angles, ambient light conditions, etc. These characteristics are then classified. In the first classifier CF, for example, the features "edge length" and "angle" are classified in order to detect manipulation; in the second classifier CF the other features "angle" and "ambient light" are classified in order to check the plausibility (sanity check; sanity check). The invention can also be implemented in such a way that, for example, quantities are classified with the first classifier and qualities are classified with the second classifier.

In addition, the plausibility check makes it possible to infer the presence of a foreign object that is not a skimming device, but merely a personal object of a user, if a change is detected in the monitored object. In the decision matrix after Fig. 3 this corresponds to the case that OUT1 * = Y and OUT2 * = N. In this case, the system can initiate automatic notification of the user or customer, for example via email or SMS. These Opportunity offers a new customer service. In this case, the operator of the self-service terminal will also be notified immediately so that the customer can keep the forgotten item for later collection. So if objects are left at the self-service terminal, email / SMS can be generated and sent automatically by accessing the customer database. In addition, for example, a silent alarm can be triggered to the staff of the self-service terminal in order to secure the forgotten object. Photos / films can also be created and warnings can be displayed on the screen to prevent theft.

The present invention has been described using the example of an ATM, but is not restricted to this, but can be applied to any type of self-service terminal.

List of reference symbols

100

Procedure with the following steps:

101

Input of image data IN and specification of features (A ... D)

110

Data processing by means of the first classifier CF

120

(parallel to this) data processing by means of a second classifier CF

111

Threshold value decision for OUT1

112

Threshold depreciation for OUT2

130

Alerting and / or notification

ATM

Self-service terminal designed as an ATM with the following operating elements: KBD keyboard, SHT cash dispenser, CSL card input funnel

CAM, CAM ', CAM "

Cameras, each aimed at a control element

CTR

Device, here control device for ATM with: IPRC image processing unit M100, M100 ', M100 "Manipulation detection modules with different classifications

ALRT

Alarm / notification unit

Claims (13)

1. Method (100) for identifying and verifying manipulation attempts on a self-service terminal (ATM) comprising at least one operating element (KBD) which is provided for users of the self-service terminal (ATM) and towards which at least one camera (CAM) is oriented, which captures the at least one operating element (KBD), wherein the image data (IN) generated by the camera (CAM) are fed to a first classifier (CF), by means of which a check is made on the basis of the image data to ascertain whether a manipulation attempt on the operating element (KDB) can be identified, wherein the image data (IN) are evaluated on the basis of first features (A, B), wherein the image data (IN) are also fed to a second classifier (SC) in parallel with the first classifier (CF), by means of which second classifier a check is made on the basis of the image data to ascertain whether the identification of a manipulation attempt is plausible, wherein the image data (IN) are evaluated on the basis of second features (C, D),
   characterized in that
   an edge detection is applied to the image data (IN) fed to the first and second classifiers (CF, SC), said edge detection relating to edges that bring about delimitation from homogeneous areas in the optical appearance of the image element (KDB) captured by the camera (CAM) by virtue of the fact that, by means of an image processing unit (IPRC), at least one edge image is created and the characteristic data thereof are compared with the pattern data of a reference edge image, and in that the first features (A, B) concern edge lengths of the operating element and the second features (C, D) concern the relative position of the edges with respect to one another.
2. Method (100) according to Claim 1, characterized in that the image data (IN) are processed in parallel and independently of one another in the classifiers (CF, SC).
3. Method (100) according to Claim 1 or 2, characterized in that the first classifier (CF) evaluates the image data (IN) on the basis of the first features (A, B) in order to obtain a first output value (OUT1), which indicates the probability of the presence of an alteration of the optical appearance of the operating element (KDB).
4. Method (100) according to Claim 1, 2 or 3, characterized in that the second classifier (SC) evaluates the image data (IN) on the basis of the second features (C, D) in order to obtain a second output value (OUT2), which indicates the probability of the presence of an alteration of the optical appearance of the operating element (KDB) that is typical of manipulations.
5. Method (100) according to either of Claims 3 and 4, characterized in that the first output value (OUT1) is compared with a first threshold value (TH1) or respectively the second output value (OUT2) is compared with a second threshold value (TH2), in order to obtain a first or respectively second binary value.
6. Method (100) according to any of the preceding claims, characterized in that an alarm is triggered if a manipulation attempt on the operating element (KDB) is identified by means of the first classifier (CF) and if the identification of the manipulation attempt is assessed as plausible by means of the second classifier (SC) .
7. Method (100) according to any of the preceding claims, characterized in that an indication, in particular notification, is generated for an operator and/or user of the self-service terminal if a manipulation attempt on the operating element (KDB) is identified by means of the first classifier (CF) and if the identification of the manipulation attempt is assessed as implausible by means of the second classifier (SC) .
8. Method (100) according to Claim 7, characterized in that the notification for the operator and/or user of the self-service terminal is sent via a communication service, in particular email or SMS.
9. Device (CTR) for identifying and verifying manipulation attempts on a self-service terminal (ATM) having at least one operating element (KBD) which is provided for users of the self-service terminal (ATM) and towards which at least one camera (CAM) is oriented, which captures the at least one operating element (KBD), wherein the device (CTR) is connected to the at least one camera (CAM) and receives image data (IN) generated by the camera (CAM), and wherein the device (CTR) comprises a data processing unit with a first classifier (CF), which checks on the basis of the image data (IN) whether a manipulation attempt on the operating element (KDB) can be identified, wherein the device (CTR) evaluates the image data (IN) on the basis of first features (A, B), wherein the data processing unit of the device (CTR) comprises a second classifier (SC), which, in parallel with the first classifier (CF), processes the image data (IN) and checks whether the identification of a manipulation attempt is plausible, wherein the second classifier is adapted to evaluate the image data (IN) on the basis of second features (C, D),
   characterized in that
   an edge detection is applied to the image data (IN) fed to the first and second classifiers (CF, SC), said edge detection relating to edges that bring about delimitation from homogeneous areas in the optical appearance of the image element (KDB) captured by the camera (CAM) by virtue of the fact that, by means of an image processing unit (IPRC), at least one edge image is created and the characteristic data thereof are compared with the pattern data of a reference edge image, wherein the first features (A, B) concern edge lengths of the operating element and the second features (C, D) concern the relative position of the edges with respect to one another.
10. Self-service terminal (ATM) comprising at least one operating element (KBD) which is provided for users of the self-service terminal (ATM) and towards which at least one camera (CAM) is oriented, and comprising a device (CTR) for identifying and verifying manipulation attempts on the self-service terminal (ATM) according to Claim 9.
11. Self-service terminal (ATM) according to Claim 10, characterized in that the self-service terminal constitutes an automatic teller machine (ATM).
12. Self-service terminal (ATM) according to Claim 10 or 12, characterized in that the at least one operating element constitutes an element suitable for manipulation, in particular a keyboard (KBD), a cash dispensing compartment (SHT) and/or a card insertion slot (CSL).
13. Self-service terminal (ATM) according to any of Claims 10 to 12, characterized in that the at least one operating element (KBD) captured by the camera (CAM) has optically unambiguously identifiable features, in particular has edges that bring about delimitation from homogeneous areas.

Images