RU2752851C1 - Method for identifying a computer user: “human or internet robot” - Google Patents

Abstract

FIELD: computer network security.SUBSTANCE: present invention relates to the security of computer networks, including unauthorized access to Internet resources, namely, to the methods for image formation and types of tasks to the user when the user passes a fully automated Turing test (CAPTCHA). This result is achieved by choosing a video sequence of a natural scene as the source of the captcha image and forming two images: the first one is the background image, the second one is the foreground image, so that the foreground image is placed in an arbitrary point in the frame or its proportions or integrity are distorted, or it is tilted, or the listed changes to the foreground image in their combination are made, then the second one is the foreground image that is superimposed on top of the first, background image and automatically a single image is formed, an “unnatural collage’, at least one image of the "unnatural collage" is shown to the user of the captcha.EFFECT: providing secure access to Internet resources.1 cl, 6 dwg

Description

Technology area

The invention relates to the security of computer networks, including unauthorized access to Internet resources, namely to methods of forming images and types of tasks for the user when the user passes the fully automated Turing test (CAPTSNA) and can be used by Internet users.

State of the art

All large and medium-sized Internet services and Web portals, mail servers, search engines, and banking systems are very attractive platforms for intruders. In most cases, at present, the actions of hackers are not aimed at gaining administrator rights to manage a web resource (protection against such direct hacks is quite reliable and comes down to simple accuracy in administering the portal). The most widespread method of organizing Internet fraud is imitation of user actions using a special program (Internet robot). For example: an Internet robot can create thousands of new accounts (i.e. register allegedly new users of a resource, for example, a free mail service), and then send out bulk advertisements, fake invoices for services, leave spam comments on forums, add friends on the VKontakte social network, publish messages that provoke the disclosure of confidential information or personal data. If there were no protection against programs (Internet robots) that imitate human actions, then without exception, all social Internet resources, including the sites of the State Services system, would have turned into information trash heaps and they simply would have been impossible to use.

(от абревиатуры САРТСНА-англ.The test for the fact that a person accesses the site has a common name - captcha.

(from the abbreviation SARTSNA-eng.

Completely Automated Public Turing test to tell Computers and Humans Apart - a fully automated public Turing test for distinguishing between computers and people) - a computer test used to determine who a user of a system is: a person or a computer. The term appeared in 2000. The main idea of the captcha is to offer the user a problem that can be easily solved by a human, but extremely difficult and time-consuming for a computer (ideally unsolvable). Since in this case the test is carried out not by a person, but by a machine, such a test is often called the inverse Turing test. As of 2013, approximately 320 million CAPTCHA are entered by users around the planet every day (see the Wikipedia article https://en.wikipedia.org/wiki/CAPTCHA). Some requirements for Internet resources that use the captcha test are set by "GOST R 52872-2012. National standard of the Russian Federation. Internet resources. Accessibility requirements for visually impaired people. "

The main measure of counteraction to Internet robots is additional user verification, which is carried out in the form of an offer to the user to perform a small test task. It is assumed that if a person tries to access the resource, then the test will be passed easily and successfully, and for an Internet robot, even a specially programmed one, the probability of completing the test task will be small. For example, on the most popular website of the Russian Internet sector (social network VKontakte), such a check is organized in the form of the following test: a person is shown a highly distorted set of four characters, above which a request is displayed to read the “code word”, and then enter a code set of characters from using the keyboard in a special window (see Fig. 1).

To make it difficult for the Internet robot to recognize the code characters in the picture, they are specially crossed out, tilted, stretched, have a discontinuous outline, etc. The imposition of more and more complex "noises" and frequent modification of distortions have to be done due to the appearance of more and more sophisticated character recognition programs. So, in the description of the invention to the patent RU No. 2608262 "DISTORTION OF SYMBOLS FOR SYMBOLIC-GRAPHIC BACK TURING TEST", it is proposed to obtain new types of graphically distorted characters by morphing (obtaining intermediate styles) based on two or more known standard fonts.

In open sources, the program code of an Internet robot is available that performs character recognition for any modern character captcha with a probability of 85% (see http: //www.opermet.m/opermews/art.shtml? Num = 47477). Site owners are forced to modify, complicate the test, introduce even more distortion and noise in the character style. However, highly complicated text captchas are easily recognized by modern recognition methods based on neural networks (see https://habrahabr.ru/post/343222/).

From the analysis of successful methods of "cracking" text captcha, conclusions can be drawn about the main shortcomings of the test based on a symbolic code word:

a) there is a limited set of characters that can be demonstrated to a person, so for the Russian language these are 33 letters and 10 numbers, the limited set of images for automatic recognition greatly facilitates the task of passing the test by an Internet robot, made, for example, based on a neural network;

b) too distorted characters are already difficult to recognize even by a person, and the threshold of permissible noise and distortion for this technology has already been reached.

In addition to captchas based on distorted characters, other methods of user verification are also common on the Internet. So, quite often, captcha appears only in the case of unusual user behavior (i.e., if there is a suspicion that an attack by an Internet robot is being carried out). For example, if a user performs a series of suspiciously fast actions of the same type, the protection of the Google search engine first suggests passing a simple test (captcha) (example at: https://patrickhlauke.github.io/recaptcha/); if the user's actions are even more threatening, then a more complex (for a robot) task appears - it is required to indicate pictures with a given content among several. For example, it is proposed to indicate with the mouse those (three or four) pictures out of 16, on which, for example, a road sign is present. Google, apparently, uses this method of checking the user (for "humanity") due to the fact that the protection by the "code word" had to be often modified, being sophisticated in distortion and noise. Google showcases a selection of 16 images from a large database, where all images are reviewed and manually classified.

The problem with this method is that attackers can easily collect a large part of the images that are stored in the Google database, then classify these images themselves, and then "train" the neural network to recognize them. To prevent this, you need to create many test images, as well as at least a hundred categories of test images. But creating a large database of classified images is a very time consuming and expensive task, even for Google. In addition, since Since the captcha image database from Google is images from the Internet, it is also not difficult for attackers to train modern learning image recognition systems to collect a training sample on the Internet. And, in addition, it turns out that free neural network training libraries from Google itself are quite suitable for use as hacking software, that is, in fact, recognition algorithms from Google (https://arxiv.org/pdf/1312.6082v4.pdf) are not bad hack Google captchas.

There are known ways to overcome the above disadvantages for the currently most common types of captcha (recall, within one type of captcha, distorted characters are displayed, in the second type it is required to indicate images (from several) with a given content).

One of the most promising ways to overcome the above drawbacks is to create a captcha, where the user is asked a question not on recognizing the object shown in the image (for example, "what number" or "indicate pictures with a dog"), but the user must give some generalized description image. For example, a captcha question can be - how many objects are in the image (such a question is proposed in the patent - RU 2663475 C1), or “indicate those images that seem unnatural” (such a question contains the captcha of this application).

The known method of forming images of Captcha according to the application US 8090219 B2 "Systems and methods for socially-based correction of tilted images", which is based on the ability of a person to recognize tilted images. The idea of whether the object in the image has a tilt from its natural location, a person always arises, because a person should know when an object can stand or lie on a horizontal surface. It seems impossible to train a computer to understand the stable vertical state of all objects that are known to the user to a human being, given the current state of the art. The number of objects that a person has an idea of their "correct" orientation is much larger than that which current algorithms can classify. But it is still not enough to classify the object, after the object is recognized for each angle in the frame, the computer program must calculate - is this object presented at an angle?

The disadvantage of the source US 8090219 B2 "Systems and methods for socially-based correction of tilted images" is that a very small fraction of images that are on the Internet can be used in such a captcha. This means that all captcha images that the user will then see must be previewed by real people, on which the operator preparing the images must select the area of the object, which is then rotated. To create a reliable CAPTCHA (when cybercriminals will not be able to receive a significant proportion of all images by bulk requests), people will need to first view and process several hundred thousand images. This means that deploying such a CAPTCHA as an Internet service will be very costly. In addition, for most objects, even without their recognition, you can specify the vertical axis. The area that the object occupies in the image (for example, a standing person) itself has an orientation - it seems to be elongated (from head to toe), and the direction of such "elongation" in a huge proportion of cases coincides with the natural vertical of the object. This property of many objects to be elongated vertically can be easily used to create an Internet bot for automatic access to the server. Thus, the method US 8090219 B2 "Systems and methods for socially-based correction of tilted images" should not be considered a reliable way to distinguish a person from an Internet robot.

Disclosure of the invention The technical problem to be solved by the claimed invention is a significant increase in the probability of recognizing a person from an Internet robot using a captcha test when granting permission to access Internet resources.

The technical result of the claimed invention is to provide secure access to Internet resources.

To achieve the technical result, a method for identifying a computer user "human or Internet robot" is proposed, which includes the steps:

(a) form a base of source images for demonstration to users,

(b) one or more images are randomly selected from the base of images, which is formed in advance,

(c) the user's screen displays the captcha question to the user in text form - in the questions to the image, one or more words from the list are used: "unnatural", "natural", "collage",

(d) comparing the parameters of the user's response actions, which are saved at stage (c) with the parameters of the image description, which are formed at stage (b),

(e) if the parameters obtained as a result of perception by the user and the parameters stored on the server do not coincide, the user is denied access to the resource, or vice versa, if the comparison is positive, the user is given access to information, while the natural video sequence is selected as the source of the captcha image scenes and form two images: the first is the background image, the second is the foreground image, so that the foreground image is placed in an arbitrary place of the frame or distorts its proportions, or distorts its integrity, or tilts it, or makes the listed changes to the foreground image plan in combination, then the second - the foreground image is superimposed on top of the first - the background image and a single image is automatically formed - "unnatural collage", at least one image of the "unnatural collage" is shown to the user with captcha.

The claimed invention proposes a method for automatic and mass synthesizing of images, which, from the point of view of a person, definitely cannot be snapshots of a natural environment. At the same time, it is not possible to create a computer program with which it would be possible to reliably distinguish the proposed "unnatural" images (collages) from natural photographs at the current state of the art. To do this, an object is “inserted” into a natural image, and the object is deformed by a pseudo-random process, while at least three parameters of such a deformation are randomly set. Thus, they achieve that there are significantly more possible variants of images of deformed objects than the number of their intended demonstrations. In the process of each of the above deformation, many characteristics acceptable to a person are violated - in terms of the normal position of the object, its proportions, integrity, etc. When synthesizing "unnatural" images, a large base of initial images is used, both of the object itself and natural images. Such a base of source images is formed on the basis of video sequences downloaded from the Internet and is constantly replenished. To provide secure access to the Internet resource, the remote user is invited to consider several images and answer the question for each of them - "is the picture a natural image?" In this case, the set of images for each user verification session is unique. Neither the synthesized and demonstrated "natural collage" (the image of a "natural collage" from the point of view of a person does not differ in any way from a natural photo), nor the synthesized and once demonstrated "unnatural collages" are shown a second time - in another user verification session.

The combination of the above essential signs of image formation and their demonstration leads to the fact that there are so many different images that are offered to be considered by remote users that it is impossible to program generalizing rules for distinguishing between "natural" and "unnatural" images, it is also impossible to form a sufficient training sample for training one of the well-known machine learning methods, for example, neural networks. At the same time, it is assumed that a person looking at each image that he sees for the first time will easily answer the question - was it artificially synthesized or is it a natural snapshot.

Brief Description of Drawings

FIG. 1 shows a "classic" text captcha, in which you need to enter the code word shown in the picture.

FIG. 2 shows how the captcha image looks for the closest prototype according to this application (US 8090219 B2), the task for the user on the captcha according to the application US 8090219 B2 is to indicate with the help of the manipulator those images on the screen that are tilted.

FIG. 3 shows an "unnatural" collage, this is an image that is clearly synthesized by computer editing, and cannot be obtained by photographing some real scene, the foreground object is located unnaturally, has no support and its integrity is violated.

FIG. 4 shows how, on the basis of two frames of two natural scenes (on one there is only a background, and on the other there is an object against the same background) by computer processing, using the "subtraction" procedure, an area of the frame ("mask") is obtained, which "covers" only the foreground object, and then apply a mask to the natural image with the object and get an image (bottom right) in which all image pixels that do not belong to the foreground object are marked as transparent.

FIG. 5 shows how, on the basis of two images - the background image and the image with the selected object, two other images are obtained, the first is a natural collage, the second is an “unnatural” collage.

FIG. 6 shows what images the user will see when responding to captcha tasks according to this application, while the user will be prompted to use the mouse or finger (for mobile devices) to indicate those images that are "unnatural".

Implementation and examples of implementation

An “unnatural collage” is an image into which an object is artificially “inserted”, and the image of this object does not match the main part of the image. For example, for an image of an "unnatural" object that is artificially inserted into a natural image:

a) violate the correct proportions;

b) exclude the necessary parts of the object;

c) the object is depicted as "suspended" without support;

d) significantly and unnaturally deform the image of the object

e) use various combinations of the above items a) -d) (see Fig. 3).

To do this, in the proposed method, the user displays several separate images on the screen when granting permission to access Internet resources.

With each user request, the following sequence of response work steps starts automatically:

- one or more images are randomly selected from the database of images, which is formed in advance;

- each selected image is rotated by an angle, the value of which is chosen at random;

- the selected image is placed on the user's screen next to the original non-tilted (natural) image;

- the user's screen displays a captcha question (or task) to the user in text form, for example, "specify those images that are tilted";

- it is further assumed that the user examines the image, reads the text of the task and performs it, while the user with the help of a manipulator "mouse" or a finger on the screen marks those images that are tilted from their natural position;

- when the user performs a sequence of responses, the user's browser generates a set of parameters characterizing the user's perception of the image, then the user's browser sends the values of these parameters to the server;

- the server receives a parametric description of the user's response and compares two sets of parameters: a) parameters characterizing the user's perception of the image on the computer screen and b);

- if the parameters obtained as a result of the perception by the user and the parameters stored on the server do not coincide, the user is denied access to the resource, or vice versa, if the comparison is positive, the user is given access to the information.

The technical result of the proposed method is achieved as follows.

In the present method, the user is prompted to distinguish an image that contains an “unnatural collage” (Fig. 3) from an image that is obtained, for example, by photographing a natural setting. At the same time, a significant factor hindering image recognition by a software robot is a set of random parameters that change the initial "natural" image, in contrast to the prototype, in which the image changes in one parameter - it rotates by some random angle.

Both "natural collages" and "unnatural collages" are composed of two pre-prepared images by "superimposing" one on top of the other: the first is a "background image" and the second is a foreground image with an opaque area. Both "natural collages" and "unnatural collages" are automatically compiled according to the program.

The area in the second image is randomly deformed, for example, stretched in a random direction, or rotated at a random angle. In addition, in the second image, the opaque part is made transparent, thereby creating a "nibble" effect. Thus, the effect of "unnatural" image is achieved when it is perceived by a person.

When synthesizing both "natural collage" and "unnatural collage", video sequences are used as the initial data. The video sequence is obtained and stored in advance by recording data from outdoor video surveillance cameras or recording training programs, which are freely available on the Internet in sufficient quantities.

At the first stage, when synthesizing both "natural collage" and "unnatural collage", a part of the original video sequence is processed so that a "background" image or a background image is formed (in Fig. 4 - the upper left frame). In this case, the method from the patent RU 2395787C2 "METHOD FOR DETECTING OBJECTS" is used. From the known patent, the following actions are used: 1) initialize the background frame, for example, with zero values; 2) take the first frame of the video sequence and then for each new frame perform for each pixel of the background frame according to some rule calculate the "background update constant", based on this constant, the current frame and the previous "background frame" calculate a new "background frame".

At the second stage of creating an “unnatural” collage (see bottom right frame in Fig. 4), using a background frame and some frame, “object detection” (upper right frame of Fig. 4) is performed according to the method described in RU 2395787 C2. From the known patent, the following actions are used: the absolute difference between the current frame and the background is calculated and binarization is carried out with a threshold frame, while the values of the pixels of the threshold frame are calculated by the formula in which the value of the pixels of the current frame is used, the value of the pixels of the background frame. As a result, a frame with a selected object is obtained (see the lower-right frame in Fig. 4).

At the third stage of automatic synthesis of the "unnatural" collage, a) the background frame of the video sequence, which was obtained at the first stage (see the upper right frame of Fig. 5), is taken and b) a frame with the selected object and a transparent background is taken (see the upper right frame of Fig. . 5). Further, on the frame with the selected object and a transparent background, distortions are made in the area of the object's image - this area is partially made transparent - the object is cropped, rotated, shifted, etc. Then the distorted object is superimposed on the background frame - and thus an "unnatural" collage is obtained ( see the lower right frame of Fig. 5). The algorithm for such image overlay is well known and is used in common computer graphics packages, for example, in Adobe Photoshop.

In order to obtain more accurate answers of human users to the captcha question according to one of the methods in this application, a "natural collage" is displayed on the user's screen (see the lower left frame of Fig. 5) next to the "unnatural collage" (see the lower right frame of Fig. . 5). It is assumed that the user will compare two frames with the same background, these frames are placed side by side, and the user will have to answer the question - "which picture is an" unnatural "collage?" or "which image is unnatural?" (see Fig. 6).

According to the main method of the present application, an image, which is assumed that a person will always perceive it on the screen as natural, is also created as an automatically synthesized “natural collage”. Such a collage is made on the basis of a "background frame" and a frame with a selected object and a transparent background, but the area of the object is not distorted or deformed in any way. Otherwise - the object is selected from the current frame and "put" back, but on the calculated "background frame" - thus getting a "natural collage". The user should not notice that this is an automatically synthesized collage, because the selected object is superimposed on the background frame without distortion, no shift, no cropping and no rotation, etc. This procedure is carried out because there are collage detection programs (photomontage). If “natural” images are not also made using collage synthesis technology (invisible from the point of view of a person), then software tools for detecting collages can be used to automatically pass (hack) captcha. Such programs will distinguish a collage from not a collage, but if only “unnatural” frames are made with collages (that is, “overlay” an object on the background), and simply frames of a video sequence without overlay are taken as natural, then programs for detecting collages will distinguish “natural "Images from" unnatural "technical features invisible to the human eye, and not in essence.

Both "natural collage" and "unnatural collage" are automatically composed of two pre-prepared images by "superimposing" one on top of the other. As one image, a "background image" (background image) is used, as the second one uses an image in which only the area that belongs to the foreground image (an object in the foreground) is opaque.

Before the "unnatural collage" is formed, the region of the front object in the second image is deformed in a random way (with random parameters, stretched in a "random" direction, a "random" rotation of the object region in the image is carried out, and also a part of the object is accidentally "cut off" or there are "holes"). In this case, a part of the opaque area of the second object is made transparent, thereby creating the effect of "biting off" the random part from the object. These means achieve the effect of "unnatural" image when it is perceived by a person.

Claims (6)

A method for identifying a computer user "human or Internet robot", which includes the following steps: (a) form a base of source images for demonstration to users, (b) one or more images are randomly selected from the base of images, which is formed in advance, (c) the user's screen displays the captcha question to the user in text form - in the questions to the image, one or more words from the list are used: "unnatural", "natural", "collage", (d) comparing the parameters of the user's response actions, which are saved at stage (c) with the parameters of the image description, which are formed at stage (b), (e) if the parameters obtained as a result of the perception by the user and the parameters stored on the server do not coincide, the user is denied access to the resource, or vice versa, if the comparison is positive, the user is given access to information, characterized in that the video sequence is selected as the source of the captcha image natural scene and form two images: the first is the background image, the second is the foreground image, so that the foreground image is placed in an arbitrary place in the frame or distorts its proportions, or distorts its integrity, or tilts it, or makes the listed image changes the foreground in combination, then the second - the foreground image is superimposed on top of the first - the background image and a single image is automatically formed - "unnatural collage", at least one image of the "unnatural collage" is shown to the user with captcha.

Images