WO2022211663A1

WO2022211663A1 - Method and system for protecting digital information displayed on a monitor with the aid of digital marks

Info

Publication number: WO2022211663A1
Application number: PCT/RU2021/000218
Authority: WO
Inventors: Артем Васильевич ВАСИЛЬЕВ; Дмитрий Алексеевич БОРИСОВ; Андрей Николаевич ЯСТРЕМБСКИЙ; Максим Андреевич КРЫЛОВ; Антон Олегович ЕРМОЛЮК
Original assignee: Публичное Акционерное Общество "Сбербанк России"
Priority date: 2021-04-01
Filing date: 2021-05-27
Publication date: 2022-10-06
Also published as: RU2768533C1

Abstract

The claimed technical solution relates to the field of protecting digital data, particularly confidential and sensitive information displayed on the screen of an electronic device, by embedding digital marks. The claimed method for protecting information displayed on a monitor of a computing device, which is implementable by a processor, contains the steps of: obtaining information to be encoded, which includes at least time and date data, as well as an identifier of the user and/or of the computing device; encoding the obtained information into a digital mark, said digital mark being a block consisting of graphic elements arranged in a geometric pattern, and the colour of the elements of said digital mark being generated in the colour scheme of the monitor, which constitutes an orthogonal basis; and generating a substrate with a set level of transparency, which is displayed on the screen of the monitor of the computing device and on which digital marks are placed, the configuration, positioning and number of said digital marks being determined by parameters that at least include the colour scheme of the monitor and the resolution thereof.

Description

METHOD AND SYSTEM FOR PROTECTING DIGITAL INFORMATION DISPLAYED ON A MONITOR USING DIGITAL LABELS

FIELD OF TECHNOLOGY

[0001] The claimed technical solution relates to the protection of digital data, in particular confidential and sensitive information displayed on the screen of an electronic device, using digital labeling technology (DM).

BACKGROUND OF THE INVENTION

[0002] The use of digital information in the field of digital information protection is a common solution in which encoded information is embedded in the image, which makes it possible to identify its ownership or the person responsible for its leakage and / or unauthorized access.

[0003] As a rule, such approaches use a given graphic element or image area containing the CM. Moreover, such a label can be both distinguishable and indistinguishable for the human eye. One example of such technology is steganography.

[0004] An analogue of the proposed solution is the principle of forming a protective layer based on the CM, disclosed in US patent 9,239,910 (Markany Inc, 01/19/2016). The solution is to create an invisible underlay containing digital marks, which is used as a background layer displayed on the device screen.

[0005] The disadvantage of the existing approach is its lack of efficiency, due to the fact that for the formation of the substrate used DM, which is a text or graphical primitive, selected from the database and used to generate the space filling. This leads to the fact that such formation of the layer becomes sensitive to quality and during the subsequent capture of the image on the screen using an external device, for example, a smartphone or camera camera, when changing the angle or capturing part of the screen with a protective layer, subsequently removing the CM and establishing the fact of leakage data becomes difficult or impossible.

SUMMARY OF THE INVENTION

[0006] The proposed approach allows solving the technical problem, which consists in reducing the stability (robustness) of the method of protecting digital data during their fixation external means from different angles and image quality, which critically affects the subsequent decoding of data from digital tags.

[0007] The technical result consists in increasing the efficiency of protecting information on the screen of devices, by improving the stability of recognition of embedded digital labels that form a substrate displayed on the screen of an electronic device.

[0008] The claimed result is achieved through the implementation of a computer-implemented method for protecting information displayed on the monitor of a computing device (CD) executed by a processor and comprising the steps of: obtaining information for encoding containing at least time data and date, as well as the user ID and / or VU; encode the received information into a digital label (DM), while the DM is a block consisting of graphic elements that are placed in a geometric pattern, and the color of the CM elements is formed in the color scheme of the monitor, which is an orthogonal basis; form a substrate with a set level of transparency, displayed on the screen of the monitor VU, on which digital marks are located, the configuration, layout and number of which is determined by parameters containing at least the color scheme of the monitor and its resolution.

[0009] In one of the particular examples of the implementation of the method, when encoding information, error correction codes and/or error correction codes are used. [0010] In another particular example of the implementation of the method, the elements of the CM of one block are located equidistant from each other.

[0011] In another particular example of the implementation of the method, blocks of digital marks on the substrate are located at a given distance from each other.

[0012] In another particular example of the implementation of the method, the thickness and brightness of the CM elements are selected based on the amount of information for encoding.

[0013] In another particular example of the implementation of the method, the distribution of blocks of the CM on the substrate is determined based on the upper left element of the CM.

[0014] In another particular example of the implementation of the method, the formation of the CM block occurs recursively with the sequential formation of each element of the CM.

[0015] In another particular example of the implementation of the method, the CM blocks located on the substrate are converted to black and white and inverted.

[0016] The claimed technical result is also achieved due to a computer-implemented method for detecting encoded through the use of digital labels of information located on the received image or part thereof, performed by the processor and containing the steps, which are: get an image containing at least part of the information displayed on the monitor VU, protected using the above method; converting the obtained image from the RGB basis to the color basis in which the CMs were formed; perform per-pixel subtraction of RGB color values in the information channel from the color values of the base channel; determine the CM in the received image and extract the information encoded in them.

[0017] In one of the particular examples of the implementation of the method, the image is received from an external device.

[0018] The claimed solution is also implemented using a computer system containing at least one processor and at least one memory storing machine-readable instructions that, when executed by the processor, perform any of the above methods.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG. 1 illustrates the general principle of the claimed solution.

[0020] FIG. 2 illustrates a block diagram of the claimed data protection method.

[0021] FIG. 3 illustrates an example of layer placement on a device screen.

[0022] FIG. 4 illustrates a flowchart of a method for decoding information from a digitally protected image.

[0023] FIG. 5 illustrates an example of capturing an image of information from a screen of a device.

[0024] FIG. 6 illustrates the principle of decoding a CM from an image.

[0025] FIG. 7 illustrates a general view of the computing device.

IMPLEMENTATION OF THE INVENTION

[0026] In FIG. 1 shows the general concept of the technical implementation of the claimed solution. The protection of sensitive and / or confidential information displayed on the screen (111) of the user's computing device (CD) (110) is carried out using the implementation of the CM (10), into which the relevant information is encoded for the subsequent determination of the location and responsible person for unauthorized obtaining information outside the protected perimeter of the infrastructure, for example, by photographing it, filming it or capturing (screenshot) an image on the screen of the VU by external devices (smartphone, camera, etc.), including with subsequent printing.

[0027] The solution can be implemented using a client-server architecture, in which information to be encoded in the CM (10) can be transmitted from the server and generated for each VU (software) individually, which will allow for a unified accounting of computing devices and timely determination of the location data leaks. The server can control connected clients - VU; assigning to each client, based on the system username of the VU, a unique ID that will be transmitted for encryption; distribution of configuration files.

[0028] As shown in FIG. 2, the claimed method (200) for protecting digital information contains a number of successive steps. At the first stage (201), information is obtained for encoding and subsequent formation of the CM (10).

[0029] As a rule, the data embedded in the CM (10) is necessary to identify the VC (110) or directly the user of this device, for example, an employee who has access to sensitive information. Such data can be: personnel number, name, user photo, IP address, MAC address, unique VU identifier. This information may be used singly or in any combination. Additionally, information about time and/or date can also be encoded, for example, the time of formation of the CM (10), the current date. The date and time information can be dynamically changed to include up-to-date information during the encoding of the CM (10).

[0030] At step (202), the formation of the CM (10) is performed. The CM (10) is a block containing static elements (11) of the graphical interface of a given bit dimension, placed in a certain pattern (for example, as shown in the figure in the form of concentric circles). The pattern of CM blocks (10) can be the same or different, while the size of the elements (11) depends on the size of the encoded information. The removal of elements (11) from each other within the CM block (10) is equal in the presented example, but a different placement principle can also be applied, laid down by the program logic of information encoding when forming the CM block (10).

[0031] The CM (10) is formed using the process of orthogonalization of the basic colors of the RGB palette (red, green, blue) in order to reduce the visibility of the marks displayed on the monitor (111) of the VU (software) for the human eye. This technology allows to provide a combination of requirements for the comfort level of the operator of the VU (110) when working with the marked screen of the monitor (111), as well as the level of brightness of the marks. When encoding information in the CM (10), the color that was considered the main one is taken (for example, the color is minimally perceived by the human eye). Next is the projection of the selected color in the basis of the primary colors of the RGB palette. After that, the vector is calculated according to the vector orthogonality criterion for the subsequent determination of the transition matrix between the primary basis (RGB) and the new color basis.

[0032] The process of encoding information in the CM (10) can occur as follows:

• Registration of the application in the network by requesting the server with confirmation of the established connection;

• Creation of a personal identifier;

• Creation of a set of additional parameters and their values required for encryption. The composition of the set is determined by the configuration file located on the server;

• Converting a personal identifier to a binary format and writing the resulting value into an encoding string;

• Converting the values of additional parameters to binary format and writing the resulting value to the encoding string. If error correction codes or error detection codes are used, then this code is calculated based on the encoding string and it is overwritten;

• Unloading a string from the coding block. Completion of work on this section of the event loop.

[0033] At step (203), the formation of the substrate (101) with the set transparency level is performed, which is displayed on the screen of the monitor VU (111). The substrate (101) occupies the entire space of the display area on the monitor screen (111), while the CM blocks (10) are located on it in a given pattern. The configuration, layout and number of CM blocks (10) are determined by parameters that include the color scheme of the monitor and its resolution. Additional options may also apply.

[0034] In FIG. 3 shows an example of a formed substrate (101) based on a plurality of CM blocks (10) with the corresponding placement of elements (11) in each of the blocks. Depending on the shift parameter of the upper left marker (11) of the CM block (10), the initial position is set for the formation of the substrate pattern (101). Further the elements (11) (markers) are applied to the transparent substrate (101) in accordance with the following parameters:

- Marker type (geometric shape);

- Brightness of the marker;

- Marker thickness.

[0035] In the presented example, using round elements (11), additional parameters are observed: the maximum marker radius, the number of concentric circles.

[0036] The process of applying elements (11) is repeated iteratively until the end of the markup of one block of the CM (10), with the achievement of a predetermined number of elements (11) in the CM block (10). The placement of elements (11) is repeated with a global shift in accordance with the shift parameter of the CM blocks (10), which is set on the server and can vary based on the achievement of the maximum possible number of solid CM blocks (10) on the monitor screen (111) that meet the requirements for placement of encoded information. Depending on the transparency parameter of the background (101) and the element type (11), a gray canvas (mask) is created with the specified transparency. The resulting mask is cloned, converted to black and white, then inverted and applied to the substrate (101) with a certain level of transparency.

[0037] Next, consider the process of decoding the information protected by the CM shown in FIG. 4 - Fig. 6. FIG. 4 illustrates a flowchart of a method (400) for decoding information from a captured image.

[0038] At the first stage (301), the computing module (eg, processor) receives an image (410) that was taken using an external device (400) or directly using a VU (110), for example, a screenshot. The image (410) may contain part or all of the information presented on the monitor screen (111), as shown in FIG. 5.

[0039] Next, the resulting image (410) goes through a processing step (302), during which the resulting image is converted from the RGB basis to the color basis in which the CM blocks (10) were formed.

[0040] After transferring the image (410) to a new basis, the information color channel and basic digital channels are extracted from it. Since the entire set of pixels can be represented as a matrix, for example, 1920x1080x3 (for a Full HD monitor), three different layers can be selected, each of which has a dimension of 1920x1080 pixels. Next, at step (303), the pixel-by-pixel subtraction of the values of the information channel selected for coding the CM (10) is performed from the corresponding b values of basic channels. This allows at step (304) to isolate the CM (10) from the rest of the noise and decode them to extract the information encoded in them, as shown schematically in FIG. 6.

[0041] In FIG. 7 is a perspective view of a computing device (500) suitable for performing methods (200, 300). Device (500) may be, for example, a server or other type of computing device that can be used to implement the claimed technical solution. Including being part of a cloud computing platform.

[0042] In the general case, the computing device (500) contains one or more processors (501), memory facilities such as RAM (502) and ROM (503), input / output interfaces (504), input devices connected by a common information exchange bus / output (505), and a device for networking (506).

[0043] The processor (501) (or multiple processors, multi-core processor) may be selected from a variety of devices currently widely used, such as Intel™, AMD™, Apple™, Samsung Exynos™, MediaTEK™, Qualcomm Snapdragon™, and etc. The processor (501) can also be a graphics processor such as Nvidia, AMD, Graphcore, etc.

[0044] RAM (502) is a random access memory and is designed to store machine-readable instructions executable by the processor (501) to perform the necessary data logical processing operations. The RAM (502) typically contains the executable instructions of the operating system and associated software components (applications, program modules, and the like).

[0045] A ROM (503) is one or more persistent storage devices such as a hard disk drive (HDD), a solid state drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media ( CD-R/RW, DVD-R/RW, BlueRay Disc, MD), etc.

[0046] Various types of I/O interfaces (504) are used to organize the operation of device components (500) and organize the operation of external connected devices. The choice of appropriate interfaces depends on the particular design of the computing device, which can be, but not limited to: PCI, AGP, PS/2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS/Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc. [0047] To ensure user interaction with the computing device (500), various I/O information means (505) are used, for example, a keyboard, a display (monitor), a touch screen, a touchpad, a joystick, a mouse, a light pen, stylus, touchpad, trackball, speakers, microphone, augmented reality, optical sensors, tablet, indicator lights, projector, camera, biometric identification tools (retinal scanner, fingerprint scanner, voice recognition module), etc. [0048] The networking means (506) enables data communication by the device (500) via an internal or external computer network, such as an Intranet, Internet, LAN, and the like. As one or more means (506) can be used, but not limited to: Ethernet card, GSM modem, GPRS modem, LTE modem, 5G modem, satellite communication module, NFC module, Bluetooth and / or BLE module, Wi-Fi module and others

[0049] Additionally, satellite navigation tools in the device (500) can also be used, for example, GPS, GLONASS, BeiDou, Galileo.

[0050] The submitted application materials disclose preferred examples of the implementation of the technical solution and should not be interpreted as limiting other, particular examples of its implementation that do not go beyond the scope of the requested legal protection, which are obvious to specialists in the relevant field of technology.

Claims

FORMULA

1. A computer-implemented method for protecting information displayed on the monitor of a computing device (VU), performed by a processor and comprising the steps of:

- receive information for encoding, containing at least data about time and date, as well as a user ID and/or VU;

- encode the received information into a digital label (DM), while the CM is a block consisting of graphic elements that are placed in a geometric pattern, and the color of the CM elements is formed in the color scheme of the monitor, which is an orthogonal basis;

- form a substrate with a set level of transparency, displayed on the screen of the monitor VU, on which digital labels are located, the configuration, layout and number of which is determined by parameters containing at least the color scheme of the monitor and its resolution.

2. The method of claim 1, wherein error correction codes and/or error correction codes are used in encoding the information.

3. The method according to claim 1, in which the elements of the CM of one block are located equidistant from each other.

4. The method according to claim 3, in which the blocks of digital marks on the substrate are located at a given distance from each other.

5. The method according to claim 1, in which the thickness and brightness of the CM elements are selected based on the amount of information for encoding.

6. The method of claim 4, wherein the distribution of CM blocks on the substrate is determined based on the upper left CM element.

7. The method according to claim 3, in which the formation of the CM block occurs recursively with the sequential formation of each element of the CM.

8. The method according to claim 1, in which the CM blocks located on the substrate are converted to black and white and inverted.

9. A computer-implemented method for detecting information encoded by using digital marks located on the received image or its part, performed using a processor and comprising the steps of:

- receive an image containing at least part of the information displayed on the monitor VU, protected using the method according to any one of paragraphs. 1-8; - converting the resulting image from the RGB basis to the color basis in which the CMs were formed;

- perform per-pixel subtraction of RGB color values in the information channel from the color values of the base channel;

- determine the CM in the received image and extract the information encoded in them.

10. The method of claim 9, wherein the image is acquired from an external device.

11. The system for protecting information displayed on the monitor of the VU, containing at least one processor and at least one memory storing machine-readable instructions that, when executed by the processor, implement the method according to any one of paragraphs. 1-8.

12. A system for detecting information encoded by using digital marks, comprising at least one processor and at least one memory storing machine-readable instructions that, when executed by the processor, implement the method according to any one of paragraphs. 9-10.