EA043172B1 - METHOD AND SYSTEM FOR PROTECTING DIGITAL INFORMATION DISPLAYED ON THE SCREEN OF ELECTRONIC DEVICES USING DYNAMIC DIGITAL LABELS - Google Patents

Description

The invention relates to the field of protection of digital data, in particular confidential and sensitive information displayed on the screen of an electronic device, through the introduction of digital labels (DM).

State of the art

The use of digital media in the field of digital information protection is a common solution in which encoded information is embedded in the image, which makes it possible to identify its ownership or the person responsible for its leakage and/or unauthorized access.

As a rule, such approaches use a given graphic element or an image area containing a CM. Moreover, such a label can be both distinguishable and indistinguishable for the human eye. One example of an indistinguishable label is steganography.

An analogue of the proposed solution is the principle of forming a protective layer based on the CM, disclosed in US patent 9239910 (Markany Inc, 01/19/2016). The solution is to create an invisible protective layer consisting of digital marks, which is used as a background layer, displayed on the device screen and invisible to the user.

The disadvantage of the existing approach is its lack of efficiency, due to the fact that a CM is used to form the protective layer, which is a text or graphic primitive selected from the database and used to subsequently generate space filling. This leads to the fact that such formation of the layer becomes sensitive to quality and during the subsequent capture of the image on the screen using an external device, for example, a smartphone or camera camera, when changing the angle or capturing part of the screen with a protective layer, subsequently removing the CM and establishing the fact of leakage data becomes difficult or impossible.

The essence of the invention

The proposed approach allows solving the technical problem, which consists in the low stability (robustness) of the digital data protection method when they are fixed by external means from different angles and image quality, which leads to a decrease in the effectiveness of this digital data protection principle. The technical result consists in increasing the efficiency of the method for protecting digital data on the screens of devices, by increasing the stability of digital labels during their dynamic scanning of the placement area on the screen with the formation of an invisible protective layer.

The claimed result is achieved through the implementation of a computer-implemented method of a computer-implemented method for protecting data displayed on the screen of a computing device (CD) executed by a processor and containing the steps at which a digital label (DM) is formed in the form of a data block containing encoded information, according to at least identifying the user of the WU, and the encoded information is presented in the form of dynamic graphic elements that scan the area of their placement on the WU screen;

forming an invisible protective layer, consisting of a set of said data blocks, covering a large part of the display area of the RT screen, and overlaying the protective layer on the display area of the RT screen. In one of the particular examples of the implementation of the method in the CM, additional information about the date and time is encoded.

In another particular example of the implementation of the method, when scanning with dynamic graphic elements, the color of the pixels in the area of their placement is analyzed.

In another particular example of the implementation of the method, based on the color of the pixel, the brightness of the graphic element is adjusted.

In another particular example of the implementation of the method, the correction is carried out based on the parameters of each pixel in the area where the graphic element is placed in the RGB color scheme.

In another particular example of the implementation of the method, the RGB color scheme is converted to HSV with the calculation of the parameters of hue H, saturation S and brightness V.

In another particular example of the implementation of the method, the obtained brightness parameter V is changed by the value Δ.

In another particular example of the implementation of the method, the obtained brightness parameter V _Δ together with the parameters H and S are converted into the RGB color scheme. In another particular example of the implementation of the method, the dynamics of displaying graphical elements is selected based on the screen refresh rate of the VU. The claimed technical result is also achieved due to a computer-implemented method for processing protected data, which is performed using a processor and contains the steps at which an image is obtained containing at least a part of a screen image with information protected using the method according to any one of claims 1- 9;

decoding information based on the CM from the obtained image, which performs a pixel-by-pixel decomposition of the image into threshold images;

- 1 043172 perform the identification of graphic elements that form the CM;

restoring information from the CM based on the bit sequence formed by the graphic elements.

In one of the particular examples of the implementation of the method, each threshold image is formed as an average spectrum of the RGB palette of the corresponding threshold P, on the basis of which M - Ap images are created, on which each image pixel satisfies the ^{PeN condition:} ! ^ρΔ ρ- , ^{ρ + Δ} ρ}, where P is the current threshold, A e Ν _ threshold detection delta.

In another particular example of the implementation of the method, when forming threshold images, each pixel is converted to the HSV format, after which the saturation index S is extracted, on the basis of which N - As images are created, on which each image pixel satisfies the condition s e R: {s - As,. s + As}, where S is the saturation of the pixel, ^ E K is the saturation detection delta.

In another particular example of the implementation of the method, when forming threshold images, pixels are extracted from the processed image, each pixel is converted to the HSV format, the brightness index V is extracted, on the basis of which N - Δν images are created, in which each image pixel satisfies the condition E R. {V - Αν, ..., V + Αν}, where V - pixel brightness, Av G R - brightness detection delta.

The claimed solution is also implemented using a system for protecting data displayed on the screen of a computing device, which contains at least one processor and at least one memory containing machine-readable instructions that, when executed by the processor, carry out the above methods.

Brief description of the drawings

Fig. 1 illustrates the general principle of the claimed solution.

Fig. 2 illustrates a block diagram of the claimed data protection method.

Fig. 3A illustrates an example of a guard layer data block.

Fig. The ST illustrates an example of placing a layer on a device screen.

Fig. 4 illustrates a flow diagram of a method for decoding information from a digitally protected image.

Fig. 5 illustrates an example of capturing an image of information from a screen of a device.

Fig. 6 illustrates the principle of decoding a DM from an image.

Fig. 7 illustrates a general view of the computing device.

Implementation of the invention

In FIG. 1 shows the general concept of the technical implementation of the claimed solution. Protection of sensitive and/or confidential information displayed on the screen (111) of the user's computing device (CD) (110), for example, a computer, laptop, tablet, etc. Data protection on the screen (111) is carried out using the introduction of the CM (10), into which the relevant information is encoded for the subsequent determination of the location and the responsible person who leaked or unauthorized receipt of information outside the protected infrastructure perimeter, for example, by photographing, video filming or capturing ( screenshot) images on the screen using external devices (smartphone, camera, etc.), including subsequent printouts of pictures with the information received. Each CM (10) is a block of data that forms an invisible protective layer (101) that covers most or all of the display area on the screen (111) of the VU (110).

As shown in FIG. 2, the claimed method (200) for protecting digital information contains a number of successive steps. At the first stage (201) the formation of the CM (10) is carried out. Any type of information, for example, textual, graphic, or combinations thereof, can be embedded in the CM (10) using algorithms for binary (binary) data coding. CM (10) is formed from a set of graphic elements (11, 12) that encode information. The number of elements (11, 12) in each CM (10) may vary based on the amount of data to be encoded, for example, encoding 32 bits of information requires the formation of a 4x8 CM (10) block. In the presented example of the CM (10), the elements (11, 12) are presented in the form of oblique lines, however, their shape and placement principle may be different, ensuring compliance with the principle of distinguishing information encoding in the form of logical 0 and 1.

As a rule, the data embedded in the CM (10) is necessary to identify the VC (110) or directly the user of this device, for example, an employee who has access to sensitive information. Such data can be: personnel number, name, user photo, IP address, MAC address, unique VU identifier. This information may be used singly or in any combination. Additionally, information about time and/or date can also be encoded, for example, the time of formation of the CM (10), the current date. The date and time information can be dynamically changed to include up-to-date information during the encoding of the CM (10).

CM (10) is an element of the graphical interface of a given bit dimension, while the graphic elements (11, 12) that form the CM (10) are dynamic and carry out

- 2 043172 constant or periodic scanning of the space of their placement. Space scanning is carried out based on the screen refresh rate (111). One element (11) of the CM is a logical 0, the second (12) is a logical 1. The different principle of orientation of the elements (11, 12) makes it possible to delimit them in space and form a pattern of the CM block (10) for encoding information inside it.

At step (202), a protective layer (101) is formed in the form of placing CMs (10) that fill most or all of the screen space (111). The number of CM (10) is determined based on the resolution of the screen (111) and the dimension of the CM (10). As a rule, CMs (10) are chosen to be of equal dimension, but they can also have an alternating order of their placement to form a layer (101), which also does not violate the technical embodiment of the claimed solution in terms of subsequent data identification in the decoding process.

The protective layer (101) is formed invisible (transparent) so as not to be visible to the human eye and to hide the fact that data protection has been applied on the screen (111).

The transparency of the layer (101) of the created pattern from the CM (10) can be formed by various ways of adjusting graphic images, for example, by adjusting the opacity of the image (opacity), using the Alpha channel in the RGBA palette, using the Alpha channel in the HSLA palette.

Also, in another particular aspect, the transparency of the layer (101) of the created pattern from the CM (10) is formed by changing the brightness characteristics (by a certain parameter - brightness delta) of the pixels on which the mark will be applied.

The selection of parameters for forming an invisible layer (101) is performed to overcome the difference threshold (https://en.wikipedia.org/wiki/Just-noticeable_difference), making it invisible to a normal user.

After the formation of the protective layer (101) at step (203), it is superimposed on the display area of the screen (111) of the VU (110), so that the entire screen coverage area is covered by the superimposed layer (101). The application of this approach allows to determine the value of the CM (10) in any part of the screen (111) regardless of the coordinate or scale of the screenshot or photocopy.

In FIG. 3A, 3B show an example of a formed protective layer (101) based on a plurality of CMs (10). The presented example of the CM (10) can be created using information encoding in contrast channels of the color scheme, for example, RGB, HSL, ARGB, etc.

Each element (11, 12) of the PA (10) is dynamic and scans the space located behind it in order to prepare the visibility of the graphic elements (11, 12) pixel by pixel, ensuring that their brightness is allocated to a given delta. To do this, a pixel Р is taken in the image on the screen (111), for which its parameters P(R, G, B) are calculated. Next, the parameters are converted into parameters P(H,S,V), after which the parameter V is extracted from P. Next, the parameter V of the pixel P is changed to delta d, V*=V +/- d, for example, d increases if V<0.5 and decreases if V is different. Moreover, if the pixels behind the graphic elements (11, 12) of the CM (10) have a dark tint, the program logic of the CM formation algorithm (10) increases the brightness, if it is light, it decreases. Next, the pixel parameters P(H,S,V*) are converted to P(R,G,B) in the image. Due to this, each pixel of the label has a brightness delta, imperceptible to the human eye, but recognized later by the decoder. This allows you to adjust the stability of the formed protective layer (101) from the marks and at the same time make it indistinguishable for the user.

Next, consider the process of decoding information protected by the CM shown in Fig. 4-6.

Fig. 4 illustrates a flowchart of a method (300) for decoding information from a captured image. At the first stage (301), the computing module (for example, a processor) receives an image (410), which was made using an external device (400) and contains part or all of the information presented on the screen (111) of the VU (110), as follows shown in Fig. 5. Next, the resulting image (410) goes through the processing step (302). In the first stage of step (302), the image is decomposed pixel-by-pixel into threshold images (step 3021) or thresholds. The threshold is the arithmetic mean of the RGB values of the image channels. Since the CM (10) is a homogeneous object that has the same delta, which causes it to appear on the image at a certain threshold value in an arbitrary region of the decoded image (410), which allows you to well display artifacts in the photograph caused by photographing the screen (111) , which has a specific refresh rate that does not match the aperture opening time of the device (400) that is used to capture photos or videos. In this case, the CM itself (10) is an artifact. Each threshold image is formed as an average spectrum of the RGB palette of the corresponding threshold P, on the basis of which M - Ap images are created, on which each image pixel satisfies the condition r-dr, ,.,ρ + rr}, where P is the current threshold, 4p e N - threshold detection delta.

Also, in another particular aspect, the formation of threshold images can be performed by a method in which each pixel is converted to the HSV format, after which the saturation index S is extracted, on the basis of which N - Δs images are created, in which each image pixel satisfies the condition ^{s eR:} is - As,..., s + As}, where S is the pixel saturation, As e R is the saturation detection delta.

In another way, when forming threshold images, pixels are extracted from the processed image, each pixel is converted to the HSV format, the brightness index V is extracted, on the basis of which N - Av images are created, on which each image pixel satisfies the ^{VeR condition:} {v-Δν,. .., v + Av}, where V - pixel brightness, Av GR - brightness detection delta.

At the next stage (3022) of decoding, graphic elements (11, 12) that form one or more CMs (10) that have entered the image (410) are identified. During processing, the location of elements (11, 12) is determined for subsequent accumulation and formation of their sequence for stage (3023), at which information is restored from the CM based on the bit sequence formed by the identified graphic elements (11, 12). The preprocessed image is created in the following way. At each threshold, the locations of the recognized elements (11) and (12) are marked, they are stored in the device memory (RAM), while the designations for both elements (11) and (12) are different, for example, for (11) - red geometric figure (square), for (12) - green. On the new canvas (image), all the elements found above (11,12) are placed from memory, and the placement occurs in accordance with the color scheme described above. In the event of a collision, the choice is made of the geometric figure that has the largest number of times in a given place.

As shown in FIG. 6, as a result of decoding the preprocessed image (410), the CM (10) is determined on the protective layer (101) and information is extracted from the elements (11, 12), encoded in the bit sequence of the CM (10), by which the ownership of the VU (110) to a particular employee is established , as well as any other additional information encoded in the CM (10).

In FIG. 7 is a perspective view of a computing device (500) suitable for performing methods (200, 300). Device (500) may be, for example, a server or other type of computing device that can be used to implement the claimed technical solution. Including being part of a cloud computing platform.

In the general case, the computing device (500) contains one or more processors (501) connected by a common information exchange bus, memory facilities such as RAM (502) and ROM (503), input/output interfaces (504), input/output devices ( 505), and a device for networking (506).

The processor (501) (or multiple processors, multi-core processor) may be selected from a variety of devices currently widely used, such as Intel™, AMD™, Apple™, Samsung Exynos™, MediaTEK™, Qualcomm Snapdragon™, etc. . The processor (501) can also be a graphics processor such as Nvidia, AMD, Graphcore, etc.

RAM (502) is a random access memory and is designed to store machine-readable instructions executable by the processor (501) to perform the necessary data logical processing operations. The RAM (502) typically contains the executable instructions of the operating system and associated software components (applications, program modules, and the like).

The ROM (503) is one or more persistent storage devices such as a hard disk drive (HDD), a solid state drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media (CD-R /RW, DVD-R/RW, BlueRay Disc, MD), etc.

Various types of I/O interfaces (504) are used to organize the operation of device components (500) and organize the operation of external connected devices. The choice of appropriate interfaces depends on the particular design of the computing device, which can be, but not limited to: PCI, AGP, PS/2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS/Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

To ensure user interaction with the computing device (500), various means (505) of I/O information are used, for example, a keyboard, a display (monitor), a touch screen, a touchpad, a joystick, a mouse, a light pen, a stylus, a touchpad, trackball, speakers, microphone, augmented reality tools, optical sensors, tablet, indicator lights, projector, camera, biometric identification tools (retina scanner, fingerprint scanner, voice recognition module), etc.

The networking means (506) enables data communication by the device (500) via an internal or external computer network, such as an Intranet, Internet, LAN, and the like. As one or more means (506) can be used, but not limited to: Ethernet card, GSM modem, GPRS modem, LTE modem, 5G modem, satellite communication module, NFC module, Bluetooth and / or BLE module, Wi-Fi module and others

Additionally, satellite navigation tools in the device (500) can also be used, for example, GPS, GLONASS, BeiDou, Galileo. The submitted application materials disclose preferred examples of the implementation of the technical solution and should not be interpreted as limiting other, particular examples of its implementation that do not go beyond the scope of the requested legal protection, which are obvious to specialists in the relevant field of technology.

Claims (15)

1. A computer-implemented method for protecting data displayed on the screen of a computing device (CD), performed by a processor and containing the steps at which a digital label (DM) is formed in the form of a data block containing encoded information at least identifying the user of the CC, moreover the encoded information is presented in the form of dynamic graphic elements that scan the area of their placement on the screen of the VU; form an invisible protective layer, consisting of a set of mentioned data blocks, covering most of the display area of the screen WU; and overlaying the protective layer on the display area of the screen RT. 2. The method according to claim 1, characterized in that information about the date and time is additionally encoded in the CM. 3. The method according to claim 1, characterized in that when scanning with dynamic graphic elements, the color of the pixels in the area of their placement is analyzed. 4. The method according to claim 3, characterized in that, based on the color of the pixel, the brightness of the graphical element is adjusted. 5. The method according to claim 4, characterized in that the correction is carried out based on the parameters of each pixel in the area where the graphic element is located in the RGB color scheme. 6. The method according to claim 5, characterized in that the RGB color scheme is converted to HSV with the calculation of the parameters of tone H, saturation S and brightness V. 7. The method according to claim 6, characterized in that the obtained brightness parameter V is changed by the value A. 8. The method according to claim 7, characterized in that the resulting brightness parameter V _A together with the parameters H and S are converted into an RGB color scheme. 9. The method according to claim 1, characterized in that the dynamics of the display of graphic elements is selected based on the refresh rate of the screen of the VU. 10. A computer-implemented method for processing protected data, performed by a processor and comprising the steps of obtaining an image containing at least a portion of a screen image with information protected using the method according to any one of claims 1 to 9; decoding information on the basis of the CM from the obtained image, in which the pixel-by-pixel decomposition of the image into threshold images is performed; perform the identification of graphic elements that form the CM; restoring information from the CM based on the bit sequence formed by the graphic elements. 11. The method according to claim 10, characterized in that each threshold image is formed as an average spectrum of the RGB palette of the corresponding threshold P, on the basis of which Μ-Δρ images are created, in which each image pixel satisfies the condition ^Ρ(ΞΝ 1 ^ρΔ Ρ' ρ + δρ), where P is the current threshold, ^δ ρ ^{e Ν} is the threshold detection delta. 12. The method according to claim 10, characterized in that during the formation of threshold images, each pixel is converted to the HSV format, after which the saturation index S is extracted, on the basis of which N-As images are created, in which each image pixel satisfies the condition s e R: {s - As, , s + As}, where S is the saturation of the pixel, As e r is the saturation detection delta. 13. The method according to claim 10, characterized in that when generating threshold images, pixels are extracted from the processed image, each pixel is converted to HSV format, the brightness index V is extracted, based on which N-Av images are created, in which each image pixel satisfies condition ^VeR {v-Δν, , ν + Δν}, where V is the brightness of the pixel, Δν e r is the brightness detection delta. 14. A system for protecting data displayed on the screen of a computing device, comprising at least one processor and at least one memory containing machine-readable instructions, which, when executed by the processor, carry out the method according to any one of claims 1-9. 15. A system for processing secure data, comprising at least one processor and at least one memory containing machine-readable instructions, which, when executed by the processor, carry out the method according to any one of claims 10-13.