EA043227B1 - METHOD AND SYSTEM FOR PROTECTING DIGITAL INFORMATION DISPLAYED ON THE SCREEN OF ELECTRONIC DEVICES - Google Patents

Description

Technical field

The claimed technical solution relates to the protection of digital data, in particular confidential and sensitive information displayed on the screen of an electronic device, through the introduction of digital labels (DM).

State of the art

The use of digital media in the field of digital information protection is a common solution in which encoded information is embedded in the image, which makes it possible to identify its ownership or the person responsible for its leakage and/or unauthorized access.

As a rule, such approaches use a given graphic element or an image area containing a CM. Moreover, such a label can be both distinguishable and indistinguishable for the human eye. One example of such technology is steganography.

An analogue of the proposed solution is the principle of forming a protective layer based on the CM, disclosed in US patent 9239910 (Markany Inc, 01/19/2016). The solution is to create an invisible protective layer consisting of digital marks, which is used as a background layer, displayed on the device screen and invisible to the user.

The disadvantage of the existing approach is its lack of efficiency, due to the fact that a CM is used to form the protective layer, which is a text or graphic primitive selected from the database and used to subsequently generate space filling. This leads to the fact that such formation of the layer becomes sensitive to quality and during the subsequent capture of the image on the screen using an external device, for example, a smartphone or camera camera, when changing the angle or capturing part of the screen with a protective layer, subsequently removing the CM and establishing the fact of leakage data becomes difficult or impossible.

The essence of the invention

The proposed approach allows solving the technical problem, which consists in reducing the stability (robustness) of the digital data protection method when they are fixed by external means from different angles and image quality, which critically affects the subsequent decoding of data from digital labels.

The technical result consists in increasing the efficiency of protecting information on the screen of devices, by improving the stability of recognition of embedded digital labels in the form of a fractal pattern that forms a protective layer.

The claimed result is achieved through the implementation of a computer-implemented method for protecting data displayed on the screen of computing devices (CD), performed using a processor and containing the steps at which a digital label (DM) is formed in the form of a graphical interface element of a given bit dimension, containing at least as far as data identifying the VU and / or the user of the VU, and fiducial points;

form a transparent protective layer that is not perceived by the human eye, which is formed by placing the CM in the form of a fractal pattern that fills the entire space of the screen VU;

the formed protective layer is superimposed on the image displayed on the WU screen.

In one of the particular examples of the implementation of the method, the CM is formed by coding in the contrast channels of at least one color scheme. In another particular example of the implementation of the method, the fractal pattern of the layer is formed by the recursive method of constructing the CM.

In another particular example of the implementation of the method, the CM has at least one nesting level.

In another particular example of the implementation of the method, the CM additionally contains time information.

In another particular example of the implementation of the method, data identifying the user is selected from the following group: personnel number, name, photograph, or combinations thereof. In another particular example of the implementation of the method, the data identifying the WU is selected from the following group: IP address, MAC address, unique identifier, or combinations thereof.

The claimed technical result is also achieved through a computer-implemented method for identifying data protected using the above method, which is performed using a processor and includes the steps at which an image is obtained containing at least part of the information displayed on the screen of the WU;

carry out pre-processing of the received image, during which it processes the image geometry, perform channel correction in at least one color palette;

the preprocessed image is decoded, during which the reference points of the CM are identified and information is extracted from the CM.

In one of the particular examples of the implementation of the method, at the stage of processing the image geometry, its rotation and perspective correction are performed.

- 1 043227

In another particular example of the implementation of the method, perspective correction performs pixel-by-pixel binary interpolation of image channels of at least one color scheme and corrects the image size to maintain its proportions. In another particular example of the implementation of the method, the image is received from an external device.

Each of the above methods may also be implemented by a computer system comprising at least one processor and at least one memory storing machine readable instructions that, when executed by the processor, perform any of the above methods.

Brief description of the figures

Fig. 1 illustrates the general principle of the claimed solution.

Fig. 2 illustrates a block diagram of the claimed data protection method.

Fig. 3A illustrates an example of a fractal guard layer.

Fig. 3B illustrates an example of layer placement on a device screen.

Fig. 4 illustrates a flow diagram of a method for decoding information from a digitally protected image.

Fig. 5 illustrates an example of capturing an image of information from a screen of a device.

Fig. 6 illustrates the principle of decoding a DM from an image.

Fig. 7 illustrates a general view of the computing device.

Implementation of the invention

In FIG. 1 shows the general concept of the technical implementation of the claimed solution. The protection of sensitive and/or confidential information displayed on the screen (111) of the computing device (CD) (110) of the user is carried out by introducing the CM (10), into which the relevant information is encoded for the subsequent determination of the location and responsible person for unauthorized receipt of information outside the protected perimeter of the infrastructure, for example, by photographing it, filming it or capturing (screenshot) an image on the VU screen by external devices (smartphone, camera, etc.), including with subsequent printing.

As shown in FIG. 2, the claimed method (200) for protecting digital information contains a number of successive steps. At the first stage (201) the formation of the CM (10) is carried out. Any type of information, such as text, graphic, or combinations thereof, can be embedded in the CM (10) using algorithms for binary (binary) data coding. CM (10) is formed from a combination of two contrasting colors. As a rule, the data embedded in the CM (10) is necessary to identify the VC (110) or directly the user of this device, for example, an employee who has access to sensitive information. Such data can be a personnel number, name, user photo, IP address, MAC address, unique identifier of the VU. This information may be used singly or in any combination. Additionally, information about time and/or date can also be encoded, for example, the time of formation of the CM (10), the current date. The date and time information may be dynamically changed to include up-to-date information during the coding of the CM (10).

CM (10) is a static element of the graphical interface of a given bit dimension. In addition to information for identifying the fact of photographing information on the screen (111), one or more reference points (11) are also introduced into the CM (10), which are necessary for the subsequent identification of the location of the CM (10).

Further, from the CM (10) at the stage (202) a protective layer (101) is formed in the form of a fractal principle of placement of the CM (10), filling the entire screen space (111). The number of CMs (10) and the principle of fractal placement are determined based on the screen resolution (111) and the dimension of the CM (10). As a rule, CMs (10) are chosen to be of equal dimension, but may also have an alternating order from placement to form a layer (101), which also does not violate the technical embodiment of the claimed solution in terms of subsequent data identification.

The protective layer (101) is formed invisible (transparent) so as not to be visible to the human eye and to hide the fact that data protection has been applied on the screen (111). The transparency of the layer (101) of the created fractal pattern can be formed by various ways of adjusting graphic images, for example, by adjusting the opacity of the image (opacity), using the Alpha channel in the RGBA palette, using the Alpha channel in the HSLA palette. The selection of parameters for forming an invisible layer (101) is performed to overcome the discrimination threshold (https://en.wikipedia.org/wiki/Justnoticeable_difference). making it invisible to the average user.

After the formation of the protective layer (101) at step (203), it is superimposed on the display area of the screen (111) of the VU (110), so that the entire screen coverage area is covered by the superimposed layer (101). The application of this approach allows to determine the value of the CM (10) in any part of the screen (111) regardless of the coordinate or scale of the screenshot or photocopy.

In FIG. 3A-3B show an example of a formed protective layer (101) based on a plurality of CMs (10) with the corresponding placement of fiducial points (11). The presented example of the CM (10) can be created using information encoding in contrast channels of the color scheme, for example, RGB, HSL, ARGB, etc. The fractal design of the layer pattern (101) can be formed by the recursive method of constructing (arranging) the CM (10) and have one or more levels of nesting (recursion) of the CM (10) into each other.

As mentioned above, in the example shown in FIG. 3A, when forming the CM (10), two contrasting colors are used, one of which represents the background of the CM (10) - logical 0, and the second is contrasting from the background color, representing logical 1. The creation of the protective layer (101) starts from the 0th recursion level, at which the background of the CM (10) is selected larger than the displayed area (111) on the screen (110). After that, the recursion cycle begins, during which the size of the rectangle describing one bit of the CM (10) (hereinafter referred to as the bit rectangle) is determined in accordance with the system of equations ixy = 1x'y' x x' I Y ~ Y' where x is the length CM background, y is the width of the CM background, x' is the length of the bit rectangle, y' is the width of the bit rectangle, 1 is the length of the bit array of the encoded message. Further, the CM (10) begins to be filled with bit rectangles, where, as mentioned above, 0 is the color of the rectangle similar to the background of the label, 1 is the color of the rectangle contrasting with the background of the label.

The CM (10) is filled with all bits (over the entire CM background), during which the n-th level label is formed. The recursion cycle is repeated anew, for the next CM size (10) one bit rectangle of the current CM (10) is taken. The cycle is repeated until the size of the CM (10) is equal to 1x1 pixel. CMs (10) in the form of bit rectangles are traversed until all CMs (10) at all recursion levels reach the end of the recursion.

Next, consider the process of decoding information protected by the CM shown in Fig. 4-6. Fig. 4 illustrates a flowchart of a method (400) for decoding information from a captured image.

At the first stage (301), the computing module (for example, a processor) receives an image (410), which was made using an external device (400) and contains part or all of the information presented on the screen (111) of the VU (110), as follows shown in Fig. 5.

Next, the resulting image (410) goes through a processing step (302), during which a geometric transformation of the image is performed, in particular rotation and/or perspective correction. Perspective correction is performed in two stages: pixel-by-pixel binary interpolation of the color channels of the image (for example, RGB) to change the perspective of the image and correction of the image size to maintain proportions according to the formula topWidth + bottomWidth targetWidth =----- -l 2 deltaX;

leftHeight + rightHeight targetHeight =---------------l· 2 deltaY, where deltaX is the length of the horizontal projection of the segment between the intersection points of the quadrilateral diagonals and its centroid, deltaY is the length of the vertical projection of the segment between the intersection points diagonals of the quadrangle and its centroid, topWidth - length of the horizontal projection of the top face of the quadrilateral, bottomWidth - length of the horizontal projection of the bottom face of the quadrilateral, leftHeight - length of the vertical projection of the left face of the quadrilateral, rightHeight - length of the vertical projection of the right face of the quadrilateral.

Next, the chroma/saturation (saturation) of the resulting image is changed in at least one color palette to increase the contrast of the image as a whole and detect the presence of the CM (10) in a particular area of the image (410).

After that, the mutual correction of the color channels of the received image (410) is carried out to highlight the contrasting colors selected during the coding of the CM (10). One of the contrasting colors selected during coding of the CM (10) is removed in each pixel of the resulting image.

At step (303), the position of the CM (10) is determined using the fiducial points (11) identified in the image (410). The centers of reference points (11) are determined, around which a circle of a given radius is constructed, calculated from the radius of the reference points and the length of the encoded message, on the perimeter of which the centers of other reference points are determined (11). From the angle of the segment between two centers to the horizontal and the length of this segment, the angle of inclination and the size of the CM (10) are calculated, respectively.

- 3 043227

From the known length of the message and the known size of the CM (10), the size of the bit rectangle of the CM (10) is calculated.

As shown in FIG. 6, as a result of decoding the preprocessed image (410), the CMs (10) on the guard layer (101) are determined at all levels of recursion (depending on the quality of the input image).

Steps (302)-(303) are performed multiple times at different saturation change settings and color changes. The result is the values that are most often repeated for different parameters. This ensures reliable decoding of the color of each pixel of the resulting image in accordance with the laws of normal distribution at various correction parameters.

After reliable determination of the color of each pixel in the resulting image (410), the background color of the given CM is determined by the color of the reference points (11), and the color of the given bit rectangle is determined by the color of the reference point of the CM inscribed in the bit rectangle of this CM. Having information about the color of each bit rectangle in the given CM and the background color of the given CM, a bit array of the encoded message is constructed.

From the bit array of the encoded message, knowing the encoding algorithm of this message, the bit array is converted into useful information, encoded in the CM.

This principle of protection and subsequent decoding of information from the protective layer (101) makes it possible to more effectively recognize the encoded information from the CM (10) and establish the VU (110) and/or the person responsible for the leakage of confidential and/or sensitive information.

In FIG. 7 is a perspective view of a computing device (500) suitable for performing methods (200, 300). Device (500) may be, for example, a server or other type of computing device that can be used to implement the claimed technical solution. Including being part of a cloud computing platform.

In the general case, the computing device (500) contains one or more processors (501) connected by a common information exchange bus, memory facilities such as RAM (502) and ROM (503), input/output interfaces (504), input/output devices ( 505), and a device for networking (506).

The processor (501) (or multiple processors, multi-core processor) may be selected from a variety of devices currently widely used, such as Intel™, AMD™, Apple™, Samsung Exynos™, MediaTEK™, Qualcomm Snapdragon™, etc. . The processor (501) can also be a graphics processor such as Nvidia, AMD, Graphcore, etc.

RAM (502) is a random access memory and is designed to store machine-readable instructions executable by the processor (501) to perform the necessary data logical processing operations. The RAM (502) typically contains the executable instructions of the operating system and associated software components (applications, program modules, and the like).

The ROM (503) is one or more persistent storage devices such as a hard disk drive (HDD), a solid state drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media (CD-R /RW, DVD-R/RW, BlueRay Disc, MD), etc.

Various types of I/O interfaces (504) are used to organize the operation of device components (500) and organize the operation of external connected devices. The choice of appropriate interfaces depends on the particular design of the computing device, which can be, but not limited to: PCI, AGP, PS/2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS/Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc. Various means (505) of I/O information are used to ensure user interaction with the computing device (500), for example, a keyboard, a display (monitor), a touch screen, a touchpad, a joystick, a mouse, a light pen, a stylus, a touchpad, a trackball , speakers, microphone, augmented reality tools, optical sensors, tablet, light indicators, projector, camera, biometric identification tools (retinal scanner, fingerprint scanner, voice recognition module), etc.

The networking means (506) enables the communication of data by the device (500) via an internal or external computer network, such as an Intranet, Internet, LAN, and the like. As one or more means (506), an Ethernet card, a GSM modem, a GPRS modem, an LTE modem, a 5G modem, a satellite communication module, an NFC module, a Bluetooth and/or BLE module, a Wi-Fi module, and others

Additionally, satellite navigation tools in the device (500) can also be used, such as GPS, GLONASS, BeiDou, Galileo. The submitted application materials disclose preferred examples of the implementation of the technical solution and should not be interpreted as limiting other, particular examples of its implementation that do not go beyond the scope of the requested legal protection, which are obvious to specialists in the relevant field of technology.

Claims (13)

1. A computer-implemented method for protecting data displayed on the screen of computing devices (CD), performed using a processor and containing the steps at which a digital label (DM) is formed in the form of a graphical interface element of a given bit dimension, containing at least data identifying VU and / or user VU, and fiducial points; form a transparent protective layer that is not perceived by the human eye, which is formed by placing the CM in the form of a fractal pattern that fills the entire space of the screen VU; the formed protective layer is superimposed on the image displayed on the WU screen. 2. The method according to claim 1, characterized in that the CM is formed by coding in contrast channels of at least one color scheme. 3. The method according to claim 1, characterized in that the fractal pattern of the layer is formed by the recursive method of constructing the CM. 4. The method according to claim 1, characterized in that the CM has at least one nesting level. 5. The method according to claim 1, characterized in that the CM additionally contains time information. 6. The method according to claim 1, characterized in that the data identifying the user is selected from the following group: personnel number, name, photograph, or combinations thereof. 7. The method according to claim 1, characterized in that the data identifying the WU is selected from the following group: IP address, MAC address, unique identifier, or combinations thereof. 8. A computer-implemented method for identifying data protected using the method according to any one of claims 1 to 7, performed by a processor and comprising the steps of obtaining an image containing at least part of the information displayed on the screen of the WU; carry out pre-processing of the obtained image, during which the image geometry is processed, the channels are corrected in at least one color palette; the preprocessed image is decoded, during which the reference points of the CM are identified and information is extracted from the CM. 9. The method according to claim 8, characterized in that at the stage of processing the image geometry, its rotation and perspective correction are performed. 10. The method according to claim 9, characterized in that the perspective correction performs pixel-by-pixel binary interpolation of the image channels of at least one color scheme and corrects the image size to maintain its proportions. 11. The method according to claim 8, characterized in that the image is obtained from an external device. 12. The system for protecting data displayed on the screen of the VU, containing at least one processor and at least one memory storing machine-readable instructions, which, when executed by the processor, carry out the method according to any one of claims 1-7. 13. The system for identifying data displayed on the screen of the VU, containing at least one processor and at least one memory storing machine-readable instructions, which, when executed by the processor, carry out the method according to any one of claims 9-11.