CN117574336A - Real-time screen watermarking method and system for tracking screen shooting leakage - Google Patents
Real-time screen watermarking method and system for tracking screen shooting leakage Download PDFInfo
- Publication number
- CN117574336A CN117574336A CN202311527940.1A CN202311527940A CN117574336A CN 117574336 A CN117574336 A CN 117574336A CN 202311527940 A CN202311527940 A CN 202311527940A CN 117574336 A CN117574336 A CN 117574336A
- Authority
- CN
- China
- Prior art keywords
- watermark
- synchronous
- information
- template
- screen
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000001360 synchronised effect Effects 0.000 claims abstract description 139
- 230000004044 response Effects 0.000 claims abstract description 53
- 239000000872 buffer Substances 0.000 claims abstract description 28
- 230000006870 function Effects 0.000 claims description 29
- 238000001514 detection method Methods 0.000 claims description 18
- 238000010586 diagram Methods 0.000 claims description 16
- 230000000873 masking effect Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 230000009466 transformation Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000001228 spectrum Methods 0.000 claims description 5
- 108091026890 Coding region Proteins 0.000 claims description 4
- 239000000969 carrier Substances 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 description 7
- 229920002430 Fibre-reinforced plastic Polymers 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 239000011151 fibre-reinforced plastic Substances 0.000 description 2
- 238000002347 injection Methods 0.000 description 2
- 239000007924 injection Substances 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000012148 binding buffer Substances 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/10—Protecting distributed programs or content, e.g. vending or licensing of copyrighted material ; Digital rights management [DRM]
- G06F21/16—Program or content traceability, e.g. by watermarking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K19/00—Record carriers for use with machines and with at least a part designed to carry digital markings
- G06K19/06—Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code
- G06K19/06009—Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code with optically detectable marking
- G06K19/06037—Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code with optically detectable marking multi-dimensional coding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0021—Image watermarking
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Technology Law (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- General Engineering & Computer Science (AREA)
- Editing Of Facsimile Originals (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a real-time screen watermarking method and a system for tracking secret divulgence of screen shooting, which can be used for obtaining evidence of a photo of a screen shot by stealth. The method uses a preset watermark template consisting of an information watermark and a synchronous watermark, and uses key information such as user ID, IP address and the like as watermark information to encode the corresponding watermark template through a QR code encoding mode and inverse Fourier transform. The minimum perceived difference model is adopted, and a pre-generated watermark template is directly embedded into a rear buffer zone of a desktop window manager exchange chain, so that the imperceptibility of the watermark is ensured, and the real-time refreshing of the watermark is realized. And realizing automatic perspective correction by a synchronous response indexing method, and positioning an information watermark template embedding region. Finally, the information watermark is extracted from the discrete fourier transform domain of the noise component. The method can effectively resist various screen candid photographing means, and realize high-frequency refreshing of the watermark and robustness of watermark evidence taking under the condition that normal display of the screen is not affected.
Description
Technical Field
The invention belongs to the technical field of information security, and particularly relates to a real-time screen watermarking method and system for tracking screen shooting leakage based on a preset watermarking template.
Background
With the popularity of digital office and smart phones, information acquisition and transmission become very convenient, but new security problems also occur. Unauthorized use of smart phones to capture sensitive content on a screen has become a common means of information leakage. Conventional information protection techniques cannot prevent such screen shots from leaking. To solve this problem, screen watermarking techniques have been proposed for tracking leakage sources. The visible watermark can affect the user experience and is easily erased. Therefore, the imperceptible screen watermarking technique is of more practical value.
The existing imperceptible image watermarking technology is mainly aimed at conventional image processing attacks, but is not designed for screen shooting attacks. The distortion caused by screen shots is different from print scanning or print camera attacks. A number of screen shot robust watermarking methods have been proposed, which can be classified into video and still image methods according to watermark carrier. Video watermarking methods are used to track illegal camera recordings, but are not suitable for capturing scenes of screen content.
The still image method can be classified into a limited and an unlimited method according to the manner of perspective distortion processing. A limited approach requires capturing the entire host image and performing manual perspective correction to extract the watermark. These methods have limited applicability and require prior knowledge of the shape of the host image. The unlimited method can automatically extract the watermark under perspective distortion, but has higher algorithm complexity, is not suitable for real-time application, and can only protect the data embedded with the watermark in advance.
Disclosure of Invention
The invention aims to: aiming at the problems of information leakage safety caused by popularization of digital office and smart phones in modern office environments and based on unauthorized capturing of sensitive content on a screen, the invention aims to provide a real-time screen watermarking method and system for tracking screen shooting leakage, and aims to realize protection of screen content and tracking of leakage sources.
The technical scheme is as follows: in order to achieve the above purpose, the invention provides a real-time screen watermarking method for tracking the leakage of screen shooting, which comprises the following steps:
generating a watermark template, including an information watermark template, a first synchronous watermark template and a second synchronous watermark template; the information watermark template is generated by inverse Fourier transform after being coded by user information QR comprising user ID, IP address and operation time; the first synchronous watermark template and the second synchronous watermark template are generated by preset synchronous watermark information through inverse Fourier transform, wherein the information of the first synchronous watermark template and the second synchronous watermark template are respectively embedded in the second radius and the angle of the amplitude spectrum of the Fourier transform domain, and the preset radius and the preset angle of the first quadrant and the preset angle of the third quadrant;
according to the screen size, the information watermark template and the synchronous watermark template are adaptively combined to generate a screen watermark template;
acquiring a function address of a switching chain switching function in a desktop window manager process, acquiring a rear buffer address of the desktop window manager switching chain, calculating a JND threshold value of rear buffer data by adopting a simplified minimum perceived difference model, and directly embedding a screen watermark template generated in advance into a rear buffer according to the threshold value;
when a photo image of a shot screen is acquired, detecting a multi-scale synchronous watermark template to obtain the positions of a first synchronous watermark template and a second synchronous watermark template, realizing automatic perspective correction based on the positions of the synchronous watermark templates, positioning an information watermark template embedding area, and obtaining embedded user information.
Further, the watermark template generation comprises the following steps:
generating a QR code from user information by using a QR code algorithm, and recording code elements in a QR code coding region into a binary watermark information sequence W line by line 1 ；
Determining the information watermark block size L according to the screen size MXN 1 ×L 1 And a synchronization watermark block size L 2 ×L 2 ，L 1 >L 2 ；
Will L 1 ×L 1 The zero matrix of (a) is regarded as the fourier transform domain amplitude map of the information watermark block, based onDetermining the location of embedded watermark information based onEmbedding watermark information, and generating an information watermark template through iDFT; wherein W is 1 (j) Represents W 1 The j-th element, l 1 Represents W 1 Length, R represents intermediate frequency radius, M 1 (x, y) represents the amplitude coefficient, k after embedding the watermark 1 Is constant and represents watermark embedding strength;
will L 2 ×L 2 Regarding the zero matrix of the synchronous watermark block as a Fourier transform domain amplitude diagram, setting at least 15 coefficients as carriers of synchronous watermark information at least 3 radii and 5 equally spaced angles preset in the second and fourth quadrants based on M 2 (x,y)＝k 2 Embedding watermark information, generating a first synchronous watermark template a through iDFT, setting at least 15 coefficients as carriers of the synchronous watermark information at least 3 radiuses and 5 equally-spaced angles preset in the first quadrant and the third quadrant, and setting the coefficients based on M 2 (x,y)＝k 2 Embedding watermark information, and generating a second synchronous watermark template b through iDFT; wherein M is 2 (x, y) represents the amplitude coefficient, k after embedding the watermark 2 Is constant and represents watermark embedding strength.
Further, the self-adaptive combination of the information watermark template and the synchronous watermark template generates a screen watermark template, which is specifically as follows:
let the screen size be MxN, according to the formulaDetermining the position of an information watermark template, wherein->And->Respectively representing the number of rows and columns which can be accommodated, L 1 Representing the side length of the information watermark template; m is m 1 And n 1 Respectively represent all c 1 ×c 2 An index of rows and columns in the information watermark;
based on the formulaDetermining a position of a first synchronous watermark template a; wherein m is 2 ，n 2 Index representing rows and columns in a synchronized watermark, L 2 Representing the side length of the synchronous watermark template;
three second synchronous watermark templates b are uniformly embedded between every two first synchronous watermark templates a.
Further, the method for acquiring the address of the rear buffer area of the desktop window manager exchange chain comprises the following steps: the method comprises the steps that a Present function address and a GetFrameStatisticInternational function address are obtained in a dynamic link library to be injected in a continuous matching mode of machine codes; the Present function performs HOOK in an inline HOOK mode; and calculating a switching chain address, wherein a switching chain base address is directly obtained by the address offset of the Present function, a switching chain constant is obtained by the address offset of the GetFrameStatisticInternational function, a switching chain pointer address is obtained according to the switching chain constant offset, and then a back buffer area address to be displayed is obtained.
Further, when the simplified minimum perceived difference model calculates the JND threshold, the following Sobel operators are adopted to calculate the horizontal gradient G of the two-dimensional image of the buffer zone respectively 1 Vertical gradient G 2 Thereby deriving a contrast masking threshold based on the gradient and the background brightness;
finally based on the formula JND (x, y) =p L (x,y)+P c (x,y)-a*min(P L (x,y),P c (x, y)) calculate JND threshold, where P L (x,y)、P c (x, y) represents the brightness masking threshold and the contrast masking threshold at the position (x, y), respectively, a being a coefficient.
Further, the two-dimensional image object I in the post-write buffer is calculated based on the following formula b ：
I b (x,y)＝Bg(x,y)+M w (x,y)*JND(x,y)*η
Wherein B is g Represents average background brightness, M w Representing a binarized watermark template, η representing watermark embedding strength.
Further, the detection of the multi-scale synchronous watermark template comprises the following steps:
converting the photographed image I into a luminance image I a ；
Image I a Scaling to multiple scales and computing image I at each scale a Is the noise component I of (1) n ；
On each scale of the image, using a window of a predetermined pixel size, at predetermined intervals, for I n Detecting the sliding blocks one by one;
in the Fourier transform domain amplitude diagram of each slider i, a first quadrant preset angle range and a second quadrant preset angle range are recorded, and the maximum specified number of amplitude values in the preset radius range area are respectively recorded as V according to the amplitude values in the second quadrant or the first quadrant l (i) And V r (i) Based on the formulaCalculating a slider window synchronization response value, wherein R s,i Synchronous response index, N, representing the corresponding detection window of the ith slider on scale s l,1 And N r,1 Respectively represent V l (i) And V r (i) The number of amplitude values is recorded;
determining an optimal scale according to the synchronous response indexes of each scale, and determining synchronous response points of each window in the optimal scale;
searching all synchronous response points, finding all point sets of information watermark positions which can be used for correction, wherein each point set comprises four points which can form a quasi-rectangle, cutting out an area formed by the point sets, and performing perspective transformation;
and calculating a synchronous response index in the sheared area by using the sliding window to determine a synchronous response point of the second synchronous watermark template.
Further, the automatic perspective correction is realized based on the synchronous watermark template position, which comprises the following steps:
for synchronous response points representing a first synchronous watermark template and a second synchronous watermark template surrounding the message watermark template, reducing the interval of sliding windows in the area taking the first synchronous watermark template and the second synchronous watermark template as the center, and carrying out synchronous response detection; recording the maximum preset number of amplitude values within the preset radius and angle range of the second quadrant for the synchronous response points of the first synchronous watermark template; for a synchronous response point of the second synchronous watermark template, recording the maximum preset number of amplitude values within the preset radius and angle range of the first quadrant; the refined synchronization response index for each coordinate is calculated based on the following formula:
R a,i ＝sum(V a (i))
R b,i ＝sum(V b (i))
wherein V is a (i) And R is a,i The amplitude value set of the detection window corresponding to the ith slide block of the first synchronous watermark template and the accurate correction synchronous response index are respectively V b (i) And R is b,i The method comprises the steps that an amplitude value set of a detection window corresponding to an ith sliding block of a second synchronous watermark template is respectively and accurately corrected to a synchronous response index;
and performing perspective transformation on the coordinates of the region maximum fine correction synchronous response index, and taking the transformed coordinates as a fine correction result.
Further, the method for acquiring the embedded user information comprises the following steps:
extracting information blocks, using noise components of the brightness images, and converting the information blocks into DFT domains; recording maximum amplitude value V in 3 x 3 range with each watermark embedding position as center max (j) Wherein j represents the jth watermark bit; based on the formulaDetermining the extracted watermark information bit W' (j), wherein +.>Sum sigma T Respectively represent the mean and standard deviation, k, of all amplitude coefficient values within the embedded range 3 Is a constant; and performing QR decoding on W' (j) to obtain user watermark information.
The invention also provides a computer system, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program realizes the steps of the real-time screen watermarking method for tracking the screen shooting leakage when being loaded to the processor.
The beneficial effects are that: the method has the characteristics and technical advantages that: (1) The method is designed for real-time screen shooting scenes, and a preset watermark template is directly embedded into a rear buffer zone of a desktop window manager exchange chain in a mode of a HOOK exchange chain exchange function, so that the refreshing rate requirement of screen watermarks is met, and continuous screen content is ensured to be perceived by human eyes; (2) The watermark technology enhances the imperceptibility of the watermark through the minimum perceptible model, and does not obviously influence the visual experience of the user. It is more difficult to delete or modify by a leaky person than a visible watermark; (3) By adopting the self-adaptive embedding method, the watermark can be kept stable under various distortion conditions, and has good attack resistance; (4) The method is suitable for screen shooting scenes of various static images and videos. Whatever content is displayed on the screen, whether it is a picture or a video, the watermark protection provided by the method can be realized; (5) The specific position of the synchronous watermark is calculated by a synchronous response indexing method, the accurate positions of the two synchronous watermarks are obtained by synchronous response detection, a watermark extraction method which is more accurate and does not need manual perspective correction can be realized based on 16 position coordinate algorithms of the two synchronous watermarks, the watermark can be automatically extracted under perspective distortion, and complex operation is not needed.
Drawings
FIG. 1 is a general flow chart of a method of an embodiment of the present invention;
FIG. 2 is a watermark template generation flow diagram of an embodiment of the invention;
FIG. 3 is a watermark embedding flow diagram of an embodiment of the present invention;
FIG. 4 is a flow chart of watermark information extraction and detection according to an embodiment of the invention;
FIG. 5 is an exemplary diagram of watermark information QR codes in accordance with an embodiment of the present invention;
FIG. 6 is a template diagram of an information watermark and a synchronization watermark using an embodiment of the invention;
FIG. 7 is a schematic diagram of a watermark template using an embodiment of the invention;
FIG. 8 is a diagram of an example watermark template using an embodiment of the invention;
FIG. 9 is a screen watermark image using an embodiment of the invention;
fig. 10 is a captured image obtained using an embodiment of the present invention;
FIG. 11 is a synchronous response thermodynamic diagram using an embodiment of the present invention;
FIG. 12 is a watermark information extraction area screenshot using an embodiment of the invention;
fig. 13 is an exemplary diagram of an information watermark two-dimensional code generated using extraction of an embodiment of the present invention;
fig. 14 is a screenshot of extracted watermark information using an embodiment of the invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples.
The embodiment of the invention discloses a real-time screen watermarking method for tracking screen shooting leakage based on a preset template, a simplified JND algorithm, a HOOK function and a self-adaptive embedding method, which mainly comprises watermark template generation, watermark embedding, watermark extraction and detection, and comprises the following specific processes:
1) Generating a watermark template, wherein the watermark template comprises an information watermark template, a first synchronous watermark template a and a second synchronous watermark template b; and according to the screen size, the information watermark template and the synchronous watermark template are adaptively combined to generate the screen watermark template. The information watermark template is generated by inverse Fourier transform after being coded by user information QR comprising user ID, IP address and operation time; the first synchronous watermark template and the second synchronous watermark template are generated by the preset synchronous watermark information through inverse Fourier transform, wherein the information of the first synchronous watermark template and the information of the second synchronous watermark template are respectively embedded in the second radius and the angle of the amplitude spectrum of the Fourier transform domain, and the preset radius and the preset angle of the first quadrant and the preset angle of the third quadrant. The specific steps are shown in fig. 2, including:
step 11: acquiring and marking the input watermark information as W 0 ；
Step 12: according to W 0 Generating a QR code by using a QR code algorithm, and recording the code elements in the QR code coding region row by row as a binary watermark information sequence W 1 ；
Step 13: the screen size is MxN, and the information watermark block size is L based on the formula (1) 1 ×L 1 ；
L 1 ＝floor(min(M,N)*0.47) (1)
Step 14: will L 1 ×L 1 The zero matrix with the size is regarded as a Fourier transform domain amplitude diagram of the information watermark block, the position of the embedded watermark information is determined based on the formula (2), the watermark information is embedded based on the formula (3), and the iDFT generates an information watermark template; wherein L is 1 Represents the side length, W 1 (j) Represents W 1 The j-th element, l 1 Represents W 1 The length, R, represents the intermediate frequency radius, in this case the value (132,145,159,172), M 1 (x, y) represents the amplitude coefficient, k after embedding the watermark 1 Representing a constant representing the watermark embedding strength;
step 15: the screen size is MxN, and the synchronous watermark block size L is obtained based on the formula (4) 2 ×L 2 ；
L 2 ＝floor(min(M,N)*0.06) (4)
The generation L 1 And L 2 The scaling factor of (2) is set based on template typesetting and the size proportion of the information watermark and the synchronous watermark, and fine adjustment can be performed.
Step 16: will L 2 ×L 2 The zero matrix with the size is regarded as a Fourier transform domain amplitude diagram of the synchronous watermark block, the radii 20, 24 and 28 in the first quadrant of the DFT domain amplitude spectrum are selected, the angles between 35 and 55 degrees are taken as intervals, 15 coefficients are taken in total, 15 coefficients are taken at the corresponding positions of the third quadrant, watermark information is embedded based on a formula (5), and the iDFT generates a synchronous watermark template b; selecting 15 coefficients in total at the positions of radii 20, 24 and 28 in the second quadrant of the DFT domain amplitude spectrum, taking 5 degrees as intervals at the positions of angles of 125-145 degrees, taking 15 coefficients at the corresponding positions of the fourth quadrant, embedding watermark information based on a formula (5), and generating different synchronous watermark templates a by iDFT; wherein M is 2 (x, y) represents the amplitude coefficient, k after embedding the watermark 2 Representing a constant representing the watermark embedding strength;
M 2 (x,y)＝k 2 (5)
step 17: the screen size is MxN, and the information watermark position is determined according to formulas (6) and (7), wherein c 1 And c 2 Representing the number of rows and columns respectively which can be accommodated, the screen image can accommodate c in total 1 ×c 2 A personal information watermark; m is m 1 And n 1 Respectively represent all c 1 ×c 2 Row and column indices in the individual information watermarks. The remaining uncovered areas of the screen watermark template are filled with incomplete information watermarks.
Step 18: determining a synchronous watermark a position based on formula (8); wherein m is 2 ，n 2 Representation ofSynchronizing the row and column indices in the watermark.
Step 19: three synchronous watermarks b are uniformly embedded between every two synchronous watermarks a.
2) Watermark embedding, obtaining a function address of a switching chain switching function in a desktop window manager process, obtaining a rear buffer address of the switching chain of the desktop window manager, calculating a JND threshold value of rear buffer data by adopting a simplified minimum perceived difference model, and directly embedding a screen watermark template generated in advance into a rear buffer according to the threshold value. The specific steps are shown in fig. 3, including:
step 21, obtaining addresses of a Present function and a GetFrameStatisticInternational function in a dynamic link library to be injected in a continuous matching mode through a machine code;
step 22: the Present function performs HOOK in an inline HOOK mode;
step 23: calculating a switching chain address, wherein a switching chain base address can be directly obtained by the address offset of a Present function, a switching chain constant is obtained by the address offset of a GetFrameStatisticInternational function, a switching chain pointer address is obtained according to the switching chain constant offset, and then a rear buffer area address to be displayed is obtained;
step 24: copying the post-buffer texture to the temporary buffer based on the post-buffer address:
step 25: binding an OpenCV object with an OpenCL buffer object, and then writing texture data into the two-dimensional image object by using a texture data reading method of OpenCL;
step 26: calculating an average background luminance Bg of the two-dimensional image object Bg based on the formula (9) using OpenCV, wherein (x, y) represents a position of a pixel, h (n, m) represents a weighted 5×5 low-pass filter, and a weight coefficient is 32, as shown in the formula (10);
step 27: calculating a luminance masking threshold P based on the average background luminance and formula (11) L ；
Step 28: sobel operator Sob based on 5*5 template shown in formula (12) 1 And Sob 2 And equations (13) and (14), calculate the horizontal gradient G of the image 1 Vertical gradient G of image 2 ；
Step 29: calculating an image gradient G based on formula (15);
step 210: calculating functions alpha (x, y) and k (x, y) related to the background brightness Bg based on the formula (16) and the formula (17);
α(x,y)＝0.0001*Bg(x,y)+0.115 (16)
step 211: computing a contrast masking threshold P for the screen image bg based on equation (18) c Wherein γ represents a parameter for balancing objective quality and perceptual characteristics of the image;
P c (x,y)＝G(x,y)*γ*α(x,y)+k(x,y) (18)
step 212: masking the luminance by a threshold value P based on formula (19) L And a contrast masking threshold P c And combining, and calculating the JND threshold value of the image.
JND(x,y)＝P L (x,y)+P c (x,y)-0.3*min(P L (x,y),P c (x,y)) (19)
Step 213: computing a two-dimensional image object I based on equation (20) b Wherein M is w The method comprises the steps of representing a binarization watermark template, JND representing the minimum perceived difference, eta representing watermark embedding strength, and finally writing a two-dimensional image object into a binding buffer area;
I b (x,y)＝Bg(x,y)+M w (x,y)*JND(x,y)*η (20)
step 214: writing the processed data from the OpenCL buffer object back to the texture of the rear buffer to realize the updating after watermark drawing;
step 215: and writing the dynamic link library into a desktop window manager process in an injection mode, so that the screen watermark embedding process is operated in the desktop window manager.
3) And when the watermark extraction and detection are carried out and the photo image of the shot screen is obtained, the detection of the multi-scale synchronous watermark template is carried out, the positions of the first synchronous watermark template and the second synchronous watermark template are obtained, automatic perspective correction is realized based on the positions of the synchronous watermark templates, and the embedded user information is obtained by positioning the embedded area of the information watermark template. The specific steps are shown in fig. 4, including:
step 31: acquiring a shooting image I and converting the shooting image I into a brightness image I a ；
Step 32: image I a Scaling to 100%,90%,80%,70%, and 60% for a total of 5 scales, and calculating a panning brightness image I on each scale based on formula (21) i Is the noise component I of (1) n Wherein H is w Representing a 5 x 5 spatial domain wiener filter;
I n ＝I a -H w ·I a (21)
step 33: on each scale image, using a 86 x 86 pixel window, at 25 pixel intervals, for I n And detecting the sliding blocks one by one. When each window is detected, I will be n Switching to a DFT domain;
step 34: in the Fourier transform domain amplitude map of each slider i, the first quadrant 30 DEG to 60 DEG, the second quadrant 120 DEG to 150 DEG, and the maximum 15 amplitude values in the radius range 18 to 30 are recorded, and the values lying in the second quadrant or the first quadrant according to the 15 amplitude values are recorded as V respectively l (i) And V r (i) Calculating a slider window synchronization response value R based on equation (22) s,i Wherein R is s,i A synchronous response index, V, representing the ith detection window on the s-scale l (i) And V r (i) The number of the recorded amplitude values is respectively represented as N l,1 And N r,1 ；
Step 35: calculating a synchronous response index for each scale based on equation (23), wherein R s Synchronous response values, σ, representing the scale s (100%, 90%,80%,70% and 60%) s Representing the scale R s,i Standard deviation of (2);
R s ＝σ s *s 2 (23)
step 36: based on formula (24), calculate R s Maximum value R m Corresponding dimension s m 。
R m ＝max(R s ) (24)
Step 37: r is selected based on equation (25) m,i Greater than a set threshold T in the region m Peak point of (2) as synchronous response point P m,i WhereinR represents m,i Is the average value of (2);
step 38: retrieving all synchronization response points P m,i All sets of points for the location of the information watermark that can be used for correction are found as follows, each set of points containing four points that can form a rectangle-like shape. From P m,i In which points P are selected in descending order of their synchronisation response values a . At P m,i Finding a point P among the remaining points in the map b So thatThe angle with the X-axis is less than 35 degrees and has a minimum +.>Also, find a point P c Make->An angle with the Y-axis of less than 35 degrees and having a minimum valueFind a point P d So that->Between->And->Between (I)>Is greater than->And->But is smaller thanAnd->A kind of electronic device. If four points are successfully found, then at set S a Is recorded as the nth point setn represents the number of found point sets. This process continues for the next point until all points have been searched. According to S a Maximum synchronous response values of four points in each point set, in descending order for S a Sequencing and outputting the obtained S a ；
Step 39: cutting out a set of specific pointsFour areas with slightly larger points are subjected to perspective transformation;
step 310: in this region, using a window of 86×86 pixels of the same size, the sliding interval is 25 pixels, R is selected according to equation (22) and equation (25) m,i Less than a specified threshold-T m Peak point P of (2) b,i As a synchronization response point for the synchronization watermark b.
Step 311: for 16 points around the message watermark representing the sync watermarks a and b, a window of 86 x 86 pixels size is used, with a sliding interval of 2 pixels, in a 150 x 150 region centered around them, for sync response detection. For the synchronous response point of a, recording the maximum 15 amplitude values V with the radius of the second quadrant being 18-30 and the angle being 120-150 degrees a (i) A. The invention relates to a method for producing a fibre-reinforced plastic composite For the synchronous response point b, recording the maximum 15 amplitude values V with the radius of 18-30 and the angle of 30-60 degrees in the first quadrant b (i) A. The invention relates to a method for producing a fibre-reinforced plastic composite Calculating a refined synchronization response index for each coordinate based on equation (26);
R i,i ＝sum(V a (i))
R b,i ＝sum(V b (i)) (26)
step 312: taking the coordinates of the maximum fine correction synchronous response indexes of the 16 areas as the fine correction results, and recording the coordinates as a new point set S';
step 313: perspective correction is performed on 16 points in the S' and an area of the information watermark is extracted.
Step 314: the information blocks are extracted, and the noise component of the luminance image thereof is used to convert it to the DFT domain. Recording maximum amplitude value V in 3 x 3 range with each watermark embedding position as center max (j) Wherein j represents the jth watermark bit; determining the extracted watermark information bits W' (j) based on equation (27), whereinSum sigma T Respectively represent [130,175 ]]Mean and standard deviation, k, of all amplitude coefficient values in the range 3 Representing a constant parameter.
Step 315: and performing QR decoding on W' (j) to obtain watermark information.
A screen image is selected as experimental data, and an exemplary description is made with respect to the whole process of generating, embedding, detecting, etc. of the watermark template. In this example, a 2560×1440 pixel screen image was selected as the experimental data.
(1) Watermark template generation
Step one: reading watermark information W composed of key information such as user ID, IP address, time and the like 0 : '0101341391040363291748202305201605', and generating a QR code image (as shown in fig. 5) based on a QR code algorithm, and recording the coding elements of the QR code coding region row by row as a binary watermark information sequence W 1 [0,0,0,1,0,0,1,…,1,1,1,0,1,0,1,0]；
Step two: calculating to obtain the information watermark block size 676;
step three: in the Fourier transform domain amplitude diagram of the information watermark block, the information watermark embedding radius is set to 132,145,159,172, and the watermark embedding strength k 1 =10000, obtaining the amplitude coefficient M after watermark embedding according to the formula 1 (x, y), iDFT generates an information watermark template (e.g., (a) of fig. 6);
step four: calculating to obtain the size of the synchronous watermark block 86 x 86;
step five: in the Fourier transform domain amplitude diagram of the synchronous watermark block, the synchronous watermark embedding radius is set to 20, 24 and 28, and the watermark embedding intensity k 2 =20000, obtaining the amplitude coefficient M after embedding the watermark according to the formula 2 (x, y), iDFT generates a synchronous watermark template (e.g., (b) of fig. 6);
step six: combining the information watermark block and the synchronous watermark block according to the screen size 2560 x 1440 to generate a watermark template T w (as in fig. 7);
(2) Watermark template embedding
Step one: the method comprises the steps that a Present function address and a GetFrameStatisticInternational function address are obtained in a dynamic link library to be injected in a continuous matching mode of machine codes;
step two: hook present function
Step three: calculating a switching chain address, wherein a switching chain base address can be directly obtained by the address offset of a Present function, a switching chain constant is obtained by the address offset of a GetFrameStatisticInternational function, a switching chain pointer address is obtained according to the switching chain constant offset, and then a rear buffer area address to be displayed is obtained;
step four: calculating JND threshold
[[3.68019723 3.57055375 3.40016583...3.37252749 3.52138304 3.64024025]
[3.53609403 3.35260418 3.0422761...3.11346151 3.36736762 3.5419132]
[3.45709342 3.16986167 2.35125309...2.28881704 3.11300685 3.411417]
...；
Step five: writing the texture data back to the back buffer;
step six: writing the dynamic link library into a desktop window manager process in an injection mode, so that a screen watermark embedding process is operated in the desktop window manager; (as in FIG. 9)
(3) Watermark extraction and detection
Step one: from the captured image (as in fig. 10), 5 noise components I of different scales are acquired n ；
Step two: i for each scale n Detecting a synchronous watermark a; (as in FIG. 11)
[[28601.2521 34568.5541...3346.3155 2254.5559]
[11642.8847 7852.4426...41553.3786 38642.5164]
[7768.4298 5568.3495...2256.8843 3486.3315]
…；
Step three: calculating R for each scale s (45376.5512, 33846.8859, 29764.3546, 30346.8721, 41067.3492) selecting the best scale s m ；
Step four: at the best scale s m Next, a synchronization response threshold T is calculated m (55894.6721) acquiring a synchronization response point P m,i ；
Step five: retrieving all P m,i Construct set S a ；
Step six: based on point setPerforming perspective transformation; (as in FIG. 12 (a))
Step seven: according to the synchronization corresponding threshold-T m And acquiring a response point of the synchronous watermark b.
Step eight: obtaining response value V of synchronous watermarks a and b a (i) And V b (i) Obtaining accurate coordinates of a and b according to the accurate correction synchronous response index;
[[54521.2611 45887.5656...60752.4385 62334.7168]
[26842.7042 19842.4832...50273.4618 47615.3419]
[18035.4765 8642.5437...9641.4795 13476.7349]
step nine: extracting the region of the information watermark and obtaining watermark information W' (j) according to an exact coordinate perspective transformation (as in fig. 12 (b)) [0,0,0,1,0,0,1, …,1,1,1,0,1,0,1,0];
step ten: QR decoding is performed on W' (j) (see fig. 13) to obtain watermark information. (as in FIG. 14)
The embodiment of the invention discloses a computer system which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program realizes the steps of the real-time screen watermarking method for tracking the screen shooting leakage when being loaded to the processor.
Claims (10)
1. The real-time screen watermarking method for tracking the screen shooting leakage is characterized by comprising the following steps of:
generating a watermark template, including an information watermark template, a first synchronous watermark template and a second synchronous watermark template; the information watermark template is generated by inverse Fourier transform after being coded by user information QR comprising user ID, IP address and operation time; the first synchronous watermark template and the second synchronous watermark template are generated by preset synchronous watermark information through inverse Fourier transform, wherein the information of the first synchronous watermark template and the second synchronous watermark template are respectively embedded in the second radius and the angle of the amplitude spectrum of the Fourier transform domain, and the preset radius and the preset angle of the first quadrant and the preset angle of the third quadrant;
according to the screen size, the information watermark template and the synchronous watermark template are adaptively combined to generate a screen watermark template;
acquiring a function address of a switching chain switching function in a desktop window manager process, acquiring a rear buffer address of the desktop window manager switching chain, calculating a JND threshold value of rear buffer data by adopting a minimum perceived difference model, and directly embedding a screen watermark template generated in advance into a rear buffer according to the threshold value;
when a photo image of a shot screen is acquired, detecting a multi-scale synchronous watermark template to obtain the positions of a first synchronous watermark template and a second synchronous watermark template, realizing automatic perspective correction based on the positions of the synchronous watermark templates, positioning an information watermark template embedding area, and obtaining embedded user information.
2. The method for real-time screen watermarking for screen shot leakage traceability according to claim 1, wherein the watermark template generation includes the steps of:
generating a QR code from user information by using a QR code algorithm, and recording code elements in a QR code coding region into a binary watermark information sequence W line by line 1 ；
Determining the information watermark block size L according to the screen size MXN 1 ×L 1 And a synchronization watermark block size L 2 ×L 2 ，L 1 >L 2 ；
Will L 1 ×L 1 The zero matrix of (a) is regarded as the fourier transform domain amplitude map of the information watermark block, based onDetermining the location of embedded watermark information based onEmbedding watermark information, and generating an information watermark template through iDFT; wherein W is 1 (j) Representing w 1 The j-th element, l 1 Represents W 1 Length, R represents intermediate frequency radius, M 1 (x, y) represents the amplitude coefficient, k after embedding the watermark 1 Is constant and represents watermark embedding strength;
will L 2 ×L 2 Regarding the zero matrix of the synchronous watermark block as a Fourier transform domain amplitude diagram, setting at least 15 coefficients as carriers of synchronous watermark information at least 3 radii and 5 equally spaced angles preset in the second and fourth quadrants based on M 2 (x,y)＝k 2 Embedding watermark information, generating a first synchronous watermark template a through iDFT, setting at least 15 coefficients as carriers of the synchronous watermark information at least 3 radiuses and 5 equally-spaced angles preset in the first quadrant and the third quadrant, and setting the coefficients based on M 2 (x,y)＝k 2 Embedding watermark information, and generating a second synchronous watermark template b through iDFT; wherein M is 2 (x, y) represents the amplitude coefficient after embedding the watermark,k 2 is constant and represents watermark embedding strength.
3. The method for real-time screen watermarking for screen shot leakage traceability according to claim 1, wherein the self-adaptive combination of the information watermarking template and the synchronous watermarking template generates a screen watermarking template, specifically:
let the screen size be MxN, according to the formulaDetermining the position of an information watermark template, wherein->And->Respectively representing the number of rows and columns which can be accommodated, L 1 Representing the side length of the information watermark template; m is m 1 And n 1 Respectively represent all c 1 ×c 2 An index of rows and columns in the information watermark;
based on the formulaDetermining a position of a first synchronous watermark template a; wherein m is 2 ，n 2 Index representing rows and columns in a synchronized watermark, L 2 Representing the side length of the synchronous watermark template;
three second synchronous watermark templates b are uniformly embedded between every two first synchronous watermark templates a.
4. The method for watermarking a real-time screen for tracking a screen shot leakage according to claim 1, wherein the method for acquiring a rear buffer address of a desktop window manager switching chain is as follows: the method comprises the steps that a Present function address and a GetFrameStatisticInternational function address are obtained in a dynamic link library to be injected in a continuous matching mode of machine codes; the Present function performs HOOK in an inline HOOK mode; and calculating a switching chain address, wherein a switching chain base address is directly obtained by the address offset of the Present function, a switching chain constant is obtained by the address offset of the GetFrameStatisticInternational function, a switching chain pointer address is obtained according to the switching chain constant offset, and then a back buffer area address to be displayed is obtained.
5. The method for real-time screen watermarking for screen shot leakage traceability according to claim 1, wherein when the minimum perceived difference model calculates a JND threshold, the following sobel operators are adopted to calculate the horizontal gradient G of the two-dimensional image of the buffer region respectively 1 Vertical gradient G 2 Thereby deriving a contrast masking threshold based on the gradient and the background brightness;
finally based on the formula JND (x, y) =p L (x,y)+P c (x,y)-a*min(P L (x,y),P c (x, y)) calculate JND threshold, where P L (x,y)、P c (x, y) represents the brightness masking threshold and the contrast masking threshold at the position (x, y), respectively, a being a coefficient.
6. The method for real-time screen watermarking for screen shot leakage traceability according to claim 5, wherein the two-dimensional image object I in the post-write buffer is calculated based on the following formula b ：
I b (x,y)＝Bg(x,y)+M w (x,y)*JND(x,y)*η
Wherein B is g Represents average background brightness, M w Representing a binarized watermark template, η representing watermark embedding strength.
7. The method for real-time screen watermarking for screen shot disclosure traceability according to claim 1, wherein the detection of the multi-scale synchronous watermark template includes the steps of:
converting the photographed image I into a luminance image I a ；
Image I a Scaling to multiple scales and computing image I at each scale a Is the noise component I of (1) n ；
On each scale of the image, using a window of a predetermined pixel size, at predetermined intervals, for I n Detecting the sliding blocks one by one;
in the Fourier transform domain amplitude diagram of each slider i, a first quadrant preset angle range and a second quadrant preset angle range are recorded, and the maximum specified number of amplitude values in the preset radius range area are respectively recorded as V according to the amplitude values in the second quadrant or the first quadrant l (i) And V r (i) Based on the formulaCalculating a slider window synchronization response value, wherein R s,i Synchronous response index, N, representing the corresponding detection window of the ith slider on scale s l,1 And N r,1 Respectively represent V l (i) And V r (i) The number of amplitude values is recorded;
determining an optimal scale according to the synchronous response indexes of each scale, and determining synchronous response points of each window in the optimal scale;
searching all synchronous response points, finding all point sets of information watermark positions which can be used for correction, wherein each point set comprises four points which can form a quasi-rectangle, cutting out an area formed by the point sets, and performing perspective transformation;
and calculating a synchronous response index in the sheared area by using the sliding window to determine a synchronous response point of the second synchronous watermark template.
8. The method for real-time screen watermarking for screen shot leakage traceability according to claim 1, wherein the automatic perspective correction is realized based on the synchronous watermark template position, comprising the following steps:
for synchronous response points representing a first synchronous watermark template and a second synchronous watermark template surrounding the message watermark template, reducing the interval of sliding windows in the area taking the first synchronous watermark template and the second synchronous watermark template as the center, and carrying out synchronous response detection; recording the maximum preset number of amplitude values within the preset radius and angle range of the second quadrant for the synchronous response points of the first synchronous watermark template; for a synchronous response point of the second synchronous watermark template, recording the maximum preset number of amplitude values within the preset radius and angle range of the first quadrant; the refined synchronization response index for each coordinate is calculated based on the following formula:
R a,i ＝sum(V d (i))
R b,i ＝sum(V b (i))
wherein V is a (i) And R is a,i The amplitude value set of the detection window corresponding to the ith slide block of the first synchronous watermark template and the accurate correction synchronous response index are respectively V b (i) And R is b,i The method comprises the steps that an amplitude value set of a detection window corresponding to an ith sliding block of a second synchronous watermark template is respectively and accurately corrected to a synchronous response index;
and performing perspective transformation on the coordinates of the region maximum fine correction synchronous response index, and taking the transformed coordinates as a fine correction result.
9. The method for real-time screen watermarking for tracking screen shot leakage according to claim 1, wherein the method for acquiring the embedded user information is as follows:
extracting information blocks, using noise components of the brightness images, and converting the information blocks into DFT domains; recording maximum amplitude value V in 3 x 3 range with each watermark embedding position as center max (j) Wherein j represents the jth watermark bit; based on the formulaDetermining the extracted watermark information bit W' (j), wherein +.>Sum sigma T Respectively represent the mean and standard deviation, k, of all amplitude coefficient values within the embedded range 3 Is a constant; and performing QR decoding on W' (j) to obtain user watermark information.
10. A computer system comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the computer program when loaded to the processor implements the steps of the real-time screen watermarking method for traceback of screen shots disclosure according to any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311527940.1A CN117574336A (en) | 2023-11-16 | 2023-11-16 | Real-time screen watermarking method and system for tracking screen shooting leakage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311527940.1A CN117574336A (en) | 2023-11-16 | 2023-11-16 | Real-time screen watermarking method and system for tracking screen shooting leakage |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117574336A true CN117574336A (en) | 2024-02-20 |
Family
ID=89894860
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311527940.1A Pending CN117574336A (en) | 2023-11-16 | 2023-11-16 | Real-time screen watermarking method and system for tracking screen shooting leakage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117574336A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118396828A (en) * | 2024-05-10 | 2024-07-26 | 北京国隐科技有限公司 | Screen watermark processing method, screen watermark extracting method and screen watermark extracting device |
-
2023
- 2023-11-16 CN CN202311527940.1A patent/CN117574336A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118396828A (en) * | 2024-05-10 | 2024-07-26 | 北京国隐科技有限公司 | Screen watermark processing method, screen watermark extracting method and screen watermark extracting device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Fang et al. | Screen-shooting resilient watermarking | |
| Zong et al. | Robust histogram shape-based method for image watermarking | |
| CN110599388B (en) | Blind robust digital watermark embedding and detecting method based on positioning point assistance | |
| CN107888925B (en) | A kind of embedding grammar and detection method of digital video hiding information | |
| Duan et al. | High-capacity image steganography based on improved FC-DenseNet | |
| Hashemzadeh | Hiding information in videos using motion clues of feature points | |
| CN117574336A (en) | Real-time screen watermarking method and system for tracking screen shooting leakage | |
| Chen et al. | Screen-cam robust and blind watermarking for tile satellite images | |
| CN107622497A (en) | Image cropping method, apparatus, computer-readable recording medium and computer equipment | |
| CN110796586A (en) | Blind watermarking method and system based on digital dot matrix and readable storage medium | |
| Wandji et al. | Detection of copy-move forgery in digital images based on DCT | |
| CN108156408B (en) | Digital watermark embedding and extracting method and system for video data | |
| CN112700363A (en) | Self-adaptive visual watermark embedding method and device based on region selection | |
| Dong et al. | Watermark-preserving keypoint enhancement for screen-shooting resilient watermarking | |
| CN106600516B (en) | Image embedding method based on digital fingerprint | |
| Juarez-Sandoval et al. | Digital image ownership authentication via camouflaged unseen-visible watermarking | |
| CN104156909B (en) | The robust reversible watermark insertion of image content-based and detection method | |
| CN106055632B (en) | Video authentication method based on scene frame fingerprints | |
| Qi et al. | An adaptive visible watermark embedding method based on region selection | |
| Yakushev et al. | Docmarking: Real-Time Screen-Cam Robust Document Image Watermarking | |
| Bashier et al. | Graphical password: Pass-images Edge detection | |
| CN113628091B (en) | Safety information extraction method and device for electronic display screen content reproduction scene | |
| Zhu et al. | Lite Localization Network and DUE-Based Watermarking for Color Image Copyright Protection | |
| Evsutin et al. | An improved video watermarking algorithm with extraction using a mobile device camera | |
| Reyes-Ruiz et al. | Robust Dual Digital Watermark Applied to Antique Digitized Cinema Images: Resistant to Print-Scan Attack |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |