CN111583086A - Adaptive digital image watermarking and repairing method based on AMBTC - Google Patents
Adaptive digital image watermarking and repairing method based on AMBTC Download PDFInfo
- Publication number
- CN111583086A CN111583086A CN202010343943.XA CN202010343943A CN111583086A CN 111583086 A CN111583086 A CN 111583086A CN 202010343943 A CN202010343943 A CN 202010343943A CN 111583086 A CN111583086 A CN 111583086A
- Authority
- CN
- China
- Prior art keywords
- block
- image
- matrix
- watermark
- ambtc
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0021—Image watermarking
- G06T1/0028—Adaptive watermarking, e.g. Human Visual System [HVS]-based watermarking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2201/00—General purpose image data processing
- G06T2201/005—Image watermarking
- G06T2201/0065—Extraction of an embedded watermark; Reliable detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2201/00—General purpose image data processing
- G06T2201/005—Image watermarking
- G06T2201/0203—Image watermarking whereby the image with embedded watermark is reverted to the original condition before embedding, e.g. lossless, distortion-free or invertible watermarking
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention provides an AMBTC-based self-adaptive digital image watermarking and repairing method. Firstly, carrying out AMBTC encoding on an original image to obtain a conventional AMBTC compression code (32bits), and carrying out bitmap compression on the conventional AMBTC compression code to obtain a novel AMBTC compression code (24bits), namely recovering a watermark; and secondly, embedding the recovered watermark into the original image by using an information hiding technology based on a magic matrix to obtain a watermark image. At a receiving end, extracting a recovery watermark from an image to be verified, and decoding an AMBTC image; and comparing the difference between the image to be verified and the decoded AMBTC image to realize image tampering positioning and repairing. The method successfully resists several common attacks, and ensures the accuracy of tampering and positioning and the higher quality of the repaired image on the premise of ensuring the quality of the high-quality watermark image.
Description
Technical Field
The invention provides an adaptive digital image watermark and a repairing method based on AMBTC, which are researched in the field of digital watermark and repairing.
Background
The challenge of protecting personal privacy while enjoying the convenience of digitization is now also becoming increasingly important. In addition, with the development of image processing and analysis tools, it becomes easier to manipulate image content almost unconsciously. Despite the increasing awareness of the rights, nothing prevents intentional and unintentional tampering. Such a scheme forms the basis of studying image verification techniques to verify the integrity of the image, and to locate tampered areas in the image and restore the image to a satisfactory perceived quality.
In general, image verification techniques can be divided into two categories: a digital signature based method and a fragile watermark based method. A digital signature method based on a cryptographic algorithm is designed to generate a corresponding digital signature by encrypting a hash result of an image feature with a private key. The image authentication process may be implemented by comparing a hash result generated from the image to be verified with an original hash result decrypted using the public key. These algorithms check the integrity of the image very well, since even with minor modifications to the image, the result of the hash is very sensitive to input and can hardly be forged. However, the greatest disadvantage of such methods is that they do not locate the tampered area, nor do they recover the image after it has been attacked. Meanwhile, methods based on fragile watermarks have attracted much research attention. Fragile watermarks are designed for image authentication, image tamper localization and image restoration. Also, a major requirement of fragile watermarks is that the design of the watermark should be extremely sensitive to any modification of the image content. Typically, a watermark is generated by extracting image features from the image content, the watermark being pre-divided into uniformly sized sub-blocks and combined with a pseudo-random sequence. The generated watermark is then embedded into the original image and is sensitive to any kind of content modification. The method can accurately position the tampered area suffering from various attacks, such as cutting attack, copying and pasting attack, collaging attack and vector quantization attack.
At present, the research of image verification and restoration by using digital watermarking has made great progress. However, they reduce the quality of the watermark image by introducing large amounts of redundant information that reduces the security of the hidden watermark. Therefore, the invention aims to realize high-quality watermark image quality on the premise of ensuring the accuracy of tampering positioning and higher quality of the repaired image.
Disclosure of Invention
In order to provide high-quality watermark image quality, ensure the accuracy of tampering positioning and the higher quality of a repaired image, the invention provides an AMBTC-based adaptive digital image watermark and a repairing method. Firstly, carrying out AMBTC encoding on an original image to obtain a conventional AMBTC compression code (32bits), and carrying out bitmap compression on the conventional AMBTC compression code to obtain a novel AMBTC compression code (24bits), namely recovering a watermark; and secondly, embedding the recovered watermark into the original image by using an information hiding technology based on a magic matrix to obtain a watermark image. At a receiving end, extracting a recovery watermark from an image to be verified, and decoding an AMBTC image; and comparing the difference between the image to be verified and the decoded AMBTC image to realize image tampering positioning and repairing.
The technical scheme of the invention comprises the following steps:
a self-adaptive digital image watermarking and repairing method based on AMBTC is used for image transmission between a sending end and a receiving end, and the specific method comprises the following steps: :
at the transmitting end, embedding a digital watermark in an original image to be transmitted according to S11-S15:
s11, dividing the original image I into a plurality of non-overlapping image blocks, wherein the size of each image block is 4 multiplied by 4;
s12, performing AMBTC compression on each image block X of the original image to obtain a three-tuple code of the block, wherein a is a low-order quantization value, B is a high-order quantization value, and B is a bitmap;
s13, re-compressing the bitmap B of the image block X according to a preset reference compression matrix RCM, wherein the RCM is a matrix formed by 0 and 1, the size of the matrix is the same as that of the bitmap B, and if the bit in the RCM is 1, the bit value of the corresponding position in the bitmap B is not compressed and still remains in B'; if the bit in the RCM is 0, the bit value of the corresponding position in the bitmap B is compressed; compressing the bitmap B to obtain a bitmap B ', thereby obtaining novel AMBTC compression codes (a, B, B') of the block X as a recovery watermark of the block X;
s14, after obtaining the novel AMBTC compression codes of all blocks of the original image, taking the block recovery watermark as a unit, carrying out Arnold scrambling transformation in a one-to-one mapping mode, and scrambling the recovery watermark of each block X into blocks Y at other positions;
s15, generating a magic matrix RM, converting the recovered watermark (a, B') transformed into block Y into binary form, and concatenating into a 24-bit bitstream BR; cutting the pixels of the block Y into 8 groups of pixel pairs in a raster scanning mode, sequentially extracting 3-bit recovery watermarks from the BR, and embedding the recovery watermarks into one group of pixel pairs of the block Y according to a magic matrix RM to obtain the pixel pairs of the watermark image; when the 24-bit stream BR is completely embedded into 8 groups of different pixel pairs, the embedding of the recovery watermark information of the current block Y is completed; after the recovery watermarks of all blocks of the original image I are completely embedded, obtaining a watermark image I' for sending to a receiving end;
at the receiving end, verifying the received image to be verified according to S21-S27, and positioning and repairing the tampered position:
s21, after receiving the sent image at the receiving end, taking the image as an image II to be verified, and dividing the image II into a plurality of non-overlapping image blocks, wherein the size of each image block is 4 multiplied by 4;
s22, generating a magic matrix RM which is the same as that in the sending end, cutting the pixels of each image block Y of the image II to be verified into 8 groups of pixel pairs in a raster scanning mode, extracting 3-bit watermark information in each group of pixel pairs by searching the magic matrix RM, and connecting the watermarks extracted from the 8 groups of pixel pairs in series to form a recovered watermark bit stream BR' stored in the block Y;
S24, restoring watermarkPerforming inverse Arnold transformation to transform the recovered watermark originally stored in block Y back to the position of block X;
s25 recovery watermark in block X after inverse Arnold transformationReconstructing the AMBTC image block of the block X by using a self-adaptive weight prediction method to obtain a reconstructed block XC;
s26, calculating the difference D between the pixel values of the block XC and the block X; if D is larger than or equal to the threshold value T, the block X is a tampered block and corresponds to a position mark '1' in the tampering mark matrix TM; if D is smaller than the threshold value T, the block X is an untampered block and is marked with '0' at the corresponding position in the tampered mark matrix TM;
s27, if the tampered block does not exist in the image II to be verified, taking the image II to be verified as a final image; if the tampered block exists in the image II to be verified, implementing a neighborhood elimination method on the tampered mark matrix TM according to the continuous characteristics of the tampering behaviors, and adjusting the element marks of the neighborhood center by using marks of other elements in the neighborhood of the matrix elements to obtain a tampered mark correction matrix TM';
and S28, performing image restoration on the block which is determined to be tampered according to the tampering mark correction matrix TM', and taking the restored image as a final image.
On the basis of the technical scheme, the steps of the invention can be further realized in the following specific mode.
Preferably, in S12, the sizes of a and B are both 8 bits, the size of B is 16 bits, and the formula for calculating the triplet is as follows:
wherein: x is the number ofiRepresenting the ith pixel value, t, in an image block0The pixel value in the block is smaller than the average value of the pixel values of the blockNumber of (1), t1Is that the pixel value in the block is not less thanNumber of pixels, BiRepresenting the bit value of the i-th pixel in bitmap B, operatorIndicating a rounding down.
Preferably, in S13, if the bit in RCM is 0, the bit value at the corresponding position in bitmap B is compressed, and B' represents "NA".
Preferably, in S14, the arnold scrambling transformation formula is as follows:
wherein (r)i,ci) The coordinates of the current block after the ith round of Arnold transformation are represented, (m, N) is a set of preset positive integers, and N is the space of Arnold transformation.
Preferably, in S15, the magic matrix RM is a tortoise shell matrix with a size of 256 × 256.
Preferably, the specific method of S25 is:
s251: recovery watermarking in block X after inverse Arnold transformationDecoding is carried out, and the corresponding bitmap in the AMBTC image block X after decodingThere is a pixel value at a value of 1, and the corresponding bitmapNo pixel value at value 0;
s252: for each compressed non-value pixel in the block X, adopting an adaptive weight prediction method to predict the pixel value, wherein the prediction process is as follows:
first, for each non-valued pixel p to be predicted in block XxThe neighborhood of (2) calculates the mean of all valued pixels in the neighborhood:
wherein: p is a radical ofiIs pxI is more than or equal to 1 and less than or equal to M, M is pxThe total number of pixels having value in the neighborhood of (a);
then, p is calculatedxEach having a value pixel p in the neighborhood of (2)iCorresponding varianceiAnd according to the varianceiDetermining each valued pixel piCorresponding prediction weights wi,i∈[1,2,…,M]:
Finally, each non-valued pixel p to be predicted in block X is calculatedxPixel value of (a):
preferably, the specific method of S26 is:
s261: calculating the pixel value difference D between the reconstructed block XC and the block X before reconstruction, wherein the calculation formula is as follows:
wherein p isXC(i, j) and pX(i, j) respectively represents the pixel values at coordinates (i, j) of block XC and block X, Q is a weighting factor, which is a constant;
s262: comparing the pixel value difference D with a preset threshold value, if D is greater than or equal to the threshold value T, determining the block X as a tampered block, and marking '1' at a corresponding position in a tampered marking matrix TM; if D is less than the threshold value T, the block X is determined to be an untampered block, and a position mark '0' is correspondingly marked in the tampered mark matrix TM;
s263: and (5) carrying out tampering identification detection on each block in the image II to be verified according to S261 and S262, and finally obtaining a tampering mark matrix TM.
Preferably, the specific method of S27 is:
s271: traversing the tampering mark matrix TM, if the tampering mark matrix TM does not have an element marked as '1', determining that a tampered block does not exist in the image II to be verified, and taking the received image II to be verified as a final image; if the tampering mark matrix TM has an element marked as '1', continuously executing a neighborhood elimination method according to S272;
s272, selecting one element of the tampering marked matrix TM to correct, and counting the number N of the elements marked as ' 1 ' in the 3 × 3 neighborhood of the element if the selected element is marked as ' 11If N is present1If the mark of the selected element is '0', counting the number N of elements marked as '1' in the 3 × 3 neighborhood of the element2If N is present2Greater than or equal to 6 thenModifying the mark of the selected element to '1';
s273: according to the correction method of S272, sequentially traversing each element in the tamper mark matrix TM to finally obtain a tamper mark correction matrix TM';
preferably, the specific method of S28 is:
s281: determining tampered blocks in the image II to be verified according to the tampering marker correction matrix TM', and repairing each tampered block to obtain a final image; wherein:
if any block X is judged to be tampered, but the corresponding block Y storing the recovery watermark is judged not to be tampered, replacing the block X with the pixel value of the corresponding block in XC;
if any block X is judged to be tampered and the corresponding block Y storing the recovery watermark is also judged to be tampered, repairing the block X by using a neighborhood image repairing technology.
Compared with the prior art, the invention has the following beneficial effects:
the method ensures high-precision tampering positioning performance and high-quality repaired images, provides considerable watermark image quality, and greatly reduces the degree of distortion of original images caused by embedding watermarks. The method can successfully resist several common attacks, and on the premise of ensuring the quality of the high-quality watermark image, the accuracy of tampering positioning and the higher quality of the repaired image are ensured. The method can be used for performing copyright authentication on the watermark image, and specifically comprises tampering behavior positioning and tampering area repairing.
Drawings
Fig. 1 is a bitmap compression process.
Figure 2 is a schematic diagram of the magic matrix RM.
Figure 3 is a schematic diagram of a process of recovering watermark embedding based on a magic matrix.
Fig. 4 is an example of adaptive weighted pixel prediction.
Fig. 5 is an exemplary diagram of tamper location.
FIG. 6 is an exemplary diagram of a neighborhood elimination method.
FIG. 7 is a performance analysis of PSNR at various tamper rates for the present invention and method [1] of Kim et al: (a) peppers, (b) Babon, (c) Lena, (d) Boat + Couple.
Detailed Description
The invention will be further elucidated and described with reference to the drawings and the detailed description.
In a preferred embodiment of the present invention, an adaptive digital image watermarking and restoration method based on AMBTC is provided for image transmission between a sending end and a receiving end. The method recovers the tampered image using an adaptive weight AMBTC compression technique. First, the image is divided into blocks for processing, and each block first generates a set of AMBTC compression codes: including two 8-bit quantized values and a 16-bit bitmap. Further, the 16-bit bitmap is compressed into an 8-bit bitmap according to the reference compression matrix, and the 8-bit bitmap and two 8-bit quantization values form a recovery watermark. And then, embedding the scrambled recovered watermark into the original image by using an information hiding technology based on a magic matrix to form a watermark image, and transmitting the watermark image to a receiving end by a transmitting end. At a receiving end, a tampering detection stage is firstly carried out, and a two-stage tampering detection strategy is executed to realize the accuracy of tampering positioning; then a self-recovery phase is entered, and the recovery of the tampered region is performed by using an adaptive weight-based recovery strategy that achieves good quality in the reconstructed image and an image inpainting technique.
The following description will be made in detail with reference to the accompanying drawings, taking an 8-bit grayscale image as an example.
At a transmitting end, embedding a digital watermark in an original image to be transmitted according to S11-S15, wherein the specific processes of S11-S15 are as follows:
s11, dividing the original image I into a plurality of non-overlapping image blocks, wherein the size of each image block is 4 multiplied by 4;
s12, performing AMBTC compression on each image block X of the original image, resulting in a triplet code (32bits) of the block, denoted (a, B), where a is a low-order quantized value (8 bits), B is a high-order quantized value (8 bits), and B is a bitmap (16 bits).
The formula for the triplet (a, B) is as follows:
wherein: x is the number ofiRepresenting the ith pixel value, t, in an image block0The pixel value in the block is smaller than the average value of the pixel values of the blockNumber of (1), t1Is that the pixel value in the block is not less thanNumber of pixels, BiRepresenting the bit value of the i-th pixel in bitmap B, operatorIndicating a rounding down.
S13, recompressing the bitmap B of the image block X according to a preset reference compression matrix RCM, where RCM is a matrix composed of 0 and 1, and the matrix size is the same as that of the bitmap B, and the specific rule is: if the bit in RCM is 1, the bit value of the corresponding position in bitmap B is not compressed and still remains in B'; if the bit in the RCM is 0, the bit value of the corresponding position in the bitmap B is compressed, and is represented by ' NA ' in the bitmap B '. The bitmap B is compressed to obtain a bitmap B ', and thus a new AMBTC compressed code (a, B') of the block X is obtained as a recovery watermark for the block X.
In this embodiment, the bitmap B is shown in fig. 1(a), the reference compression matrix RCM is shown in fig. 1(B), and the bitmap B ' obtained after compression is shown in fig. 1(c), in which 8 bits are compressed and marked with ' NA '.
S14, after obtaining the new AMBTC compression codes of all blocks of the original image, performing arnold scrambling transformation in a one-to-one mapping manner with the block recovery watermark as a unit, and scrambling the recovery watermark of each block X into blocks Y at other positions. In the present invention, the formulation of the arnold scrambling transformation is as follows:
wherein (r)i,ci) The coordinates of the current block after the ith round of Arnold transformation are represented, (m, N) is a set of preset positive integers, and N is the space of Arnold transformation.
Thus, the arnold transform can implement one-to-one map scrambling. For example, the recovery watermark for block X is scrambled to the location of block Y, …, and the recovery watermark for block Z is scrambled to the location of block X. The aim is to store the recovery information of the block in a block far away from the block as much as possible so as to facilitate the aim of later image verification and restoration.
And S15, generating a magic matrix RM. Converting the recovered watermark (a, B') transformed into block Y into binary form and concatenating into a 24-bit bitstream BR; cutting the pixels of the block Y into 8 groups of pixel pairs in a raster scanning mode, sequentially extracting 3-bit recovery watermarks from the BR, and embedding the recovery watermarks into one group of pixel pairs of the block Y according to a magic matrix RM to obtain the pixel pairs of the watermark image; when the 24-bit stream BR is completely embedded with 8 different pixel pairs, the embedding of the recovered watermark information of the current block Y is completed.
And embedding the recovery watermark into all blocks of the original image I according to the mode, and obtaining a watermark image I' after all the blocks are completely embedded for sending to a receiving end.
In this embodiment, the magic matrix RM may adopt a turtle shell matrix (turtle shell) with a size of 256 × 256, as shown in fig. 2, fig. 3 shows the adaptive weights in this embodimentIn the example of the double-pixel prediction, in the raster scanning mode, two adjacent pixels in 4 pixels in the same row in a block form a group of pixel pairs. 8 groups of different pixel pairs are embedded with watermark w through magic matrix1、w2、w3、…、w8. The process of embedding a watermark by a magic matrix belongs to the prior art, and can be referred to in detail in the prior art documents ChangC.C., Liu Y., Nguyen T.S., "A novel vertical shell based scheme for data linking," in processing of 2014 content information linking and multimedia signal processing IEEE 2014: 89-93.
To this end, a digital watermark may be embedded in an original image at a transmitting end and then transmitted to a receiving end. At a receiving end, whether the received image is tampered or not needs to be verified, and the method mainly comprises the steps of extracting a recovery watermark from the image to be verified and decoding an AMBTC image; and comparing the difference between the image to be verified and the decoded AMBTC image to realize image tampering positioning and repairing. The specific process of the sending end is as follows S21-S27:
and S21, after receiving the sent image at the receiving end, taking the image as an image II to be verified, and dividing the image II into a plurality of non-overlapping image blocks, wherein the size of each image block is 4 multiplied by 4.
S22, generating a magic matrix RM (fig. 2) the same as that in the sending end, cutting the pixels of each image block Y of the image II to be verified into 8 groups of pixel pairs in the same raster scanning manner as that in the sending end, extracting 3-bit watermark information in each group of pixel pairs by searching the magic matrix RM, and concatenating the watermarks extracted from the 8 groups of pixel pairs to form a recovered watermark bit stream BR' stored in the block Y.
S23, parsing the recovery watermark corresponding to the block Y from the bit stream BRAfter the watermark is recovered for each block, all recovered watermark information can be obtained.
S24, restoring watermarkAn inverse Arnold transform is performed in order to transform the recovered watermark originally stored in block Y (i.e., the new AMBTC compression code for block X) back to the location of block X.
S25 recovery watermark in block X after inverse Arnold transformationAnd reconstructing the AMBTC image block of the block X by using the self-adaptive weight prediction method to obtain a reconstructed block XC. The specific method of the step is as follows:
s251: recovery watermarking in block X after inverse Arnold transformationDecoding is carried out, and the corresponding bitmap in the AMBTC image block X after decodingThere is a pixel value at a value of 1, and the corresponding bitmapNo pixel value at value 0;
s252: for each compressed non-value pixel in the block X, adopting an adaptive weight prediction method to predict the pixel value, wherein the prediction process is as follows:
first, for each non-valued pixel p to be predicted in block XxThe neighborhood of (2) calculates the mean of all valued pixels in the neighborhood:
wherein: p is a radical ofiIs pxI is more than or equal to 1 and less than or equal to M, M is pxThe total number of pixels having value in the neighborhood of (a);
then, p is calculatedxEach having a value pixel p in the neighborhood of (2)iCorresponding varianceiAnd according to the varianceiDetermining each valued pixel piCorresponding prediction weights wi,i∈[1,2,…,M]:
Finally, each non-valued pixel p to be predicted in block X is calculatedxPixel value of (a):
FIG. 4 shows a specific prediction example of adaptive weighted pixels, which can be first determined from the triplet of one block XDecoding the partial pixel values of the AMBTC image block, and the corresponding bitmap can be seen from FIG. 4(a)The value is 0 and there is no pixel value, which is NA, so this portion of pixel values needs to be predicted. Fig. 4(b), (c), and (d) show neighborhood valued pixels used for predicting pixels p (3,2), p (4,1), and p (2,1), respectively, from which corresponding predicted values can be obtained according to the adaptive weight prediction method described above. In the process, the neighborhood valued pixel weights for obtaining the predicted values are generated in a self-adaptive mode, so that the recovered image quality is improved.
S26, the same block in the image to be verified and the decoded AMBTC image has two sets of pixel values, which are block X and block XC, respectively, and the difference D between the pixel values of block XC and block X can be calculated as a criterion for determining whether the block is tampered. The specific judgment rule is as follows: if D is larger than or equal to the threshold value T, the block X is a tampered block and corresponds to a position mark '1' in the tampering mark matrix TM; if D is smaller than the threshold T, the block X is an untampered block and marked with '0' at the corresponding position in the tamper flag matrix TM. The specific method of the step is as follows:
s261: calculating the pixel value difference D between the reconstructed block XC and the block X before reconstruction, wherein the calculation formula is as follows:
wherein p isXC(i, j) and pX(i, j) respectively represents the pixel values at coordinates (i, j) of block XC and block X, Q is a weighting factor, which is a constant;
s262: comparing the pixel value difference D with a preset threshold value, if D is greater than or equal to the threshold value T, determining the block X as a tampered block, and marking '1' at a corresponding position in a tampered marking matrix TM; if D is less than the threshold value T, the block X is determined to be an untampered block, and a position mark '0' is correspondingly marked in the tampered mark matrix TM;
s263: and (5) carrying out tampering identification detection on each block in the image II to be verified according to S261 and S262, and finally obtaining a tampering mark matrix TM.
Fig. 5 shows a corresponding example of the AMBTC image block of the reconstructed block X, in which fig. 5(a) is a judgment example of a tampered block, and fig. 5(b) is a judgment example of a non-tampered block. In this embodiment, Q is 64, and the threshold T is 3. Of course, the specific values of the two parameters also need to be adjusted according to practical application.
S27, if the tampered block does not exist in the image II to be verified, taking the image II to be verified as a final image; if the tampered block exists in the image II to be verified, a neighborhood elimination method is implemented on the tampered mark matrix TM according to the continuous characteristics of the tampering behaviors, the element marks of the neighborhood center are adjusted by using marks of other elements in the neighborhood of the matrix elements, and a tampered mark correction matrix TM' is obtained. The neighborhood elimination method has the effects of obtaining a more accurate tampered positioning diagram and reducing misjudgment. The specific method of the step is as follows:
s271: traversing the tampering mark matrix TM, if the tampering mark matrix TM does not have an element marked as '1', determining that a tampered block does not exist in the image II to be verified, and taking the received image II to be verified as a final image; if the tampering mark matrix TM has an element marked as '1', continuously executing a neighborhood elimination method according to S272;
s272, selecting one element of the tampering marked matrix TM to correct, and counting the number N of the elements marked as ' 1 ' in the 3 × 3 neighborhood of the element if the selected element is marked as ' 11If N is present1If the mark of the selected element is '0', counting the number N of elements marked as '1' in the 3 × 3 neighborhood of the element2If N is present2Greater than or equal to 6, the selected element is marked as '1'.
Fig. 5 shows two specific tamper correction examples. As shown in fig. 5(a), the central element thereof is labeled as '1', and only 1 element among 8 elements in its neighborhood is labeled as '1', so N1Less than 4, which indicates that the block corresponding to the element has less tampered blocks around the block and may be a false judgment, the mark of the central element is corrected to ' 0 ', that is, the block is not actually tampered, and the corrected value is recorded in the correction matrix TM '. As shown in fig. 5(b), the central element is labeled as '0', and among the 8 elements in its neighborhood, 7 elements are labeled as '1', so N2Greater than 6 indicates that the element is substantially tampered with around the corresponding tile, and the central tile is also typically tampered with, so the mark of its central element is modified to '1', i.e., the tile is also actually tampered with. By the method, the characteristic that tampering behaviors usually have continuity can be utilized, the tampering marking matrix TM is corrected, and the corrected value is recorded into the correction matrix TM'. If the condition is not satisfied, the element value of the tampering marking matrix TM is still recorded in the correction matrix TM', and is kept unchanged.
S273: and according to the correction method of S272, sequentially traversing each element in the tamper mark matrix TM to finally obtain the tamper mark correction matrix TM'. Of course, in practical application, an additional matrix TM' may not be needed, and the matrix TM may be modified directly.
And S28, performing image restoration on the block which is determined to be tampered according to the tampering mark correction matrix TM', and taking the restored image as a final image. The specific method of the step is as follows:
s281: determining tampered blocks in the image II to be verified according to the tampering marker correction matrix TM', and repairing each tampered block to obtain a final image; wherein:
if any block X is judged to be tampered, but the corresponding block Y storing the recovery watermark is judged not to be tampered, replacing the block X with the pixel value of the corresponding block in XC;
if any block X is judged to be tampered and the corresponding block Y storing the recovery watermark is also judged to be tampered, repairing the block X by using a neighborhood image repairing technology (image inpainting).
It should be noted that, in the above steps, the blocks X and Y belong to a common block representation manner, which is introduced for convenience of description, but does not refer to a specific difference in the image.
In order to show the effects achieved by the present invention, the method is applied to a specific embodiment, the specific steps are not described again, and the specific parameters and technical effects are mainly shown below.
Examples
In this embodiment, a digital watermark is embedded in an original image to be transmitted according to the foregoing steps S11 to S15, and then the received image to be verified is verified according to steps S21 to S27, and the tampered position is located and repaired. The specific results are shown below:
a. performance analysis
Introducing a large number of watermarks in an image can reduce the quality of the watermarked image, which can draw more unnecessary attention. Thus, to overcome this problem, the method of the present invention provides better quality in the watermark image, while providing better resilience to tampered images. Table 1 shows the experimental results of the present invention in terms of PSNR, SSIM (structural similarity) and NCC (normalized cross-correlation) of watermark images. In addition, the present invention can provide a quite good restored image quality, and the corresponding experimental results are shown in table 2.
Performance analysis in terms of PSNR: in one aspect, Kim et al method [1] embeds the AMBTC code into the 2LSBs of all pixels in the original image and inserts a bitmap into the 1 LSB. Their method provides good watermark image quality with PSNR close to 44.15 dB. Accordingly, the present invention proposes a fragile watermarking scheme for image self-recovery using an improved AMBTC technique. The method reduces the length of the watermark while still maintaining a high quality restored image. The PSNR average of the watermark image is about 49.76 dB. On the other hand, the restored image quality obtained by the present invention is about 32.30dB, which is also 0.41dB higher than that of Kim et al method [1 ]. This shows that the proposed scheme is able to obtain good watermark images and recovered images.
Performance analysis in terms of SSIM and NCC. As can be seen from Table 1, both scheme [1] and the present invention provide average SSIM and NCC values in excess of 0.9970. This indicates that the original image and the watermark image have a very high degree of similarity. In addition, the present invention provides the restored image with SSIM and NCC values slightly higher than scheme [1], with average values of 0.9558 and 0.9901. This means that our method provides a high structural similarity between the original image and the restored image.
TABLE 1 PSNR, SSIM and NCC Performance analysis of watermark images
TABLE 2 Performance analysis of PSNR, SSIM and NCC of restored images
FIG. 7 shows a series of comparisons between the present invention and the method of Kim et al in terms of tamper rate and restored image quality. First, as we expect, our method provides better watermark image quality than Kim et al in all cases. Also, in most cases, our method provides better image restoration than Kim et al, even with a high rate of tampering. FIGS. 7(a) - (c) show experiments by performing cropping attacks on Peppers, Baboon and Lena images. In fig. 7(a), the restored image quality of the present invention is superior to [1 ]. The average distance of PSNR for both methods is about 0.67 dB. In fig. 7(b), the restored image qualities of the two methods are close to each other. In particular, the Kim et al method has a tamper rate of 19.66% which gives a 0.66dB better recovery quality than our proposed method. This is because the Baboon image is a matte image, and therefore the accuracy when estimating the pixel values using the adaptive weight-based method is slightly insufficient. However, obtaining substantially improved image quality of the watermark image is a good choice at the expense of some recovery quality. Further, as shown in FIG. 7(c), the results are similar to those of FIG. 7 (b). Overall, the PSNR distance for both methods is about 0.30 dB. Fig. 7(d) shows an experiment performed on a Boat image by copying regions with different tamper rates from the Couple image and pasting them into the Boat image. The PSNR difference between the two approaches is about 0.48 dB. In summary, experimental results demonstrate that our method provides a positive and effective image self-recovery capability.
b robustness versus visualization
Table 3 shows a comparison of the recovery capabilities of several fragile watermarking methods [1-7 ]. The comparison between them is summarized in the following three aspects:
and comparing the watermark images. The present invention provides better watermark image quality relative to [1-7 ]. Zhang et al method [2] and Qian et al method [3] provide lower quality watermark images because they introduce more redundant information to improve the performance of tamper detection and recovery. Method [4] of Hemid et al, method [5] of Molina-Garcia et al, and method [6] of Hemid et al use a variable capacity recovery watermark generation mechanism or a bitwise embedding strategy to improve watermark image quality. Although scheme [7] provides the highest quality watermark image, with a PSNR of 57dB, it cannot recover the tampered image.
Image restoration performance comparison. The invention also provides a relatively good restored image quality with an average PSNR of 34.65dB, where TR is set to 5%, 10%,. 50%. It is inferior to Qian et al's method [3 ]. However, the tolerable rate of tampering is 35% lower for the Qian et al approach than for the other six approaches. In addition, the other four methods [2, 4-6] provide relatively low restored image quality, ranging from 24dB to 27 dB. Furthermore, in both Kim et al method [1] and the present invention, the generation of the watermark is based on AMBTC. Both methods are capable of restoring the content in the tampered area while maintaining a high restored image quality. Thus, the present invention is considered to be robust and imperceptible.
In summary, the proposed method provides better watermark image quality and better resilience to tampered images.
Table 3 compares the recovery performance with the protocols [1-7 ].
The above comparative methods are specifically described in the following references:
[1]Kim C.,Shin D.,Yang C.N.,“Self-embedding fragile watermarkingscheme to restoration of a tampered image using AMBTC,”Personal andUbiquitous Computing,2018,22(1):11-22.
[2]Zhang X.,Wang S.Feng,G.,“Fragile watermarking scheme withextensive content restoration capability,”International Workshop on DigitalWatermarking,2009:268-278.
[3]Qian Z.,Feng G.,Zhang X.,Wang S.,“Image self-embedding with high-quality restoration capability,”Digital Signal Processing,2001,21(2):278-286.
[4]Hemida O.,Huo Y.,He H.,Chen F.,“A restorable fragile watermarkingscheme with superior localization for both natural and text images,”Multimedia Tools and Applications,2019,78(9):12373-12403.
[5]Molina-Garcia J.,Garcia-Salgado B.P.,Ponomaryov V.,Reyes-Reyes R.,Sadovnychiy S.,Cruz-Ramos C.,“An effective fragile watermarking scheme forcolor image tampering detection and self-recovery,”Signal Processing:ImageCommunication,2020,81:115725.
[6]Hemida O.,He H.,“A self-recovery watermarking scheme based onblock truncation coding and quantum chaos map,”Multimedia Tools andApplications,2020:1-31.
[7]Gul E.,Ozturk S.,“A novel hash function based fragile watermarkingmethod for image integrity,”Multimedia Tools and Applications,2019,78(13):17701-17718.
therefore, the method successfully resists several common attacks, and can ensure the accuracy of tampering and positioning and the higher quality of the repaired image on the premise of ensuring the quality of the high-quality watermark image.
The above-described embodiments are merely preferred embodiments of the present invention, which should not be construed as limiting the invention. Various changes and modifications may be made by one of ordinary skill in the pertinent art without departing from the spirit and scope of the present invention. Therefore, the technical scheme obtained by adopting the mode of equivalent replacement or equivalent transformation is within the protection scope of the invention.
Claims (9)
1. A self-adaptive digital image watermarking and repairing method based on AMBTC is used for image transmission between a sending end and a receiving end, and is characterized in that:
at the transmitting end, embedding a digital watermark in an original image to be transmitted according to S11-S15:
s11, dividing the original image I into a plurality of non-overlapping image blocks, wherein the size of each image block is 4 multiplied by 4;
s12, performing AMBTC compression on each image block X of the original image to obtain a three-tuple code of the block, wherein a is a low-order quantization value, B is a high-order quantization value, and B is a bitmap;
s13, re-compressing the bitmap B of the image block X according to a preset reference compression matrix RCM, wherein the RCM is a matrix formed by 0 and 1, the size of the matrix is the same as that of the bitmap B, and if the bit in the RCM is 1, the bit value of the corresponding position in the bitmap B is not compressed and still remains in B'; if the bit in the RCM is 0, the bit value of the corresponding position in the bitmap B is compressed; compressing the bitmap B to obtain a bitmap B ', thereby obtaining novel AMBTC compression codes (a, B, B') of the block X as a recovery watermark of the block X;
s14, after obtaining the novel AMBTC compression codes of all blocks of the original image, taking the block recovery watermark as a unit, carrying out Arnold scrambling transformation in a one-to-one mapping mode, and scrambling the recovery watermark of each block X into blocks Y at other positions;
s15, generating a magic matrix RM, converting the recovered watermark (a, B') transformed into block Y into binary form, and concatenating into a 24-bit bitstream BR; cutting the pixels of the block Y into 8 groups of pixel pairs in a raster scanning mode, sequentially extracting 3-bit recovery watermarks from the BR, and embedding the recovery watermarks into one group of pixel pairs of the block Y according to a magic matrix RM to obtain the pixel pairs of the watermark image; when the 24-bit stream BR is completely embedded into 8 groups of different pixel pairs, the embedding of the recovery watermark information of the current block Y is completed; after the recovery watermarks of all blocks of the original image I are completely embedded, obtaining a watermark image I' for sending to a receiving end;
at the receiving end, verifying the received image to be verified according to S21-S27, and positioning and repairing the tampered position:
s21, after receiving the sent image at the receiving end, taking the image as an image II to be verified, and dividing the image II into a plurality of non-overlapping image blocks, wherein the size of each image block is 4 multiplied by 4;
s22, generating a magic matrix RM which is the same as that in the sending end, cutting the pixels of each image block Y of the image II to be verified into 8 groups of pixel pairs in a raster scanning mode, extracting 3-bit watermark information in each group of pixel pairs by searching the magic matrix RM, and connecting the watermarks extracted from the 8 groups of pixel pairs in series to form a recovered watermark bit stream BR' stored in the block Y;
S24, restoring watermarkPerforming inverse Arnold transformation to transform the recovered watermark originally stored in block Y back to the position of block X;
s25 recovery watermark in block X after inverse Arnold transformationReconstructing the AMBTC image block of the block X by using a self-adaptive weight prediction method to obtain a reconstructed block XC;
s26, calculating the difference D between the pixel values of the block XC and the block X; if D is larger than or equal to the threshold value T, the block X is a tampered block and corresponds to a position mark '1' in the tampering mark matrix TM; if D is smaller than the threshold value T, the block X is an untampered block and is marked with '0' at the corresponding position in the tampered mark matrix TM;
s27, if the tampered block does not exist in the image II to be verified, taking the image II to be verified as a final image; if the tampered block exists in the image II to be verified, implementing a neighborhood elimination method on the tampered mark matrix TM according to the continuous characteristics of the tampering behaviors, and adjusting the element marks of the neighborhood center by using marks of other elements in the neighborhood of the matrix elements to obtain a tampered mark correction matrix TM';
and S28, performing image restoration on the block which is determined to be tampered according to the tampering mark correction matrix TM', and taking the restored image as a final image.
2. The AMBTC-based adaptive digital image watermarking and repairing method according to claim 1, wherein in S12, a and B are both 8 bits in size, B is 16 bits in size, and the formula of the triplet is as follows:
wherein: x is the number ofiRepresenting the ith pixel value, t, in an image block0The pixel value in the block is smaller than the average value of the pixel values of the blockNumber of (1), t1Is that the pixel value in the block is not less thanNumber of pixels, BiRepresenting the bit value of the i-th pixel in bitmap B, operatorIndicating a rounding down.
3. The AMBTC-based adaptive digital image watermarking and repairing method according to claim 1, wherein in S13, if the bit in the RCM is 0, the bit value of the corresponding position in the bitmap B is compressed, and is represented by ' NA ' in B '.
4. The AMBTC-based adaptive digital image watermarking and repairing method according to claim 1, wherein in S14, the arnold scrambling transformation has the following formula:
wherein (r)i,ci) The coordinates of the current block after the ith round of Arnold transformation are represented, (m, N) is a set of preset positive integers, and N is the space of Arnold transformation.
5. The AMBTC-based adaptive digital image watermarking and restoration method according to claim 1, wherein in S15, the magic matrix RM is a tortoise shell matrix with 256 x 256 size.
6. The AMBTC-based adaptive digital image watermarking and repairing method according to claim 1, wherein the specific method of S25 is as follows:
s251: recovery watermarking in block X after inverse Arnold transformationDecoding is carried out, and the corresponding bitmap in the AMBTC image block X after decodingThere is a pixel value at a value of 1, and the corresponding bitmapNo pixel value at value 0;
s252: for each compressed non-value pixel in the block X, adopting an adaptive weight prediction method to predict the pixel value, wherein the prediction process is as follows:
first, for each non-valued pixel p to be predicted in block XxThe neighborhood of (2) calculates the mean of all valued pixels in the neighborhood:
wherein: p is a radical ofiIs pxI is more than or equal to 1 and less than or equal to M, M is pxThe total number of pixels having value in the neighborhood of (a);
then, p is calculatedxEach having a value pixel p in the neighborhood of (2)iCorresponding varianceiAnd according to the varianceiDetermining each valued pixel piCorresponding prediction weights wi,i∈[1,2,…,M]:
Finally, each non-valued pixel p to be predicted in block X is calculatedxPixel value of (a):
7. the AMBTC-based adaptive digital image watermarking and repairing method according to claim 1, wherein the specific method of S26 is as follows:
s261: calculating the pixel value difference D between the reconstructed block XC and the block X before reconstruction, wherein the calculation formula is as follows:
wherein p isXC(i, j) and pX(i, j) respectively represents the pixel values at coordinates (i, j) of block XC and block X, Q is a weighting factor, which is a constant;
s262: comparing the pixel value difference D with a preset threshold value, if D is greater than or equal to the threshold value T, determining the block X as a tampered block, and marking '1' at a corresponding position in a tampered marking matrix TM; if D is less than the threshold value T, the block X is determined to be an untampered block, and a position mark '0' is correspondingly marked in the tampered mark matrix TM;
s263: and (5) carrying out tampering identification detection on each block in the image II to be verified according to S261 and S262, and finally obtaining a tampering mark matrix TM.
8. The AMBTC-based adaptive digital image watermarking and repairing method according to claim 1, wherein the specific method of S27 is as follows:
s271: traversing the tampering mark matrix TM, if the tampering mark matrix TM does not have an element marked as '1', determining that a tampered block does not exist in the image II to be verified, and taking the received image II to be verified as a final image; if the tampering mark matrix TM has an element marked as '1', continuously executing a neighborhood elimination method according to S272;
s272, selecting one element of the tampering marked matrix TM to correct, and counting the number N of the elements marked as ' 1 ' in the 3 × 3 neighborhood of the element if the selected element is marked as ' 11If N is present1If the mark of the selected element is '0', counting the number N of elements marked as '1' in the 3 × 3 neighborhood of the element2If N is present2If the number of the selected elements is more than or equal to 6, the mark of the selected elements is modified to be '1';
s273: and according to the correction method of S272, sequentially traversing each element in the tamper mark matrix TM to finally obtain the tamper mark correction matrix TM'.
9. The AMBTC-based adaptive digital image watermarking and repairing method according to claim 1, wherein the specific method of S28 is as follows:
s281: determining tampered blocks in the image II to be verified according to the tampering marker correction matrix TM', and repairing each tampered block to obtain a final image; wherein:
if any block X is judged to be tampered, but the corresponding block Y storing the recovery watermark is judged not to be tampered, replacing the block X with the pixel value of the corresponding block in XC;
if any block X is judged to be tampered and the corresponding block Y storing the recovery watermark is also judged to be tampered, repairing the block X by using a neighborhood image repairing technology.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010343943.XA CN111583086B (en) | 2020-04-27 | 2020-04-27 | AMBTC-based self-adaptive digital image watermark and restoration method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010343943.XA CN111583086B (en) | 2020-04-27 | 2020-04-27 | AMBTC-based self-adaptive digital image watermark and restoration method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111583086A true CN111583086A (en) | 2020-08-25 |
CN111583086B CN111583086B (en) | 2023-05-26 |
Family
ID=72113144
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010343943.XA Active CN111583086B (en) | 2020-04-27 | 2020-04-27 | AMBTC-based self-adaptive digital image watermark and restoration method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111583086B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113807996A (en) * | 2021-08-06 | 2021-12-17 | 中国人民解放军战略支援部队信息工程大学 | Image self-embedding semi-fragile watermark method and system based on block truncation coding |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103426141A (en) * | 2013-08-09 | 2013-12-04 | 深圳大学 | Method and system for authenticating image contents |
CN104284190A (en) * | 2014-11-05 | 2015-01-14 | 安徽大学 | Compressed image steganography encoding method based on AMBTC high and low average optimization |
US20160132985A1 (en) * | 2014-06-10 | 2016-05-12 | Sam Houston State University | Rich feature mining to combat anti-forensics and detect jpeg down-recompression and inpainting forgery on the same quantization |
CN106780278A (en) * | 2016-11-25 | 2017-05-31 | 陕西师范大学 | A kind of combination zero watermarking and the self- recoverage image encryption and decryption method of block sort fusion |
CN110290390A (en) * | 2019-06-06 | 2019-09-27 | 绍兴聚量数据技术有限公司 | The information concealing method of Modulo-two operation and Hamming code based on AMBTC |
-
2020
- 2020-04-27 CN CN202010343943.XA patent/CN111583086B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103426141A (en) * | 2013-08-09 | 2013-12-04 | 深圳大学 | Method and system for authenticating image contents |
US20160132985A1 (en) * | 2014-06-10 | 2016-05-12 | Sam Houston State University | Rich feature mining to combat anti-forensics and detect jpeg down-recompression and inpainting forgery on the same quantization |
CN104284190A (en) * | 2014-11-05 | 2015-01-14 | 安徽大学 | Compressed image steganography encoding method based on AMBTC high and low average optimization |
CN106780278A (en) * | 2016-11-25 | 2017-05-31 | 陕西师范大学 | A kind of combination zero watermarking and the self- recoverage image encryption and decryption method of block sort fusion |
CN110290390A (en) * | 2019-06-06 | 2019-09-27 | 绍兴聚量数据技术有限公司 | The information concealing method of Modulo-two operation and Hamming code based on AMBTC |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113807996A (en) * | 2021-08-06 | 2021-12-17 | 中国人民解放军战略支援部队信息工程大学 | Image self-embedding semi-fragile watermark method and system based on block truncation coding |
CN113807996B (en) * | 2021-08-06 | 2023-07-07 | 中国人民解放军战略支援部队信息工程大学 | Image self-embedding semi-fragile watermark method and system based on block truncation coding |
Also Published As
Publication number | Publication date |
---|---|
CN111583086B (en) | 2023-05-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Yin et al. | Reversible data hiding in encrypted images based on pixel prediction and bit-plane compression | |
Shi et al. | Lossless data hiding: Fundamentals, algorithms, and applications | |
CN108876697B (en) | Pixel-level image authentication, tampering detection and recovery method | |
US20080276089A1 (en) | Content Authentication and Recovery Using Digital Watermarks | |
CN111787335B (en) | Reversible information hiding method based on AMBTC compression technology and Huffman coding | |
Kumar et al. | Reversible data hiding: A contemporary survey of state-of-the-art, opportunities and challenges | |
Chang et al. | An effective image self-recovery based fragile watermarking using self-adaptive weight-based compressed AMBTC | |
Lin et al. | Digital images watermarking by vector quantization | |
Yang et al. | Block-based colour image steganography using smart pixel-adjustment | |
CN114745475A (en) | Robust reversible information hiding method for encrypted image | |
Liu et al. | Adaptive feature calculation and diagonal mapping for successive recovery of tampered regions | |
Chen et al. | Reversible authentication and cross-recovery of images using (t, n)-threshold and modified-RCM watermarking | |
CN111583086B (en) | AMBTC-based self-adaptive digital image watermark and restoration method | |
Datta et al. | Two-layers robust data hiding scheme for highly compressed image exploiting AMBTC with difference expansion | |
CN111741301A (en) | Recompression and information hiding method based on AMBTC compression coding | |
Zhang et al. | Blind dual watermarking scheme using stucki kernel and spiht for image self-recovery | |
CN114511437B (en) | Watermark embedding and image self-recovery method based on reference matrix and LSB replacement | |
Kumar et al. | A reversible high embedding capacity data hiding technique for hiding secret data in images | |
Chen et al. | Self-recovery fragile watermarking scheme with variable watermark payload | |
Zhong et al. | A Novel Fragile Watermark-Based Image Authentication Scheme for AMBTC-Compressed Images. | |
Yang et al. | Reversible data hiding in the spatial and frequency domains | |
Lin et al. | Novel image authentication scheme for AMBTC-compressed images | |
Hu et al. | Secure image hiding scheme based upon vector quantization | |
Chen et al. | An image authentication and recovery method using optimal selection of block types | |
Imran et al. | A robust method for encrypted data hiding technique based on neighborhood pixels information |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |