CN114782462A - Semantic weighting-based image information hiding method - Google Patents
Semantic weighting-based image information hiding method Download PDFInfo
- Publication number
- CN114782462A CN114782462A CN202210220784.3A CN202210220784A CN114782462A CN 114782462 A CN114782462 A CN 114782462A CN 202210220784 A CN202210220784 A CN 202210220784A CN 114782462 A CN114782462 A CN 114782462A
- Authority
- CN
- China
- Prior art keywords
- image
- secret
- semantic
- secret image
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 230000006870 function Effects 0.000 claims abstract description 81
- 238000012549 training Methods 0.000 claims abstract description 60
- 230000008447 perception Effects 0.000 claims abstract description 53
- 238000007781 pre-processing Methods 0.000 claims abstract description 28
- 238000012360 testing method Methods 0.000 claims abstract description 28
- 230000005540 biological transmission Effects 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 10
- 238000011156 evaluation Methods 0.000 claims abstract description 6
- 230000000694 effects Effects 0.000 claims description 26
- 238000000605 extraction Methods 0.000 claims description 25
- 230000004927 fusion Effects 0.000 claims description 21
- 238000013139 quantization Methods 0.000 claims description 19
- 238000004458 analytical method Methods 0.000 claims description 15
- 238000013528 artificial neural network Methods 0.000 claims description 14
- 230000004913 activation Effects 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 12
- 238000004891 communication Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 9
- 238000013507 mapping Methods 0.000 claims description 8
- 238000013210 evaluation model Methods 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 6
- 238000010586 diagram Methods 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 230000001953 sensory effect Effects 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 230000004075 alteration Effects 0.000 claims description 2
- 238000005303 weighing Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 9
- 230000008859 change Effects 0.000 description 6
- 238000013459 approach Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioethics (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a semantic weighting-based image information hiding method, and belongs to the technical field of information hiding. The method mainly contributes to providing an image information hiding scheme for processing the secret images from a high-order characteristic level, so that the method faces the requirement of safe transmission of massive secret images and a strong third-party steganalysis technology, realizes a large load on the premise of ensuring the quality of the reconstructed secret images, and reduces the influence of the embedded secret images on the perception characteristics of the carrier images to the greatest extent. Firstly, an image information hiding system model is built, then a network model of an image information hiding method based on semantic weighting is built, the network model comprises a semantic preprocessing network, a hiding network and a deciphering network, a reverse gradient propagation model and a loss function model are built according to the requirement of training the network model, and finally network training, testing and system performance evaluation are carried out. The evaluation result shows that compared with the existing image information hiding method without considering semantic weighting, the scheme provided by the invention has better performance.
Description
Technical Field
The invention belongs to the technical field of information hiding, and particularly relates to an image information hiding method based on semantic weighting.
Background
With the development of computer network technology and the popularization of intelligent devices, a great amount of secret information related to personal privacy or organization confidentiality is transmitted in a public channel and is easily stolen and attacked by a third party, so that a serious security problem is caused. The information hiding technology embeds the secret information into a natural carrier, does not change the perception characteristic of the carrier, completes the transmission of the secret information through the transmission of the secret carrier on a public channel, and is considered as an effective means for guaranteeing the information security. Among them, information hiding using an image as a carrier is called image information hiding, and is the most widely used information hiding technology at present.
The early image information hiding method directly modifies the lowest bit of the carrier image pixel value into the secret information, the method has small influence on the visual effect of the carrier image and low complexity, but the generated secret-carrying image has obvious histogram statistical characteristics and is easy to crack by a third party by using a steganalysis technology. A mature self-adaptive image information hiding method firstly analyzes the embedding cost of each pixel point in a carrier image, then selects an area with complex texture and content, and embeds secret information by using STC (Sync-Trellis Codes).
The development of the deep neural Network and the intellectualization of the edge-side device enable a method for hiding image information by using the deep neural Network, which mainly comprises an image information hiding method based on an Automatic Encoder (AE) and an image information hiding method based on a generation countermeasure Network (GAN). The AE-based image information hiding method realizes embedding and reconstruction of the secret image with the same size as the carrier image through end-to-end training of the encoder and the decoder, the method is large in load capacity, and the visual effect of generating the secret image needs to be further improved. The image information hiding method based on the GAN introduces a third-party steganalysis technology to carry out countermeasure training, so that the security is good, but the load capacity of the carrier image is small.
In the face of the need of safe transmission of massive secret information and a powerful third-party steganalysis technology, an image information hiding method which is large in load capacity, good in hiding effect and safe is needed. The existing methods do not consider the change of the local semantic importance of secret information, take a secret image as an example, only process the secret image from a low-order pixel level, and completely embed all extracted features into a carrier image. However, different regions of the secret image have different semantic importance: regions with complex textures and contents have higher semantic importance, and more features need to be reserved for reconstruction; the semantic importance of the region with simple texture and content is lower, and better reconstruction effect can be realized only by partial features. Since redundant embedding of secret image features may affect the perceptual properties of the carrier image, it is necessary to process the secret image from a high-order feature level, considering image information hiding methods based on semantic weighting.
Disclosure of Invention
The invention provides an image information hiding method based on semantic weighting to solve the problems, which combines an automatic encoder and provides an image information hiding method based on semantic weighting, wherein extracted secret image features are subjected to semantic clipping according to the semantic importance of a secret image, and the clipped secret image features are embedded into a carrier image with the same size as the secret image to generate a secret-carrying image.
The method comprises the following specific steps:
step one, constructing an image information hiding system model;
the system model comprises a sending end and a receiving end.
The whole system comprises the following processing procedures: 1) a sending end randomly selects a carrier image and a secret image, performs feature extraction and semantic importance analysis on the secret image to generate a semantic importance graph representing semantic importance degrees of different regions of the secret image, cuts the features of the secret image according to the semantic importance graph, and only retains key features influencing the quality of a secret image reconstructed by a receiving end; 2) the sending end performs channel connection on semantic weighting, namely, the secret image features subjected to semantic clipping and a carrier image, realizes depth fusion and embedding by using a depth neural network, generates a secret-carrying image with a perception characteristic similar to that of the carrier image, and transmits the secret-carrying image to the receiving end through a public channel; 3) and the receiver receives the secret image, extracts the secret image features from the secret image by using the deep neural network, and recovers to obtain the reconstructed secret image.
The main evaluation indexes of the system comprise the similarity degree of the perceptual characteristics of the carrier image and the secret image, and the perceptual characteristic degree of the secret image and the reconstructed secret image.
Secondly, constructing a network model of an image information hiding method based on semantic weighting;
the network model comprises a semantic preprocessing network, a hidden network and a decryption network. The sending end utilizes a semantic preprocessing network and a hidden network to realize the embedding of the secret image into the carrier image to generate the secret-carrying image, and the receiving end utilizes a decryption network to recover the secret-carrying image to obtain a reconstructed secret image. The detailed network model modeling steps are as follows:
step 201, constructing a semantic preprocessing network model;
the purpose of the semantic pre-processing network is as follows: 1) in the invention, the sizes of the secret image and the carrier image are both 64 multiplied by 3 as an example, wherein 64 respectively represents the number of pixels of the image width and the image height, and 3 represents the number of RGB channels of the image; 2) extracting the features of the secret image; 3) and generating a semantic importance graph representing the semantic importance degree of different areas of the secret image, cutting the features of the secret image according to the semantic importance graph, and connecting and embedding the features of the secret image with the carrier image on a channel.
The network model comprises a feature extraction module, a semantic importance analysis module and a feature clipping module.
The feature extraction module receives a secret image of size 64 × 64 × 3 and outputs features of the secret image of size 64 × 64 × 65, and the network configuration includes two convolution modules (Conv blocks). Conv Block is a multi-scale feature fusion network, and features of images in different scales are extracted through a plurality of different convolution kernels and then feature fusion is carried out. Specifically, the sizes of convolution kernels are respectively 3 × 3, 4 × 4 and 5 × 5, the number of convolution kernel channels is respectively 50, 10 and 5, the convolution step length is 1, the activation functions are ReLU functions, and after feature extraction of the three scales is completed, the extracted features are connected in channel dimensions.
The semantic importance analysis module takes the secret image features with the size of 64 multiplied by 65 output by the feature extraction module as input, and outputs a secret image semantic importance map with the size of 64 multiplied by 1. The network structure is composed of two Conv blocks and a convolution layer, the size of a convolution kernel of the convolution layer is 3 multiplied by 3, the number of the convolution kernel channels and the convolution step length are both 1, and an activation function is a Sigmoid function. Analyzing semantic information of each pixel point of the secret image through Conv Block, calculating the semantic importance degree of each pixel point through the convolution layer, and mapping the semantic importance degree into a value between 0 and 1 through a Sigmoid function to obtain a semantic importance graph of the secret image. The larger the semantic importance graph value at a certain pixel point is, the more important the pixel point is represented, and the more the number of characteristic channels need to be reserved and embedded into the carrier image for reconstruction.
The feature clipping module takes the secret image features with the size of 64 multiplied by 65 output by the feature extraction module and the secret image semantic importance map with the size of 64 multiplied by 1 output by the semantic importance analysis module as input, and outputs the secret image features with the size of 64 multiplied by 65 and subjected to semantic weighted clipping. The module is formed by mask generation and dot multiplication cutting, a semantic importance mask with the same size as the secret image feature is generated by a secret image semantic importance graph, then the mask and the secret image feature are subjected to dot multiplication, key features required for reconstructing a secret image are reserved, and redundant features are cut, namely the feature value is set to be 0, so that the effect of well reconstructing the secret image is achieved, and the influence of embedded secret image features on the perception characteristic of a carrier image is reduced.
Assuming that the features of the secret image output by the feature extraction module are F (W, H, C), and the semantic importance map of the secret image output by the semantic importance analysis module is M (W, H,1), each feature value of the features of the secret image output by the feature clipping module should be expressed as:
w and H respectively represent the width and height of an image or a feature, C represents the number of channels of the feature, F' represents a secret image feature subjected to semantic weighted clipping by the feature clipping module, and (W, H, C) represents a certain position in the feature map, W is 1.
Specifically, mask generation first needs to make quantization approximation on the semantic significance map, and is divided into L quantization levels:
where Q denotes a quantization operation, L denotes a certain quantization level, and L ═ 1.
And expanding the semantic importance map M (W, H,1) into a semantic importance mask M' (W, H, C) with the same size as the features of the secret image after mask generation, wherein each position of the semantic importance mask is 0 or 1, 0 represents that the feature point of the secret image corresponding to the position needs to be cut, and 1 represents that the feature point of the secret image corresponding to the position needs to be reserved. The larger the semantic importance map weight of a certain pixel point is, the more the number of channels with the semantic importance mask value of 1 of the pixel point is, and the more the secret image features reserved and embedded in the carrier image are. The semantic importance mask may have position values expressed as:
performing point multiplication on the semantic importance mask and the secret image characteristics by point multiplication cutting to realize semantic weighting, reserving key characteristics influencing the effect of reconstructing the secret image, abandoning redundant characteristics, and obtaining each characteristic value of the cut secret image characteristics as follows:
F′(w,h,c)＝F(w,h,c)*M(w,h,c)
step 202, constructing a hidden network model;
the hidden network aims to realize the embedding and the depth fusion of the secret image characteristics and the carrier image, generate the carrier image with similar perception characteristics to the carrier image, transmit the carrier image in a public channel and avoid the perception, the stealing and the attack of a third party. Since the human sensory system is insensitive to high frequency of the image or subtle changes of texture and content complex areas, the hidden network is responsible for analyzing and determining the area and embedding the secret image features.
The hidden network firstly needs to connect the secret image features which are output by the semantic preprocessing network and subjected to semantic weighted clipping with the size of 64 × 64 × 65 and the carrier image with the size of 64 × 64 × 3 on a channel to obtain a preliminary combination of the secret image with the size of 64 × 64 × 68 and the carrier image, and then outputs the secret image with the size of 64 × 64 × 3 by taking the preliminary combination as input. The network model structure is composed of five Conv blocks and a convolution layer, the convolution kernel size of the convolution layer is 3 x 3, the number of convolution kernel channels is 3, the convolution step length is 1, and the activation function is a Tanh function. Firstly, determining a region suitable for embedding the secret image features in the carrier image through Conv Block, realizing the depth fusion of the region and the region, and then mapping the fusion result to a value between-1 and 1 through a Tanh function to obtain the secret-carrying image.
Step 203, constructing a decryption network model;
the purpose of the decryption network is to process the received secret-carrying image, extract the embedded secret image features from it, and recover the reconstructed secret image. The restored reconstructed secret image should have similar perception characteristics with the original secret image of the sender, so that the receiver can perform subsequent tasks by using the secret image to realize the communication function of the information hiding book.
The decryption network receives the encrypted image of 64 × 64 × 3 size as an input, and outputs a reconstructed secret image of 64 × 64 × 3 size. The network structure of the convolutional code comprises five Conv blocks and a convolutional layer, the size of a convolutional kernel of the convolutional layer is 3 multiplied by 3, the number of channels of the convolutional kernel is 3, the convolution step is 1, and an activation function is a Tanh function. Firstly, analyzing the secret-carrying image characteristics through Conv Block, extracting the embedded secret image characteristics subjected to semantic weighting cutting from the secret-carrying image characteristics, filling and recovering the secret image characteristics, and mapping the extraction result into a value between-1 and 1 through a Tanh function to obtain the secret-carrying image.
And determining a region suitable for embedding the secret image features in the carrier image, realizing the depth fusion of the region and the region, and mapping the fusion result into a value between-1 and 1 through a Tanh function to obtain a reconstructed secret image.
Thirdly, constructing a reverse gradient propagation model of the image information hiding method based on semantic weighting;
end-to-end training of the network model is needed for image information hiding, and continuous iteration of reverse gradient propagation is needed for training of the network model. Both a hidden network and a decryption network in a network model can realize reverse gradient propagation, but the reverse gradient propagation is interrupted because quantization and semantic significance map channel expansion in a mask generation module of a semantic preprocessing network are not conducive, and the gradient in front of the module is zero at any place and cannot be iterated. To solve this problem, mask generation is re-expressed as:
wherein the content of the first and second substances,
representing an ceiling function.
The mask-generated inverse gradient can be expressed as:
note that the unrepresented approach is still employed in forward propagation, and this re-representation is only used in backward gradient propagation. Therefore, the network model can be guided everywhere, the reverse gradient can complete transmission, and end-to-end training can be carried out.
Step four, constructing a loss function model;
the network model training relies on a loss function model, and the similarity degree of the perceptual characteristics of the carrier image and the secret image and the perceptual characteristic degree of the secret image and the reconstructed secret image need to be considered. The similarity degree of the perception characteristics of the carrier image and the secret-carrying image determines the information hiding effect and the safety, and the higher the similarity degree is, the better the safety is, and the better the hiding effect is. The perception characteristic degree of the secret image and the reconstructed secret image determines the realization degree of the communication function, the higher the similarity degree is, the better the transmission effect of the secret image is, and the higher the processing precision of the subsequent task of the receiving end is. If only one party is considered, the network increasingly tends to ensure the similarity degree of the perception characteristics, and if the other party is ignored, the generated secret image may not contain the secret image in extreme cases, resulting in communication failure, or may only contain the secret image, resulting in perception, stealing and attack by a third party.
Therefore, the loss function model comprises two parts, and the degree of similarity of the perceptual characteristics of the two images is measured by Mean Square Error (MSE), and the smaller the MSE, the higher the degree of similarity. The MSE calculation formula is as follows:
wherein x and y represent two images of the degree of similarity of the perceptual properties to be compared, respectively.
Assuming that C and C 'represent the carrier image and the secret image, respectively, and S' represent the secret image and the reconstructed secret image, respectively, the loss function can be expressed as:
Loss(C,C′,S,S′)＝Loss_C+β*Loss_S＝MSE(C,C′)+β*MSE(S,S′)
the Loss functions of the similarity degree of the carrier image and the secret image and the similarity degree of the reconstructed secret image are respectively represented by the Loss _ C and the Loss _ S, and the weight of the two is represented by the beta and is used for balancing the quality of the secret image and the reconstructed secret image. The larger beta represents the more critical the quality of the reconstructed secret image.
Constructing a deep neural network training model on the basis of a network model and a loss function model of the semantic weighting-based image information hiding method;
step 501, constructing a data set;
and selecting a Tiny-ImageNet data set. The data set has 200 image classes, each image class comprises 500 training images, 50 verification images and 50 test images, and the data set is one of the most common data sets in the field of computer vision at present.
In order to ensure that the network model has reasonable and uniform input, all images are randomly cut into sizes of 64 multiplied by 3, the value ranges of RGB channels from 0 to 255 are mapped into the value ranges from 0 to 1, the images are standardized, the images are in accordance with the standard normal distribution with the mean value of 0 and the standard deviation of 1, and the convergence speed of the network model is accelerated. Normalization can be expressed as:
wherein x denotes image data, μxRepresenting the mean, σ, of the imagexRepresenting the standard deviation of the image.
10000 training images are randomly selected from the processed Tiny-ImageNet data set to serve as a training set, the training set is divided into two groups, and a secret image-carrier image pair is formed, so that training of the network model is carried out.
Step 502, setting hyper-parameters;
based on UBUNTU operating system and NVIDIA GTX1080 display card build training environment, utilize PyTorch frame to realize the network model, its main super parameter sets up as follows: 1) the Batch Size (Batch Size) is set to 64, and the number of training rounds (Epoch) is set to 300; 2) the semantic preprocessing network, the hidden network and the decryption network are optimized by using an Adam optimizer, and the learning rate is set to be 0.001; 3) the semantic significance map quantization level L is set to 13; 4) the loss function weights β are set to 0.75, 1 and 1.25.
Step six, constructing a deep neural network test and system performance evaluation model on the basis of a network model and a training model of a semantic weighting-based image information hiding method;
step 601, constructing a network test model;
10000 test images in the processed Tiny-ImageNet data set are used as a test set, and the test set is divided into two groups to form a secret image-carrier image pair, so that the network model is tested. The test model hyper-parameters are the same as during training.
Step 602, constructing a system performance evaluation model;
the image information hiding system performance comprises the load amount of the carrier image, the similarity degree of the perceptual characteristics of the carrier image and the secret image, and the perceptual characteristic degree of the secret image and the reconstructed secret image.
The carrier image load has an objective performance index absolute embedding capacity and a relative embedding capacity. The absolute embedding capacity refers to the number of bytes of the secret image contained in the carrier image, and the relative embedding capacity refers to the number of bytes of the secret image contained in the carrier image per pixel on average.
The Similarity degree of the two image sensing characteristics includes an Average Pixel Deviation (APD) of objective performance indexes, an Average Pixel Square Deviation (APSD) of Average Pixel Square deviations, a Peak Signal to Noise Ratio (PSNR) and a Structural Similarity Index (SSIM).
APD refers to the deviation of the average per-pixel per RGB channel value of the two images, and APSD refers to the square of the deviation of the average per-pixel per RGB channel value of the two images. The smaller the APD and APSD values are, the higher the similarity degree of the two image sensing characteristics is.
PSNR refers to the ratio of the maximum power of a signal to the power of the noise of the signal, and a secret image embedded in a carrier image can be regarded as noise affecting the quality of the carrier image, and a carrier image embedded in a secret image can be regarded as noise affecting the quality of the secret image. The PSNR value is larger, the similarity degree of the two image perception characteristics is higher, and the calculation formula is as follows:
where n represents the number of bits per pixel.
The SSIM measures the correlation between different areas of an image in three aspects of brightness, contrast and structure. The larger the SSIM value is, the higher the similarity degree of the two image perception characteristics is, and the calculation formula is as follows:
SSIM(x,y)＝[l(x,y)α·c(x,y)β·s(x,y)γ]
where x and y represent two images respectively, l (x, y) represents brightness comparison, c (x, y) represents contrast comparison, s (x, y) represents texture comparison,. mu.xAnd muyDenotes the mean value of x and y, σxAnd σyDenotes the standard deviation, σ, of x and yxyRepresents the covariance, c1、c2And c3Is normally done, avoiding denominator being 0.α, β, and γ are generally set to 1, and there is a calculation formula as follows:
the similarity degree of the perception characteristics of the two images has subjective performance indexes, and the residual errors of the two images are compared with the histogram statistical characteristics. And evaluating whether color difference, artifacts and the like exist in the residual errors of the two images, wherein the smaller the residual error is, the better the visual effect is. And evaluating the third-party steganalysis resistance by using the histogram statistical characteristics, wherein the closer the histogram statistical characteristics are, the better the steganalysis resistance is.
The invention has the advantages that:
(1) a network model comprises a semantic preprocessing network, a hidden network and a decryption network, so that on the premise of ensuring the quality of a reconstructed secret image, a larger load and a better hidden effect are realized, and the safety of information hiding is improved;
(2) a semantic preprocessing network for processing a secret image from a high-level feature level, unlike the existing method for processing only from a low-level pixel level, the network takes into account the change of local semantic importance of the secret image, reserves more features for areas with complex textures and contents for reconstruction, crops redundant features for areas with simple textures and contents, and avoids the influence of embedding on the quality of a carrier image;
(3) an image information hiding method based on semantic weighting is characterized in that the similarity degree of the perception characteristics of a carrier image and a secret image and the perception characteristic degrees of the secret image and a reconstructed secret image are considered to construct a loss function, training and testing are carried out through a Tiny-ImageNet data set, and the performance of a system is evaluated according to various objective and subjective indexes.
Drawings
FIG. 1 is a schematic diagram of an image information hiding system model constructed by the present invention;
FIG. 2 is a schematic diagram of a network model of an image information hiding method based on semantic weighting constructed by the invention;
FIG. 3 is a schematic diagram of a network structure of a convolution module in a constructed network model according to the present invention;
FIG. 4 is a graph of convergence performance and convergence result of training rounds with a loss function weight β of 1 according to the present invention;
FIG. 5 is a graph of hiding and reconstructing effects and training round number performance and results when the loss function weight β is taken as 1 according to the present invention;
FIG. 6 shows the residual between the carrier image and the secret image, and between the secret image and the reconstructed secret image when the loss function weight β is 1;
fig. 7 shows the histogram statistical feature comparison between the carrier image and the secret image, and between the secret image and the reconstructed secret image when the loss function weight β is 1.
Detailed Description
In order that the technical principles of the invention may be more clearly understood, embodiments of the invention are described in detail below with reference to the accompanying drawings.
An Image information hiding Method (A Semantic-Based Image steganographic Method) Based on Semantic weighting is applied to an Image information hiding system and comprises a sending end and a receiving end. The sending end embeds the secret image into the carrier image, the secret image is transmitted to the receiving end through transmitting the secret image, perception of a third party is avoided, and the receiving end recovers and reconstructs the secret image from the received secret image so as to carry out follow-up tasks. The existing method is difficult to meet the requirements of efficient transmission of massive secret images and the requirements of safe transmission of a powerful third-party steganalysis technology, the secret images are processed only from a low-order pixel level, the change of the local semantic importance of the secret images is not considered, and all the secret image features are embedded into the carrier images. The method processes the secret image from a high-order feature level, considers the local semantic importance of the secret image, reserves more features for the region with complex texture and content to be reconstructed, and cuts redundant features for the region with simple texture and content to avoid the influence on the carrier image.
According to the above method concept, the challenges faced by the present invention include: 1) the semantic importance of different regions of different images is different, a reasonable and general image semantic importance extraction method needs to be considered, and specific implementation of retention and cutting features according to local semantic importance change is designed; 2) because the end-to-end training of the network model depends on the condition that the network model can be guided everywhere, the gradient can not be propagated reversely under the condition that the network model can not be guided, and in order to prevent the training failure, a reverse gradient propagation model needs to be designed for the operation that the network model can not be guided.
The whole process comprises six steps of constructing an image information hiding system model, constructing a network model of an image information hiding method based on semantic weighting, constructing a reverse gradient propagation model, constructing a loss function model, training according to the network model and the loss function, and testing according to the network model and a training result. The method comprises the steps of constructing a semantic weighting-based network model of the image information hiding method, constructing a semantic preprocessing network model, constructing a hidden network model and constructing a decryption network model, training according to the network model and a loss function, constructing a data set and setting hyper-parameters, and testing according to the network model and a training result, constructing a testing machine and constructing a system performance evaluation model.
The method comprises the following specific steps:
step one, constructing an image information hiding system model;
as shown in fig. 1, the system model includes a transmitting end and a receiving end.
The whole system comprises the following processing procedures: 1) a sending end randomly selects a carrier image and a secret image, performs feature extraction and semantic importance analysis on the secret image to generate a semantic importance graph representing semantic importance degrees of different regions of the secret image, cuts the features of the secret image according to the semantic importance graph, and only retains key features influencing the quality of a secret image reconstructed by a receiving end; 2) the sending end performs channel connection on semantic weighting, namely, the secret image features subjected to semantic clipping and a carrier image, realizes depth fusion and embedding by using a depth neural network, generates a secret-carrying image with a perception characteristic similar to that of the carrier image, and transmits the secret-carrying image to the receiving end through a public channel; 3) and the receiver receives the secret image, extracts the secret image features from the secret image by using the deep neural network, and recovers to obtain the reconstructed secret image.
The main evaluation indexes of the system comprise the similarity degree of the perceptual characteristics of the carrier image and the secret image, and the perceptual characteristic degree of the secret image and the reconstructed secret image.
Secondly, constructing a network model of an image information hiding method based on semantic weighting;
as shown in fig. 2, the network model includes a semantic preprocessing network, a hidden network, and a decryption network. The sending end utilizes a semantic preprocessing network and a hidden network to realize the embedding of the secret image into the carrier image to generate the secret-carrying image, and the receiving end utilizes a decryption network to recover from the secret-carrying image to obtain a reconstructed secret image. The detailed network model modeling steps are as follows:
step 201, constructing a semantic preprocessing network model;
the purpose of the semantic pre-processing network is as follows: 1) in the invention, the secret image and the carrier image are both 64 multiplied by 3 as an example, wherein 64 respectively represents the number of pixels of image width and height, and 3 represents the number of RGB channels of the image; 2) extracting the features of the secret image; 3) and generating a semantic importance graph representing the semantic importance degree of different areas of the secret image, cutting the features of the secret image according to the semantic importance graph, and connecting and embedding the features of the secret image with the carrier image on a channel.
The network model comprises a feature extraction module, a semantic importance analysis module and a feature clipping module.
The feature extraction module receives a secret image having a size of 64 × 64 × 3, outputs features of the secret image having a size of 64 × 64 × 65, and has a network structure including two convolution modules (Conv blocks). As shown in fig. 3, Conv Block is a multi-scale feature fusion network, and features of different scales of an image are extracted through a plurality of different convolution kernels and then feature fusion is performed. Specifically, the sizes of convolution kernels are respectively 3 × 3, 4 × 4 and 5 × 5, the number of convolution kernel channels is respectively 50, 10 and 5, the convolution step length is 1, the activation functions are ReLU functions, and after feature extraction of the three scales is completed, the extracted features are connected in channel dimensions.
The semantic importance analysis module takes the secret image features with the size of 64 multiplied by 65 output by the feature extraction module as input, and outputs a secret image semantic importance map with the size of 64 multiplied by 1. The network structure is composed of two Conv blocks and a convolution layer, the size of a convolution kernel of the convolution layer is 3 multiplied by 3, the number of the convolution kernel channels and the convolution step length are both 1, and an activation function is a Sigmoid function. Analyzing semantic information of each pixel point of the secret image through Conv Block, calculating the semantic importance degree of each pixel point through the convolution layer, and mapping the semantic importance degree into a value between 0 and 1 through a Sigmoid function to obtain a semantic importance graph of the secret image. The larger the semantic importance graph value at a certain pixel point is, the more important the pixel point is represented, and the more the number of characteristic channels need to be reserved and embedded into the carrier image for reconstruction.
The feature clipping module takes the secret image features with the size of 64 multiplied by 65 output by the feature extraction module and the secret image semantic importance map with the size of 64 multiplied by 1 output by the semantic importance analysis module as input, and outputs the secret image features with the size of 64 multiplied by 65 and subjected to semantic weighted clipping. The module is formed by mask generation and dot multiplication cutting, firstly, a semantic importance mask with the same size as the secret image characteristic is generated by a secret image semantic importance graph, then, dot multiplication is carried out on the mask and the secret image characteristic, the key characteristic required by secret image reconstruction is reserved, the redundant characteristic is cut, namely, the characteristic value is set to be 0, and therefore the effect of the embedded secret image characteristic on the carrier image perception characteristic is reduced while the good secret image reconstruction effect is achieved.
Assuming that the features of the secret image output by the feature extraction module are F (W, H, C), and the semantic importance map of the secret image output by the semantic importance analysis module is M (W, H,1), each feature value of the features of the secret image output by the feature clipping module should be expressed as:
w and H respectively represent the width and height of an image or a feature, C represents the number of channels of the feature, F' represents a secret image feature subjected to semantic weighting clipping by the feature clipping module, W, H, C) represents a certain position in a feature map, W is 1.
Specifically, mask generation first needs to make quantization approximation on the semantic significance map, which is divided into L quantization levels:
where Q denotes a quantization operation, L denotes a certain quantization level, and L ═ 1.
And expanding the semantic importance map M (W, H,1) into a semantic importance mask M' (W, H, C) with the same size as the secret image features, wherein each position of the semantic importance mask takes a value of 0 or 1, 0 represents that the secret image feature point corresponding to the position needs to be cut, and 1 represents that the secret image feature point corresponding to the position needs to be reserved. The larger the semantic importance map weight of a certain pixel point is, the more channels with the semantic importance mask value of 1 are, and the more secret image features are reserved and embedded into the carrier image. The semantic importance mask may have position values expressed as:
performing point multiplication on the semantic importance mask and the secret image characteristics by point multiplication cutting to realize semantic weighting, reserving key characteristics influencing the effect of reconstructing the secret image, abandoning redundant characteristics, and obtaining each characteristic value of the cut secret image characteristics as follows:
F′(w,h,c)＝F(w,h,c)*M(w,h,c)
step 202, constructing a hidden network model;
the hidden network aims to realize the embedding and the depth fusion of the secret image characteristics and the carrier image, generate the carrier image with similar perception characteristics to the carrier image, transmit the carrier image in a public channel and avoid the perception, the stealing and the attack of a third party. Since the human sensory system is insensitive to high frequency of the image or subtle changes of texture and content complex areas, the hidden network is responsible for analyzing and determining the area and embedding the secret image features.
The hidden network firstly needs to connect the secret image features which are output by the semantic preprocessing network and subjected to semantic weighted clipping with the size of 64 × 64 × 65 and the carrier image with the size of 64 × 64 × 3 on a channel to obtain a preliminary combination of the secret image with the size of 64 × 64 × 68 and the carrier image, and then outputs the secret image with the size of 64 × 64 × 3 by taking the preliminary combination as input. The network model structure is composed of five Conv blocks and a convolution layer, the convolution kernel size of the convolution layer is 3 x 3, the number of convolution kernel channels is 3, the convolution step length is 1, and the activation function is a Tanh function. Firstly, determining a region suitable for embedding the secret image features in the carrier image through Conv Block, realizing the deep fusion of the region and the region, and then mapping the fusion result to a value between-1 and 1 through a Tanh function to obtain the secret-carrying image.
Step 203, constructing a decryption network model;
the purpose of the decryption network is to process the received secret-carrying image, extract the embedded secret image features from it, and recover the reconstructed secret image. The restored reconstructed secret image has similar perception characteristics with the original secret image of the sender, so that the receiver can perform subsequent tasks by using the secret image, and the communication function of the information hiding book is realized.
The decryption network receives the encrypted image of 64 × 64 × 3 as an input, and outputs a reconstructed secret image of 64 × 64 × 3. The network structure of the convolutional code comprises five Conv blocks and a convolutional layer, the size of a convolutional kernel of the convolutional layer is 3 multiplied by 3, the number of channels of the convolutional kernel is 3, the convolution step is 1, and an activation function is a Tanh function. Firstly, analyzing the secret-carrying image characteristics through Conv Block, extracting the embedded secret image characteristics subjected to semantic weighting cutting from the secret-carrying image characteristics, filling and recovering the secret image characteristics, and mapping the extraction result into a value between-1 and 1 through a Tanh function to obtain a reconstructed secret image.
Thirdly, constructing a reverse gradient propagation model of the image information hiding method based on semantic weighting;
end-to-end training of the network model is needed for image information hiding, and continuous iteration of reverse gradient propagation is needed for training of the network model. Both a hidden network and a decryption network in a network model can realize reverse gradient propagation, but the reverse gradient propagation is interrupted because quantization and semantic significance map channel expansion in a mask generation module of a semantic preprocessing network are not conducive, and the gradient in front of the module is zero at any place and cannot be iterated. To solve this problem, mask generation is re-expressed as:
wherein, the first and the second end of the pipe are connected with each other,
representing an ceiling function.
The mask-generated inverse gradient can be expressed as:
note that the unrepresented approach is still adopted in forward propagation, and this re-representation is only used in backward gradient propagation. Therefore, the network model is everywhere conductive, the reverse gradient can complete transmission, and end-to-end training can be carried out.
Step four, constructing a loss function model;
the network model training relies on a loss function model, and the similarity degree of the perceptual characteristics of the carrier image and the secret image and the perceptual characteristic degree of the secret image and the reconstructed secret image need to be considered. The similarity degree of the perception characteristics of the carrier image and the secret-carrying image determines the information hiding effect and the safety, and the higher the similarity degree is, the better the safety is, and the better the hiding effect is. The perception characteristic degree of the secret image and the reconstructed secret image determines the realization degree of the communication function, the higher the similarity degree is, the better the transmission effect of the secret image is, and the higher the processing precision of the subsequent task of the receiving end is. If only one party is considered, the network increasingly tends to ensure the similarity degree of the perception characteristics, and if the other party is ignored, the generated secret image may not contain the secret image in extreme cases, resulting in communication failure, or may only contain the secret image, resulting in perception, stealing and attack by a third party.
Therefore, the loss function model comprises two parts, and the degree of similarity of the perceptual characteristics of the two images is measured by Mean Square Error (MSE), and the smaller the MSE, the higher the degree of similarity. The MSE calculation formula is as follows:
wherein x and y respectively represent two images with similar perception characteristics to be compared.
Assuming that C and C 'represent the carrier image and the secret image, respectively, and S' represent the secret image and the reconstructed secret image, respectively, the loss function can be expressed as:
Loss(C,C′,S,S′)＝Loss_C+β*Loss_S＝MSE(C,C′)+β*MSE(S,S′)
the Loss functions of the similarity degree of the carrier image and the secret image and the similarity degree of the reconstructed secret image are respectively represented by Loss _ C and Loss _ S, and the weight beta represents the weight of the two, so that the quality of the secret image and the reconstructed secret image is balanced. The larger beta represents the more critical the quality of the reconstructed secret image.
Constructing a deep neural network training model on the basis of a network model and a loss function model of the semantic weighting-based image information hiding method;
step 501, constructing a data set;
and selecting a Tiny-ImageNet data set. The data set comprises 200 image classes, wherein each image class comprises 500 training images, 50 verification images and 50 test images, and is one of the most commonly used data sets in the field of computer vision at present.
In order to ensure that the network model has reasonable and uniform input, all images are randomly cut into sizes of 64 multiplied by 3, the value ranges of RGB channels from 0 to 255 are mapped into the value ranges from 0 to 1, the images are standardized, the images are in accordance with the standard normal distribution with the mean value of 0 and the standard deviation of 1, and the convergence speed of the network model is accelerated. Normalization can be expressed as:
where x denotes image data, μxRepresenting the mean, σ, of the imagexRepresenting the standard deviation of the image.
10000 training images are randomly selected from the processed Tiny-ImageNet data set to serve as a training set, the training set is divided into two groups evenly, and a secret image-carrier image pair is formed, so that the training of the network model is carried out.
Step 502, setting hyper-parameters;
based on UBUNTU operating system and NVIDIA GTX1080 display card build training environment, utilize PyTorch frame to realize the network model, its main super parameter sets up as follows: 1) the Batch Size (Batch Size) is set to 64, and the number of training rounds (Epoch) is set to 300; 2) the semantic preprocessing network, the hidden network and the decryption network are optimized by using an Adam optimizer, and the learning rate is set to be 0.001; 3) the semantic significance map quantization level L is set to 13; 4) the penalty function weights β are set to 0.75, 1 and 1.25.
Constructing a deep neural network test and system performance evaluation model on the basis of a network model and a training model of the semantic weighting-based image information hiding method;
step 601, constructing a network test model;
and taking 10000 test images of the processed Tiny-ImageNet data set as a test set, and equally dividing the test set into two groups to form a secret image-carrier image pair so as to test the network model. The test model hyper-parameters are the same as during training.
Step 602, constructing a system performance evaluation model;
the image information hiding system performance comprises the load amount of the carrier image, the similarity degree of the perceptual characteristics of the carrier image and the secret image, and the perceptual characteristic degree of the secret image and the reconstructed secret image.
The carrier image loading has an objective performance index absolute embedding capacity and a relative embedding capacity. The absolute embedding capacity refers to the number of bytes of the secret image contained in the carrier image, and the relative embedding capacity refers to the number of bytes of the secret image contained in the carrier image per pixel on average.
The Similarity degree of the two image sensing characteristics includes an Average Pixel Deviation (APD) as an objective performance Index, an Average Square Deviation (APSD) as an Average Pixel Deviation, a Peak Signal to Noise Ratio (PSNR) as a Peak Signal to Noise Ratio (PSNR), and a Structural Similarity Index (SSIM).
APD refers to the deviation of the average per-pixel per RGB channel value of the two images, and APSD refers to the square of the deviation of the average per-pixel per RGB channel value of the two images. The smaller the APD and APSD values are, the higher the similarity degree of the two image sensing characteristics is.
PSNR refers to the ratio of the maximum power of a signal to the power of the noise of the signal, and a secret image embedded in a carrier image can be regarded as noise affecting the quality of the carrier image, and a carrier image embedded in a secret image can be regarded as noise affecting the quality of the secret image. The PSNR value is larger, the similarity degree of the two image perception characteristics is higher, and the calculation formula is as follows:
where n represents the number of bits per pixel.
The SSIM measures the correlation between different areas of an image in three aspects of brightness, contrast and structure. The larger the SSIM value is, the higher the similarity degree of the two image perception characteristics is, and the calculation formula is as follows:
SSIM(x,y)＝[l(x,y)α·c(x,y)β·s(x,y)γ]
where x and y represent two images respectively, l (x, y) represents brightness comparison, c (x, y) represents contrast comparison, s (x, y) represents texture comparison,. mu.xAnd muyDenotes the mean, σ, of x and yxAnd σyDenotes the standard deviation, σ, of x and yxyRepresents the covariance, c1、c2And c3Is normally done, avoiding denominator being 0.α, β, and γ are generally set to 1, and there is a calculation formula as follows:
the similarity degree of the perception characteristics of the two images has subjective performance indexes, and the residual errors of the two images are compared with the histogram statistical characteristics. And evaluating whether color difference, artifacts and the like exist in the residual errors of the two images, wherein the smaller the residual error is, the better the visual effect is. And evaluating the third-party steganalysis resistance by the histogram statistical characteristics, wherein the closer the histogram statistical characteristics are, the better the steganalysis resistance is.
Taking the value of the loss function weight β as 1 as an example, the convergence performance and the convergence result of the training round number (Epoch) are shown in fig. 4. The loss function value of the network model decreases as the number of training rounds increases, reaching convergence after approximately 150 epochs.
Taking the value of the loss function weight β of 1 as an example, the hiding and reconstructing effect and the performance and result of the training rounds are shown in fig. 5. The loss function value of the network model decreases as the number of training rounds increases, reaching convergence after approximately 150 epochs. The network model of (a) is only initialized and is not trained, and the generated secret image and the reconstructed secret image are empty; (b) the network model generates a secret image carrying and a reconstructed secret image first-seen prototype after being trained by 1 Epoch; (c) the network model generates a secret-carrying image and a reconstructed secret image with better effect after being trained by 20 epochs, but the secret image is secretly visible in the secret-carrying image, the secret image and the reconstructed secret image have obvious chromatic aberration, and details are lost; (d) and after the network model training is finished, the carrier image is highly similar to the secret-carrying image, the secret image is highly similar to the reconstructed secret image, and the difference is not perceived from the visual effect, so that the function of hiding the image information is realized.
The invention aims to reduce the influence of the embedded secret image on the perception characteristic of the carrier image on the premise of ensuring the quality of the reconstructed secret image, realize larger load and better hiding effect and improve the safety of information hiding. In order to embody the advantages of the invention, objective performance index comparison is carried out on the carrier image load, the similarity degree of the carrier image and the perception characteristics of the secret image, and the similarity degree of the secret image and the perception characteristics of the reconstructed secret image with the existing image information hiding method without considering semantic weighting.
The invention embeds the secret image with the size of 64 multiplied by 3 into the carrier image with the size of 64 multiplied by 3, the absolute embedding capacity is 12288 bytes, the relative embedding capacity is 1 byte per pixel per channel, the invention is the same as the prior image information hiding method based on the automatic encoder without considering semantic weighting, is superior to the image information hiding method based on the generation countermeasure network, and has larger carrier image load capacity.
The objective comparison result of the similarity degree of the image perception characteristics of the image information hiding method disclosed by the invention and the existing image information hiding method without considering semantic weighting is shown in table 1, wherein beta represents the weight of a loss function. Along with the reduction of the value of the beta, the larger the proportion of the similarity degree of the perception characteristics of the carrier image and the secret-carrying image in the loss function is, the more the network model tends to generate the secret-carrying image with better hiding effect, and the safer the information hiding is. Compared with the existing image information hiding method without considering semantic weighting, the generated secret-carrying image has better quality on all beta values and all objective performance indexes, and has higher similarity with the perception characteristic of the carrier image. When the value of beta is 0.75 and 1.25, the quality of the reconstructed secret image generated by the method is superior to that of the existing image information hiding method without considering semantic weighting on all objective performance coordinates.
In order to visually embody the effect of the invention, subjective performance index evaluation is carried out on residual errors of the carrier image and the secret image, the secret image and the reconstructed secret image and histogram statistical characteristic comparison of the carrier image and the secret image, the secret image and the reconstructed secret image.
Fig. 6 shows the residual errors of the carrier image and the secret image, and the secret image and the reconstructed secret image of the invention, and the residual errors of the carrier image, the secret image, the carrier image and the secret image, the residual errors of the carrier image and the secret image, the reconstructed secret image, the residual errors of the secret image and the reconstructed secret image, and the residual errors of the secret image and the reconstructed secret image are respectively amplified by five times from left to right. The carrier image and the secret image are similar in visual effect, the secret image and the reconstructed secret image are small in color difference, the residual error between the carrier image and the secret image is irrelevant to the secret image, and the secret image and the reconstructed secret image are randomly distributed on all pixel points.
Fig. 7 shows the histogram statistical characteristics of the carrier image and the secret image, and the histogram statistical characteristics of the secret image and the reconstructed secret image of the present invention, compared with each other, and the histogram statistical characteristics of the carrier image, the secret image, the carrier image histogram statistical characteristics, the histogram statistical characteristics of the secret image, the reconstructed secret image, the histogram statistical characteristics of the secret image, and the histogram statistical characteristics of the reconstructed secret image are respectively from left to right. The histogram statistical characteristics of the carrier image and the secret image, and the histogram statistical characteristics of the secret image and the reconstructed secret image are similar, and the difference is uniformly distributed in all RGB channels, so that the method has the capability of resisting third-party steganalysis.
TABLE 1
In summary, by implementing the image information hiding method based on semantic weighting, the influence of the embedded secret image on the perception characteristic of the carrier image can be reduced on the premise of ensuring the quality of the reconstructed secret image, so that a larger load and a better hiding effect are realized, the safety of information hiding is improved, the requirement of safe transmission of massive secret information is met, and a powerful third-party steganalysis technology is met. Compared with the existing image information hiding method which only processes from a low-order pixel level and does not consider semantic weighting, the method processes the characteristics of the secret image from a high-order characteristic level according to the local semantic importance change of the secret image, reserves more characteristics for areas with complex textures and contents for reconstruction, and cuts redundant characteristics for the areas with simple textures and contents to avoid influencing the perception characteristics of the carrier image.
The foregoing is a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can be made, such as changing the network model infrastructure, replacing the convolution module with other network structures such as U-Net, dense network, etc., or applying the semantic preprocessing network of the present invention to video or text information hiding, and these improvements and modifications are also considered as the protection scope of the present invention.
Claims (3)
1. A semantic weighting-based image information hiding method specifically comprises the following steps:
step one, constructing an image information hiding system model;
the system model comprises a sending end and a receiving end;
the whole system comprises the following processing procedures: 1) a sending end randomly selects a carrier image and a secret image, performs feature extraction and semantic importance analysis on the secret image to generate a semantic importance graph representing semantic importance degrees of different regions of the secret image, cuts the features of the secret image according to the semantic importance graph, and only retains key features influencing the quality of a secret image reconstructed by a receiving end; 2) the sending end performs channel connection on semantic weighting, namely the secret image features subjected to semantic clipping and a carrier image, realizes deep fusion and embedding by using a deep neural network, generates a secret image with a perception characteristic similar to that of the carrier image, and transmits the secret image to the receiving end through a public channel; 3) the receiver receives the secret image, extracts the secret image features from the secret image by using the deep neural network, and recovers to obtain a reconstructed secret image;
the main evaluation indexes of the system comprise the similarity degree of the perception characteristics of the carrier image and the secret image, and the perception characteristic degrees of the secret image and the reconstructed secret image;
secondly, constructing a network model of an image information hiding method based on semantic weighting;
the network model comprises a semantic preprocessing network, a hidden network and a decryption network, wherein a sending end uses the semantic preprocessing network and the hidden network to realize the embedding of the secret image into the carrier image and generate the secret-carrying image, a receiving end uses the decryption network to recover the secret-carrying image to obtain a reconstructed secret image, and the detailed network model modeling step comprises the following steps:
step 201, constructing a semantic preprocessing network model;
the purpose of the semantic pre-processing network is as follows: 1) in the invention, the secret image and the carrier image are both 64 multiplied by 3 as an example, wherein 64 respectively represents the number of pixels of image width and height, and 3 represents the number of RGB channels of the image; 2) extracting the features of the secret image; 3) generating a semantic importance graph representing the semantic importance degree of different areas of the secret image, cutting the features of the secret image according to the semantic importance graph, and connecting and embedding the features of the secret image with a carrier image on a channel;
the network model comprises a feature extraction module, a semantic importance analysis module and a feature cutting module;
the method comprises the steps that a feature extraction module takes a secret image with the size of 64 x 3 as input and outputs the secret image features of 64 x 65, a network structure is composed of two convolution modules (Conv Block), the Conv Block is a multi-scale feature fusion network, the features of different scales of the image are extracted through a plurality of different convolution kernels, feature fusion is carried out, the sizes of the convolution kernels are respectively 3 x 3, 4 x 4 and 5 x 5, the number of the channels of the convolution kernels is respectively 50, 10 and 5, the convolution step size is 1, all activation functions are ReLU functions, and after feature extraction of the three scales is completed, the extracted features are connected in the channel dimensions;
the semantic importance analysis module takes the secret image features of 64 multiplied by 65 output by the feature extraction module as input, and outputs a secret image semantic importance map of 64 multiplied by 1, the network structure is composed of two Conv blocks and a convolution layer, the convolution kernel of the convolution layer is 3 multiplied by 3, the number of channels of the convolution kernel and the convolution step length are both 1, the activation function is a Sigmoid function, the semantic information of each pixel point of the secret image is analyzed through the Conv blocks, then the semantic importance degree of each pixel point is calculated by the convolution layer, and the Sigmoid function is mapped into a value between 0 and 1 to obtain the semantic importance map of the secret image, the larger the value of the semantic importance map at a certain pixel point is, the more important the pixel point is represented, and the more the number of feature channels which need to be reserved and embedded into the carrier image so as to be reconstructed is increased;
the feature clipping module takes a secret image feature with the size of 64 multiplied by 65 output by the feature extraction module and a secret image semantic significance map with the size of 64 multiplied by 1 output by the semantic significance analysis module as input, outputs a secret image feature with the size of 64 multiplied by 65 after semantic weighted clipping, and consists of mask generation and dot multiplication clipping, wherein the semantic significance mask with the size same as that of the secret image feature is generated by the secret image semantic significance map, then the mask and the secret image feature are subjected to dot multiplication, key features required for reconstructing the secret image are reserved, and clipping redundancy features are set to be 0, so that the effect of well reconstructing the secret image is ensured, and meanwhile, the influence of embedding the secret image feature on the perception characteristic of the carrier image is reduced;
assuming that the features of the secret image output by the feature extraction module are F (W, H, C), and the semantic importance map of the secret image output by the semantic importance analysis module is M (W, H,1), each feature value of the features of the secret image output by the feature clipping module should be expressed as:
w and H respectively represent the width and height of an image or a feature, C represents the number of channels of the feature, F' represents a secret image feature subjected to semantic weighted clipping by a feature clipping module, and (W, H, C) represents a certain position in a feature map, W is 1, a.
Specifically, mask generation first needs to make quantization approximation on the semantic significance map, and is divided into L quantization levels:
wherein Q denotes a quantization operation, L denotes a certain quantization level, and L ═ 1., L;
and expanding a semantic importance map M (W, H,1) into a semantic importance mask M' (W, H, C) with the same size as the features of the secret image, wherein each position of the semantic importance mask takes a value of 0 or 1, 0 represents that the feature point of the secret image corresponding to the position needs to be cut, 1 represents that the feature point of the secret image corresponding to the position needs to be reserved, the larger the semantic importance map weight of a certain pixel point is, the more channels with the semantic importance mask value of 1 are, the more secret image features are reserved and embedded into the carrier image, and the values of each position of the semantic importance mask can be expressed as:
and performing point multiplication on the semantic importance mask and the secret image characteristics by point multiplication cutting to realize semantic weighting, reserving key characteristics influencing the effect of reconstructing the secret image, abandoning redundant characteristics, and expressing each characteristic value of the cut secret image characteristics as follows:
F′(w,h,c)＝F(w,h,c)*M(w,h,c)
step 202, constructing a hidden network model;
the hidden network is used for analyzing and determining the area and embedding the secret image features because a human sensory system is insensitive to fine changes of high frequency or texture of the image and a complex area of content;
the hidden network firstly needs to connect the secret image features subjected to semantic weighting cutting and 64 × 64 × 65 output by the semantic preprocessing network and the carrier image with the size of 64 × 64 × 3 on a channel to obtain a secret image and a carrier image which are preliminarily combined, then the secret image and the carrier image are input to output a secret image with the size of 64 × 64 × 3, the structure of a network model of the hidden network is composed of five Conv blocks and a convolution layer, the size of a convolution kernel of the convolution layer is 3 × 3, the number of the channels of the convolution kernel is 3, the convolution step size is 1, an activation function is a Tanh function, an area suitable for embedding the secret image features in the carrier image is determined through Conv blocks, the depth fusion of the two is realized, and then a fusion result is mapped to a value between-1 and 1 through the Tanh function to obtain the secret image;
step 203, constructing a decryption network model;
the purpose of the decryption network is to process the received secret-carrying image, extract the embedded secret image characteristics from the secret-carrying image, and recover the reconstructed secret image, wherein the recovered reconstructed secret image has similar perception characteristics with the original secret image of the sender, so that the receiver can use the secret image to perform subsequent tasks, and the communication function of the information hiding book is realized;
the decryption network takes a received secret carrying image with the size of 64 multiplied by 3 as an input and outputs a reconstructed secret image with the size of 64 multiplied by 3, the network structure of the decryption network is composed of five Conv blocks and a convolution layer, the convolution kernel of the convolution layer is 3 multiplied by 3, the number of the channels of the convolution kernel is 3, the convolution step is 1, an activation function is a Tanh function, the secret carrying image characteristics are analyzed through the Conv blocks, the embedded secret image characteristics subjected to semantic weighted clipping are extracted from the secret carrying image characteristics and are filled and restored, and then the extraction result is mapped to a value between-1 and 1 through the Tanh function, so that the secret carrying image is obtained;
determining a region suitable for embedding the secret image features in the carrier image, realizing the depth fusion of the region and the region, and mapping the fusion result to a value between-1 and 1 through a Tanh function to obtain a reconstructed secret image;
step three, constructing a reverse gradient propagation model of the image information hiding method based on semantic weighting;
the network model needs to be trained end to realize image information hiding, the network model needs to be trained in a way of continuous iteration of reverse gradient propagation, both a hidden network and a decryption network in the network model can realize the reverse gradient propagation, but the reverse gradient propagation is interrupted because quantization and semantic significance map channel expansion in a mask generation module of a semantic preprocessing network are not conducive, the gradient in front of the module is zero, iteration cannot be carried out, and in order to solve the problem, mask generation is represented again as follows:
wherein, the first and the second end of the pipe are connected with each other,
represents an upward rounding function;
the mask-generated inverse gradient can be expressed as:
note that a non-representation mode is still adopted in forward propagation, and the representation is only used in reverse gradient propagation, so that the network model can be conducted everywhere, the reverse gradient can complete transmission, and end-to-end training can be performed;
step four, constructing a loss function model;
the network model training depends on a loss function model, the similarity degree of the perception characteristics of the carrier image and the secret-carrying image needs to be considered, and the perception characteristic degrees of the secret image and the reconstructed secret image, the perception characteristic similarity degree of the carrier image and the secret image determines the information hiding effect and the safety, the higher the similarity degree is, the better the safety is, the better the hiding effect is, the perception characteristic degree of the secret image and the reconstructed secret image determines the communication function realization degree, the higher the similarity degree is, the better the secret image transmission effect is, the higher the processing precision of the subsequent task of the receiving end is, if only one party is considered, the network tends to ensure the similarity degree of the perception characteristic increasingly, and ignores the other party, and the secret-carrying image generated under the extreme condition may not contain a secret image to cause communication failure, or only contains the secret image to cause perception, stealing and attack by a third party;
therefore, the loss function model comprises two parts, the degree of similarity of the perceptual characteristics of the two images is measured by Mean Square Error (MSE) of the two images, the smaller the MSE is, the higher the degree of similarity is, and the MSE calculation formula is as follows:
wherein x and y respectively represent two images with similar degrees of perceptual characteristics to be compared;
assuming that C and C 'represent the carrier image and the secret image, respectively, and S' represent the secret image and the reconstructed secret image, respectively, the loss function can be expressed as:
Loss(C,C′,S,S′)＝Loss_C+β*Loss_S＝MSE(C,C′)+β*MSE(S,S′)
the Loss functions of the similarity degree of the carrier image and the secret-carrying image and the similarity degree of the secret image and the reconstructed secret image are respectively represented by Loss _ C and Loss _ S, beta represents the weight of the two and is used for weighing the quality of the secret-carrying image and the reconstructed secret image, and the larger beta represents the more critical the quality of the reconstructed secret image;
constructing a deep neural network training model on the basis of a network model and a loss function model of a semantic weighting-based image information hiding method;
step 501, constructing a data set;
selecting a Tiny-ImageNet data set, wherein the data set comprises 200 image classes, each image class comprises 500 training images, 50 verification images and 50 test images, and the Tiny-ImageNet data set is one of the most common data sets in the field of computer vision at present;
in order to ensure that the network model has reasonable and uniform input, firstly all images are randomly cut into sizes of 64 multiplied by 3, then the value range of RGB channels from 0 to 255 is mapped into the value range from 0 to 1, the images are standardized to be in accordance with the standard normal distribution with the mean value of 0 and the standard deviation of 1, the convergence speed of the network model is accelerated, and the standardization can be expressed as follows:
where x denotes image data, μxRepresenting the mean, σ, of the imagexRepresents the standard deviation of the image;
10000 training images are randomly selected from the processed Tiny-ImageNet data set to serve as a training set, the training set is divided into two groups evenly to form a secret image-carrier image pair, and therefore training of the network model is conducted;
step 502, setting hyper-parameters;
building a training environment based on a UBUNTU operating system and an NVIDIA GTX1080 graphics card, and realizing a network model by using a PyTorch frame, wherein main super parameters are set as follows: 1) the Batch Size (Batch Size) is set to 64, and the number of training rounds (Epoch) is set to 300; 2) the semantic preprocessing network, the hidden network and the decryption network are optimized by using an Adam optimizer, and the learning rate is set to be 0.001; 3) the semantic significance map quantization level L is set to 13; 4) the loss function weights β are set to 0.75, 1 and 1.25;
constructing a deep neural network test and system performance evaluation model on the basis of a network model and a training model of the semantic weighting-based image information hiding method;
step 601, constructing a network test model;
using 10000 test images of the processed Tiny-ImageNet data set as a test set, and dividing the test set into two groups to form a secret image-carrier image pair so as to test the network model, wherein the hyper-parameters of the test model are the same as those during training;
step 602, constructing a system performance evaluation model;
the image information hiding system performance comprises carrier image load capacity, the degree of similarity of the perceptual characteristics of the carrier image and the secret image, the degree of perceptual characteristics of the secret image and the reconstructed secret image,
the carrier image load capacity comprises an objective performance index absolute embedded capacity and a relative embedded capacity, wherein the absolute embedded capacity refers to the number of bytes of a carrier image containing secret images, and the relative embedded capacity refers to the number of bytes of the secret images contained in each pixel of the carrier image on average;
the Similarity degree of the two image sensing characteristics comprises an Average APD (angular position Deviation), an Average APSD (angular position Square Deviation), a Peak Signal to Noise Ratio (PSNR) and a Structural Similarity Index (SSIM);
APD means the deviation of the average value of each pixel per RGB channel value of the two images, APSD means the deviation square of the average value of each pixel per RGB channel value of the two images, and the smaller the values of APD and APSD are, the higher the similarity degree of the perception characteristics of the two images is;
PSNR refers to the ratio of the maximum power of a signal to the noise power of the signal, a secret image embedded in a carrier image can be regarded as noise affecting the quality of the carrier image, the carrier image embedded in the secret image can be regarded as noise affecting the quality of the secret image, the higher the PSNR value is, the higher the similarity degree of the perception characteristics of the two images is, and the calculation formula is as follows:
wherein n represents the number of bits per pixel;
the SSIM is used for comparing and measuring the correlation among different regions of the image in the aspects of brightness, contrast and structure, the larger the SSIM value is, the higher the similarity degree of the perception characteristics of the two images is, and the calculation formula is as follows:
SSIM(x,y)＝[l(x,y)α·c(x,y)β·s(x,y)γ]
where x and y represent two images respectively, l (x, y) represents brightness comparison, c (x, y) represents contrast comparison, s (x, y) represents texture comparison,. mu.xAnd muyDenotes the mean, σ, of x and yxAnd σyDenotes the standard deviation, σ, of x and yxyRepresents the covariance, c1、c2And c3Is a well-known one, avoids the denominator being 0, and α, β, and γ are generally set to 1, and then there is a calculation formula as follows:
the similarity degree of the perception characteristics of the two images is compared with the residual error of the two images and the histogram statistical characteristic, the residual error of the two images is evaluated to determine whether chromatic aberration, artifacts and the like exist, the smaller the residual error is, the better the visual effect is, the histogram statistical characteristic is evaluated to resist third-party steganalysis capability, and the closer the histogram statistical characteristic is, the better the anti-steganalysis capability is.
2. The semantic weighting-based image information hiding method according to claim 1, wherein the semantic preprocessing network model in the second step; since different regions of the secret image have different semantic importance levels: regions with complex textures and contents have higher semantic importance, and more features need to be reserved for reconstruction; and secondly, processing the secret image from a high-order feature level to generate a semantic importance map representing different regions of the secret image, generating a semantic importance mask consisting of 0 and 1 according to the semantic importance map, and cutting the secret image features for connecting and embedding with the carrier image on a channel.
3. The semantic weighting based image information hiding method of claim 1, wherein the inverse gradient propagation model in step three; because an end-to-end training network model is needed for realizing image information hiding, the training of the network model needs continuous iteration of reverse gradient propagation, quantization of a mask generation module of a semantic preprocessing network in the network model and semantic significance diagram channel expansion are not conductive, the reverse gradient propagation is interrupted, the gradient in front of the module is zero and cannot be iterated, and mask generation is represented again, so that the reverse gradient propagation can be carried out.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210220784.3A CN114782462A (en) | 2022-03-08 | 2022-03-08 | Semantic weighting-based image information hiding method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210220784.3A CN114782462A (en) | 2022-03-08 | 2022-03-08 | Semantic weighting-based image information hiding method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114782462A true CN114782462A (en) | 2022-07-22 |
Family
ID=82423984
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210220784.3A Pending CN114782462A (en) | 2022-03-08 | 2022-03-08 | Semantic weighting-based image information hiding method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114782462A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116958006A (en) * | 2023-09-19 | 2023-10-27 | 湖北微模式科技发展有限公司 | Equal-size image superposition algorithm based on pixel bidirectional fusion |
| CN117560455A (en) * | 2024-01-11 | 2024-02-13 | 腾讯科技（深圳）有限公司 | Image feature processing method, device, equipment and storage medium |
-
2022
- 2022-03-08 CN CN202210220784.3A patent/CN114782462A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116958006A (en) * | 2023-09-19 | 2023-10-27 | 湖北微模式科技发展有限公司 | Equal-size image superposition algorithm based on pixel bidirectional fusion |
| CN116958006B (en) * | 2023-09-19 | 2024-01-02 | 湖北微模式科技发展有限公司 | Equal-size image superposition algorithm based on pixel bidirectional fusion |
| CN117560455A (en) * | 2024-01-11 | 2024-02-13 | 腾讯科技（深圳）有限公司 | Image feature processing method, device, equipment and storage medium |
| CN117560455B (en) * | 2024-01-11 | 2024-04-26 | 腾讯科技（深圳）有限公司 | Image feature processing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110189334B (en) | Medical image segmentation method of residual error type full convolution neural network based on attention mechanism | |
| CN110084734B (en) | Big data ownership protection method based on object local generation countermeasure network | |
| CN114782462A (en) | Semantic weighting-based image information hiding method | |
| CN110517329B (en) | Deep learning image compression method based on semantic analysis | |
| CN109993678B (en) | Robust information hiding method based on deep confrontation generation network | |
| CN111429355A (en) | Image super-resolution reconstruction method based on generation countermeasure network | |
| CN111598762A (en) | Generating type robust image steganography method | |
| CN111681154B (en) | Color image steganography distortion function design method based on generation countermeasure network | |
| CN105389770A (en) | Method and apparatus for embedding and extracting image watermarking based on BP and RBF neural networks | |
| CN112634117A (en) | End-to-end JPEG domain image steganography method based on generation countermeasure network | |
| CN112711763B (en) | Color image scrambling encryption method based on countermeasure automatic encoder | |
| CN113947136A (en) | Image compression and classification method and device and electronic equipment | |
| WO2023103715A1 (en) | Image processing method and apparatus, and electronic device | |
| CN115170915A (en) | Infrared and visible light image fusion method based on end-to-end attention network | |
| CN113222108A (en) | Target detection processing method, device and equipment | |
| CN110728186A (en) | Fire detection method based on multi-network fusion | |
| Ruivo et al. | Double-deep learning-based point cloud geometry coding with adaptive super-resolution | |
| CN109413298B (en) | A kind of information concealing method of the more basis representations of adaptive secret information | |
| CN113326531B (en) | Robust efficient distributed face image steganography method | |
| CN113051617A (en) | Privacy protection method based on improved generation countermeasure network | |
| CN114549302A (en) | Image super-resolution reconstruction method and system | |
| CN113538199B (en) | Image steganography detection method based on multi-layer perception convolution and channel weighting | |
| Sapiro et al. | Morphological image coding based on a geometric sampling theorem and a modified skeleton representation | |
| CN117579837B (en) | JPEG image steganography method based on countermeasure compression image | |
| CN116403026A (en) | Style migration method based on multi-category style extraction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |