CN116090010A - Context contact-based text generation type steganography method - Google Patents
Context contact-based text generation type steganography method Download PDFInfo
- Publication number
- CN116090010A CN116090010A CN202310121433.1A CN202310121433A CN116090010A CN 116090010 A CN116090010 A CN 116090010A CN 202310121433 A CN202310121433 A CN 202310121433A CN 116090010 A CN116090010 A CN 116090010A
- Authority
- CN
- China
- Prior art keywords
- text
- layer
- steganography
- output
- word
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000012549 training Methods 0.000 claims abstract description 24
- 239000000969 carrier Substances 0.000 claims abstract description 17
- 239000013598 vector Substances 0.000 claims description 73
- 230000008569 process Effects 0.000 claims description 20
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000004927 fusion Effects 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 2
- 238000000605 extraction Methods 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 13
- 238000012360 testing method Methods 0.000 description 6
- 241000282414 Homo sapiens Species 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 230000001953 sensory effect Effects 0.000 description 2
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Bioethics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The invention discloses a text generation type steganography method based on context contact, and belongs to the technical field of information hiding. The method comprises the following steps: obtaining a Chinese popular lyric data set from a public network, storing each song in a contracted format, randomly selecting k text carriers from the songs to carry out digital serialization, and sequentially inputting a text steganography network model, wherein the model comprises an encoding end and a decoding end, a threshold recursion unit and an attention module are used, and a reconstructed steganography text is output; through continuous training, the error between the reconstructed steganographic text and the original text is continuously reduced, and the parameters of the encoder and the decoder are reversely updated; so that the trained model generates steganographic text embedded with secret information. The invention considers the serialization characteristics of the texts, the context relation among the texts, the statistical characteristics and semantic characteristics of the text carrier and the like when generating the steganography texts, so that the steganography texts are more hidden, and high-quality text steganography and secret information extraction are realized.
Description
Technical Field
The invention belongs to the technical field of information hiding, relates to text steganography, and in particular relates to a text generation type steganography method based on context contact.
Background
The information hiding can embed the secret information in the public media without changing the perceptibility of the information, and then the transmission of the secret information is completed by transmitting the carrier embedded with the secret information on the public channel. Information hiding, in addition to hiding secret information and communication behavior, can also be used for digital watermarking to address requirements such as copyright protection and tamper authentication. Unlike cryptography, which only hides the content of information and makes the encrypted information obscure, information hiding not only hides the content of information but also hides the existence of information, thereby improving the security of secret information to some extent. Information is hidden by a carrier in the form of various digital media such as images, video, voice and text, but most of the information hiding research is related to images and video.
In the computer age, information hiding is not only indistinguishable to the naked human eye, but also foolproof of computer identification. One branch of information hiding—steganography, embedding information by changing the spatial information or transform domain coefficients of the carrier, may cause significant differences in the statistical index. Whereas steganalysis methods are mainly performed by fitting statistical features. Accordingly, the steganography method reduces the change of the statistical characteristics caused by modification as much as possible by continuously changing the embedding mode, so that the modification of the carrier is not easy to be found; at the same time, however, steganalysis methods continue to construct more efficient statistical features for analysis of the vector. The two methods are continuously promoted to advance, and the development of the two methods gradually reaches the bottleneck period along with the time, so that the steganography method is difficult to find a more excellent embedding method by utilizing the traditional mode so as to ensure that the embedding method cannot be detected by the steganography analysis method.
Regarding the performance index of the steganography method, the existing work mainly considers the aspects of hiding efficiency, imperceptibility, hiding capacity, robustness, complexity and the like. Hiding efficiency refers to the time required to generate a certain number of carriers embedded with secret information; imperceptibility refers to the difference between a carrier embedded with secret information and an original carrier, and the greater the difference is, the worse the imperceptibility is, and different indexes such as perplexity (PPL), earth's' Distance (EMD) and the like can be adopted for measurement; the hidden capacity refers to the amount of secret information embedded in the carrier, and is evaluated by using an Embedding Rate (ER); robustness refers to whether the secret information can be recovered when the carrier embedded with the secret information is interfered; the complexity refers to the resources required for running the steganography model, and the index is important for some scenes with large data volume and high real-time performance.
The rise of deep learning has assisted the rapid development of various industries, and especially the simple combination attempt of Convolutional Neural Network (CNN) and steganography analysis has succeeded, so that experts can think about whether more deep learning technology can be applied to steganography. Through deep learning, the steganography algorithm can get rid of expert knowledge in preprocessing of many original data, and the countermeasure of the steganography analysis algorithm is added into the training process of the steganography model, so that the steganography algorithm has higher safety.
However, in the prior art, the information hiding model based on deep learning is mostly developed around the image, on one hand, because the image has a larger redundant space to hide information, on the other hand, the image processing tool is more and the hiding effect is very visual, and the convolutional neural network is designed for the image initially. Text is used as the most widely used media and is a carrier object with great development potential, but the characteristic of low redundancy of text information makes the research of a text steganography algorithm still have many blank parts at present, so that the research of a text steganography method based on deep learning, especially natural language processing, is also necessary.
Disclosure of Invention
In view of the above problems, the present invention provides a context association based text generation steganography method based on RNN (recurrent neural network) based on encoder-decoder architecture, which allows attention mechanisms in a transducer to be considered, realizes steganography of text, automatically generates steganography text embedded with a binary bit stream of secret information, and can extract secret information with high quality.
The text generation type steganography method based on the context contact comprises the following steps:
step one, acquiring a Chinese lyric data set, and storing each song in the format of song name, singer and lyrics;
step two, randomly selecting k text carriers from the Chinese lyric data set, wherein each text carrier is a lyric of a sentence or a song name of a song; converting the text carrier into a sequence signal, inputting the sequence signal into a text steganography network model, and reconstructing k backward-shifted serialization texts through an encoder and a decoder; k is a positive integer;
the text steganography network model comprises an encoder and a decoder, wherein the encoder network comprises an embedded layer and a threshold recursion unit GRU layer, and the decoder network comprises an embedded layer, a GRU layer, an attention module, a linear splicing layer, a linear layer and a Softmax output layer; inputting the sequence signal of the ith text carrier into an encoder to obtain an encoder output vector E i,o And decoder GRU layer initial hidden state, inputting the sequence signal of the (i+1) th text carrier into decoder, inputting E i,o And the output vector D of the GRU layer of the decoder i,o Input to the attention module to obtain a corrected output vector D considering the influence of different word weights in the sentence ′ i,o The method comprises the steps of carrying out a first treatment on the surface of the Will correct the output vector D ′ i,o And the original output vector D i,o Obtaining a shift output sequence through the linear splicing layer, the linear layer and the Softmax output layer;
training the text steganography network model to enable the shift output sequence and the original shift input sequence of the decoder to reach an error range threshold value, and updating parameters of an encoder network and a decoder network;
generating a secret-carrying text by using the trained text steganography network model;
random acquisition ofTaking a text carrier with song name as the input of the coding end, the input of the sentence head identifier of the decoding end, and obtaining a d through a text steganography network model o Maintaining an output vector; the elements in the output vector are ordered from high to low according to probability, each dimension element corresponds to a word in the corpus, and the front K=2 in the output vector is selected according to the preset embedded bit number b b Constructing a candidate pool by words corresponding to the dimension elements; selecting words from the candidate pool according to the secret information binary bit stream and outputting the words; and adding the output word to the decoding end for input, continuously generating an output vector by using the text steganography network model, repeating the word selection process, and continuously generating the steganography text.
In the second step, converting the text carrier into a sequence signal, including: sequentially taking k text carriers, performing word segmentation on each text carrier to obtain the corresponding ID of each word in a corpus vocabulary, and then representing the text carrier by using a sequence signal with a fixed length L, wherein elements in the sequence signal are IDs corresponding to the words; when the number of words contained in the text carrier is smaller than L, the insufficient positions in the sequence signal are filled with 0, otherwise the first L words of the text carrier are intercepted.
In the second step, the text steganography network model performs the following processing on the input k sequence signals: (1) Respectively passing k sequence signals through word2vec word embedding layers of an encoder to obtain sequence signal matrixes of k text carriers; (2) Inputting the sequence signal matrix of the ith text carrier into the GRU layer of the encoder to obtain an output vector of the encoder and an initial hidden state of the GRU layer of the decoder; (3) Inputting the sequence signal of the (i+1) th text carrier into a decoder, and obtaining a sequence signal matrix c through a decoder embedding layer i+1 And input the GRU layer, the initial hidden state of the decoder GRU layer is E i,h Obtaining an output vector D after passing through the GRU layer i,o The method comprises the steps of carrying out a first treatment on the surface of the (4) Output vector E from encoding end i,o And the output vector D of the GRU layer of the decoder i,o Inputting the corrected output vector into an attention module to obtain a corrected output vector considering the influence of different word weights in sentences; (5) Will new output vector D ′ i,o And the original output vector D i,o Through the linear splicing layer, the linear layer and SoAnd the ftmax output layer is used for obtaining the reconstructed shift output sequence and generating a steganographic text sequence signal.
The invention has the advantages and positive effects that: according to the text generation type steganography method based on the context relation, from the perspective of a human sensory system, the serialization characteristics of texts, the context relation among the texts, the statistical characteristics and semantic characteristics of a text carrier and the like are considered, so that the statistical difference and the semantic difference between the generated steganography text embedded with secret information and an original text are small, the distinction is not large for human beings in practice, and high-quality text steganography is realized. The method trains the text steganography network model by using the cross entropy loss function, so that the shifted output sequence and the shifted input sequence of the original decoding end reach an error range threshold value, thereby realizing effective secret information extraction.
Drawings
FIG. 1 is a flow chart of a context contact based text generation method of the present invention;
FIG. 2 is a schematic diagram of the training and testing process of the context contact based text generation steganography network of the present invention;
FIG. 3 is a schematic diagram of the internal structure of the network model according to the present invention;
FIG. 4 is a schematic diagram of parameters inside a network model according to the present invention;
FIG. 5 is a schematic diagram of a steganographic text generated after embedding secret information using the text-generated steganographic method of the present invention;
FIG. 6 is a schematic representation of the present model in text steganography.
Detailed Description
The invention will be described in further detail with reference to the drawings and examples.
In order to describe the features and advantages of the present invention in detail, the following description will be made of the actual application of the present invention in terms of the entire process from training to application.
The existing cyclic neural network can train on a large number of text carriers based on encoder-decoder architecture by utilizing the characteristic of good serialization modeling, and automatically generates a steganographic text embedded with a secret information binary bit stream after completion. Based on the method, the attention mechanism in the transducer is considered, the text generation type steganography method based on the context contact is designed, the text can be steganographically, the steganographically embedded with the secret information binary bit stream can be automatically generated, and the steganographically can be extracted with high quality.
As shown in FIG. 1, the context contact-based text generation steganography method of the present invention is described in the following five steps.
Step one, a Chinese lyric data set is collected from a public network and preprocessed, so that each song is stored in a dictionary in the format of song name, singer and lyrics.
In the embodiment of the invention, the Chinese lyric data set selects the public popular Chinese music from the network, so that the consistency of text formats in the data set is required to be ensured for training of a steganography model; if the above requirements are not met, data preprocessing is required to meet the requirements.
And secondly, randomly selecting k text carriers containing song names and lyrics in a plurality of songs from the preprocessed Chinese lyrics data set, wherein each text carrier is one sentence of lyrics or the song name of one song, inputting the text carriers into a text steganography network model in a sequence mode, and reconstructing k backward-shifted serialized texts through an encoder and a decoder.
As shown in fig. 2 and 3, the text steganographic network model of the present invention, which may be referred to as a T-GRU network model, includes an encoding-side network E and a decoding-side network D. In the training process, the ith text carrier is input into the encoding end network E, the (i+1) th text carrier is input into the decoding end network D, a reconstructed shift output text corresponding to the encoding end input text is generated, the difference between the reconstructed text and the decoding end input text is compared, a cross entropy loss function is used for calculating a loss value and further propagating a gradient in the opposite direction, training is continued, and the direct difference between the two text carriers is reduced. In the test process, except that the gradient is not counter-propagated, other steps are consistent with the training process, and finally the loss value is obtained through loss function calculation.
As shown in fig. 3, the encoding end network E of the text steganographic network model of the present invention includes an embedding layer and a threshold recursion unit (Gated Recurrent Unit, GRU), and the decoding end network D includes an embedding layer, a GRU, an attention module, a linear splicing layer, a linear layer, and a Softmax output. The invention is innovative in that the RNN variant GRU is fused with the Seq2Seq sequence model to construct a new model, and meanwhile, an attention module is also provided in the model, so that when the steganographic text is generated, the influence of words in sentences on subsequent generation is considered, and the relation between context sentences is considered, so that the steganographic text has stronger concealment.
As shown in fig. 4, the network parameters of the network model of the present invention are shown.
Step 201, sequentially taking k text carriers, and respectively performing word segmentation processing to form a sequence signal;
the sequence signals of the ith text vector are as follows:
f i ＝(f i (1),…,f i (t),…,f i (L)),i＝1,…,k；
l represents a preset fixed length after serialization, and N represents the number of words contained in the text segment at the encoding end. If N<L, filling 0 at the subsequent L-N positions; if N is greater than or equal to L, the first L words are fetched. f (f) i (t) represents the ID of the t-th word corresponding in the corpus vocabulary.
The method of the invention carries out word segmentation processing on the input Chinese text carrier, queries the corresponding ID of each word after word segmentation in a corpus vocabulary, and replaces the word with a digital ID to obtain a digital sequence signal corresponding to the text carrier.
Step 202, after k sequence signals respectively pass through word2vec word embedding layers of the coding end, obtaining sequence signal matrixes of k text carriers;
sequence signal matrix c of the ith text carrier i The acquisition process is as follows:
firstly, loading word2vec word embedding matrix pre-trained on a large-scale corpus
Where n is the number of words in the corpus and d is the vectorization dimension, i.e., embedding dimension, corresponding to each word.
Then, for the sequence signal f i Each word f of (1) i (t) searching a corresponding d-dimensional word vector in the word embedding matrix C to obtain a sequence signal f i Corresponding sequence signal matrix
Step 203, inputting the sequence signal matrix to a threshold recursion unit GRU layer of the encoding end to obtain the corresponding output vector of the encoding end and the initial hidden state of the decoding end GRU layer.
Encoding end output vector E of ith text carrier i,o Initial hidden state E of GRU layer at decoding end i,h The acquisition process is as follows:
first, a theoretical calculation formula is obtained according to the internal structure of the GRU:
z t ＝σ(W z [h t-1 ,x t ]),
r t ＝σ(W r [h t-1 ,x t ]),
wherein x is t An input vector at the time t is h t-1 And h t The hidden state vectors calculated for the gate are updated for time t-1 and time t respectively,
representing the hidden state vector calculated based on the reset gate. z t To update the gate output, r t In order to reset the gate output,
respectively update gates z t Reset gate r t Vector->
The parameters are learned during training. Long-term dependent memory capacity through reset gate r t Update door z t To ensure.
Let c i ＝(x 1 ,…,x t ,…,x L ) Corresponding to the input containing L moments, initial h 0 Set to 0, thereby obtaining the output vector of the encoding end
Initial hidden state E of GRU layer at decoding end i,h ＝h L ；E i,o ＝(h 1 ,…,h t ,…,h L ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein d h H is the output dimension after passing through the GRU layer L And outputting the hidden state at the moment L in the GRU layer.
Step 204, similar to the above steps, the (i+1) th text carrier sequence signal is input at the decoding end, and the output vector D is obtained after passing through the GRU layer i,o ；
The acquisition process is as follows:
firstly, word segmentation processing is performed on the (i+1) th text carrier input by the decoding end, and the same as step 201, a serialized input signal is obtained:
f i+1 ＝(f i+1 (1),…,f i+1 (t),…,f i+1 (L)),i＝1,…,k；
through word2vec word embedding layer, the sequence signal matrix c of the (i+1) th text carrier is obtained in the same way as step 202 i+1 。
Then, when passing through the GRU layer at the decoding end, the initial hidden state h at the time 0 Not set to 0, but to E i,h Accordingly, the output vector of the GRU layer at the decoding end can be obtained
Step 205, the output vector of the encoding end and the output vector of the decoding end after passing through the GRU layer are input into an attention module, and a corrected output vector considering the influence of different word weights in the sentence is obtained.
The acquisition process is as follows:
the output vector of the ith text carrier after passing through the encoding end is known as E i,o ＝(h 1 ,…,h t ,…,h L ) The output vector of the (i+1) th text carrier after passing through the GRU layer at the decoding end can be expressed as D i,o ＝(h ′ 1 ,…,h ′ t ,…,h ′ L ). Thus, for D i,o The t signal in (2) can obtain the correlation coefficient e between the t signal and the output vector of the encoding end t ：
e t ＝[a(h ′ t ,h 1 ),a(h ′ t ,h 2 ),..,a(h ′ t ,h L )]
a(h ′ t ,h i )＝tanh(W 1 h i +W 2 h ′ t )
Wherein a is a linear layer function, weight parameter matrix
After softmax activation function, the correlation coefficient e is obtained t Weight coefficient corresponding to each dimension of
Then, by fusion E i,o And the weight coefficient +.>
Obtaining a new vector c t 。
Finally, get the sum D i,o Matrix vector D with influence weights of the same dimension ′ i,o ＝(c 1 ,…,c t ,…,c L )。
In fig. 4, the attention module includes a score calculation layer, a score fusion layer, and a Dropout layer.
In step 206, after the new output vector and the original output vector pass through the splicing layer, the linear layer and the softmax activation function, a steganographic text sequence signal is obtained.
D corresponding to the sequence signal of the (i+1) -th input text carrier ′ i,o And D i,o Directly splicing to obtain a reconstruction vector D i,concat ：
Next, D is carried out i,concat The dimension reduction processing is realized through the full-connection linear layer, and the shift output sequence f of the (i+1) th text carrier after reconstruction is obtained through the softmax activation function ′ i ＝(f i (2),…,f i (t),…,f i (L)),i＝1,…,k。
The k text carriers are sequentially input into a text steganography network model, and the text steganography network model is processed by an encoder and a decoder to output a shifted sequence signal, so that the corresponding text can be further obtained through conversion.
Step three, continuously training a network model by utilizing a data set so that the shifted output sequence f ′ i Shift input sequence f with original decoding end ′ i+1 ＝(f i+1 (2),…,f i+1 (t),…,f i+1 (L)), i=1, …, k reaches the error range threshold, and the parameters of the encoder network and decoder network are updated in reverse.
The loss function used for training is a cross entropy loss function:
the cross entropy loss function for a single sample is:
wherein y is a real label,
to predict tags.
Here, the shift input sequence f to the original decoding end is required ′ i+1 ＝(f i+1 (2),…,f i+1 (t),…,f i+1 (L)), i=1, …, k and the output sequence f after the network model ′ i ＝(f i (2),…,f i (t),…,f i (L)), i=1, …, k performs cross entropy loss function calculation, and performs superposition summation processing after calculating a loss value of each single sample to obtain a total loss value.
And step four, using a trained network model, obtaining a candidate pool according to the input text carrier, the preset embedded bit number and the secret information binary bit stream B= "01110000101001 …", constructing a binary tree, and selecting corresponding words as output.
Randomly selecting a song, inputting a text carrier with the content of song name at the encoding end, inputting a sentence identifier at the decoding end, and obtaining a d after a network model o And D, outputting the vector by dimension, and selecting a word corresponding to the bit string in the b according to a certain coding rule to be used as a steganographic text embedded with secret information.
The process of deriving the output word using the trained web model includes the following:
(4.1) first, the resultant d o The elements in the dimension output vector are sequenced from high probability to low probability, each dimension element represents a word in the corpus, and the front K=2 is selected according to the preset embedded bit number b b Word construction candidate pool cp= [ m ] corresponding to dimension vector 1 ,…,m K ]；
(4.2) then constructing a complete binary tree or a Huffman tree that takes into account the probability differences for the words in the candidate pool, one word for each leaf node; selecting the corresponding word from the candidate pool as output means selecting the corresponding word in the binary tree as final output according to the bit string in the secret information bit stream.
And fifthly, re-inputting the output word to the decoding end, and repeating the steps continuously to obtain the secret information embedded steganographic text meeting the requirements.
Adding the word obtained in the fourth step into an input sequence of a decoding end, re-inputting a network model, inputting the network model by the encoding end, repeating the above process, and selecting the next word through the network model until the selected word is an end-of-sentence identifier, thereby obtaining a complete steganographic text.
And (3) taking the whole sentence steganographic text as an input part of the coding end, taking a sentence head identifier as a decoding end for input, re-inputting the network model, repeating the steps four and five to obtain a new steganographic text, and continuously repeating until a complete song is automatically obtained or the preset total sentence number of the lyrics is reached.
When secret information is extracted, key sentences and sentence head identifiers which generate a secret text are input into a text steganography network model, and secret information binary bit streams are reconstructed to complete the extraction process.
Examples:
the invention uses a Chinese lyrics data set in experiments. The data set is composed of popular Chinese music collected from public networks and is mainly used for researching the text generation problem in the field of natural language processing; the dataset contained a representative 7259 song works of 40 singers (20 singers each for men and women), a total of 223042 lyrics, 1189292 lyrics. All songs were read as per 9:1 dividing text data for constructing training set and test set.
In the experiment, the Chinese lyrics in the data set are processed by jieba segmentation, python based on Pytorch is adopted, and meanwhile, the training process is accelerated by using a Geforce RTX 3090GPU and a CUDA 11.2. The coding end and the decoding end of the model map each word to a 200-dimensional vector, each GRU layer is two layers, each layer comprises 200 GRU units, and tan h is adopted as a nonlinear activation function sigma in each GRU unit. In model training, in order to strengthen regularization, avoid over fitting, add dropout mechanism and adopt Adam optimization method.
A round training process: and randomly selecting k text carriers containing song names and song words from the training set, wherein each text carrier is one song word or song name of one song, inputting the text carriers into a network model in a sequence mode, and reconstructing k backward serialized texts through an encoder and a decoder.
After reconstruction, for the ith text carrier, the shifted output sequence f is calculated using a cross entropy loss function ′ i Shift input sequence f with original decoding end ′ i+1 Errors between them.
Finally, calculating the parameter gradients of the encoder network and the decoder network according to the errors, and updating the parameter values according to the Adam optimizer and the learning rate. Also, during each round of training, the model is tested with a test set, and during the test, the network parameter gradient does not need to be updated.
Training a plurality of epochs until the loss value of the test set is not reduced, completing the training process, deriving a model, and performing secret information embedding and extracting operations.
In the embedding process, d for each output o And constructing a candidate pool by using the dimension vector, selecting a word output corresponding to the binary bit stream b= "01110000101001 …" of the secret information according to the constructed binary tree, and continuously and repeatedly generating a steganographic text to realize the embedding of the secret information.
And when the extraction process is carried out, inputting the key sentence and the sentence head identifier for generating the secret text into the network model, and reconstructing the secret information binary bit stream.
As shown in fig. 5, in order to generate the steganographic text by using the method of the present invention, from the perspective of human sensory system, the present invention considers the serialization characteristics of the text, the context relation between the texts, the statistical characteristics and semantic characteristics of the text carrier, etc., so that the statistical difference and the semantic difference between the generated steganographic text embedded with secret information and the original text are smaller, and are in fact not very differentiated for human beings.
As shown in fig. 6, the text steganography method and RNN method of the present invention have a contrasting effect. When the context contact-based text generation type steganography method is adopted to generate the steganography text, the hiding efficiency is lower than that of RNN, but because the network model is more complex, more time is required for training and embedding. However, the network model used in the invention is better than the performance of RNN in terms of imperceptibility and hidden capacity, and the invention has a certain innovation and has a certain use value in practice.
Other than the technical features described in the specification, all are known to those skilled in the art. Descriptions of well-known components and well-known techniques are omitted so as to not unnecessarily obscure the present invention. The embodiments described in the above examples are not intended to represent all the embodiments consistent with the present application, and on the basis of the technical solutions of the present invention, various modifications or variations may be made by those skilled in the art without the need for inventive efforts, while remaining within the scope of the present invention.
Claims (8)
1. A method of text generation based on context contacts, comprising the steps of:
step one, acquiring a Chinese lyric data set, and storing each song in the format of song name, singer and lyrics;
step two, randomly selecting k text carriers from the Chinese lyric data set, wherein each text carrier is a lyric of a sentence or a song name of a song; converting the text carrier into a sequence signal, inputting the sequence signal into a text steganography network model, and reconstructing k backward-shifted serialization texts through an encoder and a decoder; k is a positive integer;
the text steganography network model comprises an encoder and a decoder, wherein the encoder network comprises an embedding layer and a threshold recursion unit GRU layer, and the decoder network comprises the embedding layer and GRU layer, attention module, linear splice layer, linear layer and Softmax output layer; inputting the sequence signal of the ith text carrier into an encoder to obtain an encoder output vector E i,o And decoder GRU layer initial hidden state, inputting the sequence signal of the (i+1) th text carrier into decoder, inputting E i,o And the output vector D of the GRU layer of the decoder i,o Input to the attention module to obtain a corrected output vector D 'considering the influence of different word weights in the sentence' i,o The method comprises the steps of carrying out a first treatment on the surface of the Will correct the output vector D' i,o And the original output vector D i,o Obtaining a shift output sequence through the linear splicing layer, the linear layer and the Softmax output layer;
training the text steganography network model to enable the shift output sequence and the original shift input sequence of the decoder to reach an error range threshold value, and updating parameters of an encoder network and a decoder network;
generating a secret-carrying text by using the trained text steganography network model;
randomly acquiring a text carrier with song name as an encoding end input, a decoding end input sentence head identifier, and acquiring a d through a text steganography network model o Maintaining an output vector; the elements in the output vector are ordered from high to low according to probability, each dimension element corresponds to a word in the corpus, and the front K=2 in the output vector is selected according to the preset embedded bit number b b Constructing a candidate pool by words corresponding to the dimension elements; selecting words from the candidate pool according to the secret information binary bit stream and outputting the words; and adding the output word to the decoding end for input, continuously generating an output vector by using the text steganography network model, repeating the word selection process, and continuously generating the steganography text.
2. The method of claim 1, wherein in the second step, converting the text carrier into the sequence signal includes: sequentially taking k text carriers, performing word segmentation on each text carrier to obtain the corresponding ID of each word in a corpus vocabulary, and then representing the text carrier by using a sequence signal with a fixed length L, wherein elements in the sequence signal are IDs corresponding to the words; when the number of words contained in the text carrier is smaller than L, the insufficient positions in the sequence signal are filled with 0, otherwise the first L words of the text carrier are intercepted.
3. The method according to claim 1 or 2, wherein in the second step, the text steganographic network model performs the following processing on the input k sequence signals:
(1) Respectively passing k sequence signals through word2vec word embedding layers of an encoder to obtain sequence signal matrixes of k text carriers;
acquiring a word2vec word embedded matrix C after pre-training, and setting the sequence signal of an ith text carrier as f i For f i Searching a corresponding d-dimensional word vector in the word embedding matrix C to obtain a sequence signal matrix C of an ith text carrier i ；
(2) Inputting the sequence signal matrix of the ith text carrier into the GRU layer of the encoder to obtain an output vector of the encoder and an initial hidden state of the GRU layer of the decoder;
set the sequence signal matrix c of the ith text carrier i ＝(x 1 ,…,x t ,…,x L ) L is the length of the sequence signal and corresponds to L moments; the calculation formula of the GRU layer is as follows:
z t ＝σ(W z [h t-1 ，x t ])
r t ＝σ(W r [h t-1 ，x t ])
wherein h is t-1 And h t The hidden state vectors calculated for the gate are updated for time t-1 and time t respectively,
representing a hidden state vector calculated based on the reset gate; z t To update the gate output, r t To reset the gate output, W z 、W r W is update gate, reset gate, vector +.>
Training parameters of (a); initial h 0 Setting to 0;
obtaining an encoder output vector E i,o ＝(h 1 ,…,h t ,…,h L ) Decoder GRU layer initial hidden state E i,h ＝h L ，h L Outputting hidden states at the moment L in the GRU layer of the encoder;
(3) Inputting the sequence signal of the (i+1) th text carrier into a decoder, and obtaining a sequence signal matrix c through a decoder embedding layer i+1 And input the GRU layer, the initial hidden state of the decoder GRU layer is E i,h Obtaining an output vector D after passing through the GRU layer i,o ；
(4) Output vector E from encoding end i,o And the output vector D of the GRU layer of the decoder i,o Inputting the corrected output vector into an attention module to obtain a corrected output vector considering the influence of different word weights in sentences, wherein the corrected output vector comprises the following components:
let the output vector D of the (i+1) th text carrier after passing through the GRU layer at the decoding end i,o ＝(h′ 1 ,…,h′ t ,…,h′ L ) Calculate D i,o T-th signal of (b) and encoder output vector E i,o Is a correlation coefficient e of (2) t The following are provided:
e t ＝[a(h′ t ，h 1 )，a(h′ t ，h z )，..，a(h′ t ，h L )]
a(h′ t ，h i )＝tanh(W 1 h i +W 2 h′ t )
wherein a is a linear layer function, W 1 ,W 2 Is a weight parameter matrix; after softmax activation function, the correlation coefficient e is obtained t Weight coefficient corresponding to each dimension of
The following are provided:
fusion E i,o Weight coefficient
Obtaining a new vector c t The following are provided:
finally, get the sum D i,o Matrix vectors D 'with influence weights of the same dimension' i,o ＝(c 1 ,…,c t ,…,c L )。
(5) Will new output vector D' i,o And the original output vector D i,o Obtaining a reconstructed shift output sequence through the linear splicing layer, the linear layer and the Softmax output layer, and generating a steganographic text sequence signal;
d 'is formed on the linear splicing layer' i,o And D i,o Directly splicing to obtain a reconstruction vector D i,concat The method comprises the steps of carrying out a first treatment on the surface of the At the full-connection linear layer pair D i,concat Performing dimension reduction treatment; the reconstructed shifted output sequence is obtained at the Softmax output layer through the Softmax activation function.
4. The method of claim 3, wherein in the third step, the shift output sequence of the text steganographic network model output is set to be f' i ＝(f i (2),…,f i (t),…,f i (L)), i=1, …, k; the original shifted input sequence of the decoder is denoted f' i+1 ＝(f i+1 (2),…,f i+1 (t),…,f i+1 (L)), i=1, …, k; calculating a loss value for each individual sample using a cross entropy loss functionAnd then carrying out superposition summation to obtain a total loss value.
5. The method according to claim 1, wherein in the fourth step, a complete binary tree or a huffman tree considering probability difference is constructed for the words in the candidate pool, each leaf node corresponds to one word, and the corresponding word is selected from the binary tree or the huffman tree according to the binary bit stream of the secret information and output.
6. The method according to claim 1 or 5, wherein in the fourth step, the method comprises:
(1) Adding the output word into the input sequence of the decoding end, re-inputting the text steganography network model, enabling the input of the encoding end to be unchanged, generating an output vector by utilizing the text steganography network model, selecting the next word according to the secret information binary bit stream, and repeating the process until the selected word is an end-of-sentence identifier, so as to obtain a complete steganography text;
(2) And (3) inputting the whole sentence steganography text as an encoding end, inputting a sentence head identifier as a decoding end, re-inputting the text steganography network model, repeating the process (1) to obtain a new steganography text, and continuously repeating until a complete song is automatically obtained or a preset total number of lyrics is reached.
7. The method of claim 1, wherein the method, when extracting the secret information from the generated secret text, inputs the key sentence and the sentence head identifier of the generated secret text into the text steganography network model to reconstruct the secret information binary bit stream.
8. The method of claim 1, wherein in the first step, a set of chinese lyrics data is collected from the public network and preprocessed for each song to make the text format in the set of data consistent.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310121433.1A CN116090010A (en) | 2023-02-01 | 2023-02-01 | Context contact-based text generation type steganography method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310121433.1A CN116090010A (en) | 2023-02-01 | 2023-02-01 | Context contact-based text generation type steganography method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116090010A true CN116090010A (en) | 2023-05-09 |
Family
ID=86206318
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310121433.1A Pending CN116090010A (en) | 2023-02-01 | 2023-02-01 | Context contact-based text generation type steganography method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116090010A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118468822A (en) * | 2024-07-10 | 2024-08-09 | 成都佳发安泰教育科技股份有限公司 | Target field text generation method and system |
-
2023
- 2023-02-01 CN CN202310121433.1A patent/CN116090010A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118468822A (en) * | 2024-07-10 | 2024-08-09 | 成都佳发安泰教育科技股份有限公司 | Target field text generation method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111816156B (en) | Multi-to-multi voice conversion method and system based on speaker style feature modeling | |
| CN113158665B (en) | Method for improving dialog text generation based on text abstract generation and bidirectional corpus generation | |
| CN110060690B (en) | Many-to-many speaker conversion method based on STARGAN and ResNet | |
| CN110751698B (en) | Text-to-image generation method based on hybrid network model | |
| CN109711121B (en) | Text steganography method and device based on Markov model and Huffman coding | |
| Yang et al. | GAN-TStega: Text steganography based on generative adversarial networks | |
| CN112348911B (en) | Semantic constraint-based method and system for generating fine-grained image by stacking texts | |
| Yang et al. | Rits: Real-time interactive text steganography based on automatic dialogue model | |
| CN113987129A (en) | Digital media protection text steganography method based on variational automatic encoder | |
| CN114549850A (en) | Multi-modal image aesthetic quality evaluation method for solving modal loss problem | |
| CN111429893A (en) | Many-to-many speaker conversion method based on Transitive STARGAN | |
| CN112580370B (en) | Mongolian nerve machine translation method integrating semantic knowledge | |
| CN114581770A (en) | TransUnnet-based automatic extraction processing method for remote sensing image building | |
| CN116090010A (en) | Context contact-based text generation type steganography method | |
| CN115906857A (en) | Chinese medicine text named entity recognition method based on vocabulary enhancement | |
| CN115510236A (en) | Chapter-level event detection method based on information fusion and data enhancement | |
| CN112069809A (en) | Missing text generation method and system | |
| CN114896371A (en) | Training method and device of natural language processing model | |
| CN118095430A (en) | Inference model, data processing method, device and medium | |
| CN117851565A (en) | Text visual question-answering method and system based on multi-source interaction | |
| CN112182162A (en) | Personalized dialogue method and system based on memory neural network | |
| CN116259289A (en) | Automatic music description generation method | |
| CN112989845B (en) | Chapter-level neural machine translation method and system based on routing algorithm | |
| CN115270917A (en) | Two-stage processing multi-mode garment image generation method | |
| CN115358220A (en) | Document level relation extraction method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |