CN113852434B - LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system - Google Patents

LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system Download PDF

Info

Publication number
CN113852434B
CN113852434B CN202111113281.8A CN202111113281A CN113852434B CN 113852434 B CN113852434 B CN 113852434B CN 202111113281 A CN202111113281 A CN 202111113281A CN 113852434 B CN113852434 B CN 113852434B
Authority
CN
China
Prior art keywords
layer
output
lstm
block
input
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111113281.8A
Other languages
Chinese (zh)
Other versions
CN113852434A (en
Inventor
张皓天
姜园
张琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sun Yat Sen University
Original Assignee
Sun Yat Sen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sun Yat Sen University filed Critical Sun Yat Sen University
Priority to CN202111113281.8A priority Critical patent/CN113852434B/en
Publication of CN113852434A publication Critical patent/CN113852434A/en
Application granted granted Critical
Publication of CN113852434B publication Critical patent/CN113852434B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B17/00—Monitoring; Testing
    • H04B17/30—Monitoring; Testing of propagation channels
    • H04B17/391—Modelling the propagation channel
    • H04B17/3912—Simulation models, e.g. distribution of spectral power density or received signal strength indicator [RSSI] for a given geographic region
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/02—Arrangements for optimising operational condition
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/06—Testing, supervising or monitoring using simulated traffic
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Electromagnetism (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The invention discloses an LSTM and ResNet aided deep learning end-to-end intelligent communication method, which uses a long-term and short-term memory unit in an end-to-end intelligent communication system to perform joint coding and decoding and modulation and demodulation processing on a grouping bit sequence in an actual communication system so as to improve the error rate performance of the system; meanwhile, the invention provides an application of a residual network structure in the end-to-end intelligent communication system to effectively improve the convergence rate of the neural network and avoid the problems of gradient disappearance and gradient explosion which possibly occur.

Description

LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system
Technical Field
The invention relates to the technical field of communication, in particular to an LSTM and ResNet assisted deep learning end-to-end intelligent communication method and system.
Background
As a powerful machine learning algorithm, the deep learning technique can efficiently learn linear or nonlinear mapping required by a user, and has been widely used in various fields such as image classification, speech recognition, language translation, and the like. On the other hand, the future communication system is also proposed in the global first 6G peak held in 3 months in 2019, and needs to be enhanced and deeply fused with artificial intelligence and machine learning so as to further improve the performance of the communication system and realize intelligent communication.
Currently existing studies of deep learning in combination with communication systems are generally limited to optimizing one module in the communication system, such as demodulation, decoding, etc., using deep learning. The idea that the different modules are respectively optimized is likely to not obtain a global optimal value, so that the upper performance limit of the system can be limited. Meanwhile, the traditional optimization algorithm and the modularized optimization algorithm based on deep learning are difficult to obtain satisfactory performance under the condition that the communication environment is unknown, and the practicability of the high-speed communication system such as 5G in a complex environment is greatly limited. Finally, in recent years, the communication technology is updated rapidly, and the construction cost of the communication infrastructure and the compatibility and upgradeability of equipment are also tested, so that the advantage that the intelligent communication system can switch the neural network structure and adjust the parameters according to the requirement at low cost is more highlighted.
In order to exert the advantage of deep learning while solving the above-mentioned problems, an end-to-end intelligent communication system based on deep learning is proposed. The system regards the whole transmission system as a black box model, and realizes the joint optimization of each component part of a receiving and transmitting end through a neural network, thereby providing a globally optimal end-to-end intelligent communication system comprising the realization of coding, modulating, demodulating and decoding processes. Compared with the traditional communication system and the machine learning auxiliary communication system optimized by the split modules, the end-to-end intelligent communication system can effectively adapt to the unknown communication environment and the nonlinearity imposed by communication equipment or channels. Meanwhile, due to the use of the deep learning technology, the communication system can quickly and cheaply optimize the communication algorithm and parameters.
However, the existing end-to-end intelligent communication system still has the defects, wherein the most important is that the improvement for communication application is lacking in the structure of the used neural network, so that the system still has a large improvement space in performance.
The publication date is 2019, 11, 15 and the Chinese patent with publication number CN110460402A discloses an end-to-end communication system establishment method based on deep learning. The method of the invention is divided into two stages. Firstly, a self-encoder neural network is established, a channel layer is complicated, and a random number simulation is used as a training set to perform preliminary training on the network so as to obtain a coding mode with adaptability to channel interference. The USRP is used for collecting a large amount of communication data under the actual channels and is used as a training set to train the decoding layer independently, so that the decoding layer has better performance for the communication under the actual conditions. Also, this patent, while using deep learning, lacks improvement for communication applications in the neural network architecture used.
Disclosure of Invention
The primary purpose of the invention is to provide an LSTM and ResNet assisted deep learning end-to-end intelligent communication method, which effectively utilizes user data and improves the communication performance of the system.
It is a further object of the present invention to provide an LSTM and ResNet assisted deep learning end-to-end intelligent communication system
In order to solve the technical problems, the technical scheme of the invention is as follows:
an LSTM and ResNet assisted deep learning end-to-end intelligent communication method comprises the following steps:
s1: randomly generating binary data as data to be sent by a user;
s2: setting a simulation channel environment;
s3: initializing a deep neural network, wherein the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data generated randomly in the step S1 is output to the noise layer after being processed by the sending block;
s4: the noise layer scrambles and denoises the signal processed by the sending block according to the simulation channel environment set in the step S2, and sends the signal to the receiving block;
s5: the receiving block obtains estimated user data after processing opposite to the transmitting block;
s6: training the deep neural network according to the difference between the estimated user data obtained in the step S5 and the data to be transmitted by the user in the step S1;
s7: setting the transmitting block after training on the communication transmitting end, taking binary data bits to be transmitted as input vectors of the transmitting block, transmitting output vectors of the transmitting block as transmitting signals into a wireless channel, setting the receiving block after training on the communication receiving end, taking the receiving block as input vectors of the receiving block after the receiving end receives signals, and obtaining estimated user transmitting data after processing.
Preferably, in step S2, a channel simulation environment is set, specifically:
when the actual channel environment is known, simulation is performed directly according to the probability distribution of the known actual channel environment;
if the actual channel environment is unknown, the simulation is performed by adaptively fitting against the generation network and learning the probability distribution of the unknown channel.
Preferably, in step S3, the transmitting block of the deep neural network includes an LSTM layer, a full connection layer, and a regularization layer, where an input of the LSTM layer is data to be transmitted by a user, an output of the LSTM layer is overlapped with an input of the LSTM layer and then used as an input of the full connection layer, to form a res nets structure, an output of the full connection is an input of the regularization layer, and an output of the regularization layer is an input of a noise layer.
Preferably, the user data to be transmitted is firstly converted into one-hot vector and then input to the LSTM layer of the transmission block.
Preferably, the LSTM layer passes through an external input vector x at time t t And an output vector h at time t-1 t-1 Calculating to obtain an output vector h at the time t t ：
f t ＝σ(W f ·[h t-1 ,x t ]+b f )
i t ＝σ(W i ·[h t-1 ,x t ]+b i )
o t ＝σ(W o ·[h t-1 ,x t ]+b o )
h t ＝o t ⊙tanh(c t )
In the formula, [ h ] t-1 ,x t ]Represents x t ，h t-1 Is a concatenation vector of W f 、b f 、W i 、b i 、W o 、b o As a trainable parameter, σ (. Cndot.) represents a sigmoid activation function, tan h (. Cndot.) represents a tan h activation function, and as such, indicates multiplication of corresponding elements; c t-1 A memory cell vector at time t-1; f (f) t 、i t 、o t Respectively a forgetting gate, an input gate and an output gate vector, f t Responsible for controlling c t-1 Which elements need to be reserved and which elements need to be weakened, i t Deciding alternative update vector +.>Which elements need to be added to the cell vector at the current time, f t 、i t 、/>The three co-operate to generate a new memory cell vector c t The method comprises the steps of carrying out a first treatment on the surface of the Next, o t Determining tanh (c) t ) Which of them are output to obtain an output vector h at time t t 。
Preferably, the receiving block of the deep neural network includes two full connection layers and one LSTM unit with a res net structure, wherein:
the input of one full-connection layer is the output of a noise layer, the output is the input of an LSTM unit with a ResNet structure, the output of the LSTM unit with the ResNet structure is overlapped with the output of one full-connection layer to be used as the input of the other full-connection layer, the output of the other full-connection layer is a probability vector obtained by mapping that data to be sent by a user are firstly converted into one-hot vectors, the serial number of the largest element in the probability vector is judged as user sending information, and estimated user data are obtained according to the serial number.
Preferably, step S6 trains the deep neural network according to the difference between the estimated data obtained in step S5 and the binary data randomly generated in step S1, specifically:
calculating a loss function value between estimated user data output by the deep neural network and data to be transmitted by a user according to a preset loss function, quantitatively measuring the difference between actual output and expected output, and calculating the bias of the loss value to each trainable parameter in the deep neural network through a back propagation algorithm, so that the parameters of the deep neural network are updated by a preset optimization algorithm, and the difference between the estimated user data and the data to be transmitted by the user of the deep neural network is reduced.
Preferably, in step S7, if the actual channel changes slowly, retraining with low time cost and low computation cost is performed by a method including migration learning, and the synchronization of deep neural network training and deployment application is implemented by a means including parallel training.
An LSTM and resnet assisted deep learning end-to-end intelligent communication system, wherein the system uses the LSTM and resnet assisted deep learning end-to-end intelligent communication method, the system comprising:
the generation module is used for randomly generating binary data serving as data to be sent by a user;
the channel simulation module is used for setting a simulation channel environment;
the initialization network module is used for initializing a deep neural network, the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated by the generating module are output to the noise layer after being processed by the sending block;
the noise module is used for scrambling and noise-adding the signal processed by the sending block according to the simulation channel environment set by the channel simulation module and sending the signal to the receiving block;
a receiving processing module, in which the receiving block obtains estimated user data after processing opposite to the sending block;
the training module trains the deep neural network according to the difference between the estimated user data obtained by the receiving and processing module and the data to be sent by the user in the generating module;
the deployment module sets the transmitting block after training on the communication transmitting end, takes binary data bits to be transmitted as input vectors of the transmitting block, transmits output vectors of the transmitting block as transmitting signals to the wireless channel, sets the receiving block after training on the communication receiving end, and obtains estimated user transmitting data after processing after the receiving block receives the signals as input vectors of the receiving block.
Preferably, the transmitting block of the deep neural network includes an LSTM layer, a full connection layer, and a regularization layer, where an input of the LSTM layer is data to be transmitted by a user, an output of the LSTM layer is overlapped with an input of the LSTM layer and then is used as an input of the full connection layer, a resnet structure is formed, an output of the full connection is an input of the regularization layer, and an output of the regularization layer is an input of a noise layer.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the invention applies a long short-term memory (LSTM) unit to carry out joint coding and decoding and modulation and demodulation processing on the grouping bit sequence in the actual communication system, and effectively utilizes the characteristics of user data to improve the error rate (symbol error rate, SER) performance of the system; meanwhile, the invention provides that a residual error network (residual networks, resNet) structure is applied to an end-to-end intelligent communication system to effectively improve the convergence speed of the neural network and avoid the problems of gradient disappearance and gradient explosion which possibly occur.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
Fig. 2 is a schematic diagram of a deep neural network according to the present invention.
Fig. 3 is a schematic structural diagram of the LSTM layer.
FIG. 4 is a schematic diagram of the ResNet structure.
Fig. 5 is a diagram illustrating SER performance comparison between a communication system using the method of the present invention and conventional hamming code, BPSK modulation under AWGN channel.
Fig. 6 is a diagram illustrating SER performance comparison between a rayleigh fast fading channel and a conventional hamming code, BPSK modulated communication system using the method of the present invention.
Fig. 7 is a diagram for comparing SER performance with conventional hamming code using the method of the present invention under different channel estimation accuracy conditions.
Fig. 8 is a diagram showing the comparison of SER performance of a communication system in a deep neural network with and without the addition of a res net structure in an AWGN channel according to the present invention.
Fig. 9 is a schematic diagram of a system structure according to the present invention.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the present patent;
for the purpose of better illustrating the embodiments, certain elements of the drawings may be omitted, enlarged or reduced and do not represent the actual product dimensions;
it will be appreciated by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical scheme of the invention is further described below with reference to the accompanying drawings and examples.
Example 1
The embodiment provides an LSTM and ResNet assisted deep learning end-to-end intelligent communication method, as shown in FIG. 1, comprising the following steps:
s1: randomly generating binary data as data to be sent by a user;
s2: setting a simulation channel environment;
s3: initializing a deep neural network, wherein the deep neural network comprises a sending block, a noise layer and a receiving block as shown in fig. 2, and binary data generated randomly in the step S1 is output to the noise layer after being processed by the sending block;
s4: the noise layer scrambles and denoises the signal processed by the sending block according to the simulation channel environment set in the step S2, and sends the signal to the receiving block;
s5: the receiving block obtains estimated user data after processing opposite to the transmitting block;
s6: training the deep neural network according to the difference between the estimated user data obtained in the step S5 and the data to be transmitted by the user in the step S1;
s7: setting the transmitting block after training on the communication transmitting end, taking binary data bits to be transmitted as input vectors of the transmitting block, transmitting output vectors of the transmitting block as transmitting signals into a wireless channel, setting the receiving block after training on the communication receiving end, taking the receiving block as input vectors of the receiving block after the receiving end receives signals, and obtaining estimated user transmitting data after processing.
The steps S1 to S6 are training phases of the present invention, and are used for optimizing trainable parameters in the deep neural network, and the step S7 is a deployment application phase of the present invention, in which the deep neural network can directly read the optimized parameters, so as to implement encoding and modulation of bit data to be transmitted by a user at a transmitting end and demodulation and decoding of a received signal at a receiving end in real time.
In step S2, a channel simulation environment is set, specifically:
when the actual channel environment is known, such as a gaussian channel, a rayleigh fading channel and the like, scrambling and noise are directly carried out on an output signal s (t) of a neural network sending block according to the probability distribution of the known actual channel environment to obtain a receiving signal r (t), namely r (t) =i (s (t)), wherein I (·) represents a mapping function from an input signal to an output signal, which is set according to the actual known channel environment in the training process;
if the actual channel environment is unknown, the simulation is performed by adaptively fitting against the generation network and learning the probability distribution of the unknown channel.
The sending block of the deep neural network in step S3 is shown in fig. 3, and includes an LSTM layer, a full connection layer, and a regularization layer, where an input of the LSTM layer is data to be sent by a user, an output of the LSTM layer is overlapped with an input of the LSTM layer and then is used as an input of the full connection layer, a ResNets structure is formed, an output of the full connection is an input of the regularization layer, and an output of the regularization layer is an input of a noise layer.
The user data to be sent is converted into one-hot vector, and then is input into LSTM layer of the sending block, where the vector has only one bit element of 1 and the rest of all are 0, for example, assuming that 2-bit data is used as a group to process user data, d represents 2-bit information, which has 4 possibilities, if the sending end sends 3 rd information, one-hot vector l d ＝[0,0,1,0]。
The LSTM layer passes through the external input vector x at the time t t And an output vector h at time t-1 t-1 Calculating to obtain an output vector h at the time t t ：
f t ＝σ(W f ·[h t-1 ,x t ]+b f )
i t ＝σ(W i ·[h t-1 ,x t ]+b i )
o t ＝σ(W o ·[h t-1 ,x t ]+b o )
h t ＝o t ⊙tanh(c t )
In the formula, [ h ] t-1 ,x t ]Represents x t ，h t-1 Is a concatenation vector of W f 、b f 、W i 、b i 、W o 、b o As a trainable parameter, σ (. Cndot.) represents a sigmoid activation function, tan h (. Cndot.) represents a tan h activation function, and as such, indicates multiplication of corresponding elements; c t-1 Memory cell at time t-1A metavector; f (f) t 、i t 、o t Respectively a forgetting gate, an input gate and an output gate vector, f t Responsible for controlling c t-1 Which elements need to be reserved and which elements need to be weakened, i t Deciding alternative update vector +.>Which elements need to be added to the cell vector at the current time, f t 、i t 、/>The three co-operate to generate a new memory cell vector c t The method comprises the steps of carrying out a first treatment on the surface of the Next, o t Determining tanh (c) t ) Which of them are output to obtain an output vector h at time t t ；
The LSTM layer is used for capturing dynamic characteristics of input vectors, so that correlation of user data in each time period can be effectively utilized to perform encoding, decoding and modulation demodulation, and reliability of information transmission is improved; even in the special case that the user data in each time period are completely independent, the LSTM layer assisted neural network can still realize joint code modulation operation on the user data in a plurality of time periods. Therefore, the overall optimization advantage is exerted, and meanwhile, the constraint length of the codes is increased under the condition of maintaining the code rate unchanged, so that the reliability of the system is effectively improved.
The full-connection layer is responsible for further carrying out nonlinear operation on the output of the LSTM layer, so that the depth of the neural network is deepened, and the learning capacity of the neural network is improved. And a regularization layer in the transmitting block, which is used for carrying out normalization operation on the coded and modulated signals so as to maintain the power of the transmitted signals at a certain value.
The receiving block of the deep neural network is responsible for learning and performing the inverse operation of the transmitting block to recover the user data bits as correctly as possible from the interfered received signal, comprising two fully connected layers and an LSTM unit with a res net structure, wherein:
the input of one full-connection layer is the output of a noise layer, the output is the input of an LSTM unit with a ResNet structure, the output of the LSTM unit with the ResNet structure is overlapped with the output of one full-connection layer to be used as the input of the other full-connection layer, the output of the other full-connection layer is a probability vector obtained by mapping that data to be sent by a user are firstly converted into one-hot vectors, the serial number of the largest element in the probability vector is judged as user sending information, and estimated user data are obtained according to the serial number.
As shown in fig. 4, in the resuts structure, assuming that the input vector is x, the mapping that is desired to be implemented is G (x), and the actual output of the target layer is F (x). Then, before introducing the ResNet structure, it is necessary to train so that F (x) =G (x); after the ResNet structure is introduced, the training object is changed to be H (x) =F (x) +x=G (x), that is, F (x) =G (x) -x. By introducing the structure, the deep neural network can realize residual error learning on the appointed layer, so that the degradation of the neural network is avoided, the possibility of gradient disappearance problem in the training process is reduced, and the convergence efficiency of the neural network is improved.
Step S6, training the deep neural network according to the difference between the estimated data obtained in the step S5 and the binary data randomly generated in the step S1, wherein the training method specifically comprises the following steps:
calculating a loss function value between estimated user data output by the deep neural network and data to be transmitted by a user according to a preset loss function, quantitatively measuring the difference between actual output and expected output, and calculating the bias of the loss value to each trainable parameter in the deep neural network through a back propagation algorithm, so that the parameters of the deep neural network are updated by a preset optimization algorithm, and the difference between the estimated user data and the data to be transmitted by the user of the deep neural network is reduced.
In the step S6, attention is paid to the deep neural network in the training phase, and the transmitting block and the receiving block are training-optimized together. In the deployment application stage after training, the transmitting block and the receiving block of the neural network are deployed at two positions of the transmitting end and the receiving end respectively so as to realize coding, modulation and demodulation and decoding processing of user bit data respectively.
In step S7, if the actual channel changes slowly, retraining with low time cost and low calculation cost is performed by a method including migration learning, and synchronous performance of deep neural network training and deployment application is achieved by a method including parallel training.
In a specific embodiment, the wireless channel on which the signal propagates is assumed to be an AWGN channel, while 4 bits of user information are required to be transmitted over 7 channels. Under this condition, the LSTM and ResNet aided deep learning end-to-end intelligent communication system provided by the invention is compared with the traditional Hamming code coding and BPSK modulation communication system in terms of SER performance, and the result is shown in FIG. 5. Where Q in the legend represents the number of groups of 4 bits of user information that are processed by the LSTM unit in combination at a time. According to fig. 5, the SER performance of the communication system using the method of the present invention is superior to that of the conventional communication system for hard decision decoding under AWGN channel, and the effectiveness of the system of the present invention is demonstrated similarly to the conventional communication system for maximum likelihood decoding. In addition, compared with a neural network using LSTM units without ResNet structures, the neural network using LSTM units with ResNet structures can obtain better SER performance, which shows that the ResNet structures adopted by the invention can effectively improve the reliability of the end-to-end intelligent communication system.
Under the condition that other conditions are unchanged, the wireless channel for signal propagation is set as a Rayleigh fast fading channel, and under the condition, the LSTM and ResNet assisted deep learning end-to-end intelligent communication system provided by the invention is compared with the conventional communication system with Hamming code coding and BPSK modulation to carry out SER performance, and the result is shown in figure 6. According to the result, under the condition of more complex communication environment, compared with the traditional communication system, the end-to-end intelligent communication system can obtain better SER performance, and the performance gain is improved along with the increase of Q. This further demonstrates that the system of the present invention has better reliability and reflects that LSTM units can indeed improve SER performance of information transmission by jointly processing multiple sets of user data bits.
Radio channel setting for signal propagation under otherwise unchanged conditionsFor Rayleigh fast fading channels, SER performance of the system of the present invention and a conventional communication system under different channel estimation accuracy conditions was tested to reflect robustness of the system of the present invention. The results are shown in FIG. 7. Assuming that the actual channel coefficient is h and the channel estimation accuracy rate is ρ, estimating the channel coefficientCan be expressed as:
where ε is a complex Gaussian variable with a mean of 0 and a variance of 1 and ρ is a constant between 0 and 1. In the training phase, ρ is fixed to 1, indicating that the channel estimation during training is completely accurate; in the test stage, SER performance of each system under different rho conditions is tested to reflect the robustness of the system when the training conditions differ from the test conditions. As can be seen from fig. 7, when the channel estimation is not perfect and the test condition deviates from the training condition, the system of the present invention can still obtain SER performance better than the conventional communication system when q=4, 8, which proves that the system has good robustness.
And when other conditions are unchanged, the SER performance of the neural network with and without the ResNet structure is added after the number of each training round is measured, so that the convergence efficiency of the system is reflected. According to the illustration in fig. 8, in the training process, compared with the neural network without the ResNet structure, the deep neural network adopted by the system of the invention can more quickly improve the SER performance before reaching the saturation state, and can obtain better SER performance when the training reaches the saturation state. Therefore, the neural network adopted by the system has higher convergence efficiency and lower training time cost.
Example 2
The embodiment provides an LSTM and res-assisted deep learning end-to-end intelligent communication system, which is characterized in that the system uses the LSTM and res-assisted deep learning end-to-end intelligent communication method described in embodiment 1, as shown in fig. 9, and the system includes:
the generation module is used for randomly generating binary data serving as data to be sent by a user;
the channel simulation module is used for setting a simulation channel environment;
the initialization network module is used for initializing a deep neural network, the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated by the generating module are output to the noise layer after being processed by the sending block;
the noise module is used for scrambling and noise-adding the signal processed by the sending block according to the simulation channel environment set by the channel simulation module and sending the signal to the receiving block;
a receiving processing module, in which the receiving block obtains estimated user data after processing opposite to the sending block;
the training module trains the deep neural network according to the difference between the estimated user data obtained by the receiving and processing module and the data to be sent by the user in the generating module;
the deployment module sets the transmitting block after training on the communication transmitting end, takes binary data bits to be transmitted as input vectors of the transmitting block, transmits output vectors of the transmitting block as transmitting signals to the wireless channel, sets the receiving block after training on the communication receiving end, and obtains estimated user transmitting data after processing after the receiving block receives the signals as input vectors of the receiving block.
The transmitting block of the deep neural network comprises an LSTM layer, a full-connection layer and a regularization layer, wherein the input of the LSTM layer is data to be transmitted by a user, the output of the LSTM layer and the input of the LSTM layer are overlapped to be used as the input of the full-connection layer, a ResNet structure is formed, the full-connection output is the input of the regularization layer, and the output of the regularization layer is the input of the noise layer.
The same or similar reference numerals correspond to the same or similar components;
the terms describing the positional relationship in the drawings are merely illustrative, and are not to be construed as limiting the present patent;
it is to be understood that the above examples of the present invention are provided by way of illustration only and not by way of limitation of the embodiments of the present invention. Other variations or modifications of the above teachings will be apparent to those of ordinary skill in the art. It is not necessary here nor is it exhaustive of all embodiments. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the invention are desired to be protected by the following claims.

Claims (6)

1. An LSTM and ResNet assisted deep learning end-to-end intelligent communication method is characterized by comprising the following steps:
s1: randomly generating binary data as data to be sent by a user;
s2: setting a simulation channel environment;
s3: initializing a deep neural network, wherein the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data generated randomly in the step S1 is output to the noise layer after being processed by the sending block;
s4: the noise layer scrambles and denoises the signal processed by the sending block according to the simulation channel environment set in the step S2, and sends the signal to the receiving block;
s5: the receiving block obtains estimated user data after processing opposite to the transmitting block;
s6: training the deep neural network according to the difference between the estimated user data obtained in the step S5 and the data to be transmitted by the user in the step S1;
s7: setting the transmitting block after training on a communication transmitting end, taking binary data bits to be transmitted as input vectors of the transmitting block, transmitting output vectors of the transmitting block as transmitting signals into a wireless channel, setting the receiving block after training on a communication receiving end, taking the receiving block as the input vectors of the receiving block after the receiving end receives signals, and obtaining estimated user transmitting data after processing;
the transmitting block of the deep neural network in the step S3 comprises an LSTM layer, a full-connection layer and a regularization layer, wherein the input of the LSTM layer is data to be transmitted by a user, the output of the LSTM layer and the input of the LSTM layer are overlapped to be used as the input of the full-connection layer to form a ResNet structure, the full-connection output is the input of the regularization layer, and the output of the regularization layer is the input of a noise layer;
the user data to be sent is firstly converted into one-hot vectors and then is input into an LSTM layer of a sending block;
the receiving block of the deep neural network comprises two full connection layers and an LSTM unit with a ResNet structure, wherein:
wherein the input of one full-connection layer is the output of a noise layer, the output is the input of an LSTM unit with a ResNet structure, the output of the LSTM unit with the ResNet structure is overlapped with the output of one full-connection layer to be used as the input of the other full-connection layer, the output of the other full-connection layer is a probability vector obtained by mapping that data to be sent by a user is firstly converted into a one-hot vector, the serial number of the largest element in the probability vector is judged as user sending information, and estimated user data is obtained according to the serial number;
step S6, training the deep neural network according to the difference between the estimated data obtained in the step S5 and the binary data randomly generated in the step S1, wherein the training method specifically comprises the following steps:
calculating a loss function value between estimated user data output by the deep neural network and data to be transmitted by a user according to a preset loss function, quantitatively measuring the difference between actual output and expected output, and calculating the bias of the loss value to each trainable parameter in the deep neural network through a back propagation algorithm, so that the parameters of the deep neural network are updated by a preset optimization algorithm, and the difference between the estimated user data and the data to be transmitted by the user of the deep neural network is reduced.
2. The LSTM and ResNets-assisted deep learning end-to-end intelligent communication method according to claim 1, wherein the channel simulation environment is set in step S2, specifically:
when the actual channel environment is known, simulation is performed directly according to the probability distribution of the known actual channel environment;
if the actual channel environment is unknown, the simulation is performed by adaptively fitting against the generation network and learning the probability distribution of the unknown channel.
3. The LSTM and ResNets assisted deep learning end-to-end intelligent communication method of claim 2, wherein said LSTM layer is configured to pass an external input vector x at time t t And an output vector h at time t-1 t-1 Calculating to obtain an output vector h at the time t t ：
f t ＝σ(W f ·[h t-1 ，x t ]+b f )
i t ＝σ(W i ·[h t-1 ，x t ]+b i )
o t ＝σ(W o ·[h t-1 ，x t ]+b o )
h t ＝o t ⊙tanh(c t )
In the formula, [ h ] t-1 ，x t ]Represents x t ，h t-1 Is a concatenation vector of W f 、b f 、W i 、b i 、W o 、b o For trainable parameters, σ (·) represents a sigmoid activation function, tan h (·) represents a tan h activation function, and pair represents multiplication of corresponding elements; c t-1 A memory cell vector at time t-1; f (f) t 、i t 、o t Respectively a forgetting gate, an input gate and an output gate vector, f t Responsible for controlling c t-1 Which elements need to be reserved and which elements need to be weakened, i t Deciding alternative update vector +.>Which elements need to be added to the cell vector at the current time, f t 、i t 、/>The three co-operate to generate a new memory cell vector c t The method comprises the steps of carrying out a first treatment on the surface of the Next, o t Determining tanh (c) t ) Which of them are output to obtain an output vector h at time t t 。
4. The LSTM and ResNets-assisted deep learning end-to-end intelligent communication method according to claim 1, wherein in step S7, if the actual channel changes slowly, retraining with low time cost and low computation cost is performed by a method including migration learning, and synchronization between deep neural network training and deployment application is achieved by a method including parallel training.
5. An LSTM and resnet assisted deep learning end-to-end intelligent communications system using the LSTM and resnet assisted deep learning end-to-end intelligent communications method of any of claims 1 to 4, the system comprising:
the generation module is used for randomly generating binary data serving as data to be sent by a user;
the channel simulation module is used for setting a simulation channel environment;
the initialization network module is used for initializing a deep neural network, the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated by the generating module are output to the noise layer after being processed by the sending block;
the noise module is used for scrambling and noise-adding the signal processed by the sending block according to the simulation channel environment set by the channel simulation module and sending the signal to the receiving block;
a receiving processing module, in which the receiving block obtains estimated user data after processing opposite to the sending block;
the training module trains the deep neural network according to the difference between the estimated user data obtained by the receiving and processing module and the data to be sent by the user in the generating module;
the deployment module sets the transmitting block after training on the communication transmitting end, takes binary data bits to be transmitted as input vectors of the transmitting block, transmits output vectors of the transmitting block as transmitting signals to the wireless channel, sets the receiving block after training on the communication receiving end, and obtains estimated user transmitting data after processing after the receiving block receives the signals as input vectors of the receiving block.
6. The LSTM and ResNets assisted deep learning end-to-end intelligent communication system of claim 5, wherein the transmission block of the deep neural network includes an LSTM layer, a fully connected layer, and a regularized layer, wherein an input of the LSTM layer is data to be transmitted by a user, an output of the LSTM layer is overlapped with an input of the LSTM layer and is used as an input of the fully connected layer, a ResNets structure is formed, an output of the fully connected layer is an input of the regularized layer, and an output of the regularized layer is an input of a noise layer.
CN202111113281.8A 2021-09-18 2021-09-18 LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system Active CN113852434B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111113281.8A CN113852434B (en) 2021-09-18 2021-09-18 LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111113281.8A CN113852434B (en) 2021-09-18 2021-09-18 LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system

Publications (2)

Publication Number Publication Date
CN113852434A CN113852434A (en) 2021-12-28
CN113852434B true CN113852434B (en) 2023-07-25

Family

ID=78979321

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111113281.8A Active CN113852434B (en) 2021-09-18 2021-09-18 LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system

Country Status (1)

Country Link
CN (1) CN113852434B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230267337A1 (en) * 2022-02-24 2023-08-24 Protopia AI, Inc. Conditional noise layers for generating adversarial examples

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109767759A (en) * 2019-02-14 2019-05-17 重庆邮电大学 End-to-end speech recognition methods based on modified CLDNN structure
CN110460402A (en) * 2019-07-15 2019-11-15 哈尔滨工程大学 A kind of end-to-end communication system method for building up based on deep learning

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109767759A (en) * 2019-02-14 2019-05-17 重庆邮电大学 End-to-end speech recognition methods based on modified CLDNN structure
CN110460402A (en) * 2019-07-15 2019-11-15 哈尔滨工程大学 A kind of end-to-end communication system method for building up based on deep learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于ResNet-BLSTM的端到端语音识别;胡章芳;徐轩;付亚芹;夏志广;马苏东;;计算机工程与应用(第18期);全文 *

Also Published As

Publication number Publication date
CN113852434A (en) 2021-12-28

Similar Documents

Publication Publication Date Title
Park et al. Learning to demodulate from few pilots via offline and online meta-learning
Zhang et al. Deep learning-based beamspace channel estimation in mmWave massive MIMO systems
Hanna et al. Signal processing-based deep learning for blind symbol decoding and modulation classification
Nguyen et al. Application of deep learning to sphere decoding for large MIMO systems
CN113381828B (en) Sparse code multiple access random channel modeling method based on condition generation countermeasure network
Morocho-Cayamcela et al. Learning to communicate with autoencoders: Rethinking wireless systems with deep learning
CN113852434B (en) LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system
CN114337911A (en) Communication method based on neural network and related device
Thakkar et al. Deep learning and channel estimation
CN115941001A (en) Information transmission transceiving device, system and method based on MIMO system
CN113890799B (en) Underwater acoustic communication channel estimation and signal detection method based on domain countermeasure network
Ye et al. Bilinear convolutional auto-encoder based pilot-free end-to-end communication systems
CN109617655B (en) Polarization code wireless data secure transmission method based on random scrambling code assistance
Dridi et al. Blind detection of severely blurred 1d barcode
CN114759997B (en) MIMO system signal detection method based on data model double driving
Huang et al. Extrinsic neural network equalizer for channels with high inter-symbol-interference
Lu et al. Attention-Empowered Residual Autoencoder for End-to-End Communication Systems
CN112821971A (en) Time-varying channel signal detection method based on countermeasure learning
Liu et al. MIMO signal multiplexing and detection based on compressive sensing and deep learning
Njoku et al. BLER performance evaluation of an enhanced channel autoencoder
Chaudhari et al. A resnet based end-to-end wireless communication system under rayleigh fading and bursty noise channels
KR100888649B1 (en) Decoder for Detecting Transmitted Signal at MIMO system and Method thereof
Hares et al. Recurrent neural networks for pilot-aided wireless communications
Jiang et al. Adaptive semantic video conferencing for ofdm systems
McCaskey et al. Implementation of a machine learning based modulation scheme in gnuradio for over-the-air packet communications

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant