CN113852434A - LSTM and ResNet assisted deep learning end-to-end intelligent communication method and system - Google Patents

LSTM and ResNet assisted deep learning end-to-end intelligent communication method and system Download PDF

Info

Publication number
CN113852434A
CN113852434A CN202111113281.8A CN202111113281A CN113852434A CN 113852434 A CN113852434 A CN 113852434A CN 202111113281 A CN202111113281 A CN 202111113281A CN 113852434 A CN113852434 A CN 113852434A
Authority
CN
China
Prior art keywords
layer
lstm
output
input
sending
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111113281.8A
Other languages
Chinese (zh)
Other versions
CN113852434B (en
Inventor
张皓天
姜园
张琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sun Yat Sen University
Original Assignee
Sun Yat Sen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sun Yat Sen University filed Critical Sun Yat Sen University
Priority to CN202111113281.8A priority Critical patent/CN113852434B/en
Publication of CN113852434A publication Critical patent/CN113852434A/en
Application granted granted Critical
Publication of CN113852434B publication Critical patent/CN113852434B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B17/00—Monitoring; Testing
    • H04B17/30—Monitoring; Testing of propagation channels
    • H04B17/391—Modelling the propagation channel
    • H04B17/3912—Simulation models, e.g. distribution of spectral power density or received signal strength indicator [RSSI] for a given geographic region
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/02—Arrangements for optimising operational condition
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/06—Testing, supervising or monitoring using simulated traffic
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Electromagnetism (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The invention discloses an LSTM and ResNet assisted deep learning end-to-end intelligent communication method, which applies a long and short term memory unit in an end-to-end intelligent communication system to carry out joint coding and decoding and modulation and demodulation processing on a grouped bit sequence in an actual communication system so as to improve the error rate performance of the system; meanwhile, the invention provides that a residual error network structure is applied in an end-to-end intelligent communication system to effectively improve the convergence speed of a neural network and avoid the problems of gradient disappearance and gradient explosion which may occur.

Description

LSTM and ResNet assisted deep learning end-to-end intelligent communication method and system
Technical Field
The invention relates to the technical field of communication, in particular to an LSTM and ResNet assisted deep learning end-to-end intelligent communication method and system.
Background
As a powerful machine learning algorithm, the deep learning technique can efficiently learn linear or non-linear mapping required by users, and has been widely applied to various fields such as image classification, speech recognition, language translation, and the like. On the other hand, the future communication system needs to be enhanced to be deeply integrated with artificial intelligence and machine learning so as to further improve the performance of the communication system and realize intelligent communication, which is also proposed in the first 6G meeting of the world held in 3 months in 2019.
The current research on deep learning in combination with communication systems is usually limited to the use of deep learning to optimize, e.g., demodulate, decode, etc., one of the modules in the communication system. The idea of optimizing different modules respectively is likely to fail to obtain a global optimum value, thereby limiting the performance upper limit that the system can achieve. Meanwhile, the traditional optimization algorithm and the modular optimization algorithm based on deep learning both have difficulty in obtaining satisfactory performance under the condition that the communication environment is unknown, so that the practicability of a high-speed communication system such as a 5G communication system in a complex environment is greatly limited. Finally, in recent years, accelerated updating of communication technologies brings examination to the construction cost of communication infrastructure and compatibility and upgradability of equipment, and the advantage that the intelligent communication system can switch the neural network structure and adjust parameters according to requirements at low cost is further highlighted.
In order to solve the above problems and simultaneously exert the advantages of deep learning, a deep learning-based end-to-end intelligent communication system is proposed. The system regards the whole transmission system as a black box model, and realizes the joint optimization of all components of a transmitting end and a receiving end through a neural network, thereby providing an end-to-end intelligent communication system which is globally optimal and comprises the realization of the processes of coding, modulating, demodulating and decoding. Compared with the traditional communication system and the machine learning auxiliary communication system optimized in a sub-module mode, the end-to-end intelligent communication system can effectively adapt to the unknown communication environment and nonlinearity imposed by communication equipment or a channel. Meanwhile, due to the use of the deep learning technology, the communication system can quickly and inexpensively optimize the communication algorithm and parameters.
However, the existing end-to-end intelligent communication system still has shortcomings, and the most important thing is that the improvement for communication application is lacked in the used neural network structure, so the system still has a large improvement space in performance.
Chinese patent publication No. CN110460402A, published as 11/15/2019, discloses a method for establishing an end-to-end communication system based on deep learning. The process of the present invention is divided into two stages. Firstly, a self-encoder neural network is established and a channel layer is complicated, and the network is initially trained by taking random number simulation as a training set to obtain an encoding mode which has adaptability to channel interference. Then, the USRP collects communication data under a large number of actual channels and uses the collected communication data as a training set to train the decoding layer independently so that the decoding layer has better performance for communication under actual conditions. Also, this patent, while using deep learning, lacks improvements for communication applications in the neural network architecture used.
Disclosure of Invention
The invention aims to provide an LSTM and ResNet assisted deep learning end-to-end intelligent communication method, which effectively utilizes user data and improves the communication performance of a system.
It is a further object of this invention to provide an LSTM and ResNet assisted deep learning end-to-end intelligent communications system
In order to solve the technical problems, the technical scheme of the invention is as follows:
an LSTM and ResNet assisted deep learning end-to-end intelligent communication method comprises the following steps:
s1: randomly generating binary data as data to be sent by a user;
s2: setting a simulation channel environment;
s3: initializing a deep neural network, wherein the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated in the step S1 is processed by the sending block and then output to the noise layer;
s4: the noise layer scrambles and adds noise to the signal processed by the sending block according to the simulation channel environment set in the step S2, and sends the signal to the receiving block;
s5: the receiving block obtains estimated user data after using the reverse processing of the sending block;
s6: training the deep neural network according to the difference between the estimated user data obtained in the step S5 and the user data to be sent in the step S1;
s7: the method comprises the steps of setting a trained sending block at a communication sending end, using binary data bits needing to be sent as input vectors of the sending block, using output vectors of the sending block as sending signals to be sent to a wireless channel, setting a trained receiving block at a communication receiving end, using the trained receiving block as the input vectors of the receiving block after the communication receiving end receives the signals, and obtaining estimated user sending data after processing.
Preferably, the step S2 sets a channel simulation environment, specifically:
when the actual channel environment is known, simulation is directly carried out according to the probability distribution of the known actual channel environment;
if the actual channel environment is unknown, the probability distribution of the unknown channel is adaptively fitted and learned through the countermeasure generation network for simulation.
Preferably, the sending block of the deep neural network in step S3 includes an LSTM layer, a fully connected layer, and a regularization layer, where an input of the LSTM layer is data to be sent by the user, an output of the LSTM layer is superimposed with an input of the LSTM layer and then used as an input of the fully connected layer to form a ResNets structure, an output of the full connection is an input of the regularization layer, and an output of the regularization layer is an input of the noise layer.
Preferably, the data to be sent of the user is converted into a one-hot vector and then input to the LSTM layer of the sending block.
Preferably, the LSTM layer passes an external input vector x at time ttAnd the output vector h at time t-1t-1Calculating to obtain an output vector h at the moment tt：
ft＝σ(Wf·[ht-1,xt]+bf)
it＝σ(Wi·[ht-1,xt]+bi)
Figure BDA0003271199900000031
Figure BDA0003271199900000032
ot＝σ(Wo·[ht-1,xt]+bo)
ht＝ot⊙tanh(ct)
Wherein [ h ]t-1,xt]Denotes xt，ht-1Splicing vector of Wf、bf、Wi、bi、
Figure BDA0003271199900000033
Wo、boFor trainable parameters, σ (-) indicates a sigmoid activation function, tanh (-) indicates a tanh activation function, and "-" indicates multiplication of corresponding elements; c. Ct-1Is the memory cell vector at time t-1; f. oft、it、otAre vector of forgetting gate, input gate, output gate, respectively, ftIs responsible for controlling ct-1Which elements need to be preserved and which elements need to be weakened, itDetermining candidate update vectors
Figure BDA0003271199900000034
Which elements of the memory cell vector need to be added to the memory cell vector at the current time, ft、it、
Figure BDA0003271199900000035
The three components act together to generate a new memory cell vector ct(ii) a Then, otDetermination of tan h (c)t) Which are used as output, thereby obtaining an output vector h at the time tt。
Preferably, the receiving block of the deep neural network comprises two fully connected layers and one LSTM unit with a ResNets structure, wherein:
the input of one full connection layer is the output of a noise layer, the output is the input of an LSTM unit with a ResNet structure, the output of the LSTM unit with the ResNet structure is superposed with the output of one full connection layer to be used as the input of the other full connection layer, the output of the other full connection layer is a probability vector obtained by mapping that user data to be sent is firstly converted into a one-hot vector, the serial number of the largest element in the probability vector is judged to be user sending information, and estimated user data is obtained according to the probability vector.
Preferably, in step S6, the deep neural network is trained according to the difference between the estimation data obtained in step S5 and the binary data randomly generated in step S1, specifically:
the method comprises the steps of calculating a loss function value between estimated user data output by a deep neural network and user data to be sent according to preset loss functions, quantitatively measuring the difference between actual output and expected output, calculating the partial derivative of a loss value to each trainable parameter in the deep neural network through a back propagation algorithm, updating the parameters of the deep neural network by using a preset optimization algorithm, and reducing the difference between the estimated user data of the deep neural network and the user data to be sent.
Preferably, in step S7, if the actual channel has slow variation, retraining with low time cost and low computation cost is performed by a method including transfer learning, and performing deep neural network training and deploying applications synchronously by a means including parallel training.
An LSTM and ResNets assisted deep learning end-to-end intelligent communication system, wherein the system uses the LSTM and ResNets assisted deep learning end-to-end intelligent communication method, the system comprising:
the generating module is used for randomly generating binary data as data to be sent by a user;
the channel simulation module is used for setting a simulation channel environment;
the device comprises an initialization network module, a data processing module and a data processing module, wherein the initialization network module is used for initializing a deep neural network, the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated by the generation module are output to the noise layer after being processed by the sending block;
the noise layer scrambles and adds noise to the signal processed by the sending block according to the simulation channel environment set by the channel simulation module and sends the signal to the receiving block;
a receiving processing module, wherein the receiving block obtains estimated user data after using the processing opposite to that of the sending block;
the training module trains the deep neural network according to the difference between the estimated user data obtained by the receiving and processing module and the data to be sent by the user in the generating module;
and the deployment module is used for setting the trained sending block at the communication sending end, using binary data bits needing to be sent as input vectors of the sending block, using output vectors of the sending block as sending signals to be sent to a wireless channel, setting the trained receiving block at the communication receiving end, using the trained receiving block as the input vectors of the receiving block after the communication receiving end receives the signals, and processing the signals to obtain estimated user sending data.
Preferably, the sending block of the deep neural network includes an LSTM layer, a full-connection layer, and a regularization layer, where an input of the LSTM layer is data to be sent by a user, an output of the LSTM layer is superimposed with an input of the LSTM layer and then used as an input of the full-connection layer to form a ResNets structure, an output of the full-connection layer is an input of the regularization layer, and an output of the regularization layer is an input of the noise layer.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the invention applies a long short-term memory (LSTM) unit to carry out joint coding and decoding and modulation and demodulation processing on a packet bit sequence in an actual communication system, and effectively utilizes the characteristics of user data to improve the performance of a Symbol Error Rate (SER) of the system; meanwhile, the invention provides that a residual network (ResNet) structure is applied in an end-to-end intelligent communication system to effectively improve the convergence speed of the neural network and avoid the problems of gradient disappearance and gradient explosion which can occur.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
FIG. 2 is a schematic diagram of a deep neural network structure according to the present invention.
Fig. 3 is a schematic diagram of the structure of the LSTM layer.
Fig. 4 is a schematic diagram of the structure of ResNets.
Fig. 5 is a schematic diagram showing the SER performance comparison of the communication system using the proposed method of the present invention and the conventional hamming code encoding and BPSK modulation under AWGN channel.
Fig. 6 is a schematic diagram showing the SER performance comparison between the method of the present invention and the conventional hamming code encoding and BPSK modulation communication system under the rayleigh fast fading channel.
Fig. 7 is a schematic diagram showing the SER performance comparison between the method of the present invention and the conventional hamming code coding under different channel estimation accuracy conditions.
Fig. 8 is a schematic diagram showing the comparison of SER performance of a communication system under a deep neural network with a ResNets structure added under AWGN channel and without the ResNets structure added.
Fig. 9 is a schematic diagram of the system structure of the present invention.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;
it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
Example 1
The embodiment provides an LSTM and ResNets assisted deep learning end-to-end intelligent communication method, as shown in fig. 1, including the following steps:
s1: randomly generating binary data as data to be sent by a user;
s2: setting a simulation channel environment;
s3: initializing a deep neural network, wherein the deep neural network comprises a sending block, a noise layer and a receiving block as shown in fig. 2, and binary data randomly generated in step S1 is processed by the sending block and then output to the noise layer;
s4: the noise layer scrambles and adds noise to the signal processed by the sending block according to the simulation channel environment set in the step S2, and sends the signal to the receiving block;
s5: the receiving block obtains estimated user data after using the reverse processing of the sending block;
s6: training the deep neural network according to the difference between the estimated user data obtained in the step S5 and the user data to be sent in the step S1;
s7: the method comprises the steps of setting a trained sending block at a communication sending end, using binary data bits needing to be sent as input vectors of the sending block, using output vectors of the sending block as sending signals to be sent to a wireless channel, setting a trained receiving block at a communication receiving end, using the trained receiving block as the input vectors of the receiving block after the communication receiving end receives the signals, and obtaining estimated user sending data after processing.
The steps S1 to S6 are training stages of the present invention, and are used to optimize trainable parameters in the deep neural network, and the step S7 is a deployment application stage of the present invention, in which the deep neural network can directly read the optimized parameters, so as to implement encoding and modulation of bit data to be sent by a user at a sending end and demodulation and decoding of a received signal at a receiving end in real time.
In step S2, a channel simulation environment is set, specifically:
when the actual channel environment is known, for example, a gaussian channel, a rayleigh fading channel, etc., output signals s (t) of a neural network transmission block are scrambled and noised directly according to the probability distribution of the known actual channel environment to obtain received signals r (t), namely r (t) I (s (t)), wherein I () represents a mapping function from input signals to output signals set according to the actual known channel environment in the training process;
if the actual channel environment is unknown, the probability distribution of the unknown channel is adaptively fitted and learned through the countermeasure generation network for simulation.
The sending block of the deep neural network in step S3 is shown in fig. 3, and includes an LSTM layer, a fully connected layer, and a regularization layer, where an input of the LSTM layer is data to be sent by a user, an output of the LSTM layer is superimposed with an input of the LSTM layer and is used as an input of the fully connected layer to form a networks structure, an output of the fully connected layer is an input of the regularization layer, and an output of the regularization layer is an input of the noise layer.
The data to be sent by the user is converted into a one-hot vector, and then input to the LSTM layer of the sending block, where only one bit element of the vector is 1 and the rest are all 0, e.g., if 2-bit data is used as a group to process the user data, d represents 2-bit information, which has 4 possibilities, and if 3 types of information are sent by the sending end, then a one-hot vector l is inputd＝[0,0,1,0]。
The LSTM layer passes through an external input vector x at time ttAnd the output vector h at time t-1t-1Calculating to obtain an output vector h at the moment tt：
ft＝σ(Wf·[ht-1,xt]+bf)
it＝σ(Wi·[ht-1,xt]+bi)
Figure BDA0003271199900000071
Figure BDA0003271199900000072
ot＝σ(Wo·[ht-1,xt]+bo)
ht＝ot⊙tanh(ct)
Wherein [ h ]t-1,xt]Denotes xt，ht-1Splicing vector of Wf、bf、Wi、bi、
Figure BDA0003271199900000073
Wo、boFor trainable parameters, σ (-) indicates a sigmoid activation function, tanh (-) indicates a tanh activation function, and "-" indicates multiplication of corresponding elements; c. Ct-1Is the memory cell vector at time t-1; f. oft、it、otAre vector of forgetting gate, input gate, output gate, respectively, ftIs responsible for controlling ct-1Which elements need to be preserved and which elements need to be weakened, itDetermining candidate update vectors
Figure BDA0003271199900000074
Which elements of the memory cell vector need to be added to the memory cell vector at the current time, ft、it、
Figure BDA0003271199900000075
The three components act together to generate a new memory cell vector ct(ii) a Then, otDetermination of tan h (c)t) Which are used as output, thereby obtaining an output vector h at the time tt；
The LSTM layer is used for capturing dynamic characteristics of input vectors, so that coding, decoding and modulation and demodulation can be performed by effectively utilizing the correlation of user data in each time period, and the reliability of information transmission is improved; even under the special condition that the user data in each time period are completely independent, the LSTM layer-assisted neural network can still realize the joint coding modulation operation on the user data in a plurality of time periods. Therefore, the advantage of global optimization is exerted, and meanwhile, the constraint length of coding is increased under the condition of keeping the code rate unchanged, so that the reliability of the system is effectively improved.
The full connection layer is responsible for further carrying out nonlinear operation on the output of the LSTM layer, so that the depth of the neural network is deepened, and the learning capability of the neural network is improved. And the regularization layer in the transmission block is used for carrying out normalization operation on the coded and modulated signals so as to maintain the power of the transmission signals at a certain value.
The receive block of the deep neural network is responsible for learning and performing the inverse operation of the transmit block to recover the user data bits from the disturbed receive signal as correctly as possible, including two fully connected layers and one LSTM unit with a ResNets structure, where:
the input of one full connection layer is the output of a noise layer, the output is the input of an LSTM unit with a ResNet structure, the output of the LSTM unit with the ResNet structure is superposed with the output of one full connection layer to be used as the input of the other full connection layer, the output of the other full connection layer is a probability vector obtained by mapping that user data to be sent is firstly converted into a one-hot vector, the serial number of the largest element in the probability vector is judged to be user sending information, and estimated user data is obtained according to the probability vector.
As shown in fig. 4, in the ResNets structure, assuming that the input vector is x, the desired implementation is mapped to g (x), and the actual output of the target layer is f (x). Then, before introducing the ResNets structure, f (x) ═ g (x) needs to be trained; after the ResNets structure is introduced, the training target is changed to h (x) ═ f (x) + x ═ g (x), that is, f (x) ═ g (x) — x. By introducing the structure, the deep neural network can realize residual learning on a specified layer, so that neural network degradation is avoided, the possibility of gradient disappearance in the training process is reduced, and the convergence efficiency of the neural network is improved.
Step S6 trains the deep neural network according to the difference between the estimated data obtained in step S5 and the binary data randomly generated in step S1, specifically:
the method comprises the steps of calculating a loss function value between estimated user data output by a deep neural network and user data to be sent according to preset loss functions, quantitatively measuring the difference between actual output and expected output, calculating the partial derivative of a loss value to each trainable parameter in the deep neural network through a back propagation algorithm, updating the parameters of the deep neural network by using a preset optimization algorithm, and reducing the difference between the estimated user data of the deep neural network and the user data to be sent.
In the step S6, attention is paid to the deep neural network in the training phase, and the sending block and the receiving block are trained and optimized together. In the deployment application stage after training, the sending block and the receiving block of the neural network are respectively deployed at the sending end and the receiving end so as to respectively realize the coding, modulation and demodulation and decoding processing of the user bit data.
In step S7, if the actual channel changes slowly, retraining with low time cost and low computation cost is performed by a method including transfer learning, and deep neural network training and deployment application are performed synchronously by a means including parallel training.
In a specific embodiment, the wireless channel over which the signal propagates is assumed to be an AWGN channel, while 4 bits of user information are required to be transmitted over 7 channels. Under the condition, the LSTM and ResNet assisted deep learning end-to-end intelligent communication system provided by the invention is compared with the SER performance of a traditional Hamming code coding and BPSK modulation communication system, and the result is shown in FIG. 5. Where Q in the legend represents the number of groups of 4-bit user information that are jointly processed by the LSTM unit at a time. As shown in fig. 5, the SER performance of the communication system using the method of the present invention is better than that of the conventional communication system of hard decision decoding under AWGN channel, similar to that of the conventional communication system of maximum likelihood decoding, which shows the effectiveness of the system of the present invention. In addition, compared with the neural network using the LSTM unit without adding the ResNet structure, the neural network using the LSTM unit with the ResNet structure can obtain better SER performance, which shows that the ResNet structure adopted by the invention can effectively improve the reliability of the end-to-end intelligent communication system.
Under other conditions, the wireless channel for signal propagation is set as a rayleigh fast fading channel, and under such conditions, the SER performance of the LSTM and renets assisted deep learning end-to-end intelligent communication system provided by the present invention is compared with the SER performance of the traditional hamming code encoding and BPSK modulation communication system, and the result is shown in fig. 6. According to the result, under more complex communication environment conditions, compared with the traditional communication system, the end-to-end intelligent communication system can obtain better SER performance, and the performance gain is improved along with the increase of Q. This further demonstrates that the system of the present invention has better reliability and reflects that the LSTM unit can indeed improve SER performance for information transmission by jointly processing multiple groups of user data bits.
When other conditions are unchanged, the wireless channel of signal propagation is set as a Rayleigh fast fading channel, and SER performance of the system and the traditional communication system under different channel estimation accuracy conditions is tested to reflect robustness of the system. The results are shown in FIG. 7. Assuming that the actual channel coefficient is h and the channel estimation accuracy is rho, estimating the channel coefficient
Figure BDA0003271199900000091
Can be expressed as:
Figure BDA0003271199900000092
wherein epsilon is a complex gaussian variable with a mean value of 0 and a variance of 1, and rho is a constant with a value between 0 and 1. In the training stage, rho is fixed to be 1, which shows that the channel estimation during training is completely accurate; and in the testing stage, the SER performance of each system under different rho conditions is tested to reflect the robustness of the system when the training condition is different from the testing condition. As can be seen from fig. 7, when the channel estimation is not perfect and the test condition deviates from the training condition, the system of the present invention can still achieve SER performance better than that of the conventional communication system when Q is 4 and 8, which proves that the system has good robustness.
And when other conditions are unchanged, measuring the SER performance of the neural network with the ResNet structure and the neural network without the ResNet structure after each training round number to reflect the convergence efficiency of the system. As shown in fig. 8, in the training process, compared with a neural network without adding a ResNets structure, the deep neural network adopted by the system of the present invention can improve the SER performance of the neural network before reaching a saturation state, and can obtain a better SER performance when the training reaches saturation. Therefore, the neural network adopted by the system has higher convergence efficiency and lower training time cost.
Example 2
The embodiment provides an LSTM and ResNets assisted deep learning end-to-end intelligent communication system, which is characterized in that the system uses the LSTM and ResNets assisted deep learning end-to-end intelligent communication method described in embodiment 1, as shown in fig. 9, the system includes:
the generating module is used for randomly generating binary data as data to be sent by a user;
the channel simulation module is used for setting a simulation channel environment;
the device comprises an initialization network module, a data processing module and a data processing module, wherein the initialization network module is used for initializing a deep neural network, the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated by the generation module are output to the noise layer after being processed by the sending block;
the noise layer scrambles and adds noise to the signal processed by the sending block according to the simulation channel environment set by the channel simulation module and sends the signal to the receiving block;
a receiving processing module, wherein the receiving block obtains estimated user data after using the processing opposite to that of the sending block;
the training module trains the deep neural network according to the difference between the estimated user data obtained by the receiving and processing module and the data to be sent by the user in the generating module;
and the deployment module is used for setting the trained sending block at the communication sending end, using binary data bits needing to be sent as input vectors of the sending block, using output vectors of the sending block as sending signals to be sent to a wireless channel, setting the trained receiving block at the communication receiving end, using the trained receiving block as the input vectors of the receiving block after the communication receiving end receives the signals, and processing the signals to obtain estimated user sending data.
The transmitting block of the deep neural network comprises an LSTM layer, a full connection layer and a regularization layer, wherein the input of the LSTM layer is data to be transmitted by a user, the output of the LSTM layer and the input of the LSTM layer are superposed to be used as the input of the full connection layer to form a ResNet structure, the output of the full connection is the input of the regularization layer, and the output of the regularization layer is the input of a noise layer.
The same or similar reference numerals correspond to the same or similar parts;
the terms describing positional relationships in the drawings are for illustrative purposes only and are not to be construed as limiting the patent;
it should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims (10)

1. An LSTM and ResNet assisted deep learning end-to-end intelligent communication method is characterized by comprising the following steps:
s1: randomly generating binary data as data to be sent by a user;
s2: setting a simulation channel environment;
s3: initializing a deep neural network, wherein the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated in the step S1 is processed by the sending block and then output to the noise layer;
s4: the noise layer scrambles and adds noise to the signal processed by the sending block according to the simulation channel environment set in the step S2, and sends the signal to the receiving block;
s5: the receiving block obtains estimated user data after using the reverse processing of the sending block;
s6: training the deep neural network according to the difference between the estimated user data obtained in the step S5 and the user data to be sent in the step S1;
s7: the method comprises the steps of setting a trained sending block at a communication sending end, using binary data bits needing to be sent as input vectors of the sending block, using output vectors of the sending block as sending signals to be sent to a wireless channel, setting a trained receiving block at a communication receiving end, using the trained receiving block as the input vectors of the receiving block after the communication receiving end receives the signals, and obtaining estimated user sending data after processing.
2. The LSTM and ResNets-assisted deep learning end-to-end intelligent communication method according to claim 1, wherein the channel simulation environment is set in step S2, specifically:
when the actual channel environment is known, simulation is directly carried out according to the probability distribution of the known actual channel environment;
if the actual channel environment is unknown, the probability distribution of the unknown channel is adaptively fitted and learned through the countermeasure generation network for simulation.
3. The LSTM and ResNets-assisted deep learning end-to-end intelligent communication method of claim 1, wherein the sending block of the deep neural network in step S3 includes an LSTM layer, a full connection layer and a regularization layer, wherein an input of the LSTM layer is data to be sent by a user, an output of the LSTM layer is superimposed with an input of the LSTM layer and is used as an input of the full connection layer to form a ResNets structure, an output of the full connection is an input of the regularization layer, and an output of the regularization layer is an input of the noise layer.
4. The LSTM and ResNets assisted deep learning end-to-end intelligent communication method according to claim 3, wherein the user data to be sent is converted into a one-hot vector before being input into the LSTM layer of the sending block.
5. The LSTM and ResNets assisted deep learning end-to-end intelligent communication method of claim 4, wherein the LSTM layer passes an external input vector x at time ttAnd the output vector h at time t-1t-1Calculating to obtain an output vector h at the moment tt：
ft＝σ(Wf·[ht-1，xt]+bf)
it＝σ(Wi·[ht-1，xt]+bi)
Figure FDA0003271199890000021
Figure FDA0003271199890000022
ot＝σ(Wo·[ht-1，xt]+bo)
ht＝ot⊙tanh(ct)
Wherein [ h ]t-1，xt]Denotes xt，ht-1Splicing vector of Wf、bf、Wi、bi、
Figure FDA0003271199890000025
Wo、boFor trainable parameters, σ (-) indicates a sigmoid activation function, tan h (-) indicates a tan h activation function, and "-" indicates multiplication of corresponding elements; c. Ct-1Is the memory cell vector at time t-1; f. oft、it、otAre vector of forgetting gate, input gate, output gate, respectively, ftIs responsible for controlling ct-1Which elements need to be preserved and which elements need to be weakened, itDetermining candidate update vectors
Figure FDA0003271199890000023
Which elements of the memory cell vector need to be added to the memory cell vector at the current time, ft、it、
Figure FDA0003271199890000024
The three components act together to generate a new memory cell vector ct(ii) a Then, otDetermination of tan h (c)t) Which are used as output, thereby obtaining an output vector h at the time tt。
6. The LSTM and ResNets assisted deep learning end-to-end intelligent communication method of claim 4, wherein the receiving block of the deep neural network comprises two fully connected layers and one LSTM unit with ResNets structure, wherein:
the input of one full connection layer is the output of a noise layer, the output is the input of an LSTM unit with a ResNet structure, the output of the LSTM unit with the ResNet structure is superposed with the output of one full connection layer to be used as the input of the other full connection layer, the output of the other full connection layer is a probability vector obtained by mapping that user data to be sent is firstly converted into a one-hot vector, the serial number of the largest element in the probability vector is judged to be user sending information, and estimated user data is obtained according to the probability vector.
7. The LSTM and ResNets assisted deep learning end-to-end intelligent communication method of claim 1, wherein step S6 is to train the deep neural network according to the difference between the estimation data obtained in step S5 and the binary data randomly generated in step S1, specifically:
the method comprises the steps of calculating a loss function value between estimated user data output by a deep neural network and user data to be sent according to preset loss functions, quantitatively measuring the difference between actual output and expected output, calculating the partial derivative of a loss value to each trainable parameter in the deep neural network through a back propagation algorithm, updating the parameters of the deep neural network by using a preset optimization algorithm, and reducing the difference between the estimated user data of the deep neural network and the user data to be sent.
8. The LSTM and ResNets assisted deep learning end-to-end intelligent communication method of claim 1, wherein in step S7, if the actual channel changes slowly, retraining with low time cost and low computation cost is performed by a method including transfer learning, and the deep neural network training and the deployment application are performed synchronously by a means including parallel training.
9. An LSTM and ResNets assisted deep learning end-to-end intelligent communications system using the LSTM and ResNets assisted deep learning end-to-end intelligent communications method of any of claims 1 to 8, the system comprising:
the generating module is used for randomly generating binary data as data to be sent by a user;
the channel simulation module is used for setting a simulation channel environment;
the device comprises an initialization network module, a data processing module and a data processing module, wherein the initialization network module is used for initializing a deep neural network, the deep neural network comprises a sending block, a noise layer and a receiving block, and binary data randomly generated by the generation module are output to the noise layer after being processed by the sending block;
the noise layer scrambles and adds noise to the signal processed by the sending block according to the simulation channel environment set by the channel simulation module and sends the signal to the receiving block;
a receiving processing module, wherein the receiving block obtains estimated user data after using the processing opposite to that of the sending block;
the training module trains the deep neural network according to the difference between the estimated user data obtained by the receiving and processing module and the data to be sent by the user in the generating module;
and the deployment module is used for setting the trained sending block at the communication sending end, using binary data bits needing to be sent as input vectors of the sending block, using output vectors of the sending block as sending signals to be sent to a wireless channel, setting the trained receiving block at the communication receiving end, using the trained receiving block as the input vectors of the receiving block after the communication receiving end receives the signals, and processing the signals to obtain estimated user sending data.
10. The LSTM and ResNets assisted deep learning end-to-end intelligent communication system of claim 9, wherein the sending block of the deep neural network comprises an LSTM layer, a fully connected layer and a regularization layer, wherein an input of the LSTM layer is data to be sent by a user, an output of the LSTM layer is superimposed with an input of the LSTM layer to be used as an input of the fully connected layer to form a ResNets structure, an output of the fully connected layer is an input of the regularization layer, and an output of the regularization layer is an input of a noise layer.
CN202111113281.8A 2021-09-18 2021-09-18 LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system Active CN113852434B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111113281.8A CN113852434B (en) 2021-09-18 2021-09-18 LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111113281.8A CN113852434B (en) 2021-09-18 2021-09-18 LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system

Publications (2)

Publication Number Publication Date
CN113852434A true CN113852434A (en) 2021-12-28
CN113852434B CN113852434B (en) 2023-07-25

Family

ID=78979321

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111113281.8A Active CN113852434B (en) 2021-09-18 2021-09-18 LSTM and ResNet-assisted deep learning end-to-end intelligent communication method and system

Country Status (1)

Country Link
CN (1) CN113852434B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023164166A1 (en) * 2022-02-24 2023-08-31 Protopia AI, Inc. Conditional noise layers for generating adversarial examples

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109767759A (en) * 2019-02-14 2019-05-17 重庆邮电大学 End-to-end speech recognition methods based on modified CLDNN structure
CN110460402A (en) * 2019-07-15 2019-11-15 哈尔滨工程大学 A kind of end-to-end communication system method for building up based on deep learning

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109767759A (en) * 2019-02-14 2019-05-17 重庆邮电大学 End-to-end speech recognition methods based on modified CLDNN structure
CN110460402A (en) * 2019-07-15 2019-11-15 哈尔滨工程大学 A kind of end-to-end communication system method for building up based on deep learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
胡章芳;徐轩;付亚芹;夏志广;马苏东;: "基于ResNet-BLSTM的端到端语音识别", 计算机工程与应用, no. 18 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023164166A1 (en) * 2022-02-24 2023-08-31 Protopia AI, Inc. Conditional noise layers for generating adversarial examples

Also Published As

Publication number Publication date
CN113852434B (en) 2023-07-25

Similar Documents

Publication Publication Date Title
Park et al. Learning to demodulate from few pilots via offline and online meta-learning
Zhang et al. Deep learning-based beamspace channel estimation in mmWave massive MIMO systems
CN112637093B (en) Signal detection method based on model-driven deep learning
Mohamed et al. Model-based: End-to-end molecular communication system through deep reinforcement learning auto encoder
CN113381828B (en) Sparse code multiple access random channel modeling method based on condition generation countermeasure network
Khadr et al. Machine learning-based massive augmented spatial modulation (ASM) for IoT VLC systems
Morocho-Cayamcela et al. Learning to communicate with autoencoders: Rethinking wireless systems with deep learning
Smith et al. A communication channel density estimating generative adversarial network
CN113890799B (en) Underwater acoustic communication channel estimation and signal detection method based on domain countermeasure network
CN113852434A (en) LSTM and ResNet assisted deep learning end-to-end intelligent communication method and system
Ye et al. Bilinear convolutional auto-encoder based pilot-free end-to-end communication systems
CN114759997B (en) MIMO system signal detection method based on data model double driving
Apalina et al. Design of Constrained Codebooks Using Autoencoder Networks
Lu et al. Attention-Empowered Residual Autoencoder for End-to-End Communication Systems
Raviv et al. Adaptive data augmentation for deep receivers
Jiang et al. Adaptive semantic video conferencing for ofdm systems
Liu et al. MIMO signal multiplexing and detection based on compressive sensing and deep learning
CN112821971A (en) Time-varying channel signal detection method based on countermeasure learning
Chaudhari et al. A resnet based end-to-end wireless communication system under rayleigh fading and bursty noise channels
Du et al. Fully dense neural network for the automatic modulation recognition
Njoku et al. BLER performance evaluation of an enhanced channel autoencoder
Yuan et al. Sparse Code Multiple Access Scheme Based on Variational Learning
Hares et al. Recurrent neural networks for pilot-aided wireless communications
Mahmoud et al. Learning for Time Series Differential Phase-Shift Keying-based Non-coherent Receivers
CN115085808B (en) VLC system time-frequency joint post-equalization method based on wavelet neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant