CN114826850B - Modulation identification method, device and equipment based on time-frequency diagram and deep learning - Google Patents

Modulation identification method, device and equipment based on time-frequency diagram and deep learning Download PDF

Info

Publication number
CN114826850B
CN114826850B CN202210422929.8A CN202210422929A CN114826850B CN 114826850 B CN114826850 B CN 114826850B CN 202210422929 A CN202210422929 A CN 202210422929A CN 114826850 B CN114826850 B CN 114826850B
Authority
CN
China
Prior art keywords
frequency
signal
modulation
network
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210422929.8A
Other languages
Chinese (zh)
Other versions
CN114826850A (en
Inventor
雷菁
刘伟
邓喆
刘鹏涛
黄英
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202210422929.8A priority Critical patent/CN114826850B/en
Publication of CN114826850A publication Critical patent/CN114826850A/en
Application granted granted Critical
Publication of CN114826850B publication Critical patent/CN114826850B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L27/00Modulated-carrier systems
    • H04L27/0012Modulated-carrier systems arrangements for identifying the type of modulation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L27/00Modulated-carrier systems
    • H04L27/0014Carrier regulation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L27/00Modulated-carrier systems
    • H04L27/26Systems using multi-frequency codes
    • H04L27/2601Multicarrier modulation systems
    • H04L27/2647Arrangements specific to the receiver only
    • H04L27/2649Demodulators
    • H04L27/26542Wavelet transform demodulators 
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L27/00Modulated-carrier systems
    • H04L27/0014Carrier regulation
    • H04L2027/0024Carrier regulation at the receiver end
    • H04L2027/0026Correction of carrier offset

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Digital Transmission Methods That Use Modulated Carrier Waves (AREA)

Abstract

The application relates to a modulation identification method and device based on a time-frequency graph and deep learning, computer equipment and a storage medium. The method comprises the following steps: performing wavelet transformation on the phase modulation signals to obtain a transformation result, transforming the magnitude of a modulus value of the transformation result into a color map, and reconstructing the color map to obtain a wavelet transformation time-frequency map of the PSK modulation signals; constructing a deep neural network according to the size of a wavelet transform time-frequency diagram, the type of a PSK modulation mode and a residual error network; the deep neural network comprises a residual error connection network and a random channel network; inputting the wavelet transformation time-frequency diagram into a residual error connection network and a random channel network to obtain a plurality of recall rates of PSK modulation modes; and carrying out fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal. By adopting the method, the modulation identification efficiency and accuracy can be improved.

Description

Modulation identification method, device and equipment based on time-frequency graph and deep learning
Technical Field
The present application relates to the field of communications technologies, and in particular, to a modulation identification method and apparatus based on a time-frequency diagram and deep learning, a computer device, and a storage medium.
Background
With the rapid development of communication technologies and the increasing maturity of technologies such as mobile communication, wireless broadband access and satellite communication, the wireless communication environment is increasingly complex, and in order to meet the complex and various requirements of users, further improve the channel utilization rate and the anti-interference capability, and prolong the transmission distance, communication signals adopt various modulation modes. At present, communication and electronic countermeasure have become the strategic high points of competition in development of various countries, however, various modulation modes make the communication signal identification technology face the bottleneck, and how a receiving end modulates and identifies an intercepted low signal-to-noise ratio communication signal becomes a challenging problem. The modulation scheme of the communication signal is an important characteristic for distinguishing the communication signal, and an information acquisition party needs to know the modulation scheme of the signal in order to know the information content of the communication signal. And modulation identification, namely judging the modulation mode adopted by the signal according to the signal received by the receiver. The signal modulation identification has important significance in the military communication fields of detection, monitoring, electronic countermeasure and the like. In the information countermeasure, the receiver usually does not know the modulation system of the received signal, and the modulation identification technology can judge the modulation system of the received signal at this moment, which is a key step of signal demodulation.
However, conventional modulation recognition is mainly classified into two categories, a decision theory-based method and a feature extraction-based recognition method. The decision-theoretic-based method converts the modulation recognition problem into a hypothesis testing problem, and optimally determines which distribution the current data obeys from a given observation sample and several possible probability distributions. The methods based on the likelihood ratio hypothesis test, although theoretically optimal, are highly sensitive to parametric models and rely on modeling of modulation type and channel, which tend to be unpredictable in real third party reception and thus impact recognition performance. The feature extraction-based method is to extract features of various modulation types, mainly comprising constellation features, cumulant features, transform domain features and the like, and to construct a classifier by using one or more types of features to realize modulation identification. Although the modulation identification method based on the constellation diagram has intuitiveness, the performance of the modulation identification method depends on the recovery of the constellation diagram, and the constellation diagram is blurred under low signal-to-noise ratio, so that the classification performance of the modulation identification method is influenced. The modulation identification based on the transform domain features is to realize signal classification by using the feature difference of signals of different modulation types in a frequency domain or a time-frequency domain, and common transform methods comprise Fourier transform, wavelet transform and the like. But the method is limited by the traditional classifier, depends on manual selection, and has the problems of weak generalization ability, low efficiency and the like.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a modulation identification method, device, computer device and storage medium based on time-frequency diagram and deep learning, which can improve the modulation identification efficiency.
A modulation identification method based on time-frequency graphs and deep learning comprises the following steps:
constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal;
adding channel loss to the frequency band signal to obtain a simulated communication signal;
filtering and frequency converting the simulation communication signal by using a filter to obtain a frequency conversion signal;
sampling the frequency conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase modulation signal;
performing wavelet transformation on the phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into a color chart, and reconstructing the color chart to obtain a wavelet transformation time-frequency chart of the PSK modulation signals;
constructing a deep neural network according to the size of a wavelet transform time-frequency diagram, the type of a PSK modulation mode and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
inputting the wavelet transform time-frequency diagram into a residual error connection network and a random channel network to obtain a plurality of recall rates of PSK modulation modes;
and carrying out fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
In one embodiment, constructing satellite communication according to a PSK modulation scheme to obtain a frequency band signal includes:
satellite communication is constructed according to a PSK modulation mode to obtain a frequency band signal of
s(t)=real(Y(t)e exp(j2πf c t))
Wherein Y (t) is a complex baseband signal, e is a natural logarithm, theta is the initial phase of the baseband signal, f c Is the carrier frequency, t is time, and j is the imaginary unit.
In one embodiment, adding channel loss to the frequency band signal to obtain an artificial communication signal comprises:
and adding frequency shift, phase shift and white noise to the frequency band signal to obtain the simulated communication signal.
In one embodiment, the filtering and frequency converting the simulated communication signal by using a filter to obtain a frequency-converted signal includes:
and filtering the simulation communication signal by adopting an FIR filter to remove out-of-band noise, and then performing digital down-conversion on the signal by using a low-frequency filter to obtain a filtered signal.
In one embodiment, the wavelet transform is performed on the phase modulation signal to obtain a transform result, the magnitude of a modulus of the transform result is converted into a color map, and the color map is reconstructed to obtain a wavelet transform time-frequency map of the PSK modulation signal, including:
wavelet analysis is carried out by utilizing Morse wavelet, wavelet transformation is carried out on the phase modulation signal after a wavelet transformation base is determined, a transformation result is obtained, the magnitude of a modulus value of the transformation result is transformed into a color image, and the color image is reconstructed to obtain a wavelet transformation time-frequency image of the PSK modulation signal.
In one embodiment, the method for constructing the deep neural network according to the size of the wavelet transform time-frequency diagram, the type of the PSK modulation mode and the residual error network further includes:
and modifying the size of an input layer of the residual error network into the size of a wavelet transform time-frequency graph, and keeping the output sizes of a full connection layer and an output layer of the residual error network consistent with the number of types of PSK modulation modes to construct a deep neural network.
In one embodiment, the fusion processing is performed on the plurality of recall rates to obtain a modulation identification result of the frequency band signal, and the method includes:
the fusion processing is carried out on a plurality of recall rates, and the modulation identification result of the frequency band signal is obtained
Figure BDA0003608678740000031
Figure BDA0003608678740000032
Wherein, P is recall, TP is the number of positive examples predicted from positive examples, FP is the number of false examples predicted from positive examples, and θ is the modulation type selected by the fusion method.
A modulation identification apparatus based on time-frequency graphs and deep learning, the apparatus comprising:
the signal construction module is used for constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal; adding channel loss to the frequency band signal to obtain a simulated communication signal;
the data preprocessing module is used for filtering and frequency converting the simulation communication signals by adopting a filter to obtain frequency conversion signals; sampling the frequency conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase modulation signal;
the wavelet transformation module is used for performing wavelet transformation on the phase modulation signals to obtain a transformation result, transforming the magnitude of a modulus value of the transformation result into a color graph, and reconstructing the color graph to obtain a wavelet transformation time-frequency graph of the PSK modulation signals;
the network construction module is used for constructing a deep neural network according to the size of the wavelet transform time-frequency diagram, the type of PSK modulation modes and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
the modulation identification module is used for inputting the wavelet transformation time-frequency diagram into the residual error connection network and the random channel network to obtain a plurality of recall rates of the PSK modulation mode; and performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal;
adding channel loss to the frequency band signal to obtain a simulated communication signal;
filtering and frequency converting the simulation communication signal by using a filter to obtain a frequency conversion signal;
sampling the frequency-conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase-modulation signal;
performing wavelet transformation on the phase modulation signals to obtain a transformation result, transforming the magnitude of a modulus value of the transformation result into a color map, and reconstructing the color map to obtain a wavelet transformation time-frequency map of the PSK modulation signals;
constructing a deep neural network according to the size of the wavelet transform time-frequency diagram, the type of the PSK modulation mode and the residual error network; the deep neural network comprises a residual error connection network and a random channel network;
inputting the wavelet transform time-frequency diagram into a residual error connection network and a random channel network to obtain a plurality of recall rates of PSK modulation modes;
and performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal;
adding channel loss to the frequency band signal to obtain a simulated communication signal;
filtering and frequency converting the simulation communication signal by using a filter to obtain a frequency conversion signal;
sampling the frequency conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase modulation signal;
performing wavelet transformation on the phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into a color chart, and reconstructing the color chart to obtain a wavelet transformation time-frequency chart of the PSK modulation signals;
constructing a deep neural network according to the size of a wavelet transform time-frequency diagram, the type of a PSK modulation mode and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
inputting the wavelet transform time-frequency diagram into a residual error connection network and a random channel network to obtain a plurality of recall rates of PSK modulation modes;
and performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
According to the modulation identification method, the modulation identification device, the computer equipment and the storage medium based on the time-frequency diagram and the deep learning, satellite communication is constructed according to a PSK modulation mode, then the obtained frequency band signal is preprocessed to obtain a simulated communication signal, the frequency conversion signal is sampled by using an up-sampling method, and the sampling result is subjected to digital frequency conversion and normalization processing to obtain a phase modulation signal; the method comprises the steps of performing wavelet transformation on phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into color maps, reconstructing the color maps to obtain wavelet transformation time-frequency maps of the PSK modulation signals, and enabling the time-frequency maps to visually reflect signal characteristics and change details based on the characteristics of the time-frequency maps of the signals even if the number of received sample points is small. Because the signal frequency offset is represented by moving up and down on the spectrogram, the signal timing error is represented by moving left and right on the spectrogram, the noise has little influence on the image contour, the time-frequency diagram has strong robustness on the representation of the signal characteristics, and compared with modulation identification based on high-order cumulant, the method is insensitive to the carrier estimation frequency offset. Meanwhile, a deep neural network is constructed according to the size of the wavelet transform time-frequency graph, the type of the PSK modulation mode and a residual error network, the deep learning neural network is used as a classifier, the defect of dependence on manual selection of a judgment threshold is overcome, compared with the neural network based on original data input, the time-frequency graph is generated by using few data sampling points, the modulation mode can be still correctly identified under the condition that the received signal is short, and the modulation identification efficiency and accuracy are improved.
Drawings
FIG. 1 is a schematic flow chart illustrating a modulation identification method based on a time-frequency diagram and deep learning according to an embodiment;
FIG. 2 is a diagram illustrating a frequency response of a wavelet transform basis in one embodiment;
FIG. 3 is a diagram illustrating a color scheme for a wavelet transform time-frequency diagram in an embodiment;
FIG. 4 is a diagram of a time-frequency diagram result of wavelet transform for five types of PSK signals in another embodiment;
FIG. 5 is a diagram illustrating different layer outputs after different video pictures are input into a residual connection network in one embodiment;
FIG. 6 is a diagram illustrating the performance of a residual error connection network in identifying PSK-like signals with different carrier-to-noise ratios in one embodiment;
FIG. 7 is a diagram illustrating the performance of a random access network in identifying PSK-like signals with different carrier-to-noise ratios in one embodiment;
FIG. 8 is a block diagram of a modulation recognition apparatus based on time-frequency diagrams and deep learning according to an embodiment;
FIG. 9 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In one embodiment, as shown in fig. 1, a modulation identification method based on a time-frequency diagram and deep learning is provided, which includes the following steps:
102, constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal; and adding channel loss to the frequency band signal to obtain a simulated communication signal.
PSK modulation modes comprise BPSK, QPSK,8PSK, OQPSK and pi/4-QPSK, and the frequency band signals are obtained by performing baseband forming on the frequency band signals and general complex baseband symbol mapping of the PSK modulation modes; channel loss includes frequency shift, phase shift, and white noise.
104, filtering and frequency converting the simulation communication signal by using a filter to obtain a frequency conversion signal; and sampling the frequency conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase modulation signal.
For simulation communication signals, after the central frequency and the bandwidth are determined, an FIR filter can be directly adopted to filter the signals to remove out-of-band noise, then down-conversion is carried out to obtain variable frequency signals, and PSK signals need to be identified, so the change condition of the phase reflects the modulation order, the analytic wavelet can detect the change of the phase sharply, but the scale of wavelet transformation is nonlinear, the obtained time-frequency image is deformed, the bandwidth of the signals needs to be reduced, the sampling rate of the signals needs to be increased, and input signals with different sampling rates are up-sampled to the same sampling rate, so that the input is unified and beneficial to training. And the sampling rate is increased, the signal bandwidth is reduced, the observation scale is relatively enlarged, the signal representation is fuzzified, the energy change trend is reserved, and the removal of excessive complex details is favorable for the classification of the PSK signals. The up-sampling method is that a sampling function is used to restore a continuous signal, and then a corresponding sampling point is obtained under a new sampling frequency:
Figure BDA0003608678740000071
Figure BDA0003608678740000072
wherein T is time, x (T) is a signal time domain expression, T S Is the sampling interval, ω m Is the maximum analog frequency of the signal, f s ' is the new sampling frequency, and x (n) is the corresponding sampling point obtained at the new sampling frequency.
Since the continuous wavelet transform of a complex signal is divided into two parts (clockwise part and counterclockwise part), in order to reduce the amount of input, and all the inputs are unified into one frequency bin. Therefore, the application carries out normalization processing on the baseband complex signal after interpolation is completed through digital up-conversion, the normalization frequency is 0.25, and the real part is taken to carry out wavelet transformation.
And 106, performing wavelet transformation on the modulation signals to obtain a transformation result, converting the modulus value of the transformation result into a color map, and reconstructing the color map to obtain a wavelet transformation time-frequency map of the PSK modulation signals.
The time-frequency graph is generated by utilizing wavelet transformation, the wavelet transformation can well detect the phase change condition, the phase information of the time-frequency graph cannot be lost, and the time-frequency graph can visually reflect the characteristics and the change details of signals even if the number of received sample points is small based on the characteristics of the time-frequency graph of the signals. Because the signal frequency offset is represented by moving up and down on the spectrogram, the signal timing error is represented by moving left and right on the spectrogram, the noise has little influence on the image contour, the time-frequency diagram has strong robustness on the representation of the signal characteristics, and compared with modulation identification based on high-order cumulant, the method is insensitive to the carrier estimation frequency offset and has higher accuracy of modulation identification.
The continuous wavelet transform is defined as follows:
Figure BDA0003608678740000081
where s (t) is the signal, ψ (t) is the selected wavelet basis function (Morse wavelet is selected, the specific expression is defined below), constants a and b are called scale parameter and translation parameter, then the absolute value of the obtained wavelet transform value is taken first and normalized to (0, 1), then the value is used to determine the color at the corresponding point, the maximum value corresponds to gray, the minimum value corresponds to black, the specific color scheme is shown in FIG. 3:
Figure BDA0003608678740000082
108, constructing a deep neural network according to the size of the wavelet transform time-frequency diagram, the type of the PSK modulation mode and a residual error network; the deep neural network comprises a residual error connection network and a random channel network.
The size of an input layer of the residual error network is modified into the size of a wavelet transform time-frequency graph, and the output sizes of a full connection layer and an output layer of the residual error network are kept consistent with the number of types of PSK modulation modes to construct a deep neural network, so that the deep neural network can better classify and identify the characteristics and the modulation modes of the time-frequency graph.
Step 110, inputting the wavelet transform time-frequency diagram into a residual error connection network and a random channel network to obtain a plurality of recall rates of PSK modulation modes; and carrying out fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
And inputting the wavelet transformation time-frequency diagram into a residual error connection network and a random channel network to obtain the recall rate of the PSK modulation modes in the two networks, and fusing the recall rate of each modulation mode to obtain a modulation identification result.
According to the modulation identification method based on the time-frequency diagram and the deep learning, satellite communication is constructed according to a PSK type modulation mode, then the obtained frequency band signal is preprocessed to obtain a simulated communication signal, the frequency conversion signal is sampled by using an up-sampling method, and the sampling result is subjected to digital frequency conversion and normalization processing to obtain a phase modulation signal; the method comprises the steps of performing wavelet transformation on phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into color maps, reconstructing the color maps to obtain wavelet transformation time-frequency maps of the PSK modulation signals, and enabling the time-frequency maps to visually reflect signal characteristics and change details based on the characteristics of the time-frequency maps of the signals even if the number of received sample points is small. Because the signal frequency offset is represented by up-and-down movement on the spectrogram, the signal timing error is represented by left-and-right movement on the spectrogram, the noise has little influence on the image contour, the time-frequency graph has strong robustness on the representation of the signal characteristics, and compared with modulation identification based on high-order cumulant, the method is insensitive to the carrier estimation frequency offset. Meanwhile, a deep neural network is constructed according to the size of the wavelet transform time-frequency image, the type of the PSK modulation mode and a residual error network, the deep learning neural network is used as a classifier, the defect of dependence on manual selection of a judgment threshold is overcome, compared with a neural network based on original data input, the time-frequency image is generated by using few data sampling points, the modulation mode can be still correctly identified under the condition that the received signal is short, and the modulation identification efficiency and accuracy are improved.
In one embodiment, constructing satellite communication according to a PSK modulation scheme to obtain a frequency band signal includes:
constructing satellite communication according to PSK modulation mode to obtain frequency band signal of
s(t)=real(Y(t)e exp(j2πf c t))
Wherein Y (t) is complex baseband signal, e is natural logarithm, theta is initial phase of baseband signal, f c Is the carrier frequency, t is time, and j is the imaginary unit.
The general complex baseband symbol mapping and the band signal form of the PSK-like modulation scheme can be expressed as follows:
Figure BDA0003608678740000091
wherein s is n (t) is PSK complex baseband signal, M is PSK order, pi is circumferential rate, j is imaginary unit
OQPSK is half cycle offset in the quadrature and in-phase components compared to QPSK, and pi/4-QPSK rotates the constellation by pi/4 every one symbol cycle compared to QPSK.
The shaping filtering adopts square root raised cosine pulse filter or Gaussian filter, and the expressions of the square root raised cosine pulse filter and the Gaussian filter are respectively
Figure BDA0003608678740000101
Figure BDA0003608678740000102
Wherein
Figure BDA0003608678740000103
Wherein r is p (T) is the raised cosine function, T s Is the sampling interval, alpha is the roll-off coefficient, r s (t) is the root raised cosine function, tdust. g (t) is a Gaussian function and BT is a time-bandwidth product. log is the natural logarithm.
The expression of the frequency band signal obtained after baseband molding is as follows:
s(t)=real(Y(t)e exp(j2πf c t))
in one embodiment, adding channel loss to the frequency band signal to obtain an artificial communication signal comprises:
and adding frequency shift, phase shift and white noise to the frequency band signal to obtain the simulated communication signal.
The frequency offset caused by different clocks is expressed as follows:
Δf=offset/f clock ·f c
wherein, the offset is the number of difference points in one second of the clock of the sending end and the receiving end, f clock Is the fundamental frequency of the clock, f c For transmitting signal carrier frequency, timing drift can be obtained by obtaining real sampling frequency and sampling time by a similar method and obtaining the real sampling frequency and the sampling time by signal interpolation.
The expression for adding the frequency shift Δ f and the phase shift Δ φ is as follows:
Y'(t)=Y(t)·e jΔφ+Δft
and finally, adding larger white noise to obtain the simulated communication signal.
Indicators for the simulated communication signal data set are shown in table 1:
TABLE 1 simulation data set indices
Figure BDA0003608678740000111
In one embodiment, the filtering and frequency converting the simulated communication signal by using a filter to obtain a frequency-converted signal includes:
and filtering the simulation communication signal by adopting an FIR filter to remove out-of-band noise, and then performing digital down-conversion on the signal by using a low-frequency filter to obtain a filtered signal.
In one embodiment, the wavelet transform is performed on the phase modulation signal to obtain a transform result, the magnitude of a modulus of the transform result is converted into a color map, and the color map is reconstructed to obtain a wavelet transform time-frequency map of the PSK modulation signal, including:
wavelet analysis is carried out by utilizing Morse wavelets, wavelet transformation is carried out on the phase modulation signals after a wavelet transformation base is determined, transformation results are obtained, the magnitude of a modulus value of the transformation results is converted into a color graph, and the color graph is reconstructed to obtain a wavelet transformation time-frequency graph of the PSK modulation signals.
The phase modulated signal has phase discontinuities and in this application wavelet analysis is performed using a Morse-resolved wavelet. The Fourier transform of Morse wavelet only has a real axis part, and the specific Fourier transform expression is as follows:
Figure BDA0003608678740000112
wherein U (omega) represents unit step response, omega is angular frequency, e is natural logarithm, alpha P,γ Denotes the normalization parameter, P 2 Representing the time bandwidth product, P is proportional to the duration of the wavelet, gamma reflects the symmetry of the Morse wavelet, and the 3-hour Morse wavelet is taken to have the minimum Heisenberg area. The wavelet transform base is adopted to select the frequency range of [0.20.35 ]]The number of the spacing points between every two frequency multiplication is 48, the frequency response of the selected transformation base is shown in figure 2, continuous wavelet transformation is carried out on the communication signals, the magnitude of the modulus of the result is converted into a color map, a 128 color map is selected, and the color scheme of the wavelet transformation time-frequency map is shown in figure 3. Mapping the module value of the result to color and reconstructing the size of the picture to 50 multiplied by 2000 multiplied by 3 to obtain five-class PSK modulation signal wavelet transformation time-frequency graphs which are respectively BPSK, QPSK,8PSK, OQPSK and pi/4-QPSK from top to bottom as shown in figure 4.
In one embodiment, the method for constructing the deep neural network according to the size of the wavelet transform time-frequency diagram, the type of the PSK modulation mode and the residual error network further includes:
and modifying the size of an input layer of the residual error network into the size of a wavelet transform time-frequency graph, and keeping the output sizes of a full connection layer and an output layer of the residual error network consistent with the number of types of PSK modulation modes to construct a deep neural network.
The network construction adopts a residual error network, the size of an input layer of the modified residual error network is 50 multiplied by 2000 multiplied by 3, the output of a full connection layer and an output layer is 5, and in order to reduce the calculation amount and improve the accuracy, a random channel network unit is used for replacing a residual error block in the residual error network, wherein the random channel network unit comprises point-by-point group convolution, channel rearrangement and depth separable convolution. After the deep neural network is built, a time-frequency graph is input for training, sgdm optimizers are adopted, and specific training parameters of the two networks are respectively shown in table 2.
TABLE 2 network training parameters
Figure BDA0003608678740000131
And constructing a satellite communication signal data set by using a PSK (phase shift keying) modulation mode, carrying out data processing on the data set to obtain a time-frequency graph, verifying the deep neural network by using the time-frequency graph, and inputting the time-frequency graph into a residual connecting network and a random channel network, namely ResNet and Shufflenet. Taking a residual error connection network as an example, observing the outputs of the residual error connection network on a posing layer, a last convolutional layer and a softmax layer, and visualizing by adopting a t-distributed random field embedding (t-SNE) method to obtain the correlation between different video images input through the network as shown in fig. 5, wherein ResNet18 is the residual error connection network, it can be seen that the differences between five types of inputs on a shallow layer are not large, the distances between the five types of inputs are increased after passing through various intermediate hidden layers, five types of PSK communication signals are well separated at the softmax layer, but a part of samples are not correctly separated into proper regions to cause identification errors, five PSK type signal identification results of two networks under different signal-to-noise ratios are respectively tested, as shown in fig. 6 and 7, fig. 6 is a case where ResNet identifies different PSK type signal performances under different signal-to-noise ratios, fig. 7 is a case where shufflen is a random channel network, and it can be seen that the identification effect of two networks on five types of signals can be more than 90%.
In one embodiment, the fusion processing is performed on the plurality of recall rates to obtain a modulation identification result of the frequency band signal, and the method includes:
the fusion processing is carried out on a plurality of recall rates, and the modulation identification result of the frequency band signal is obtained
Figure BDA0003608678740000132
Figure BDA0003608678740000133
Wherein, P is recall rate, TP is the number of predicted positive examples, FP is the number of predicted false examples, and θ is the modulation type selected by the fusion method.
The specific implementation process is that according to the principle of frequency approximation probability, the recall rate of network identification of a certain signal category is regarded as a network identification result, and the posterior probability of the signal category is determined by the input signal under the condition that the network identification result is the signal of the signal category. Inputting a signal, different recognition results exist, the probability that the positive class is predicted to be the positive class is assumed to be TP, and the probability that the negative class is predicted to be the positive class is assumed to be FP, so the recall rate P calculation method and the later stage fusion principle of the two networks are as follows:
Figure BDA0003608678740000141
Figure BDA0003608678740000142
and the two networks simultaneously judge the input, then inquire the recall ratio below the network confusion matrix according to the result, select the result with higher recall ratio to become the result of later-stage fusion, and fuse the identification results of the two networks to obtain the final modulation identification result with higher accuracy.
It should be understood that, although the steps in the flowchart of fig. 1 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 1 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 8, there is provided a modulation identification apparatus based on time-frequency diagram and deep learning, including: a build signal module 802, a data pre-processing module 804, a wavelet transform module 806, a network build module 808, and a modulation identification module 810, wherein:
a signal constructing module 802, configured to construct satellite communication according to a PSK modulation scheme, to obtain a frequency band signal; adding channel loss to the frequency band signal to obtain a simulated communication signal;
the data preprocessing module 804 is configured to filter and frequency-convert the simulated communication signal by using a filter to obtain a frequency-converted signal; sampling the frequency-conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase-modulation signal;
the wavelet transform module 806 is configured to perform wavelet transform on the phase modulation signal to obtain a transform result, convert a magnitude of a modulus of the transform result into a color map, and reconstruct the color map to obtain a wavelet transform time-frequency map of the PSK modulation signal;
a network construction module 808, configured to construct a deep neural network according to the size of the wavelet transform time-frequency diagram, the type of the PSK modulation mode, and the residual error network; the deep neural network comprises a residual error connection network and a random channel network;
the modulation identification module 810 is configured to input the wavelet transform time-frequency diagram to a residual connection network and a random channel network, so as to obtain multiple recall rates of PSK modulation modes; and performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
In one embodiment, the module 802 for constructing a signal is further configured to construct satellite communication according to a PSK-like modulation scheme, so as to obtain a frequency band signal, including:
satellite communication is constructed according to a PSK modulation mode to obtain a frequency band signal of
s(t)=real(Y(t)e exp(j2πf c t))
Wherein Y (t) is complex baseband signal, e is natural logarithm, theta is initial phase of baseband signal, f c Is the carrier frequency, t is time, and j is the imaginary unit.
In one embodiment, the signal constructing module 802 is further configured to add channel loss to the frequency band signal to obtain an artificial communication signal, including:
and adding frequency shift, phase shift and white noise to the frequency band signal to obtain the simulated communication signal.
In one embodiment, the data preprocessing module 804 is further configured to filter and frequency-convert the simulated communication signal by using a filter to obtain a frequency-converted signal, including:
and filtering the simulation communication signal by adopting an FIR filter to remove out-of-band noise, and then performing digital down-conversion on the signal by using a low-frequency filter to obtain a filtered signal.
In one embodiment, the wavelet transform module 806 is further configured to perform wavelet transform on the phase-modulated signal to obtain a transform result, convert a magnitude of a modulus of the transform result into a color map, and reconstruct the color map to obtain a wavelet transform time-frequency map of the PSK-like modulated signal, where the wavelet transform time-frequency map includes:
wavelet analysis is carried out by utilizing Morse wavelet, wavelet transformation is carried out on the phase modulation signal after a wavelet transformation base is determined, a transformation result is obtained, the magnitude of a modulus value of the transformation result is transformed into a color image, and the color image is reconstructed to obtain a wavelet transformation time-frequency image of the PSK modulation signal.
In one embodiment, the network constructing module 808 is further configured to construct a deep neural network according to the size of the wavelet transform time-frequency diagram, the type of the PSK modulation mode, and the residual error network, and further includes:
and modifying the size of an input layer of the residual error network into the size of a wavelet transform time-frequency graph, and keeping the output sizes of a full connection layer and an output layer of the residual error network consistent with the number of types of PSK modulation modes to construct a deep neural network.
In one embodiment, the modulation identification module 810 is further configured to perform fusion processing on the plurality of recalls to obtain a modulation identification result of the frequency band signal, including:
the fusion processing is carried out on a plurality of recall rates, and the modulation identification result of the frequency band signal is obtained
Figure BDA0003608678740000161
Figure BDA0003608678740000162
Wherein, P is recall, TP is the number of positive examples predicted from positive examples, FP is the number of false examples predicted from positive examples, and θ is the modulation type selected by the fusion method.
For specific limitations of the modulation recognition device based on the time-frequency diagram and the deep learning, reference may be made to the limitations of the modulation recognition method based on the time-frequency diagram and the deep learning, which are not described herein again. The modules in the modulation identification device based on the time-frequency diagram and the deep learning can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 9. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program to run on the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a modulation recognition method based on time-frequency graphs and deep learning. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 9 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In an embodiment, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the method in the above embodiments when the processor executes the computer program.
In an embodiment, a computer storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, is adapted to carry out the steps of the method of the above embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct Rambus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), among others.
All possible combinations of the technical features in the above embodiments may not be described for the sake of brevity, but should be considered as being within the scope of the present disclosure as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent application shall be subject to the appended claims.

Claims (9)

1. A modulation identification method based on time-frequency graphs and deep learning is characterized by comprising the following steps:
constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal;
adding channel loss to the frequency band signal to obtain a simulated communication signal;
filtering and frequency converting the simulation communication signal by using a filter to obtain a frequency conversion signal;
sampling the frequency-conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase-modulation signal;
performing wavelet transformation on the phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into a color map, and reconstructing the color map to obtain a wavelet transformation time-frequency map of the PSK modulation signals;
constructing a deep neural network according to the size of a wavelet transform time-frequency diagram, the type of a PSK modulation mode and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
inputting the wavelet transform time-frequency diagram into the residual error connection network and a random channel network to obtain a plurality of recall rates of the PSK modulation mode;
performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal;
the fusion processing is carried out on the plurality of recall rates to obtain the modulation identification result of the frequency band signal, and the method comprises the following steps:
the plurality of recall rates are fused, and the modulation identification result of the frequency band signal is obtained
Figure FDA0003889682580000011
Figure FDA0003889682580000012
Wherein, P is recall rate, TP is the number of predicted positive examples, FP is the number of predicted false examples, and θ is the modulation type selected by the fusion method.
2. The method of claim 1, wherein constructing satellite communication according to PSK-like modulation to obtain a frequency band signal comprises:
satellite communication is constructed according to a PSK modulation mode to obtain a frequency band signal of
s(t)=real(Y(t)e exp(j2πf c t))
Wherein Y (t) is a complex baseband signal, e is a natural logarithm, theta is the initial phase of the baseband signal, f c Is the carrier frequency, t is time, and j is the imaginary unit.
3. The method of claim 1, wherein adding channel loss to the frequency band signal to obtain an emulated communication signal comprises:
and adding frequency shift, phase shift and white noise to the frequency band signal to obtain a simulated communication signal.
4. The method of any of claims 1 to 3, wherein filtering and frequency converting the simulated communication signal with a filter to obtain a frequency converted signal comprises:
and filtering the simulation communication signal by adopting an FIR filter, removing out-of-band noise, and then performing digital down-conversion on the signal by using a low-frequency filter to obtain a filtered signal.
5. The method of claim 1, wherein performing wavelet transform on the phase-modulated signals to obtain a transform result, transforming a magnitude of a modulus of the transform result into a color map, and reconstructing the color map to obtain a wavelet transform time-frequency map of the PSK-like modulated signals, comprises:
wavelet analysis is carried out by utilizing Morse wavelets, wavelet transformation is carried out on the phase modulation signals after a wavelet transformation base is determined, transformation results are obtained, the magnitude of modulus values of the transformation results is converted into a color graph, and a wavelet transformation time-frequency graph of the PSK modulation signals is obtained after the color graph is reconstructed.
6. The method of claim 5, wherein the deep neural network is constructed according to the size of the wavelet transform time-frequency diagram, the type of PSK modulation mode and the residual error network, and further comprising:
and modifying the size of an input layer of the residual error network into the size of the wavelet transformation time-frequency diagram, and keeping the output sizes of a full connection layer and an output layer of the residual error network consistent with the type number of the PSK type modulation modes to construct a deep neural network.
7. A modulation recognition device based on time-frequency graph and deep learning is characterized in that the device comprises:
the signal construction module is used for constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal; adding channel loss to the frequency band signal to obtain a simulated communication signal;
the data preprocessing module is used for filtering and frequency-converting the simulation communication signal by adopting a filter to obtain a frequency-converted signal; sampling the frequency-conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase-modulation signal;
the wavelet transform module is used for performing wavelet transform on the phase modulation signals to obtain transform results, converting the magnitude of modulus values of the transform results into a color map, and reconstructing the color map to obtain a wavelet transform time-frequency map of the PSK modulation signals;
the network construction module is used for constructing a deep neural network according to the size of the wavelet transform time-frequency diagram, the type of PSK modulation modes and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
the modulation identification module is used for inputting the wavelet transformation time-frequency diagram into the residual error connection network and the random channel network to obtain a plurality of recall rates of the PSK modulation mode; performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal; performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal, including:
the plurality of recall rates are fused, and the modulation identification result of the frequency band signal is obtained
Figure FDA0003889682580000031
Figure FDA0003889682580000032
Wherein, P is recall rate, TP is the number of predicted positive examples, FP is the number of predicted false examples, and θ is the modulation type selected by the fusion method.
8. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.
CN202210422929.8A 2022-04-21 2022-04-21 Modulation identification method, device and equipment based on time-frequency diagram and deep learning Active CN114826850B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210422929.8A CN114826850B (en) 2022-04-21 2022-04-21 Modulation identification method, device and equipment based on time-frequency diagram and deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210422929.8A CN114826850B (en) 2022-04-21 2022-04-21 Modulation identification method, device and equipment based on time-frequency diagram and deep learning

Publications (2)

Publication Number Publication Date
CN114826850A CN114826850A (en) 2022-07-29
CN114826850B true CN114826850B (en) 2022-11-25

Family

ID=82506387

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210422929.8A Active CN114826850B (en) 2022-04-21 2022-04-21 Modulation identification method, device and equipment based on time-frequency diagram and deep learning

Country Status (1)

Country Link
CN (1) CN114826850B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117743946B (en) * 2024-02-19 2024-04-30 山东大学 Signal type identification method and system based on fusion characteristic and group convolution ViT network

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103905360A (en) * 2014-02-19 2014-07-02 江苏科技大学 Non-cooperative BPSK signal decoding method with polarity judgment operation
CN113315727A (en) * 2021-03-29 2021-08-27 中山大学 Digital communication signal modulation identification method based on preprocessing noise reduction

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10644916B1 (en) * 2002-05-14 2020-05-05 Genghiscomm Holdings, LLC Spreading and precoding in OFDM
US7881397B2 (en) * 2005-03-31 2011-02-01 Teradyne, Inc. Wireless communication system
US9379924B2 (en) * 2013-12-20 2016-06-28 King Fahd University Of Petroleum And Minerals Cognitive radio spectrum sensing with improved edge detection of frequency bands
CN111585662B (en) * 2020-06-01 2021-09-07 中国科学院声学研究所东海研究站 Classification identification and parameter estimation method and system for phase modulation signal
CN112347871B (en) * 2020-10-23 2023-04-07 中国电子科技集团公司第七研究所 Interference signal modulation identification method for communication carrier monitoring system
CN112307927A (en) * 2020-10-26 2021-02-02 重庆邮电大学 BP network-based identification research for MPSK signals in non-cooperative communication
CN113537402A (en) * 2021-08-13 2021-10-22 重庆大学 Vibration signal-based converter transformer multi-scale fusion feature extraction method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103905360A (en) * 2014-02-19 2014-07-02 江苏科技大学 Non-cooperative BPSK signal decoding method with polarity judgment operation
CN113315727A (en) * 2021-03-29 2021-08-27 中山大学 Digital communication signal modulation identification method based on preprocessing noise reduction

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于小波变换的信号时频分析与重构;董航等;《中国新通信》;20160805(第15期);全文 *

Also Published As

Publication number Publication date
CN114826850A (en) 2022-07-29

Similar Documents

Publication Publication Date Title
CN108234370B (en) Communication signal modulation mode identification method based on convolutional neural network
CN109587372B (en) Invisible image steganography based on generation of countermeasure network
CN112702294B (en) Modulation recognition method for multi-level feature extraction based on deep learning
CN110569752A (en) convolutional neural network-based radar signal category determination method
CN114826850B (en) Modulation identification method, device and equipment based on time-frequency diagram and deep learning
CN114553648A (en) Wireless communication modulation mode identification method based on space-time diagram convolutional neural network
CN116992299B (en) Training method, detecting method and device of blockchain transaction anomaly detection model
Ali et al. Algorithm for automatic recognition of PSK and QAM with unique classifier based on features and threshold levels
CN112737992A (en) Underwater sound signal modulation mode self-adaptive in-class identification method
CN114422311A (en) Signal modulation identification method and system combining deep neural network and expert prior characteristics
CN114896887A (en) Frequency-using equipment radio frequency fingerprint identification method based on deep learning
Al‐Makhlasawy et al. Modulation classification in the presence of adjacent channel interference using convolutional neural networks
CN116628566A (en) Communication signal modulation classification method based on aggregated residual transformation network
CN113726711B (en) OFDM receiving method and device, and channel estimation model training method and device
CN113076925B (en) M-QAM signal modulation mode identification method based on CNN and ELM
CN114925720A (en) Small sample modulation signal identification method based on space-time mixed feature extraction network
CN114615118A (en) Modulation identification method based on multi-terminal convolution neural network
CN112183300B (en) AIS radiation source identification method and system based on multi-level sparse representation
Vagollari et al. An End-to-End Deep Learning Framework for Wideband Signal Recognition
CN115086123B (en) Modulation identification method and system based on fusion of time-frequency diagram and constellation diagram
CN113822162B (en) Convolutional neural network modulation identification method based on pseudo constellation diagram
CN113869227B (en) Signal modulation mode identification method, device, equipment and readable storage medium
CN116016071A (en) Modulation signal identification method based on double-flow fusion CNN-BiLSTM network
CN114548201A (en) Automatic modulation identification method and device for wireless signal, storage medium and equipment
CN116319210A (en) Signal lightweight automatic modulation recognition method and system based on deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant