CN114826850A - Modulation identification method, device and equipment based on time-frequency diagram and deep learning - Google Patents
Modulation identification method, device and equipment based on time-frequency diagram and deep learning Download PDFInfo
- Publication number
- CN114826850A CN114826850A CN202210422929.8A CN202210422929A CN114826850A CN 114826850 A CN114826850 A CN 114826850A CN 202210422929 A CN202210422929 A CN 202210422929A CN 114826850 A CN114826850 A CN 114826850A
- Authority
- CN
- China
- Prior art keywords
- frequency
- signal
- modulation
- network
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L27/00—Modulated-carrier systems
- H04L27/0012—Modulated-carrier systems arrangements for identifying the type of modulation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L27/00—Modulated-carrier systems
- H04L27/0014—Carrier regulation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L27/00—Modulated-carrier systems
- H04L27/26—Systems using multi-frequency codes
- H04L27/2601—Multicarrier modulation systems
- H04L27/2647—Arrangements specific to the receiver only
- H04L27/2649—Demodulators
- H04L27/26542—Wavelet transform demodulators
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L27/00—Modulated-carrier systems
- H04L27/0014—Carrier regulation
- H04L2027/0024—Carrier regulation at the receiver end
- H04L2027/0026—Correction of carrier offset
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Digital Transmission Methods That Use Modulated Carrier Waves (AREA)
Abstract
The application relates to a modulation identification method and device based on a time-frequency graph and deep learning, computer equipment and a storage medium. The method comprises the following steps: performing wavelet transformation on the phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into a color chart, and reconstructing the color chart to obtain a wavelet transformation time-frequency chart of the PSK modulation signals; constructing a deep neural network according to the size of a wavelet transform time-frequency diagram, the type of a PSK modulation mode and a residual error network; the deep neural network comprises a residual error connection network and a random channel network; inputting the wavelet transform time-frequency diagram into a residual error connection network and a random channel network to obtain a plurality of recall rates of PSK modulation modes; and performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal. By adopting the method, the modulation identification efficiency and accuracy can be improved.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a modulation identification method and apparatus based on a time-frequency diagram and deep learning, a computer device, and a storage medium.
Background
With the rapid development of communication technology and the increasing maturity of technologies such as mobile communication, wireless broadband access and satellite communication, the wireless communication environment is increasingly complex, and in order to meet the complex and various requirements of users and further improve the channel utilization rate and the anti-interference capability and prolong the transmission distance, communication signals adopt diversified modulation modes. At present, communication and electronic countermeasure have become the strategic high points of competition in development of various countries, however, various modulation modes make the communication signal identification technology face the bottleneck, and how a receiving end modulates and identifies an intercepted low signal-to-noise ratio communication signal becomes a challenging problem. The modulation scheme of the communication signal is an important characteristic for distinguishing the communication signal, and an information acquisition party needs to know the modulation scheme of the signal in order to know the information content of the communication signal. And modulation identification, namely judging the modulation mode adopted by the signal according to the signal received by the receiver. The signal modulation identification has important significance in the military communication fields of detection, monitoring, electronic countermeasure and the like. In the information countermeasure, the receiver usually does not know the modulation system of the received signal, and the modulation identification technology can judge the modulation system of the received signal at this time, which is a key step of signal demodulation.
However, conventional modulation recognition is mainly classified into two categories, a decision theory-based method and a feature extraction-based recognition method. The decision-theoretic-based method converts the modulation recognition problem into a hypothesis testing problem, and optimally determines which distribution the current data obeys from a given observation sample and several possible probability distributions. The methods based on the likelihood ratio hypothesis test, although theoretically optimal, are highly sensitive to parametric models and rely on modeling of modulation type and channel, which tend to be unpredictable in real third party reception and thus impact recognition performance. The feature extraction-based method is to extract features of various modulation types, mainly comprising constellation features, cumulant features, transform domain features and the like, and to construct a classifier by using one or more types of features to realize modulation identification. Although the modulation identification method based on the constellation diagram has intuitiveness, the performance of the modulation identification method depends on the recovery of the constellation diagram, and the constellation diagram is blurred under low signal-to-noise ratio, so that the classification performance of the modulation identification method is influenced. The modulation identification based on the transform domain features is to realize signal classification by using the feature difference of signals of different modulation types in a frequency domain or a time-frequency domain, and common transform methods comprise Fourier transform, wavelet transform and the like. But the method is limited by the traditional classifier, depends on manual selection, and has the problems of weak generalization ability, low efficiency and the like.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a modulation identification method, device, computer device and storage medium based on time-frequency diagram and deep learning, which can improve the modulation identification efficiency.
A modulation identification method based on time-frequency graphs and deep learning comprises the following steps:
constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal;
adding channel loss to the frequency band signal to obtain a simulated communication signal;
filtering and frequency converting the simulation communication signal by using a filter to obtain a frequency conversion signal;
sampling the frequency-conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase-modulation signal;
performing wavelet transformation on the phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into a color chart, and reconstructing the color chart to obtain a wavelet transformation time-frequency chart of the PSK modulation signals;
constructing a deep neural network according to the size of a wavelet transform time-frequency diagram, the type of a PSK modulation mode and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
inputting the wavelet transform time-frequency diagram into a residual error connection network and a random channel network to obtain a plurality of recall rates of PSK modulation modes;
and performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
In one embodiment, constructing satellite communication according to a PSK modulation scheme to obtain a frequency band signal includes:
satellite communication is constructed according to a PSK modulation mode to obtain a frequency band signal of
s(t)＝real(Y(t)e jθ exp(j2πf c t))
Wherein Y (t) is a complex baseband signal, e is a natural logarithm, theta is the initial phase of the baseband signal, f c Is the carrier frequency, t is time, and j is the imaginary unit.
In one embodiment, adding channel loss to the frequency band signal to obtain an artificial communication signal comprises:
and adding frequency shift, phase shift and white noise to the frequency band signal to obtain the simulated communication signal.
In one embodiment, the filtering and frequency converting the simulated communication signal by using a filter to obtain a frequency-converted signal includes:
and filtering the simulation communication signal by using an FIR filter to remove out-of-band noise, and then performing digital down-conversion on the signal by using a low-frequency filter to obtain a filtered signal.
In one embodiment, the wavelet transform is performed on the phase modulation signal to obtain a transform result, the magnitude of a modulus of the transform result is converted into a color map, and the color map is reconstructed to obtain a wavelet transform time-frequency map of the PSK modulation signal, including:
wavelet analysis is carried out by utilizing Morse wavelets, wavelet transformation is carried out on the phase modulation signals after a wavelet transformation base is determined, transformation results are obtained, the magnitude of a modulus value of the transformation results is converted into a color graph, and the color graph is reconstructed to obtain a wavelet transformation time-frequency graph of the PSK modulation signals.
In one embodiment, the method for constructing the deep neural network according to the size of the wavelet transform time-frequency diagram, the type of the PSK modulation mode and the residual error network further includes:
and modifying the size of an input layer of the residual error network into the size of a wavelet transform time-frequency graph, and keeping the output sizes of a full connection layer and an output layer of the residual error network consistent with the number of types of PSK modulation modes to construct a deep neural network.
In one embodiment, the fusion processing of the plurality of recalls to obtain the modulation identification result of the frequency band signal includes:
the fusion processing is carried out on a plurality of recall rates, and the modulation identification result of the frequency band signal is obtained
Wherein, P is recall, TP is the number of positive examples predicted from positive examples, FP is the number of false examples predicted from positive examples, and θ is the modulation type selected by the fusion method.
A modulation identification apparatus based on time-frequency graphs and deep learning, the apparatus comprising:
the signal construction module is used for constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal; adding channel loss to the frequency band signal to obtain a simulated communication signal;
the data preprocessing module is used for filtering and frequency-converting the simulation communication signals by adopting a filter to obtain frequency-converted signals; sampling the frequency-conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase-modulation signal;
the wavelet transformation module is used for performing wavelet transformation on the phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into a color map, and reconstructing the color map to obtain a wavelet transformation time-frequency map of the PSK modulation signals;
the network construction module is used for constructing a deep neural network according to the size of the wavelet transform time-frequency diagram, the type of PSK modulation modes and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
the modulation identification module is used for inputting the wavelet transformation time-frequency diagram into the residual error connection network and the random channel network to obtain a plurality of recall rates of the PSK modulation mode; and carrying out fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal;
adding channel loss to the frequency band signal to obtain a simulated communication signal;
filtering and frequency converting the simulation communication signal by using a filter to obtain a frequency conversion signal;
sampling the frequency-conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase-modulation signal;
performing wavelet transformation on the phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into a color chart, and reconstructing the color chart to obtain a wavelet transformation time-frequency chart of the PSK modulation signals;
constructing a deep neural network according to the size of a wavelet transform time-frequency diagram, the type of a PSK modulation mode and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
inputting the wavelet transform time-frequency diagram into a residual error connection network and a random channel network to obtain a plurality of recall rates of PSK modulation modes;
and performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal;
adding channel loss to the frequency band signal to obtain a simulated communication signal;
filtering and frequency converting the simulation communication signal by using a filter to obtain a frequency conversion signal;
sampling the frequency conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase modulation signal;
performing wavelet transformation on the phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into a color chart, and reconstructing the color chart to obtain a wavelet transformation time-frequency chart of the PSK modulation signals;
constructing a deep neural network according to the size of a wavelet transform time-frequency diagram, the type of a PSK modulation mode and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
inputting the wavelet transform time-frequency diagram into a residual error connection network and a random channel network to obtain a plurality of recall rates of PSK modulation modes;
and carrying out fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
According to the modulation identification method, the modulation identification device, the computer equipment and the storage medium based on the time-frequency diagram and the deep learning, satellite communication is constructed according to a PSK modulation mode, then the obtained frequency band signal is preprocessed to obtain a simulated communication signal, the frequency conversion signal is sampled by using an up-sampling method, and the sampling result is subjected to digital frequency conversion and normalization processing to obtain a phase modulation signal; the method comprises the steps of performing wavelet transformation on phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into color maps, reconstructing the color maps to obtain wavelet transformation time-frequency maps of the PSK modulation signals, and enabling the time-frequency maps to visually reflect signal characteristics and change details based on the characteristics of the time-frequency maps of the signals even if the number of received sample points is small. Because the signal frequency offset is represented by moving up and down on the spectrogram, the signal timing error is represented by moving left and right on the spectrogram, the noise has little influence on the image contour, the time-frequency diagram has strong robustness on the representation of the signal characteristics, and compared with modulation identification based on high-order cumulant, the method is insensitive to the carrier estimation frequency offset. Meanwhile, a deep neural network is constructed according to the size of the wavelet transform time-frequency graph, the type of the PSK modulation mode and a residual error network, the deep learning neural network is used as a classifier, the defect of dependence on manual selection of a judgment threshold is overcome, compared with the neural network based on original data input, the time-frequency graph is generated by using few data sampling points, the modulation mode can be still correctly identified under the condition that the received signal is short, and the modulation identification efficiency and accuracy are improved.
Drawings
FIG. 1 is a schematic flow chart illustrating a modulation identification method based on a time-frequency diagram and deep learning according to an embodiment;
FIG. 2 is a diagram illustrating the frequency response of the wavelet transform basis in one embodiment;
FIG. 3 is a diagram illustrating a color scheme for a wavelet transform time-frequency diagram according to an embodiment;
FIG. 4 is a diagram of a time-frequency diagram result of wavelet transform for five types of PSK signals in another embodiment;
FIG. 5 is a diagram illustrating different layer outputs after different video pictures are input into a residual connection network in one embodiment;
FIG. 6 is a diagram illustrating the performance of a residual error connection network in identifying PSK-like signals with different carrier-to-noise ratios in one embodiment;
FIG. 7 is a diagram illustrating the performance of a random access network in identifying PSK-like signals with different carrier-to-noise ratios in one embodiment;
FIG. 8 is a block diagram of a modulation recognition apparatus based on time-frequency diagrams and deep learning according to an embodiment;
FIG. 9 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In one embodiment, as shown in fig. 1, a modulation identification method based on time-frequency diagram and deep learning is provided, which includes the following steps:
102, constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal; and adding channel loss to the frequency band signal to obtain a simulated communication signal.
PSK modulation modes comprise BPSK, QPSK,8PSK, OQPSK and pi/4-QPSK, and the frequency band signals are obtained by performing baseband forming on the frequency band signals and general complex baseband symbol mapping of the PSK modulation modes; channel loss includes frequency shift, phase shift, and white noise.
104, filtering and frequency converting the simulation communication signal by using a filter to obtain a frequency conversion signal; and sampling the frequency conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase modulation signal.
For simulation communication signals, after the central frequency and the bandwidth are determined, an FIR filter can be directly adopted to filter the signals to remove out-of-band noise, then down-conversion is carried out to obtain variable frequency signals, and PSK signals need to be identified, so the change condition of the phase reflects the modulation order, the analytic wavelet can detect the change of the phase sharply, but the scale of wavelet transformation is nonlinear, the obtained time-frequency image is deformed, the bandwidth of the signals needs to be reduced, the sampling rate of the signals needs to be increased, and input signals with different sampling rates are up-sampled to the same sampling rate, so that the input is unified and beneficial to training. And the sampling rate is increased, the signal bandwidth is reduced, the observation scale is relatively enlarged, the signal representation is fuzzified, the energy change trend is reserved, and the removal of excessive complex details is favorable for the classification of the PSK signals. The up-sampling method is that a sampling function is used to restore a continuous signal, and then a corresponding sampling point is obtained under a new sampling frequency:
where T is time, x (T) is a signal time domain expression, T S Is the sampling interval, ω m Is the maximum analog frequency of the signal, f s ' is newX (n) is the corresponding sampling point obtained at the new sampling frequency.
Since the continuous wavelet transform of a complex signal is divided into two parts (clockwise part and counterclockwise part), in order to reduce the amount of input, and all the inputs are unified into one frequency bin. Therefore, the baseband complex signal after interpolation is subjected to digital up-conversion for normalization, the normalization frequency is 0.25, and the real part is taken for wavelet transformation.
And 106, performing wavelet transformation on the modulation signals to obtain a transformation result, converting the modulus value of the transformation result into a color map, and reconstructing the color map to obtain a wavelet transformation time-frequency map of the PSK modulation signals.
The time-frequency graph is generated by utilizing wavelet transformation, the wavelet transformation can well detect the phase change condition, the phase information of the time-frequency graph cannot be lost, and the time-frequency graph can visually reflect the characteristics and the change details of signals even if the number of received sample points is small based on the characteristics of the time-frequency graph of the signals. Because the signal frequency offset is represented by moving up and down on the spectrogram, the signal timing error is represented by moving left and right on the spectrogram, the noise has little influence on the image contour, the time-frequency diagram has strong robustness on the representation of the signal characteristics, and compared with modulation identification based on high-order cumulant, the method is insensitive to the carrier estimation frequency offset and has higher accuracy of modulation identification.
The continuous wavelet transform is defined as follows:
where s (t) is a signal, ψ (t) is a selected wavelet basis function (a morse wavelet is selected, and a specific expression is defined below), constants a and b are called a scale parameter and a translation parameter, then an absolute value is taken for the obtained wavelet transform value and normalized to (0,1), then the value is used to determine the color at the corresponding point, the maximum value corresponds to gray, the minimum value corresponds to black, and the specific color scheme is shown in fig. 3:
The size of an input layer of the residual error network is modified into the size of a wavelet transform time-frequency graph, and the output sizes of a full connection layer and an output layer of the residual error network are kept consistent with the number of types of PSK modulation modes to construct a deep neural network, so that the deep neural network can better classify and identify the characteristics and the modulation modes of the time-frequency graph.
And inputting the wavelet transformation time-frequency diagram into a residual error connection network and a random channel network to obtain the recall rate of the PSK modulation modes in the two networks, and fusing the recall rate of each modulation mode to obtain a modulation identification result.
According to the modulation identification method based on the time-frequency diagram and the deep learning, satellite communication is constructed according to a PSK modulation mode, then the obtained frequency band signals are preprocessed to obtain simulated communication signals, the frequency conversion signals are sampled by using an up-sampling method, and digital frequency conversion and normalization processing are carried out on sampling results to obtain phase modulation signals; the method comprises the steps of performing wavelet transformation on phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into color maps, reconstructing the color maps to obtain wavelet transformation time-frequency maps of the PSK modulation signals, and enabling the time-frequency maps to visually reflect signal characteristics and change details based on the characteristics of the time-frequency maps of the signals even if the number of received sample points is small. Because the signal frequency offset is represented by up-and-down movement on the spectrogram, the signal timing error is represented by left-and-right movement on the spectrogram, the noise has little influence on the image contour, the time-frequency graph has strong robustness on the representation of the signal characteristics, and compared with modulation identification based on high-order cumulant, the method is insensitive to the carrier estimation frequency offset. Meanwhile, a deep neural network is constructed according to the size of the wavelet transform time-frequency graph, the type of the PSK modulation mode and a residual error network, the deep learning neural network is used as a classifier, the defect of dependence on manual selection of a judgment threshold is overcome, compared with the neural network based on original data input, the time-frequency graph is generated by using few data sampling points, the modulation mode can be still correctly identified under the condition that the received signal is short, and the modulation identification efficiency and accuracy are improved.
In one embodiment, constructing satellite communication according to a PSK modulation scheme to obtain a frequency band signal includes:
satellite communication is constructed according to a PSK modulation mode to obtain a frequency band signal of
s(t)＝real(Y(t)e jθ exp(j2πf c t))
Wherein Y (t) is complex baseband signal, e is natural logarithm, theta is initial phase of baseband signal, f c Is the carrier frequency, t is time, and j is the imaginary unit.
The general complex baseband symbol mapping and the band signal format of the PSK-like modulation scheme can be expressed as follows:
wherein s is n (t) is PSK complex baseband signal, M is PSK order, pi is circumferential rate, j is imaginary unit
OQPSK is half cycle offset in the quadrature and in-phase components compared to QPSK, and pi/4-QPSK rotates the constellation by pi/4 every one symbol cycle compared to QPSK.
The forming filtering adopts a square root raised cosine pulse filter or a Gaussian filter, and the expressions of the square root raised cosine pulse filter and the Gaussian filter are respectively
Wherein r is p (T) is the raised cosine function, T s Is the sampling interval, alpha is the roll-off coefficient, r s (t) is the root raised cosine function, t pseudo-time. g (t) is a gaussian function and BT is a time-bandwidth product. log is the natural logarithm.
The expression of the frequency band signal obtained after baseband molding is as follows:
s(t)＝real(Y(t)e jθ exp(j2πf c t))
in one embodiment, adding channel loss to the frequency band signal to obtain an artificial communication signal comprises:
and adding frequency shift, phase shift and white noise to the frequency band signal to obtain the simulated communication signal.
The frequency offset caused by different clocks is expressed as follows:
Δf＝offset/f clock ·f c
wherein, the offset is the number of difference points in one second of the clock of the sending end and the receiving end, f clock Is the fundamental frequency of the clock, f c For transmitting the signal carrier frequency, the timing drift can be obtained by obtaining the real sampling frequency and the sampling time by a similar method and obtaining the real sampling frequency and the sampling time by signal interpolation.
The expression for adding the frequency shift Δ f and the phase shift Δ φ is as follows:
Y'(t)＝Y(t)·e jΔφ+Δft
and finally, adding larger white noise to obtain the simulated communication signal.
Indices of the simulated communication signal data set are shown in table 1:
TABLE 1 simulation data set indices
In one embodiment, the filtering and frequency converting the simulated communication signal by using a filter to obtain a frequency-converted signal includes:
and filtering the simulation communication signal by adopting an FIR filter to remove out-of-band noise, and then performing digital down-conversion on the signal by using a low-frequency filter to obtain a filtered signal.
In one embodiment, the wavelet transform is performed on the phase modulation signal to obtain a transform result, the magnitude of a modulus of the transform result is converted into a color map, and the color map is reconstructed to obtain a wavelet transform time-frequency map of the PSK modulation signal, including:
wavelet analysis is carried out by utilizing Morse wavelets, wavelet transformation is carried out on the phase modulation signals after a wavelet transformation base is determined, transformation results are obtained, the magnitude of a modulus value of the transformation results is converted into a color graph, and the color graph is reconstructed to obtain a wavelet transformation time-frequency graph of the PSK modulation signals.
The phase modulated signal has phase discontinuities and in this application wavelet analysis is performed using a Morse-resolved wavelet. The Fourier transform of Morse wavelet only has real axis part, and its specific Fourier transform expression is:
wherein U (omega) represents unit step response, omega is angular frequency, e is natural logarithm, alpha P,γ Denotes the normalization parameter, P 2 Representing the time bandwidth product, P is proportional to the duration of the wavelet, gamma reflects the symmetry of the morse wavelet, and the 3-hour morse wavelet is taken to have the smallest hesenberg area. The wavelet transform base is adopted to select the frequency range of [0.20.35 ]]The number of the spacing points between every two frequency multiplication is 48, the frequency response of the selected transformation base is shown in figure 2, continuous wavelet transformation is carried out on the communication signals, the magnitude of the modulus of the result is converted into a color map, a 128 color map is selected, and the color scheme of the wavelet transformation time-frequency map is shown in figure 3. Mapping the module value of the result to color and reconstructing the picture size to 50 × 2000 × 3 to obtain five-class PSK modulation signal wavelet transform time-frequency diagram as shown in FIG. 4 from the topBPSK, QPSK,8PSK, OQPSK, pi/4-QPSK.
In one embodiment, the method for constructing the deep neural network according to the size of the wavelet transform time-frequency diagram, the type of the PSK modulation mode and the residual error network further includes:
and modifying the size of an input layer of the residual error network into the size of a wavelet transform time-frequency graph, and keeping the output sizes of a full connection layer and an output layer of the residual error network consistent with the number of types of PSK modulation modes to construct a deep neural network.
The network construction adopts a residual error network, the size of an input layer of the modified residual error network is 50 multiplied by 2000 multiplied by 3, the output of a full connection layer and an output layer is 5, and in order to reduce the calculated amount and improve the accuracy, a random channel network unit is used for replacing a residual error block in the residual error network, wherein the random channel network unit comprises point-by-point group convolution, channel rearrangement and depth separable convolution. After the deep neural network is built, a time frequency graph is input for training, an sgdm optimizer is adopted, and specific training parameters of the two networks are respectively shown in table 2.
TABLE 2 network training parameters
And constructing a satellite communication signal data set by using a PSK (phase shift keying) modulation mode, carrying out data processing on the data set to obtain a time-frequency graph, verifying the deep neural network by using the time-frequency graph, and inputting the time-frequency graph into a residual connecting network and a random channel network, namely ResNet and Shufflenet. Taking a residual error connection network as an example, observing the outputs of the residual error connection network on a posing layer, a last convolutional layer and a softmax layer, visualizing by adopting a t-distributed random field embedding (t-SNE) method to obtain the correlation between different video graphs input through the network, as shown in fig. 5, wherein ResNet18 is the residual error connection network, it can be seen that the differences between five types of shallow input are not large, the distance between the five types of shallow input is increased after passing through each intermediate hidden layer, five types of PSK communication signals are well separated in the softmax layer, but a part of samples are not correctly separated into proper areas to cause identification errors, five PSK type signal identification results of two networks under different signal-to-noise ratios are respectively tested, as shown in fig. 6 and 7, fig. 6 is a result of ResNet identifying different PSK type signal performances under different signal-to-noise ratios, fig. 7 is a result of random channel network identifying different PSK type signals, wherein, the Shufflenet is a random channel network, and it can be seen that when the carrier-to-noise ratio is higher than 10, the identification effect of the two networks on five types of signals can reach more than 90%.
In one embodiment, the fusion processing of the plurality of recalls to obtain the modulation identification result of the frequency band signal includes:
the fusion processing is carried out on a plurality of recall rates, and the modulation identification result of the frequency band signal is obtained
Wherein, P is recall, TP is the number of positive examples predicted from positive examples, FP is the number of false examples predicted from positive examples, and θ is the modulation type selected by the fusion method.
The specific implementation process is that according to the principle of frequency approximation probability, the recall rate of network identification of a certain signal category is regarded as a network identification result, and the posterior probability of the signal category is determined by the input signal under the condition that the network identification result is the signal of the signal category. Inputting a signal, different recognition results exist, the probability that the positive class is predicted to be the positive class is assumed to be TP, and the probability that the negative class is predicted to be the positive class is assumed to be FP, so the later fusion principle of the recall rate P calculation method and the two networks is as follows:
and the two networks simultaneously judge the input, then inquire the recall ratio below the network confusion matrix according to the result, select the result with higher recall ratio to become the result of later-stage fusion, and fuse the identification results of the two networks to obtain the final modulation identification result with higher accuracy.
It should be understood that, although the steps in the flowchart of fig. 1 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 1 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 8, there is provided a modulation identification apparatus based on time-frequency diagram and deep learning, including: a build signal module 802, a data pre-processing module 804, a wavelet transform module 806, a network build module 808, and a modulation identification module 810, wherein:
a signal constructing module 802, configured to construct satellite communication according to a PSK modulation scheme, to obtain a frequency band signal; adding channel loss to the frequency band signal to obtain a simulated communication signal;
the data preprocessing module 804 is configured to filter and frequency-convert the simulated communication signal by using a filter to obtain a frequency-converted signal; sampling the frequency-conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase-modulation signal;
a wavelet transform module 806, configured to perform wavelet transform on the phase-modulated signals to obtain a transform result, convert a magnitude of a modulus of the transform result into a color map, and reconstruct the color map to obtain a wavelet transform time-frequency map of the PSK-type modulated signals;
the network construction module 808 is used for constructing a deep neural network according to the size of the wavelet transform time-frequency diagram, the type of the PSK modulation mode and the residual error network; the deep neural network comprises a residual error connection network and a random channel network;
the modulation identification module 810 is configured to input the wavelet transform time-frequency diagram to a residual connection network and a random channel network, so as to obtain multiple recall rates of PSK modulation modes; and performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
In one embodiment, the module 802 for constructing a signal is further configured to construct satellite communication according to a PSK-like modulation scheme, so as to obtain a frequency band signal, including:
satellite communication is constructed according to a PSK modulation mode to obtain a frequency band signal of
s(t)＝real(Y(t)e jθ exp(j2πf c t))
Wherein Y (t) is complex baseband signal, e is natural logarithm, theta is initial phase of baseband signal, f c Is the carrier frequency, t is time, and j is the imaginary unit.
In one embodiment, the signal constructing module 802 is further configured to add a channel loss to the frequency band signal to obtain an artificial communication signal, including:
and adding frequency shift, phase shift and white noise to the frequency band signal to obtain the simulated communication signal.
In one embodiment, the data preprocessing module 804 is further configured to filter and frequency-convert the simulated communication signal by using a filter to obtain a frequency-converted signal, including:
and filtering the simulation communication signal by adopting an FIR filter to remove out-of-band noise, and then performing digital down-conversion on the signal by using a low-frequency filter to obtain a filtered signal.
In one embodiment, the wavelet transform module 806 is further configured to perform wavelet transform on the phase-modulated signal to obtain a transform result, convert a magnitude of a modulus of the transform result into a color map, and reconstruct the color map to obtain a wavelet transform time-frequency map of the PSK-type modulated signal, including:
wavelet analysis is carried out by utilizing Morse wavelets, wavelet transformation is carried out on the phase modulation signals after a wavelet transformation base is determined, transformation results are obtained, the magnitude of a modulus value of the transformation results is converted into a color graph, and the color graph is reconstructed to obtain a wavelet transformation time-frequency graph of the PSK modulation signals.
In one embodiment, the network constructing module 808 is further configured to construct a deep neural network according to the size of the wavelet transform time-frequency diagram, the type of PSK modulation mode, and the residual error network, and further includes:
and modifying the size of an input layer of the residual error network into the size of a wavelet transform time-frequency graph, and keeping the output sizes of a full connection layer and an output layer of the residual error network consistent with the number of types of PSK modulation modes to construct a deep neural network.
In one embodiment, the modulation identification module 810 is further configured to perform fusion processing on the plurality of recalls to obtain a modulation identification result of the frequency band signal, including:
the fusion processing is carried out on a plurality of recall rates, and the modulation identification result of the frequency band signal is obtained
Wherein, P is recall, TP is the number of positive examples predicted from positive examples, FP is the number of false examples predicted from positive examples, and θ is the modulation type selected by the fusion method.
For specific limitations of the modulation recognition device based on the time-frequency diagram and the deep learning, refer to the above limitations of the modulation recognition method based on the time-frequency diagram and the deep learning, and are not described herein again. The modules in the modulation identification device based on the time-frequency diagram and the deep learning can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 9. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program to run on the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a modulation recognition method based on a time-frequency graph and deep learning. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 9 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In an embodiment, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the method in the above embodiments when the processor executes the computer program.
In an embodiment, a computer storage medium is provided, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method in the above-mentioned embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A modulation identification method based on time-frequency graphs and deep learning is characterized by comprising the following steps:
constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal;
adding channel loss to the frequency band signal to obtain a simulated communication signal;
filtering and frequency converting the simulation communication signal by using a filter to obtain a frequency conversion signal;
sampling the frequency-conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase-modulation signal;
performing wavelet transformation on the phase modulation signals to obtain transformation results, converting the magnitude of modulus values of the transformation results into a color map, and reconstructing the color map to obtain a wavelet transformation time-frequency map of the PSK modulation signals;
constructing a deep neural network according to the size of a wavelet transform time-frequency diagram, the type of a PSK modulation mode and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
inputting the wavelet transform time-frequency diagram into the residual error connection network and a random channel network to obtain a plurality of recall rates of the PSK modulation mode;
and performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
2. The method of claim 1, wherein constructing satellite communication according to PSK-like modulation to obtain a frequency band signal comprises:
constructing satellite communication according to PSK modulation mode to obtain frequency band signal of
s(t)＝real(Y(t)e jθ exp(j2πf c t))
Wherein Y (t) is complex baseband signal, e is natural logarithm, theta is initial phase of baseband signal, f c Is the carrier frequency, t is time, and j is the imaginary unit.
3. The method of claim 1, wherein adding channel loss to the frequency band signal to obtain an emulated communication signal comprises:
and adding frequency shift, phase shift and white noise to the frequency band signal to obtain a simulated communication signal.
4. The method of any of claims 1 to 3, wherein filtering and frequency converting the simulated communication signal with a filter to obtain a frequency converted signal comprises:
and filtering the simulation communication signal by adopting an FIR filter to remove out-of-band noise, and then performing digital down-conversion on the signal by using a low-frequency filter to obtain a filtered signal.
5. The method of claim 1, wherein performing wavelet transform on the phase-modulated signals to obtain transform results, transforming the magnitude of the modulus of the transform results into a color map, and reconstructing the color map to obtain a wavelet transform time-frequency map of the PSK-like modulated signals, comprises:
wavelet analysis is carried out by utilizing Morse wavelets, wavelet transformation is carried out on the phase modulation signals after a wavelet transformation base is determined, transformation results are obtained, the magnitude of modulus values of the transformation results is converted into a color graph, and a wavelet transformation time-frequency graph of the PSK modulation signals is obtained after the color graph is reconstructed.
6. The method of claim 5, wherein the deep neural network is constructed according to the size of the wavelet transform time-frequency diagram, the type of PSK modulation mode and the residual error network, and further comprising:
and modifying the size of an input layer of the residual error network into the size of the wavelet transformation time-frequency diagram, and keeping the output sizes of a full connection layer and an output layer of the residual error network consistent with the type number of the PSK type modulation modes to construct a deep neural network.
7. The method according to claim 5, wherein the fusion processing of the plurality of recalls to obtain the modulation recognition result of the frequency band signal comprises:
the plurality of recall rates are fused, and the modulation identification result of the frequency band signal is obtained
Wherein, P is recall, TP is the number of positive examples predicted from positive examples, FP is the number of false examples predicted from positive examples, and θ is the modulation type selected by the fusion method.
8. A modulation identification device based on time-frequency graph and deep learning is characterized in that the device comprises:
the signal construction module is used for constructing satellite communication according to a PSK modulation mode to obtain a frequency band signal; adding channel loss to the frequency band signal to obtain a simulated communication signal;
the data preprocessing module is used for filtering and frequency-converting the simulation communication signal by adopting a filter to obtain a frequency-converted signal; sampling the frequency-conversion signal by using an up-sampling method, and carrying out digital frequency conversion and normalization processing on a sampling result to obtain a phase-modulation signal;
the wavelet transform module is used for performing wavelet transform on the phase modulation signals to obtain transform results, converting the magnitude of modulus values of the transform results into a color map, and reconstructing the color map to obtain a wavelet transform time-frequency map of the PSK modulation signals;
the network construction module is used for constructing a deep neural network according to the size of the wavelet transform time-frequency diagram, the type of PSK modulation modes and a residual error network; the deep neural network comprises a residual error connection network and a random channel network;
the modulation identification module is used for inputting the wavelet transformation time-frequency diagram into the residual error connection network and the random channel network to obtain a plurality of recall rates of the PSK modulation mode; and performing fusion processing on the plurality of recall rates to obtain a modulation identification result of the frequency band signal.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210422929.8A CN114826850B (en) | 2022-04-21 | 2022-04-21 | Modulation identification method, device and equipment based on time-frequency diagram and deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210422929.8A CN114826850B (en) | 2022-04-21 | 2022-04-21 | Modulation identification method, device and equipment based on time-frequency diagram and deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114826850A true CN114826850A (en) | 2022-07-29 |
| CN114826850B CN114826850B (en) | 2022-11-25 |
Family
ID=82506387
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210422929.8A Active CN114826850B (en) | 2022-04-21 | 2022-04-21 | Modulation identification method, device and equipment based on time-frequency diagram and deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114826850B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117743946A (en) * | 2024-02-19 | 2024-03-22 | 山东大学 | Signal type identification method and system based on fusion characteristics and group convolution ViT network |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060222102A1 (en) * | 2005-03-31 | 2006-10-05 | Toshihide Kadota | Wireless communication system |
| CN103905360A (en) * | 2014-02-19 | 2014-07-02 | 江苏科技大学 | Non-cooperative BPSK signal decoding method with polarity judgment operation |
| US20150180689A1 (en) * | 2013-12-20 | 2015-06-25 | King Fahd University Of Petroleum And Minerals | Cognitive radio spectrum sensing with improved edge detection of frequency bands |
| CN111585662A (en) * | 2020-06-01 | 2020-08-25 | 中国科学院声学研究所东海研究站 | Classification identification and parameter estimation method and system for phase modulation signal |
| CN112307927A (en) * | 2020-10-26 | 2021-02-02 | 重庆邮电大学 | BP network-based identification research for MPSK signals in non-cooperative communication |
| CN112347871A (en) * | 2020-10-23 | 2021-02-09 | 中国电子科技集团公司第七研究所 | Interference signal modulation identification method for communication carrier monitoring system |
| US11075786B1 (en) * | 2004-08-02 | 2021-07-27 | Genghiscomm Holdings, LLC | Multicarrier sub-layer for direct sequence channel and multiple-access coding |
| CN113315727A (en) * | 2021-03-29 | 2021-08-27 | 中山大学 | Digital communication signal modulation identification method based on preprocessing noise reduction |
| CN113537402A (en) * | 2021-08-13 | 2021-10-22 | 重庆大学 | Vibration signal-based converter transformer multi-scale fusion feature extraction method |
-
2022
- 2022-04-21 CN CN202210422929.8A patent/CN114826850B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11075786B1 (en) * | 2004-08-02 | 2021-07-27 | Genghiscomm Holdings, LLC | Multicarrier sub-layer for direct sequence channel and multiple-access coding |
| US20060222102A1 (en) * | 2005-03-31 | 2006-10-05 | Toshihide Kadota | Wireless communication system |
| US20150180689A1 (en) * | 2013-12-20 | 2015-06-25 | King Fahd University Of Petroleum And Minerals | Cognitive radio spectrum sensing with improved edge detection of frequency bands |
| CN103905360A (en) * | 2014-02-19 | 2014-07-02 | 江苏科技大学 | Non-cooperative BPSK signal decoding method with polarity judgment operation |
| CN111585662A (en) * | 2020-06-01 | 2020-08-25 | 中国科学院声学研究所东海研究站 | Classification identification and parameter estimation method and system for phase modulation signal |
| CN112347871A (en) * | 2020-10-23 | 2021-02-09 | 中国电子科技集团公司第七研究所 | Interference signal modulation identification method for communication carrier monitoring system |
| CN112307927A (en) * | 2020-10-26 | 2021-02-02 | 重庆邮电大学 | BP network-based identification research for MPSK signals in non-cooperative communication |
| CN113315727A (en) * | 2021-03-29 | 2021-08-27 | 中山大学 | Digital communication signal modulation identification method based on preprocessing noise reduction |
| CN113537402A (en) * | 2021-08-13 | 2021-10-22 | 重庆大学 | Vibration signal-based converter transformer multi-scale fusion feature extraction method |
Non-Patent Citations (3)
| Title |
|---|
| 董志杰等: "雷达信号脉内调制识别新方法", 《航天电子对抗》 * |
| 董航等: "基于小波变换的信号时频分析与重构", 《中国新通信》 * |
| 黎仁刚等: "基于卷积神经网络的雷达和通信信号调制识别", 《航天电子对抗》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117743946A (en) * | 2024-02-19 | 2024-03-22 | 山东大学 | Signal type identification method and system based on fusion characteristics and group convolution ViT network |
| CN117743946B (en) * | 2024-02-19 | 2024-04-30 | 山东大学 | Signal type identification method and system based on fusion characteristic and group convolution ViT network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114826850B (en) | 2022-11-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108234370B (en) | Communication signal modulation mode identification method based on convolutional neural network | |
| CN112702294B (en) | Modulation recognition method for multi-level feature extraction based on deep learning | |
| CN110569752A (en) | convolutional neural network-based radar signal category determination method | |
| CN114826850B (en) | Modulation identification method, device and equipment based on time-frequency diagram and deep learning | |
| CN113726711B (en) | OFDM receiving method and device, and channel estimation model training method and device | |
| Ali et al. | Algorithm for automatic recognition of PSK and QAM with unique classifier based on features and threshold levels | |
| CN114548201B (en) | Automatic modulation identification method and device for wireless signal, storage medium and equipment | |
| CN114978827B (en) | Modulation identification method based on constellation diagram phase abnormal ratio correction frequency offset | |
| CN113869227B (en) | Signal modulation mode identification method, device, equipment and readable storage medium | |
| CN114943245A (en) | Automatic modulation recognition method and device based on data enhancement and feature embedding | |
| CN112737992A (en) | Underwater sound signal modulation mode self-adaptive in-class identification method | |
| CN116628566A (en) | Communication signal modulation classification method based on aggregated residual transformation network | |
| Wang et al. | A spatiotemporal multi-stream learning framework based on attention mechanism for automatic modulation recognition | |
| CN113076925B (en) | M-QAM signal modulation mode identification method based on CNN and ELM | |
| CN114615118A (en) | Modulation identification method based on multi-terminal convolution neural network | |
| CN116319210A (en) | Signal lightweight automatic modulation recognition method and system based on deep learning | |
| CN115086123B (en) | Modulation identification method and system based on fusion of time-frequency diagram and constellation diagram | |
| CN113822162B (en) | Convolutional neural network modulation identification method based on pseudo constellation diagram | |
| CN116132235A (en) | Continuous phase modulation signal demodulation method based on deep learning | |
| CN109768944A (en) | A kind of signal modulation identification of code type method based on convolutional neural networks | |
| CN111275603B (en) | Security image steganography method based on style conversion and electronic device | |
| US11507803B2 (en) | System for generating synthetic digital data for data multiplication | |
| Wong et al. | An analysis of RF transfer learning behavior using synthetic data | |
| Ren et al. | Deep Learning Based Identification Method for Signal-Level Wireless Protocol | |
| CN113242201A (en) | Wireless signal enhanced demodulation method and system based on generation classification network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |