CN114114363A

CN114114363A - Opportunistic signal sensing method and system based on time frequency and convolutional neural network and opportunistic signal positioning method

Info

Publication number: CN114114363A
Application number: CN202111311252.2A
Authority: CN
Inventors: 邓中亮; 綦航; 胡恩文; 刘延旭
Original assignee: Beijing University of Posts and Telecommunications
Current assignee: Beijing University of Posts and Telecommunications
Priority date: 2021-11-08
Filing date: 2021-11-08
Publication date: 2022-03-01

Abstract

The invention provides an opportunistic signal sensing method, an opportunistic signal sensing system and an opportunistic signal positioning method based on time frequency and convolutional neural networks, wherein the sensing method comprises the following steps: calling a plurality of sample signals of a target frequency band to represent as a first time-frequency image, setting a label for the first time-frequency image according to the signal type, and establishing as a training data set; dividing a target frequency band into a plurality of sub-frequency bands, dividing a training data set into a plurality of training data groups based on the sub-frequency bands, inputting the training data groups into a preset convolutional neural network model, and training the convolutional neural network model to obtain a perception model; receiving the opportunity signal, and inputting the second time-frequency image into the perception model; and calling a corresponding perception model according to the sub-band where the opportunistic signal is located to obtain the signal type of the opportunistic signal, and activating a corresponding receiver based on the signal type. The continuous sensing of the signals can be ensured without keeping the work of all the receivers, and the corresponding receivers are activated according to sensing results, so that the utilization rate of hardware resources is improved, and the power consumption is reduced.

Description

Opportunistic signal sensing method and system based on time frequency and convolutional neural network and opportunistic signal positioning method

Technical Field

The invention relates to the technical field of opportunistic signal positioning, in particular to an opportunistic signal sensing method, an opportunistic signal sensing system and an opportunistic signal positioning method based on time frequency and convolutional neural networks.

Background

The traditional navigation system mostly utilizes a Global Navigation Satellite System (GNSS) to perform positioning, and the full-source navigation, opportunistic signal navigation and multi-source fusion positioning adopt all radio frequency signals which can be used for positioning in an airspace to perform navigation positioning, including various non-navigation-dedicated signals, such as digital audio broadcasting, digital television broadcasting signals, amplitude modulation and frequency modulation broadcasting signals, cellular base station signals, bluetooth, ZigBee, Wi-Fi and other communication network signals.

Signal of opportunity navigation locates by receiving all potential wireless signals in the environment. The signals of opportunity include various signals which are not specially designed for navigation except for GNSS systems, such as digital audio broadcasting, digital television broadcasting signals, amplitude modulation and frequency modulation broadcasting signals, cellular base station signals, Bluetooth, ZigBee, Wi-Fi and other wireless signals. These signals are widely present in the surrounding environment and are typically used to communicate rather than being broadcast specifically for navigation. By identifying multiple types of signals, reliable positioning and navigation service is provided.

Opportunistic signal navigation requires identification of various wireless signals, because it is not known which signals are available in the environment, all types of receivers need to continuously operate as shown in fig. 3, and even if some types of signals do not exist in the surrounding environment, all types of receivers need to keep operating to ensure continuous perception of various signals, and the resource utilization rate is low.

Disclosure of Invention

In view of this, embodiments of the present invention provide an opportunistic signal sensing method, system and opportunistic signal positioning method based on time-frequency and convolutional neural networks, so as to eliminate or improve one or more defects in the prior art.

One aspect of the present invention provides an opportunistic signal sensing method based on time-frequency and convolutional neural networks, wherein receivers of multiple types of signals are in a standby state at an initial time, and the method comprises the following steps:

calling a plurality of sample signals which are pre-stored in a target frequency band, representing the sample signals as first time-frequency images, setting labels for the first time-frequency images according to the signal types of the sample signals, and establishing the first time-frequency images with the labels as training data sets;

dividing the target frequency band into a plurality of sub-frequency bands, dividing a training data set into a plurality of training data groups based on the sub-frequency bands where the sample signals are located, respectively inputting the plurality of training data groups into a preset convolutional neural network model, and training the convolutional neural network model to obtain a plurality of perception models corresponding to the number of the sub-frequency bands;

receiving an opportunity signal, representing the opportunity signal as a second time-frequency image, and calling a corresponding perception model according to a sub-frequency band where the opportunity signal is located;

and inputting the second time-frequency image into the corresponding perception model to obtain the signal type of the opportunity signal, and activating a receiver in a standby state in a corresponding category based on the signal type.

By adopting the scheme, the receivers of various types of signals are in a standby state at first, the signal types of the opportunity signals are judged, the corresponding receivers are activated according to the judged signal types, continuous sensing of various types of signals can be guaranteed without keeping the operation of all types of receivers, the utilization rate of hardware resources is improved, and the power consumption is reduced.

The sample signal and the opportunity signal are both wireless signals, and the first time-frequency image and the second time-frequency image are both time-frequency representations of the wireless signals.

And each sub-frequency band is correspondingly provided with the perception model.

In some embodiments of the invention, the step of receiving the signal of opportunity comprises:

determining the length of a receiving frequency band of a receiving window according to the length of the sub-frequency band, wherein the length of the frequency band received by the receiving window is equal to the length of the sub-frequency band;

the length of the receive window receives the range of frequency bands in which the opportunistic signal is located.

And switching the sub-frequency band where the receiving window is located according to the monitoring duration of each sub-frequency band, and when the receiving window is located in any sub-frequency band, receiving the opportunity signal in the sub-frequency band by the receiving window.

In some embodiments of the present invention, the step of invoking the corresponding perceptual model according to the sub-band where the opportunistic signal is located further comprises: and when the receiving window is positioned in any sub-band, calling a perception model corresponding to the sub-band, wherein the perception model is used for perceiving the signal type of the opportunistic signal.

In some embodiments of the invention, the step of representing the opportunity signal as a second time-frequency image further comprises graying the second time-frequency image.

In some embodiments of the invention, in the step of training the convolutional neural network model, the convolutional neural network model includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a third convolutional layer, a third pooling layer, a fourth convolutional layer, a fourth pooling layer, a first fully-connected layer, a second fully-connected layer, and an output layer, which are connected in sequence.

The structures of the perception models corresponding to different sub-bands are all the structures, and the first full connection layer and the second full connection layer of different perception models have different weight parameters.

In some embodiments of the present invention, the output ends of the first convolution layer, the second convolution layer, the third convolution layer, the fourth convolution layer, the first full-link layer and the second full-link layer are connected with a correction linear unit.

In some embodiments of the present invention, the step of inputting the plurality of training data sets into the preset convolutional neural network model further comprises: and carrying out graying processing and normalization processing on the first time-frequency image, wherein the graying processing is used for processing the original first time-frequency image into a grayscale image, and the normalization processing is used for processing the sizes of all the first time-frequency images into a uniform size.

In some embodiments of the invention, the step of training the convolutional neural network model comprises positive learning training and negative learning training,

labels in a training data set used by the learning training are class labels, and the class labels mark classes of wireless signals corresponding to the first time-frequency images;

and the label in the training data set used by the negative learning training is a negative label, and the negative label is used for marking that the wireless signal corresponding to the first time-frequency image belongs to a noise signal or an interference signal.

The noise signal or the interference signal does not belong to any signal type to be classified, and the signal type to be classified comprises wifi, Bluetooth and the like.

The category label can be represented as (0,0,1), (1,1,0), (1,0,1)), and the negative label can be represented as (0,0, 0).

The opportunistic signal sensing method based on the time frequency and the convolutional neural network firstly enables receivers of various types of signals to be in a standby state, judges the signal types of the opportunistic signals, activates the corresponding receivers according to the judged signal types of the opportunistic signals, can guarantee continuous sensing of various types of signals without keeping the operation of all types of receivers, improves the utilization rate of hardware resources and reduces power consumption.

Another aspect of the present invention provides an opportunity signal localization method, including the steps of:

activating the receiver corresponding to the signal of opportunity according to the method, and inputting the signal of opportunity into the receiver;

and extracting signal parameters of the signals of opportunity at the receiver, performing positioning calculation on the signals of opportunity based on the signal parameters, and positioning the receiver based on the result of the positioning calculation of a transmission source.

And positioning the receiver, namely positioning the receiver per se, wherein the receivers of various types are positioned at the same position, the receiver and the USRP are positioned at the same position.

Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.

It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present invention are not limited to the specific details set forth above, and that these and other objects that can be achieved with the present invention will be more clearly understood from the detailed description that follows.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention.

FIG. 1 is a schematic diagram of an embodiment of a method for opportunistic signal sensing based on time-frequency and convolutional neural networks according to the present invention;

FIG. 2 is a schematic diagram of the signal receiving step;

FIG. 3 is a schematic diagram of a prior art opportunistic signal positioning method;

FIG. 4 is a diagram illustrating a method for opportunistic signal positioning according to the present invention;

FIG. 5 is a schematic diagram of an opportunistic signal sensing apparatus based on time-frequency and convolutional neural networks according to the present invention;

FIG. 6 is a diagram illustrating target frequency division;

FIG. 7 is a schematic diagram of the structure of the inventive convolutional neural network;

FIG. 8 is a schematic diagram of an implementation of the positive and negative learning exercises of the present invention;

FIG. 9 is a graph of loss function and training accuracy for positive and negative learning training;

FIG. 10 is a schematic layout of an experiment;

FIG. 11 is a graph showing the results of the experiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the following embodiments and accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.

It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and/or processing steps closely related to the scheme according to the present invention are shown in the drawings, and other details not so relevant to the present invention are omitted.

It should be emphasized that the term "comprises/comprising" when used herein, is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.

It is also noted herein that the term "coupled," if not specifically stated, may refer herein to not only a direct connection, but also an indirect connection in which an intermediate is present.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals denote the same or similar parts, or the same or similar steps.

As shown in fig. 1, one aspect of the present invention provides a method for opportunistic signal sensing based on time-frequency and convolutional neural networks, wherein receivers of multiple types of signals are in a standby state at an initial time, and the method comprises the following steps:

step S100, calling a plurality of sample signals which are pre-stored in a target frequency band, representing the sample signals as first time-frequency images, setting labels for the first time-frequency images according to the signal types of the sample signals, and establishing the first time-frequency images with the labels as training data sets;

in some embodiments of the present invention, the sample signal is a signal received from various signal sources in advance, and the signal source may be a wifi signal source, a Zigbee signal source, a bluetooth signal source, or the like.

In some embodiments of the present invention, the first time-frequency image is a time-frequency representation of a signal, and representing the sample signal as the first time-frequency image may be implemented by a time-frequency joint characterization method.

In some embodiments of the present invention, the time-frequency joint characterization method may be a gobor (Gabor) transform method or a short-time fourier transform (STFT) method, etc.

With the above scheme, each pixel of the first time-frequency image represents the energy of the signal at a certain frequency and a certain time point; the whole image shows the wireless signal energy distribution over the whole frequency domain in a short time.

Step S200, dividing the target frequency band into a plurality of sub-frequency bands, dividing a training data set into a plurality of training data sets based on the sub-frequency bands where the sample signals are located, respectively inputting the plurality of training data sets into a preset convolutional neural network model, and training the convolutional neural network model to obtain a plurality of perception models corresponding to the number of the sub-frequency bands;

by adopting the scheme, the wireless signal is generally represented by taking time as an independent variable, can be decomposed into different frequency components through Fourier transform, and can be described and analyzed from two aspects of time domain and frequency domain, the time domain analyzes the change rule of the amplitude of the signal along with the time, pays attention to the characteristics of amplitude fluctuation, period fluctuation, local amplitude fluctuation and the like, and the frequency domain analyzes the distribution characteristics of the energy or power of the signal on the frequency band.

The time domain and the frequency domain of the signal are mutually transformed through Fourier transform and inverse transform. However, the fourier transform is an integral transform, which is only suitable for stationary signals (short-term stationary signals) and deterministic signals, and cannot reflect the time variation of the frequency characteristics of the signals. The characterization of the signal is either entirely time domain or entirely frequency domain. All the characteristics of the signal cannot be represented from the time domain and the frequency domain, and a time-frequency representation (TFR) is needed to analyze the time-varying relation of the frequency in the signal, so that the time-frequency joint characteristics of the signal are comprehensively reflected.

The non-parametric time-frequency analysis method does not need signal prior knowledge, so that the resolution of the obtained time and frequency does not depend on specific signals, and the method is more suitable for use scenes of opportunistic signal sensing. The common non-parametric time-frequency analysis method can be divided into linear and non-linear methods, typical linear time-frequency analysis includes short-time Fourier transform, continuous wavelet transform and the like, and typical non-linear time-frequency analysis includes Wigner-Ville distribution, Cohen (Cohen) distribution and the like.

The invention adopts a linear time-frequency analysis method.

According to the method, the convolutional neural network model is trained through the time-frequency image, all characteristics of signals can be shown, and the classification accuracy of the perception model is improved.

As shown in fig. 6, in some embodiments of the present invention, the target frequency band is a frequency range monitored by the present invention, the target frequency band may be a frequency band between frequency f1 and frequency f4, the sub-frequency bands may be a frequency band between frequency f1 and frequency f2, a frequency band between frequency f2 and frequency f3, and a frequency band between frequency f3 and frequency f4, f4 > f3 > f2 > f 1;

in some embodiments of the present invention, the training data set may include a first data set, a second data set, and a third data set, and the first time-frequency image corresponding to the sample signal between the frequency f1 and the frequency f2 may be divided into the first data set, the first time-frequency image corresponding to the sample signal between the frequency f2 and the frequency f3 may be divided into the second data set, and the first time-frequency image corresponding to the sample signal between the frequency f3 and the frequency f4 may be divided into the third data set.

In some embodiments of the present invention, the sensing models are all provided with a convolutional layer, a pooling layer and a fully-connected layer, and the weight parameters of the fully-connected layers of the sensing models are different.

Step S300, receiving an opportunity signal, representing the opportunity signal as a second time-frequency image, and calling a corresponding perception model according to a sub-frequency band where the opportunity signal is located;

in some embodiments of the invention, the signal of opportunity is represented as a second time-frequency image in the same way as the sample signal is represented as a first time-frequency image;

the sample signal and the opportunity signal are both wireless signals including, but not limited to, wifi signals, Zigbee signals, and bluetooth signals.

Calling a corresponding perception model according to a sub-band where the opportunistic signal is located, and calling a perception model trained by using a first data group if the opportunistic signal is in a frequency band between a frequency f1 and a frequency f2, wherein the sub-band where the opportunistic signal is located is preset in a Universal Software Radio Peripheral (USRP).

In some embodiments of the present invention, the trained perceptual model is provided in the USRP.

The USRP is a software programmable receiver for collecting signals.

Step S400, inputting the second time-frequency image into the corresponding perception model to obtain the signal type of the opportunity signal, and activating a receiver in a standby state in a corresponding category based on the signal type.

In some embodiments of the present invention, the receivers with multiple signal types preset in the present invention include, but are not limited to, a wifi signal receiver, a Zigbee signal receiver, and a bluetooth signal receiver, each receiver is in a standby state, the signal type of the signal of opportunity is identified according to the above steps, and the receivers of the corresponding category are activated according to the signal type. And when the signal sensor does not sense the signal for a long time, the receiver returns to a standby state to wait for the next activation.

By adopting the scheme, the receivers of various types of signals are in a standby state at first, the signal types of the opportunity signals are judged, the corresponding receivers are activated according to the judged signal types of the opportunity signals, continuous perception of various types of signals can be guaranteed without keeping the operation of all types of receivers, the utilization rate of hardware resources is improved, and the power consumption is reduced.

In some embodiments of the present invention, the opportunistic signal sensing method based on time-frequency and convolutional neural networks includes: a perception model training stage and a perceptron perceiving stage:

a perception model training stage: the signal perceptron calls a plurality of sample signals of a target frequency band to represent the sample signals as a first time-frequency image, and sets a label for the first time-frequency image according to the signal type to establish the sample signals as a training data set; if the target frequency band range is large, dividing the target frequency band into a plurality of sub-frequency bands, dividing a training data set into a plurality of training data groups based on the sub-frequency bands, inputting the training data groups into a preset convolutional neural network model, and training the convolutional neural network model to obtain a perception model;

a perceptron perception stage: the special signal receiver is in a standby state at the initial moment, the signal perceptron receives the opportunity signal, and the second time-frequency image is input into the trained perception model; and calling a corresponding perception model according to the sub-band where the opportunistic signal is located to obtain the signal type of the opportunistic signal, activating a special signal receiver of the corresponding type based on the signal type, and further extracting positioning information to position.

The continuous sensing of various signals can be ensured without keeping the work of all the receivers, and the corresponding receivers are activated according to sensing results, so that the utilization rate of hardware resources is improved, and the power consumption is reduced.

In some embodiments of the invention, as shown in fig. 2, the step of receiving the signal of opportunity comprises:

step S310, determining the length of a receiving frequency band of a receiving window according to the length of the sub-frequency band, wherein the length of the frequency band received by the receiving window is equal to the length of the sub-frequency band;

in some embodiments of the invention, the frequency band length of each of said sub-bands is equal.

S320, switching the sub-frequency band where the receiving window is located according to the monitoring duration of each sub-frequency band, and when the receiving window is located in any sub-frequency band, the receiving window receives the opportunity signal located in the sub-frequency band.

In some embodiments of the present invention, the monitoring duration of each of the sub-bands is a fixed value, that is, the receiving window switches the monitored sub-band every monitoring duration.

As shown in fig. 6, in some embodiments of the present invention, the monitoring duration may be 5s, that is, the monitored sub-band is switched every 5s, and if the frequency band between the frequency f1 and the frequency f2 is the first sub-band, the frequency band between the frequency f2 and the frequency f3 is the second sub-band, and the frequency band between the frequency f3 and the frequency f4 is the third sub-band, the receiving window is switched in such a way that the first sub-band is monitored for the first 5s, the second sub-band is monitored for the second 5s, and the third sub-band is monitored for the third 5 s.

By adopting the scheme, because the active frequency bands of different types of signals are different, the sensing models are respectively trained for different sub-frequency bands, the opportunistic signals are sensed by sub-frequency bands, and the classification accuracy of the trained sensing models is improved; for multiple types of signals in the same sub-band, the signals can be identified through the same perception model.

According to the method, the signals in hundreds of MHz bandwidth can be monitored simultaneously according to the sampling rate of the used USRP equipment, and by combining the method for monitoring different frequency bands of the target frequency band in a time-sharing mode, the signals in wider wide frequency bands can be sensed, various signals in the same frequency band can be identified simultaneously, and the typical example of the condition is the ISM frequency band.

In some embodiments of the present invention, the step of invoking the corresponding perceptual model according to the sub-band where the opportunistic signal is located further comprises: step S330, when the receiving window is in any sub-band, calling a perception model corresponding to the sub-band, wherein the perception model is used for perceiving the signal type of the opportunistic signal.

In some embodiments of the invention, the convolutional layers of the convolutional neural network model comprise a first convolutional layer, a second convolutional layer, and a third convolutional layer; the pooling layers include a first pooling layer, a second pooling layer and a third pooling layer; the full-link layer includes a first full-link layer and a second full-link layer.

In some embodiments of the invention, as shown in fig. 7, in the step of training the convolutional neural network model, the convolutional neural network model includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a third convolutional layer, a third pooling layer, a fourth convolutional layer, a fourth pooling layer, a first fully-connected layer, a second fully-connected layer and an output layer, which are connected in sequence.

The image classification is a technology of classifying each pixel or area in an image into one of a plurality of categories by analyzing the whole image so as to replace artificial visual interpretation. The core is that a neural network model is constructed through training to simulate the learning behavior of the human brain, and the parameter optimization of the model is realized through training iteration.

The neural network is composed of a convolutional layer, a pooling layer and a full-link layer. The theoretical basis of convolutional layers is mainly the concept of the receptive field in biology, and parameters required by neural network training can be greatly reduced. Downsampling, also known as pooling, is actually a subsampling of the image. It is used to reduce the amount of data while retaining useful information. By stacking convolutional and pooling layers, one or more fully connected layers are formed at the end, thereby achieving higher order reasoning capabilities.

In some embodiments of the present invention, the output ends of the first convolution layer, the second convolution layer, the third convolution layer, the fourth convolution layer, the first full-link layer and the second full-link layer are connected with a correction linear unit. The rectification Linear Unit may be implemented by a Linear rectification function (ReLU).

The main purpose of the convolutional layer is feature abstraction and extraction, while the pooling layer is responsible for feature fusion and dimensionality reduction. The full connection layer is responsible for logical inference, and all parameters need to be learned. The full link layer of the first layer is used for linking the output of the convolutional layer, removing spatial information, namely the number of channels, and converting the three-dimensional matrix into vectors. The output ends of the first convolution layer, the second convolution layer, the third convolution layer, the fourth convolution layer, the first full-connection layer and the second full-connection layer are all connected with the correction linear unit, so that the disappearance of the gradient and the gradient explosion can be relieved, and the training speed is accelerated.

In some embodiments of the present invention, for the present application, the color information of the time-frequency image has little meaning to the identification of the signal from the time-frequency image of the signal, and what is more critical is the relationship between the signal pattern and the spatial distribution. Therefore, the first time-frequency image and the second time-frequency image are processed into the gray images, and the training precision is improved.

In some embodiments of the present invention, when training the convolutional neural network model, training parameters need to be set, where the step of setting the training parameters includes:

the convolution kernel size of the first convolution layer is set to 11 × 16, stride 4, padding 2. The total parameter is (11 × 11) × 16 ═ 1936. The parameters represent the weights of the layers, which are convolved with the original pixels, and feature mapping is given. The output size of each convolution kernel in the first convolution layer is (224-11)/2+1 ═ 55, and the output size of the first convolution layer is 55 × 16. The first pooling layer was 3 x 3, stride 2, the pooled core of this layer had an output size of (55-3/2 +1) ═ 27, and an output size of 27 x 16. All parameters of the pooling layer are hyper-parameters and do not need to be learned. By using the steps, each convolution layer and each pooling layer can be set, and the size of each convolution layer and each pooling layer can be calculated. The first fully-connected layer and the second fully-connected layer, each with 864 neurons, also changed the parameters to learn from the convolution kernel parameters to the weight coefficients in the full-connection. The output layer is used after the fully connected layer in order to avoid overfitting. The parameters of the second convolutional layer and the third convolutional layer may be set to the same parameters as the first convolutional layer, and the parameters of the second pooling layer and the third pooling layer may be set to the same parameters as the first pooling layer.

The first time frequency image and the second video image can both be 224x224 grayscale images.

In some embodiments of the invention, the output layer uses a sigmoid classifier for classification.

Since whether each type of signal appears in the time-frequency diagram is independent, the problem of multi-label classification is solved. Sigmoid is used in the output layer instead of the original softmax function. Equivalently, a sigmoid classifier is used for each classification, the output probability is between [0 and 1], if the output probability is greater than a probability threshold, the probability threshold can be 0.5, and if the output probability of an output layer is greater than 0.5 when a certain signal type is verified, the opportunistic signal is considered as the type.

By adopting the scheme, noise signals often appear in the environment, so that the model is easy to learn the noise characteristics wrongly and overfitting is caused in the training process. Here we use a Negative Learning (NL) training method. The Negative Learning (NL) method does not require a priori knowledge of any type, quantity, etc. of noisy data. Unlike positive label data used in Positive Learning (PL) which contains target feature information of interest to the model, negative label data can tell the model about feature information of noise and interference information, thereby facilitating discrimination of noise features by the model. By combining PL and NL, accuracy can be improved while guaranteeing training speed. Using PL can quickly reduce losses and improve recognition accuracy, but eventually overfitting is easily produced. And correcting overfitting of the obtained model to noise through NL learning, and improving the identification accuracy.

In fig. 8, an initial network is an initial convolutional neural network, NL network represents the initial convolutional neural network after the first negative learning training, NL _ PL network is the initial convolutional neural network after the first positive learning training, and a trained network is a sensing model after the training.

As shown in FIG. 8, in some embodiments of the invention, the invention may use a combination of two negative learning trainings and one positive learning training to train between the positive learning training and the two negative learning trainings.

In some embodiments of the present invention, the step of training negative learning1 and negative learning2 twice and training positive learning comprises setting parameters of three times of learning including initial learning rate (learning rate), small batch size (batch size) and training iteration number (training iteration number). We set different super parameter values for 3 training processes, and perform a series of training using different parameter combinations, and the final optimal parameter values are shown in the following table:

	learning rate	batch size	training iteration number
				Negative learning 1	0.000002	30	10
Positive learning	0.0003	30	30
				Negative learning 2	0.00001	30	15

according to the hyper-parameter settings in the table, after 55 times of iterative training, the loss function and training accuracy curve of the network training process are shown in fig. 9(a), and it can be seen that through the first 40 times of iterative training, the loss function curve gradually decreases to a lower level, but overfitting occurs, and in the 41 times of negative learning training, the loss curve and the accuracy curve both have large mutation. But the correction of the overfitting of the noisy data is accomplished by a second negative learning, with the loss function and accuracy curve returning to normal levels. The test result of the finally trained network on the test set is shown in fig. 9(b), loss in fig. 9(a)/(b) is a loss function, epoch is the iteration number, and acc is the accuracy.

In some embodiments of the invention, the convolutional neural network model is trained using python and a pytorch library.

In the first experimental example, a plurality of nodes of each type are prepared, signal source equipment is arranged in a field under the condition that no external interference source exists in the experimental environment, and a signal acquisition system is started to acquire time-frequency images under different signal combinations. The data of the data set mainly considers different numbers of signal sources and combination relations. The number of the nodes of each type in the working state is variable, and all combinations among the nodes of each type are included, so that the practical channel environment is approached as much as possible. The signals are combined for each type of node, and 7 types of signals can be listed as 3 types of signals. The data set labels are used for the situations, and each label collects not less than 200 pictures.

Finally, 80% of training data sets are used as training sets, 20% of training data sets are used as verification sets, and in addition, 20 pictures are collected for each type of labels and are reserved as test sets.

The test results are shown in the following table, wherein Recognition access is the accuracy, and signal type is the signal type of opportunity:

Signal type	Wi-Fi	Bluetooth	ZigBee
				Recognition accurate	0.98888889	0.97071084	0.97752809

hardware construction of an opportunistic signal sensing method based on time frequency and convolutional neural networks is carried out on the basis of a USRP platform, and B210 USRP and a DELL notebook which are produced by ettus are adopted. The USRP is connected to a standard 2.4GHz 3dBi omnidirectional antenna and is connected to the computer via a USB3.0 interface.

USRP B210 integrates an AD9361RFIC direct conversion transceiver, providing 56MHz real-time bandwidth, with radio frequency ranging from 70MHz to 6 GHz. On-board signal processing and control of the AD9361 is performed by a Spartan6 XC6SLX150 FPGA connected to the host PC using 3.0 and communicating with the PC using UHD API for further processing of the acquired data.

The notebook is a DELL P74G notebook, which is matched with i7-8550U dual-core CPU, 8GB RAM and WIN10 operating system, the installed software is UHD 4.0 driver, UHD is hardware driver of USRP (universal software radio peripheral), LabVIEW 2020 and python 3.8.

USRP completes the sampling of wireless signals and sends data to PC through UHD and USB 3.0. And the labview software completes subsequent signal processing and interface display. The signal processing mainly comprises time-frequency image processing, image preprocessing, a perception control strategy, identification model management and signal classification. Where the signal classification runs a pre-trained CNN model by calling the python node. We implemented the CNN model based on a pytorreh library using the python language.

In order to test the signal sensing effect of the invention, a plurality of WiFi, Bluetooth and ZigBee signal source nodes are distributed in an actual scene to test the invention.

The experimental selection was made in a two-story underground car park, as there was no 2.4G wireless device deployed. Under the condition that no external interference source exists in the experimental environment, signal source equipment is arranged in different areas, and the influence of different numbers of signal sources and combinations on identification is mainly considered during arrangement.

As shown in fig. 10(a), at the B1 layer, we lay out multiple types of signals simultaneously to examine the signal perception capability of the device in a complex environment with multiple signals coexisting. As shown in fig. 10(B), at the B2 layer, we only lay out one type of signal at a time, and examine the sensing ability of the device for each type of signal. Test route in fig. 10(a) represents the test route of the tester.

The test results are shown in fig. 11, fig. 11(a) is a graph showing the test results at layer B1, and fig. 11(B) is a graph showing the test results at layer B2.

The recognition condition of the signal is represented by a line graph, blank represents that the signal is not sensed at all, a solid line represents that the signal is sensed stably, a dotted line represents that the signal sensing result is correct but sometimes not, and an arrow area represents that the sensing classification is wrong.

As shown in fig. 5, another aspect of the present invention provides an opportunistic signal sensing apparatus based on time-frequency and convolutional neural networks, which includes a signal acquisition module, a time-frequency image conversion module, a sensing control strategy module, a recognition model management module and a signal classification module,

the signal acquisition module is used for accessing the opportunity signal;

the time-frequency image conversion module is connected with the signal acquisition module and is used for representing the accessed opportunity signal as a second time-frequency image;

the signal classification module is connected with the time-frequency image conversion module and is used for inputting a second time-frequency image into a pre-trained perception model and outputting the type of the opportunity signal;

the perception control strategy module is connected with the signal acquisition module and is used for controlling the signal acquisition module to acquire the sub-band of the opportunity signal, and the sub-band can be any sub-band in a target frequency band;

the recognition model management module is used for endowing the perception model corresponding to the signal classification module according to the sub-frequency band monitored by the signal acquisition module.

In some embodiments of the present invention, the perceptual model is set in the signal classification module, and when the signal acquisition module monitors different sub-bands, different parameters are assigned to the perceptual model of the same structure. The parameters include weight parameters of the fully connected layer.

In some embodiments of the present invention, the signal obtaining module of the present application may be implemented by a USRP (Universal Software Radio Peripheral), where the USRP is connected to a PC, and the opportunistic signals are further processed by Labview Software or open source Software Radio (GNU Radio) at the PC end, so as to finally complete classification.

The USRP works including mixing, AD sampling, data caching, and the like.

In order to obtain richer signal characteristics, a one-dimensional time domain signal is converted into a two-dimensional time domain image through a time domain joint representation method, and therefore the time domain joint characteristics of the signal are obtained. Each pixel represents the energy of the wireless signal at a certain frequency and at a certain point in time. The whole image shows the wireless signal energy distribution over the whole frequency domain in a short time. The specific implementation is that the USRP (Universal Software Radio Peripheral ) transmits the acquired data to the pc, and the acquired data is further processed by labview Software or open source Software Radio (GNU Radio) to convert the original sampling data into a time-frequency image.

As shown in fig. 4, another aspect of the present invention provides an opportunity signal localization method, including the steps of:

The positioning calculation is used for positioning the position of the receiver according to the intensity data and/or the time sequence data, the positioning method can be based on the intensity data of the signal, a fingerprint positioning method is used for positioning the signal source or the position of the receiver, a dead reckoning method can also be used for continuously moving the position of a receiving point, namely the position of the receiving signal, and the position of the receiver is obtained through calculation or known movement speed and the received time sequence data on the basis of the known position.

In the current 2.4G ISM frequency band, the signal can be used freely without authorization, so that the frequency band becomes a frequency band in which various signals are preferentially used, various signals are mixed in a bandwidth of dozens of megameters, and the difficulty of signal identification is increased.

Wi-Fi, Bluetooth, ZigBee, etc. signals are used in a large amount in the positioning field in recent years, and the signals are concentrated in the ISM band of 2.4GHz, and the bandwidth is about 100 MHz. The existing positioning system mostly adopts a mode of providing special communication modules for different signals to sense the signals and extract information, and each module monitors a Signal source according to rules in each Signal communication protocol, and identifies and collects data such as Signal source ID (identity) and RSSI (Received Signal Strength Indication) and the like. The sensing process is as follows: and the AP (Access Point) sends the broadcast frame at regular time, and the slave equipment identifies the broadcast frame and acquires the signal source information by continuously scanning external paging to finish signal source perception.

A Global Navigation Satellite System (GNSS) is the most widely used positioning means at present, and a satellite is used to broadcast a positioning signal, so that a positioning navigation service in a Global range can be provided for a user. It also has some problems: 1. the signal landing level is about-130 dBm, so that the signal is easy to be maliciously interfered and deceived, 2, the signal is easy to be blocked by obstacles, and the signal is difficult to use in urban dense areas or indoor environments.

Signal of opportunity navigation is located by receiving all potential wireless signals in the environment. The signals of opportunity include various signals which are not specially designed for navigation except for GNSS systems, such as digital audio broadcasting, digital television broadcasting signals, amplitude modulation and frequency modulation broadcasting signals, cellular base station signals, Bluetooth, ZigBee, Wi-Fi and other wireless signals. These signals are widely present in the surrounding environment and are typically used to communicate rather than being broadcast specifically for navigation. By identifying the multiple types of signals, information such as signal intensity, time and the like of the signals are extracted, fusion calculation is carried out, and reliable positioning navigation service is provided. Common signal of opportunity categories are shown in the following table:

Signal	Frequency	Bandwidth
			WiFi	2.4GHz/5GHz	20MHz/40MHz/80MHz
Bluetooth	2.4GHz	1MHz
			ZigBee	2.4GHz	2MHz
DVB-T	40-200MHz	8MHz
			GMS	900,1800MHz	200kHz
Iridium	1620MHz	41.67kHz

the opportunistic signal positioning process comprises several links of signal sensing, preprocessing, information extraction and positioning resolving. Effective identification of the opportunistic signal sources becomes a primary task for opportunistic signal navigation. As can be seen from the above table, the opportunistic signals are of various types, have different distribution frequency bands, bandwidths and signal modulation modes, and bring some difficulties to the perception of the signals. The signal sensing method adopted by the existing opportunistic signal system has the modes of coherent detection, energy detection, cyclostationary feature detection and the like.

In fig. 3 and 4, RF is a receiver, preprocessing is preprocessing, signal reception is signal sensing, Feature extraction is information extraction, positioning engine is positioning calculation, and positioning result is output;

the permission controller in fig. 4 is a perceptual control policy module.

The existing coherent detection signal sensing method has high computational complexity, and needs to obtain prior information about signals, such as: modulation scheme, modulation order, pulse shape, packet format, etc. But also the correlation with the signal needs to be obtained by time, carrier or even channel synchronization. It is relatively complex to implement. The amount of computation is also large for different primary user types requiring specialized receivers.

The decision threshold of the existing energy detection signal sensing method is easily affected by noise power change and fails. When a plurality of signals are in the same frequency band, the signals are easily interfered by other co-frequency signals. The energy detection algorithm is not applicable to direct sequence spread spectrum signals as well as frequency hopping signals.

In the existing sensing method for detecting the signal with the cyclostationary characteristic, the communication signal generally comprises a carrier frequency, a frequency hopping sequence, a cyclic prefix and the like, which enable the signal to have inherent periodicity. The statistical properties of the mean value, the correlation function and the like of the signal are periodic, and the noise does not have the cyclostationarity, so that the noise and the target signal can be separated, but the cyclostationary feature detection has higher complexity and needs longer detection time.

The opportunistic signal perception method based on the time-frequency convolutional neural network is a method applied to signal perception, quickly identifies signal types, starts corresponding receivers, is used for denoising and frequency reduction of received signals through preprocessing, extracts intensity data and time sequence data used for extracting signals through information extraction, and is used for positioning the positions of the receivers according to the intensity data and/or the time sequence data through positioning by using a fingerprint positioning method based on the intensity data of the signals, and continuously moves the positions of receiving points, namely the positions of the receiving signals, and on the basis of the known previous positions, the positions of the receivers are obtained through calculation or known movement speed and the received time sequence data.

Corresponding to the opportunistic signal sensing method based on the time-frequency and convolutional neural network, the invention also provides an opportunistic signal sensing system based on the time-frequency and convolutional neural network, which comprises a computer device, wherein the computer device comprises a processor and a memory, computer instructions are stored in the memory, the processor is used for executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the device/system realizes the steps of the method.

The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of the opportunistic signal sensing method based on time-frequency and convolutional neural network. The computer readable storage medium may be a tangible storage medium such as Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein may be implemented as hardware, software, or combinations of both. Whether this is done in hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link.

It is to be understood that the invention is not limited to the specific arrangements and instrumentality described above and shown in the drawings. A detailed description of known methods is omitted herein for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications and additions or change the order between the steps after comprehending the spirit of the present invention.

Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments in the present invention.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made to the embodiment of the present invention by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A opportunistic signal sensing method based on time frequency and convolutional neural network is characterized in that receivers of multiple types of signals are in a standby state at an initial time, and the method comprises the following steps:

2. The method of claim 1, wherein the step of receiving the signal of opportunity comprises:

3. The method of claim 2, wherein the step of invoking the corresponding perceptual model according to the sub-band where the signal of opportunity is located further comprises: and when the receiving window is positioned in any sub-band, calling a perception model corresponding to the sub-band, wherein the perception model is used for perceiving the signal type of the opportunistic signal.

4. The method according to any of claims 1-3, wherein the step of representing the signal of opportunity as a second time-frequency image further comprises graying out the second time-frequency image.

5. The method of claim 4, wherein in the step of training the convolutional neural network model, the convolutional neural network model comprises a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a third convolutional layer, a third pooling layer, a fourth convolutional layer, a fourth pooling layer, a first fully-connected layer, a second fully-connected layer, and an output layer, which are connected in sequence.

6. The method of claim 5, wherein the output ends of the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer, the first fully-connected layer and the second fully-connected layer are connected with a rectification linear unit.

7. The method of claim 1, 5 or 6, wherein the step of inputting the plurality of training data sets into the preset convolutional neural network model further comprises: and carrying out graying processing and normalization processing on the first time-frequency image, wherein the graying processing is used for processing the original first time-frequency image into a grayscale image, and the normalization processing is used for processing the sizes of all the first time-frequency images into a uniform size.

8. The method of claim 7, wherein the step of training the convolutional neural network model comprises a positive learning training and a negative learning training,

and the label in the training data set used by the negative learning training is a negative label, and the noise label marks that the wireless signal corresponding to the first time-frequency image belongs to a noise signal or an interference signal.

9. An opportunity signal locating method, comprising the steps of:

activating a receiver of the class to which the signal of opportunity corresponds according to the method of any of claims 1-8, inputting the signal of opportunity into the receiver;

and extracting signal parameters of the signals of opportunity at the receiver, performing positioning calculation on a signal emission source of the signals of opportunity based on the signal parameters, and positioning the receiver based on the result of the positioning calculation of the emission source.

10. A system for opportunistic signal perception based on time-frequency and convolutional neural networks, comprising a processor and a memory, characterized in that the memory has stored therein computer instructions for executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system realizes the steps of the method according to any one of claims 1-8.