CN113670369B

CN113670369B - Wind speed measurement and wind noise detection method and device based on mobile terminal

Info

Publication number: CN113670369B
Application number: CN202110776982.3A
Authority: CN
Inventors: 黄菩臣; 黄哲龙; 潘佳辉; 周爽; 钱增辉; 李童; 韩皓
Original assignee: Nanjing University of Aeronautics and Astronautics
Current assignee: Nanjing University of Aeronautics and Astronautics
Priority date: 2021-07-09
Filing date: 2021-07-09
Publication date: 2023-01-06
Anticipated expiration: 2041-07-09
Also published as: CN113670369A

Abstract

The invention discloses a method and a device for measuring wind speed and detecting wind noise based on a mobile terminal, wherein the method comprises the following steps: acquiring data collected by a top microphone and a bottom microphone in the mobile terminal; separating data collected by the top microphone from data collected by the bottom microphone; respectively extracting the characteristics of the separated data; constructing a neural network; inputting the features extracted by the top microphone into the neural network to detect wind noise; inputting the features extracted by the bottom microphone into the neural network to measure the wind speed; according to the wind speed measurement and wind noise detection method, the data collected by the top microphone and the bottom microphone on the mobile terminal are obtained, the data are processed to extract the characteristics, and the characteristics are input into the neural network, so that the wind speed and the wind noise are measured, additional sensors are not needed, the method is applied to a specific environment, and the quality of video or audio recorded by the mobile equipment can be ensured.

Description

Wind speed measurement and wind noise detection method and device based on mobile terminal

Technical Field

The invention relates to the field of wind speed measurement and wind noise detection, in particular to a method and a device for wind speed measurement and wind noise detection based on a mobile terminal.

Background

The meteorological information such as wind speed/wind direction in modern war has crucial influence on field battle. At present, a wind speed/wind direction measuring method mainly depends on deploying a meteorological observation station in advance and carries out prediction by acquiring sampling data. Generally, the more weather station nodes are deployed, the higher the prediction accuracy, but this method is not effective in field battles because the field environment cannot deploy the required weather station nodes in advance. Therefore, how to accurately, rapidly and dynamically measure the wind speed and the wind direction without an meteorological station is an important research subject in modern war. In recent years, rapid development of mobile technology and the rise of crowd-sourcing calculations have made possible wind speed measurements based on mobile technology devices (e.g., smartphones). By utilizing the crowd-sourcing computing technology, soldiers, weaponry and the like in a battlefield can be used as a 'movable meteorological observation station' to acquire information and perform meteorological prediction.

In the existing wind speed measuring method based on the mobile terminal, an external sensor is mostly needed to be added so as to achieve the purpose of measuring wind speed and wind noise.

Disclosure of Invention

In order to solve the above problems, the present invention provides a wind speed measurement and wind noise detection method and apparatus capable of measuring wind speed and wind noise without an external device.

In order to achieve the above object, an aspect of the present invention provides a wind speed measurement and wind noise detection method based on a mobile terminal, including:

acquiring data collected by a top microphone and a bottom microphone in the mobile terminal;

separating data collected by the top microphone from data collected by the bottom microphone;

respectively extracting the characteristics of the separated data;

constructing a neural network;

inputting the features extracted by the top microphone into the neural network to detect wind noise;

inputting the features extracted by the bottom microphone into the neural network to measure the wind speed.

In the wind speed measurement and wind noise detection method, the feature extraction is performed on the separated data, and the method further includes:

respectively acquiring waveforms of data collected by a top microphone and a bottom microphone;

and performing corresponding operation on the waveform according to the feature to be extracted.

In the wind speed measurement and wind noise detection method, the characteristics further include: entropy of the top microphone, entropy of the bottom microphone, standard deviation of the top microphone, standard deviation of the bottom microphone, skewness of the top microphone, skewness of the bottom microphone, kurtosis of the top microphone, kurtosis of the bottom microphone, form factor of the top microphone, form factor of the bottom microphone, frequency standard deviation of the top microphone, frequency standard deviation of the bottom microphone, mean of the amplitudes of the top microphone over the 20,120 band, mean of the amplitudes of the bottom microphone over the 20,120 band, correlation of the two microphones, subtraction of the spectra of the two microphones, averaging of the amplitudes over the frequency 20,120.

In the above method for measuring wind speed and detecting wind noise, the neural network further comprises:

the input of the one-dimensional convolution is a matrix, the output is also a matrix, the input characteristic diagram is recorded as I (H multiplied by W), the output diagram is O, the middle part has c convolution kernels with the length of O n and is recorded as K (c multiplied by n), and then the size of the output characteristic diagram is c multiplied by max { W-n +1,1}. Let O [ i ] [ j ] represent the value of the element in row i and column j of the O matrix, and let us assume that our indices are numbered from 1, the formula for O is:

tanh activation function:

ReLU(x)＝max{0,x}

ReLU activation function:

softmax activation function:

where A is an n-dimensional vector and A [ i ] represents the ith element of vector A.

In the wind speed measurement and wind noise detection method, the inputting the features extracted by the top microphone into the neural network to measure the wind speed further includes:

firstly, mapping four one-dimensional convolution kernels with the length of 5, and obtaining a 4 multiplied by 16 characteristic diagram after ReLU activation output;

then, through eight one-dimensional convolution kernels with the length of 5, after the activation output of the ReLU, a characteristic diagram of 8 multiplied by 12 is obtained;

then, through four one-dimensional convolution kernels with the length of 5, after Tanh activation output, a 4 multiplied by 8 characteristic diagram is obtained;

then, two one-dimensional convolution kernels with the length of 5 are used, and a 2 x 4 characteristic diagram is obtained after no activation output;

and finally, obtaining a 1 multiplied by 1 characteristic diagram through a one-dimensional convolution kernel with the length of 4 without activation output, wherein the characteristic diagram has only one value, and the value is the wind speed measurement result obtained by the regression channel.

In the above method for measuring wind speed and detecting wind noise, the method further includes inputting the features extracted by the bottom microphone into the neural network to detect wind noise, and further includes:

firstly, mapping four one-dimensional convolution kernels with the length of 5, and obtaining a 4 multiplied by 16 characteristic diagram after Tanh activation output;

then, through eight one-dimensional convolution kernels with the length of 5, after Tanh activation output, an 8 multiplied by 12 characteristic diagram is obtained;

then, through four one-dimensional convolution kernels with the length of 5, after Tanh activation output, a 4 multiplied by 8 characteristic diagram is obtained; then, two one-dimensional convolution kernels with the length of 5 are used, and a 2 x 4 characteristic diagram is obtained after no activation output;

flattening the obtained 2 x 4 convolution graph to obtain a vector with the length of 8, activating and outputting softmax through a full connection layer, and mapping the vector with the size of 8 into a vector with the length of 2;

and finally, judging the position of the two-dimensional vector to have a larger value, if the value of the first position of the two-dimensional vector is larger than the value of the second position of the two-dimensional vector, judging the classification result as no wind noise, and otherwise, judging the classification result as wind noise.

In the wind speed measurement and wind noise detection method, the size of the fully-connected layer is m, the input of the fully-connected layer is a vector, and the output of the fully-connected layer is also a vector. The input vector is recorded as n-dimensional vector A, the output vector is recorded as B, the full-connection layer is recorded as W, the size of the full-connection layer is m, and the dimension of B is m. W is an n × m matrix, and B is calculated as B = a × W, where × is the matrix multiplication.

In the wind speed measurement and wind noise detection method, preferably, when the wind speed is less than 0.4m/s, there is no wind noise.

In another aspect, the present invention provides a wind speed measurement and wind noise detection device based on a mobile terminal, including:

the acquisition unit is used for acquiring data collected by a top microphone and a bottom microphone in the mobile electronic equipment;

a separation unit for separating data collected by the top microphone and data collected by the bottom microphone;

the characteristic extraction unit is used for respectively extracting the characteristics of the separated data;

the building unit is used for building a neural network;

the first measurement unit is used for inputting the features extracted by the top microphone into the neural network to detect wind noise;

and the second measuring unit is used for inputting the features extracted by the bottom microphone into the neural network to measure the wind speed.

Compared with the prior art, the invention has the beneficial effects that: according to the wind speed measurement and wind noise detection method, the data collected by the top microphone and the bottom microphone on the mobile terminal are obtained, then the data are processed to extract the characteristics, and the characteristics are input into the neural network, so that the wind speed and the wind noise are measured, an additional sensor is not needed, and the quality of video or audio recorded by the mobile equipment is ensured.

Drawings

Fig. 1 is a flowchart of a method for measuring wind speed and detecting wind noise based on a mobile terminal according to an embodiment of the present invention;

fig. 2 is a structural diagram of a neural network provided in the present embodiment;

FIG. 3 is a diagram of error data for detecting wind noise after training of a neural network in the present embodiment;

FIG. 4 is a graph of experimental data of wind speed measurement in the present embodiment;

FIGS. 5 and 6 are error data graphs of the measured wind speed after the neural network is trained in the present embodiment;

fig. 7 is a structural diagram of a wind speed measurement and wind noise detection device based on a mobile terminal according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As used in this application and the appended claims, the terms "a," "an," "the," and/or "the" are not intended to be inclusive in the singular, but rather are intended to be inclusive in the plural unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that steps and elements are included which are explicitly identified, that the steps and elements do not form an exclusive list, and that a method or apparatus may include other steps or elements.

The relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present application unless specifically stated otherwise. Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate. In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

Referring to fig. 1, the present embodiment provides a wind speed measurement and wind noise detection method based on a mobile terminal, including the following steps:

s10: acquiring data collected by a top microphone and a bottom microphone in the mobile terminal;

in this embodiment, specifically, a Native layer (local service) of android is called by an Oboe library, and then a device microphone is called to record audio, so as to realize low-delay data acquisition. When storing data, two mData arrays are used, each of which can store one minute of data. When one array is full, the UNIX timestamp at that moment is recorded while being stored with the other array. And storing the timestamp and the data of the array into a file. The time of the file name is the current time and date of the system obtained through java.util.date (), i.e. the time when the recording of data is started. The function that writes to the file is woken up every 500 ms. When the acquisition time is less than one minute, the prompt that the data acquisition time is too short and no file is generated is given. The saved file is saved with a file name of "year-month-day-hour-minute-second-time length txt", and one minute of data is recorded in each line of the file.

S20: separating data collected by the top microphone from data collected by the bottom microphone;

specifically, the sequential data is first separated by taking out the data of the arrays in which the data are stored every other data to obtain the sequential data of the two microphones. Taking the one minute data as an example, at a sampling rate of 22050, there are 22050 × 60 data for each of the two microphones per minute.

S30: respectively extracting the characteristics of the separated data;

specifically, because the data collected by the microphone is the waveform of sound, the obtained waveform file is preprocessed firstly, the storage mode of the computer to the audio is sampling points, and the value of each sampling point represents the amplitude of the waveform at the moment of the point; and then, carrying out corresponding operation on the waveform according to the features to be extracted. In the present embodiment, the features to be extracted are shown in table 1

TABLE 1

It should be noted that, since the specific operation of the waveform according to the above features belongs to the prior art in the field, it is not described herein again.

S40: constructing a neural network;

specifically, as shown in fig. 2, the structure diagram of the neural network provided in this embodiment is that the input of the one-dimensional convolution of the neural network is a matrix, the output O is also a matrix, the input feature diagram is denoted as I (H × W), the output diagram is O, c convolution kernels with n lengths exist in the middle, and the output feature diagram is denoted as K (c × n), so that the size of the output feature diagram is c × max { W-n +1,1}. Let O [ i ] [ j ] represent the value of the element in row i and column j of the O matrix, and assuming that our index is numbered from 1, the formula for O is:

tanh activation function:

ReLU(x)＝max{0,x}

ReLU activation function:

softmax activation function:

S50: inputting the features extracted by the top microphone into the neural network to detect wind noise;

flattening the obtained 2 x 4 convolution graph, namely traversing line by line to obtain a vector with the length of 8, activating and outputting softmax through a full connection layer, and mapping the vector with the size of 8 into a vector with the length of 2;

The size of the fully-connected layer is m, the input of the fully-connected layer is a vector, and the output of the fully-connected layer is also a vector. The input vector is recorded as an n-dimensional vector A, the output vector is recorded as B, the fully connected layer is recorded as W, the size of the fully connected layer is m, and the dimension of B is m. W is an n × m matrix, and B is calculated as B = a × W, where × is the matrix multiplication.

TABLE 1 comparison table of experimental value and true value of wind noise detection through neural network

Serial number	True value	Prediction value	Serial number	True value	Prediction value
							1	1	1	16	0	1
2	1	1	17	1	1
						3	1	1	18	1	1
4	1	1	19	1	1
						5	1	1	20	1	1
6	1	1	21	1	1
						7	1	1	22	1	1
8	1	1	23	1	1
						9	1	1	24	1	1
10	1	1	25	1	1
						11	1	1	26	1	1
12	1	1	27	1	1
						13	1	1	28	1	1
14	1	1	29	0	1
						15	1	1	30	1	1

Table 1 shows data measured by the neural network, where 1 represents noise, and 0 represents no noise, and it can be seen from experimental data that the detection effect of the neural network is very good.

Referring to fig. 3, fig. 3 is measured data during the training process of the neural network, wherein epoch represents a period, loss represents an error between an actual value and a measured value, and accuracy is an accuracy rate, and it can be seen from fig. 3 that the accuracy of the neural network is very high after the training process.

S60: inputting the features extracted by the bottom microphone into the neural network to measure the wind speed.

Specifically, the data of the top microphone is mapped by four one-dimensional convolution kernels with the length of 5, and is output through the ReLU activation, so that a 4 × 16 characteristic diagram is obtained;

It should be noted here that when the wind speed measurement result shows that the wind speed is lower than 0.4m/s, the wind noise is not generated regardless of the previous wind noise detection result.

Fig. 4 is specific experimental data in the embodiment, in which wind speed represents actual wind speed, CNN represents wind speed measured by the convolutional neural network, LSTM represents wind speed measured by the LSTM neural network, abscissa represents time, and ordinate represents wind speed grade, and it can be seen from fig. 3 that the wind speed measured by the neural network has a small difference from the actual wind speed.

In addition, after multiple times of training, the difference between the two values is smaller and smaller, referring to fig. 5 and fig. 6, in the figure, each epoch represents a period, namely a forward transmission and a directional transmission of all training samples, loss is the degree of inconsistency between a predicted value and a true value, namely an error, fig. 4 is an error value of the CNN neural network after training, fig. 5 is an error value of the LSTM neural network after training, and it can be seen from the two data that the loss value of the neural network model can be reduced to 1.1 for the error of the predicted wind speed after training, and the effect is good, and the error value is within an acceptable range.

On the other hand, referring to fig. 7, the present invention provides a wind speed measurement and wind noise detection apparatus based on a mobile terminal, comprising:

an acquisition unit 100, configured to acquire data collected by a top microphone and a bottom microphone in a mobile electronic device; it should be noted that, since the specific obtaining manner and process are already described in detail in step S10 of the wind speed measurement and wind noise detection method based on the mobile terminal, they are not described herein again.

A separation unit 200 for separating the data collected by the top microphone and the data collected by the bottom microphone; since the specific separation manner and process are described in detail in step S20 of the wind speed measurement and wind noise detection method based on the mobile terminal, they are not described herein again.

A feature extraction unit 300, configured to perform feature extraction on the separated data respectively; since the specific extraction method and process are already described in detail in step S30 of the wind speed measurement and wind noise detection method based on the mobile terminal, they are not described herein again.

A construction unit 400 for constructing a neural network; since the specific construction manner and process are already described in detail in step S40 of the wind speed measurement and wind noise detection method based on the mobile terminal, they are not described herein again.

A first measurement unit 500, configured to input features extracted by a top microphone into the neural network to detect wind noise; since the detailed measurement method and process are already described in detail in step S50 of the wind speed measurement and wind noise detection method based on the mobile terminal, they are not described herein again.

A second measuring unit 600, configured to input the features extracted by the bottom microphone into the neural network to measure the wind speed; since the detailed measurement method and process are already described in detail in step S60 of the wind speed measurement and wind noise detection method based on the mobile terminal, they are not described herein again.

In addition, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium may store a program, and the program, when executed, includes some or all of the steps of any one of the mobile terminal-based wind speed measurement and wind noise detection methods described in the above method embodiments.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a memory and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned memory comprises: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, which may include: flash Memory disks, read-Only memories (ROMs), random Access Memories (RAMs), magnetic or optical disks, and the like.

Exemplary flowcharts for a mobile terminal-based wind speed measurement and wind noise detection method according to embodiments of the present invention are described above with reference to the accompanying drawings. It should be noted that the numerous details included in the above description are merely exemplary of the invention and are not limiting of the invention. In other embodiments of the invention, the method may have more, fewer, or different steps, and the order, inclusion, function, etc. of the steps may be different from that described and illustrated.

Claims

1. A wind speed measurement and wind noise detection method based on a mobile terminal is characterized by comprising the following steps:

respectively extracting the characteristics of the separated data;

constructing a neural network;

inputting the features extracted by the bottom microphone into the neural network to measure the wind speed;

the respectively performing feature extraction on the separated data further comprises:

performing corresponding operation on the waveform according to the feature to be extracted;

the features include: entropy of the top microphone, entropy of the bottom microphone, standard deviation of the top microphone, standard deviation of the bottom microphone, skewness of the top microphone, skewness of the bottom microphone, kurtosis of the top microphone, kurtosis of the bottom microphone, form factor of the top microphone, form factor of the bottom microphone, frequency standard deviation of the top microphone, frequency standard deviation of the bottom microphone, mean of the amplitude of the top microphone over the [20,120] band, mean of the amplitude of the bottom microphone over the [20,120] band, correlation of the two microphones, spectral subtraction of the two microphones, mean of the amplitude over the frequency [20,120 ];

the neural network is as follows:

the input of the one-dimensional convolution is a matrix, the output is also a matrix, the input characteristic diagram is recorded as I (H multiplied by W), the output diagram is O, c convolution kernels with the length of n are arranged in the middle, the length of the convolution kernels is recorded as K (c multiplied by n), and then the size of the output characteristic diagram is c multiplied by max { W-n +1,1}; let O [ i ] [ j ] represent the value of the element in row i and column j of the O matrix, and assuming that our index is numbered from 1, the formula for O is:

tanh activation function:

ReLU activation function:

ReLU(x)＝max{0,x}

softmax activation function:

2. The method of claim 1, wherein inputting the features extracted by the top microphone into the neural network measures wind speed, further comprising:

3. The method of claim 1, wherein the features extracted by a bottom microphone are input into the neural network to detect wind noise, further comprising:

then, through eight one-dimensional convolution kernels with the length of 5, after Tanh activation output, an 8 multiplied by 12 characteristic diagram is obtained; then, through four one-dimensional convolution kernels with the length of 5, after Tanh activation output, a 4 multiplied by 8 characteristic diagram is obtained; then, two one-dimensional convolution kernels with the length of 5 are used, and a 2 x 4 characteristic diagram is obtained after no activation output;

the resulting 2 x 4 convolution is then flattened, resulting in a vector of length 8, followed by a full link layer,

the softmax activates output, and a vector with the size of 8 is mapped into a vector with the length of 2;

4. The method of claim 3, wherein the wind speed measurement and wind noise detection method comprises: the size of the full connection layer is m, the input of the full connection layer is a vector, the output of the full connection layer is also a vector, the input vector is recorded as an n-dimensional vector A, the output vector is B, the full connection layer is recorded as W, the size of the full connection layer is m, the dimension of B is m, W is an n × m matrix, the calculation formula of B is B = A × W, and x is matrix multiplication.

5. The method of claim 4, wherein the wind speed measurement and wind noise detection comprises: when the wind speed is lower than 0.4m/s, no wind noise exists.

6. The utility model provides a wind speed measurement and wind noise detection device based on mobile terminal which characterized in that includes:

the building unit is used for building a neural network;

the second measuring unit is used for inputting the features extracted by the bottom microphone into the neural network to measure the wind speed;

the neural network is as follows:

tanh activation function:

ReLU activation function:

ReLU(x)＝max{0,x}

softmax activation function:

7. A computer readable storage medium, having a computer program stored thereon, wherein the computer program, when being executed by a processor, performs the steps of a mobile terminal based wind speed measurement and wind noise detection method according to any of the claims 1 to 5.