CN118070162A - Bolt loosening monitoring method based on small sample learning voice recognition and portable detection device - Google Patents
Bolt loosening monitoring method based on small sample learning voice recognition and portable detection device Download PDFInfo
- Publication number
- CN118070162A CN118070162A CN202410222726.3A CN202410222726A CN118070162A CN 118070162 A CN118070162 A CN 118070162A CN 202410222726 A CN202410222726 A CN 202410222726A CN 118070162 A CN118070162 A CN 118070162A
- Authority
- CN
- China
- Prior art keywords
- small sample
- bolt
- data
- sample learning
- sound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 76
- 238000012544 monitoring process Methods 0.000 title claims abstract description 33
- 238000001514 detection method Methods 0.000 title claims abstract description 30
- 238000012549 training Methods 0.000 claims abstract description 43
- 230000005236 sound signal Effects 0.000 claims abstract description 30
- 230000000873 masking effect Effects 0.000 claims abstract description 25
- 238000012360 testing method Methods 0.000 claims description 37
- 230000006870 function Effects 0.000 claims description 26
- 239000013598 vector Substances 0.000 claims description 26
- 230000007246 mechanism Effects 0.000 claims description 19
- 230000003993 interaction Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 9
- 238000013135 deep learning Methods 0.000 claims description 7
- 238000013527 convolutional neural network Methods 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 5
- 238000005259 measurement Methods 0.000 claims description 5
- 238000012935 Averaging Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 230000000306 recurrent effect Effects 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 230000007787 long-term memory Effects 0.000 claims description 2
- 230000006403 short-term memory Effects 0.000 claims description 2
- 238000012795 verification Methods 0.000 abstract description 2
- 240000007651 Rubus glaucus Species 0.000 description 8
- 235000011034 Rubus glaucus Nutrition 0.000 description 8
- 235000009122 Rubus idaeus Nutrition 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000015654 memory Effects 0.000 description 3
- 238000009527 percussion Methods 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 101100455978 Arabidopsis thaliana MAM1 gene Proteins 0.000 description 1
- 229910000831 Steel Inorganic materials 0.000 description 1
- 239000000919 ceramic Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 210000000078 claw Anatomy 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 229910052742 iron Inorganic materials 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
Landscapes
- Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
Abstract
The invention provides a bolt loosening monitoring method and a portable detection device based on small sample learning voice recognition, which adopt an improved small sample learning model suitable for voice signals and a voice signal data enhancement method based on time domain/frequency domain masking, compared with a small sample learning model directly used in the image recognition field, the improved model can obtain higher bolt loosening recognition precision, and the verification is carried out through an example; in addition, the training samples are enhanced by using the proposed data enhancement method, the data enhancement method is more suitable for sound signals, the accuracy of bolt loosening identification under a small number of samples can be further improved, and finally, a model for completing training is integrated into an embedded board card, so that the portable bolt loosening detection device is realized.
Description
Technical Field
The invention relates to the field of structural health monitoring, in particular to a bolt loosening monitoring method based on small sample learning voice recognition and a portable detection device.
Background
Bolts are widely used in structures for connecting and transmitting loads, and in the aerospace field, a large number of bolts are used for one aircraft. Although the bolted connection is fairly tight, it still loosens under various complex load impacts, and the danger of loosening of bolts is enormous, as for example, the derailment event of a train in the uk Lambrigg of month 2 in 2007 is caused by loosening of the wheel nuts. Therefore, the method has important significance in monitoring bolt loosening.
At present, a Structural Health Monitoring (SHM) method is used for researching the bolt loosening monitoring problem in different structural systems, and a common technical method mainly comprises SHM technologies based on vibration, electromechanical impedance and guided wave. Since the assembled structure is typically composed of many bolts and the bolted connections are local structural elements, loosening of the bolts at local locations does not result in significant changes in overall structure dynamics. Therefore, the vibration-based SHM technology is insensitive to the change of the bolt pretightening force, so that the monitoring sensitivity of the technology is poor; the SHM technology based on electromechanical impedance has the advantages that the detection range is limited to the vicinity of the piezoelectric sensor, an expensive high-precision impedance analyzer is needed, and certain limitation exists; the SHM technology based on ultrasonic guided waves still needs to be pasted with a piezoelectric sensor, which brings inconvenience to the application of the sensor. The bolt looseness detection method, system and storage medium [ P ] Sichuan province CN114742820B,2023-06-27 ] based on deep learning, but the method needs to record a torque reference under standard torque, which is inconvenient to apply.
The frequency band of the sound signal is generally higher than that of the vibration signal, meanwhile, the piezoelectric ceramic sensor is not required to be excited, the sensitivity to damage is high, and the sound signal becomes a research hot spot in the field of structural damage monitoring. In 2020, university of houston, usa, professor Furui Wang and Gangbing Song proposed to monitor bolt looseness by a "percussion" method, which converts a voice signal into a two-dimensional mel-sound spectrogram (MFCC), and established a two-dimensional long-short-term memory neural network for identifying the mel-sound spectrogram (see document Wang F,Song G.Looseness detection in cup-lock scaffolds using percussion-based method[J].Automation in Construction,2020,118:103266.), however, the mel-sound spectrogram adopted by the method is two-dimensional data, and the established deep learning model is large and has an excessive data volume, which is difficult to apply in embedded hardware, and cannot be subjected to portable on-line detection in an actual structure, a scholars (see patent Shi Xiaowei, yue Shichao, a bolt failure detection method [ P ], jiangsu province: CN113804767B,2022-11-04 ]) adopts a similar method, also scholars (see document Zhuo Debing, cao Hui, a project mechanics, 2021,38 (09): 228-238.) converts a voice signal into a small-wave time frequency image as an input, adopts a convolutional neural network MobileNetv and a large-scale memory neural network, and a device for detecting a large amount of data, which is still required to detect a large amount of data by a bolt failure detection method [ P ], a user (see patent No. Zhuo Debing, cao Hui, a great deal of a device, a device for detecting a large amount of data required to be recorded by a bolt, and a large-amount of data is recorded by a patent, a device, and a device for detecting a large amount of a signal is required to be recorded by a signal by a patent, and a device, a large amount of a device.
It can be seen that the sound signal has higher sensitivity to structural damage, so that the detection of bolt looseness can be realized, and a sensor does not need to be adhered. However, the existing percussion method is complex in sound signal processing method, relies on a large data acquisition and processing device, is difficult to realize portable detection of loosening of engineering structure bolts, and meanwhile training of a depth network is seriously dependent on a large amount of marked data, but in the actual use process, a large number of marked samples are difficult to obtain.
Disclosure of Invention
In response to the problems noted in the background, small sample learning and data enhancement methods may be employed in portable monitoring of bolt loosening based on voice recognition. The small sample learning technique is that when a new class has only a small number of labeled samples, the old class that has been learned can help predict the new class, which can effectively reduce the amount of data needed to train the deep network model. Meanwhile, the data enhancement method can also improve the sample number, so that the network precision can be improved. However, the current small sample learning method and data enhancement method are mainly used in the field of image recognition, and the effect of directly using the small sample learning method and the data enhancement method are not ideal, so that the invention provides a bolt loosening monitoring method based on small sample learning voice recognition, and an improved small sample learning model suitable for voice signals and a voice signal data enhancement method based on time domain/frequency domain masking are adopted. Compared with a small sample learning model in the field of image recognition, the improved model can obtain higher bolt loosening recognition precision, and is verified through an example; in addition, the training samples are enhanced by using the proposed data enhancement method, and the data enhancement method is more suitable for sound signals, so that the accuracy of bolt loosening identification under a small number of samples can be further improved. And finally integrating the training model into an embedded board card to realize the portable bolt loosening detection device.
The technical scheme of the invention is as follows:
a bolt looseness monitoring method based on small sample learning voice recognition comprises the following steps:
step 1: building a small sample learning model; the small sample learning model comprises an embedded function and a measurement module; the embedded function is realized by adopting a deep learning neural network, and the measurement module is realized by adopting a cosine distance function;
Step 2: taking a bolt connection structure actually mounted on the integral structure as a target domain structure, knocking the target domain structure under different bolt torque states, and collecting sound data generated by knocking; preprocessing the collected sound data to obtain a target domain data set, and dividing the target domain data set into a support set S and a test set T, wherein the support set S is a training sample with a category label, and the category label refers to a bolt torque state;
Step 3: preparing an independent source domain structure which is the same as the target domain structure, knocking the source domain structure under different bolt torque states, and collecting sound data generated by knocking; preprocessing the collected sound data to obtain an auxiliary set A; enhancing the auxiliary set A by using a data enhancement method based on time domain and frequency domain masking to obtain an enhanced auxiliary set A 1; wherein: when the time domain masking data are enhanced, all values of a certain time period of the collected sound signals are set to be zero at random to be used as new data; in frequency masking, carrying out Fourier transform on an acquired sound signal, then selecting a frequency interval outside a central frequency band in the whole frequency band of the sound signal to mask, and carrying out inverse Fourier transform on the masked signal to obtain new data;
Step 4: training the model on the auxiliary set A 1 for a plurality of periods;
in a training period, randomly constructing a plurality of auxiliary support subsets A s and auxiliary test subsets A t from an auxiliary set A 1, wherein the number of categories and samples in the auxiliary support subset A s is the same as that of a support set S, and the number of categories and samples in the auxiliary test subset A t is the same as that of a test set T; a group of A s and A t form an episode, and the small sample learning model built in the step 1 is trained by using the built episodes in sequence;
Reconstructing A s and A t randomly in each training period, and training by using a model which is trained in the previous period; when the last training period is finished, saving the trained small sample learning model for bolt loosening monitoring of the target domain structure;
Step 5: inputting training samples in the support set S into the embedded function in the small sample learning model trained in the step 4, calculating the embedded vector of each training sample, and averaging the embedded vectors of all samples in the same category in the support set S to obtain a prototype vector of the category; and (3) inputting samples in the test set T into the embedded function in the small sample learning model trained in the step (4), calculating an embedded vector of each test sample, calculating the cosine distance between the embedded vector of each test sample and each class prototype vector, classifying according to the cosine distance, and obtaining the class of each test sample, thereby realizing bolt loosening monitoring on the bolt connection structure actually installed on the integral structure.
Further, the embedding function in the step 1 adopts a convolutional neural network, a long-term and short-term memory recurrent neural network or a multi-layer feedforward neural network.
Further, the embedding function in the step 1 adopts a convolutional neural network, and comprises 4 convolutional modules, wherein each convolutional module comprises 1 convolutional layer, 1 batch normalization layer, 1 ReLU layer and 1 maximum pooling layer.
Further, in the step 2 and the step 3, the preprocessing process for the collected sound data is as follows: and normalizing the sound loudness, cutting off silence before and after, and unifying the data length.
Further, in step 3, the specific process of performing time domain masking data enhancement is as follows:
For the preprocessed one-dimensional sound signal, randomly selecting a time domain interval [ t 0,t0 +delta t ], and giving a signal value on the interval to be 0 to obtain new data; wherein the selectable maximum value of Δt is 10% of the signal length and the selectable minimum value is one third of the selectable maximum value.
Further, in step 3, the specific process of performing frequency domain masking data enhancement is as follows:
1) Transforming the filtered one-dimensional sound signal to a frequency domain through FFT;
2) Acquiring a signal main lobe frequency range [ f min,fmax ];
3) Selecting a frequency interval [ f 0,f0 +Deltaf ] randomly from other areas in the frequency range [ f min,fmax ] except for the range [ f cen×95％,fcen multiplied by 105 percent ], and assigning the FFT amplitude in the frequency band interval to be 0; wherein f cen is the center frequency, the selectable maximum value of Δf is 15% of the signal main lobe frequency band, and the selectable minimum value is one third of the selectable maximum value;
4) And converting the signals after frequency domain masking into time domain through IFFT to obtain new data.
Based on the above method, the present invention also proposes a storage medium storing a computer executable program capable of implementing the above method when executed.
Based on the method and the storage medium, the invention also provides a portable detection device for bolt looseness monitoring, which comprises a control chip, a man-machine interaction interface, a mobile power supply and a pickup mechanism;
the pick-up mechanism is used for collecting bolt knocking sound information in the bolt loosening detection process and transmitting the sound information to the control chip;
The control chip can control the pickup mechanism to collect sound information according to the instruction received by the man-machine interaction interface; the storage medium is preset in the control chip, and can detect the sound information collected by the pickup mechanism according to the instruction received by the man-machine interaction interface to obtain a detection result and display the detection result on the man-machine interaction interface;
the mobile power supply supplies power to the control chip, the man-machine interaction interface and the pickup mechanism.
Furthermore, the control chip adopts an embedded Linux board card.
Furthermore, the embedded Linux board is connected with the mobile power supply and the man-machine interaction interface through the GPIO interface and is connected with the pickup mechanism through the USB interface.
Advantageous effects
Aiming at the portable monitoring requirement of bolt loosening, the invention provides a bolt loosening monitoring method based on small sample learning voice recognition and a portable detection device. Firstly, aiming at the practical problems that a large amount of training data are needed in the existing deep learning and other methods, and a large amount of labeled samples are difficult to obtain when the actual structure is knocked to acquire the sound signals, the invention provides the method which adopts an independent test structure similar to the detected engineering object structure to construct auxiliary set data, carries out data enhancement based on time domain/frequency domain masking, adopts an inserting training method to establish an improved prototype network, directly adopts 1-dimensional sound signals as input to learn small samples, realizes high-precision bolt loosening identification under few training samples, and overcomes the difficulty that the deep learning identification precision seriously depends on the data set scale. The embedded Linux board card and the microphone are adopted to collect sound signals on the hardware, so that the portable rapid and direct detection of bolt looseness is realized. The device adopts battery power supply, can hand-carry, and the practicality is stronger, and the cost is lower, and engineering application of being convenient for.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the invention will become apparent and may be better understood from the following description of embodiments taken in conjunction with the accompanying drawings in which:
fig. 1: schematic flow chart of the method of the invention
Fig. 2: schematic diagram of experimental test piece used in the invention
Fig. 3: bolt looseness identification program GUI
Fig. 4: modified raspberry pie (with touch screen on the upper layer, mobile power supply in the middle layer, and peripheral interface on the bottom layer of raspberry pie main board)
Fig. 5: relationship diagram between modified raspberry group hardware
Detailed Description
The following detailed description of embodiments of the invention is exemplary and intended to be illustrative of the invention and not to be construed as limiting the invention.
The embodiment is an application example for detecting bolt looseness of a certain high-speed train bottom connecting component. Because the sample data acquisition process of the high-speed train bottom connecting part is complex, the labeled sample which can be acquired is limited, the problem of typical small sample is solved, and the sound signal is different from the typical image data, the small sample learning method which is conventionally applied to the image data is not applicable to the sound signal, in this embodiment, an improved small sample learning model which is applicable to the sound signal and a sound signal data enhancement method based on time domain/frequency domain masking are adopted, and the bolt loosening monitoring method based on the small sample learning sound identification is provided, and specifically comprises the following steps:
Step 1: and constructing a small sample learning model in the Pytorch deep learning framework. Prototype networks are the most commonly used small sample learning model that contains both embedded functions and metrology modules.
The embedding function can be a common neural network, such as a convolutional neural network, a long-short-term memory recurrent neural network, and a multi-layer feedforward neural network. In this embodiment, a convolutional neural network is used as an embedding function, features of an input signal are extracted, the built embedding function includes 4 convolutional modules, and each convolutional module includes 1 convolutional layer, 1 batch normalization layer, 1 ReLU layer and 1 maximum pooling layer. The output of the embedding function is the embedding vector of the signal, and the prototype vector of the same class can be obtained by averaging the embedding vectors of all training samples of the class. When classifying, the embedded vector of the test sample is extracted by utilizing the embedded function, then the distance between the embedded vector of the test sample and the prototype vector of each category is calculated, if the distance is small, the test sample and the prototype vector are judged to be of the same kind, otherwise, the test sample and the prototype vector are not of the same kind, and further the classification is realized.
In this embodiment, the measurement module uses cosine similarity as the distance function, which is a special choice for sound signals. The distance function in the traditional prototype network model is Euclidean distance, but because the sound signal is one-dimensional, when the traditional prototype network uses the Euclidean distance to carry out classification judgment, the precision is poor, and after the cosine similarity is adopted as the distance function in an improved way, the bolt loosening recognition precision under the condition of a small sample is improved, and the verification is carried out in an example.
Step 2: taking a bolt connection structure actually mounted on the integral structure as a target domain structure, knocking the target domain structure under different bolt torque states by adopting a small iron hammer (such as a claw hammer and a bench hammer), and collecting sound data generated by knocking; preprocessing the collected sound data to obtain a target domain data set, and dividing the target domain data set into a support set S and a test set T, wherein the support set S is a training sample with a category label, and the category label refers to a bolt torque state.
In this example, the torque of the bolt was adjusted to 30n·m (tight) or 0n·m (loose) using a torque wrench, and two types were obtained.
After the sound data are collected, preprocessing is carried out on the data, the sound is normalized based on loudness, silence before and after the sound is removed, the data length is unified, a target domain data set is obtained, and the target domain data set is divided into a support set S and a test set T. S and T share the same label space, and because of the limitation of practical application conditions, the available labeled samples in the support set S are few, in the embodiment, each class has 1-5 training samples, if C classes are contained in S and K labeled samples are contained in each class, then the task is called a (C-way K-shot) small sample classification task.
Step 3: preparing an independent source domain structure which is the same as the target domain structure, knocking the source domain structure under different bolt torque states, and collecting sound data generated by knocking; preprocessing the collected sound data to obtain an auxiliary set A; enhancing the auxiliary set A by using a data enhancement method based on time domain/frequency domain masking to obtain an enhanced auxiliary set A 1; wherein: when the time domain masking data are enhanced, all values of a certain time period of the collected sound signals are set to be zero at random to be used as new data; in frequency masking, fourier transform is performed on an acquired sound signal, then a frequency interval outside a central frequency band is selected from the whole frequency band of the sound signal for masking, and the masked signal is used as new data through inverse fourier transform. The auxiliary set A 1 generally comprises a plurality of non-intersecting categories, each category has more labeled training samples, and the model further obtains higher bolt looseness identification precision after being trained on the enhanced auxiliary set A 1.
In this embodiment, for a certain high-speed train bottom connection component structure, two perforated steel plates with the size of 160×160mm and the thickness of 20mm are connected by using 4M 12 bolts as independent source domain structures, as shown in fig. 2.
In this embodiment, the specific process of performing time domain masking data enhancement is:
For the preprocessed one-dimensional sound signal, randomly selecting a time domain interval [ t 0,t0 +delta t ], and giving a signal value on the interval to be 0 to obtain new data; wherein the selectable maximum value of Δt is 10% of the signal length and the selectable minimum value is one third of the selectable maximum value.
In this embodiment, the specific process of performing frequency domain masking data enhancement is:
1) Transforming the filtered one-dimensional sound signal to a frequency domain through FFT;
2) Acquiring a signal main lobe frequency range [ f min,fmax ];
3) Selecting a frequency interval [ f 0,f0 +Deltaf ] randomly from other areas in the frequency range [ f min,fmax ] except for the range [ f cen×95％,fcen multiplied by 105 percent ], and assigning the FFT amplitude in the frequency band interval to be 0; wherein f cen is the center frequency, the selectable maximum value of Δf is 15% of the signal main lobe frequency band, and the selectable minimum value is one third of the selectable maximum value; here, it can be seen that since the energy is concentrated near the center frequency f cen, the data in the center frequency band, i.e., [ f cen×95％,fcen ×105% ], is not frequency-masked;
4) And converting the signals after frequency domain masking into time domain through IFFT to obtain new data.
Enhancement by time/frequency domain masking, the auxiliary set a is extended 8-fold to obtain an enhanced auxiliary set a 1.
Step 4: training is carried out by adopting an episode training method.
The model is trained on the auxiliary set a 1 for multiple cycles:
in a training period, randomly constructing a plurality of auxiliary support subsets A s and auxiliary test subsets A t from an auxiliary set A 1, wherein the number of categories and samples in the auxiliary support subset A s is the same as that of the support set S, and the number of categories and samples in the auxiliary test subset A t is the same as that of the test set T; a group of A s and A t form an episode, and the small sample learning model built in the step 1 is trained by using the built episodes in sequence; and when the last training period is finished, storing the trained small sample learning model for bolt loosening monitoring of the target domain structure.
It can be seen that the sampled episodes are a simulation of the test small sample classification task, and the small sample learning model built in step 1 is trained using a plurality of episodes built. The sample in the episode is input into the small sample learning model in the step 1 for training, 1 episode training is completed, N episodes need to be sampled in each Epoch of the model training, namely, after the M epochs training is finished, the model carries out M×N episode training, and when the classification precision of the episode training on the auxiliary test subset A t reaches a specified index, the model training is finished.
Step 5: inputting training samples in the support set S into the embedded function in the small sample learning model trained in the step 4, calculating the embedded vector of each training sample, and averaging the embedded vectors of all samples in the same category in the support set S to obtain a prototype vector of the category; and (3) inputting samples in the test set T into the embedded function in the small sample learning model trained in the step (4), calculating an embedded vector of each test sample, calculating the cosine distance between the embedded vector of each test sample and each class prototype vector, classifying according to the cosine distance, and obtaining the class of each test sample, thereby realizing bolt loosening monitoring on the bolt connection structure actually installed on the integral structure.
Based on the above method, the embodiment also provides a portable detection device for bolt loosening monitoring, as shown in fig. 3 to 5, which comprises a control chip, a man-machine interaction interface, a mobile power supply and a pickup mechanism.
The pick-up mechanism is used for collecting bolt knocking sound information in the bolt loosening detection process and transmitting the sound information to the control chip;
The control chip can control the pickup mechanism to collect sound information according to the instruction received by the man-machine interaction interface; the storage medium is preset in the control chip, and can detect the sound information collected by the pickup mechanism according to the instruction received by the man-machine interaction interface to obtain a detection result and display the detection result on the man-machine interaction interface;
the mobile power supply supplies power to the control chip, the man-machine interaction interface and the pickup mechanism.
In this embodiment, the control chip uses an embedded Linux board card (raspberry group). The embedded Linux board is connected with the mobile power supply and the man-machine interaction interface through the GPIO interface and is connected with the pickup mechanism through the USB interface. A deep learning frame and an operation environment thereof are installed on an embedded Linux board card (such as a raspberry pie), a small sample bolt looseness identification model embodying the method is transplanted from a computer to the raspberry pie, and meanwhile, a graphical user interface is written, so that one-key operation of looseness identification can be realized, the interface contains 'record', 'identify' buttons, and functions of recording and looseness identification are respectively realized.
When the test piece is used, the raspberry pie is started, the pickup mechanism (the external microphone) is connected to the raspberry pie, the microphone is close to the bolt to be tested of the test piece, and the bolt loosening monitoring program is operated. And (3) clicking a record button on a program interface to start recording, knocking a bolt to be detected by using a hammer, and displaying that the recording is completed above the interface to indicate that the input is completed. Clicking the "identify" button on the program interface, the top of the interface can display if the bolt is loose.
In order to illustrate the effect of the invention, the proposed improved prototype network small sample model and data enhancement method are used in the present embodiment to identify the bolt loosening condition of the structure, and compare the original prototype network, improve the identification accuracy of the prototype network and other common small sample learning methods.
The set 5 different bolt loosening conditions are shown in table 1. Different small sample learning methods use the same 4-layer convolution feature extractor and super parameters, and after the same auxiliary set is subjected to the training of the cutting, the classification tasks of the (5-way 5-shot) and (5-way 1-shot) small samples are respectively carried out, so that the loosening condition of bolts in the table 1 is identified, and the average value and standard deviation of classification accuracy after 50 times of testing are shown in the table 2.
TABLE 1 bolt loosening conditions
Table 2 test accuracy of different methods
Of these, relationNet, matchingNet, MAML and ProtoNet are commonly used small sample classification methods. From the above table, it can be seen that the improvement ProtoNet proposed in all models can achieve the highest classification accuracy after the training data is enhanced using time/frequency domain masking.
Although embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives, and variations may be made in the above embodiments by those skilled in the art without departing from the spirit and principles of the invention.
Claims (10)
1. A bolt loosening monitoring method based on small sample learning voice recognition is characterized by comprising the following steps of: the method comprises the following steps:
step 1: building a small sample learning model; the small sample learning model comprises an embedded function and a measurement module; the embedded function is realized by adopting a deep learning neural network, and the measurement module is realized by adopting a cosine distance function;
Step 2: taking a bolt connection structure actually mounted on the integral structure as a target domain structure, knocking the target domain structure under different bolt torque states, and collecting sound data generated by knocking; preprocessing the collected sound data to obtain a target domain data set, and dividing the target domain data set into a support set S and a test set T, wherein the support set S is a training sample with a category label, and the category label refers to a bolt torque state;
Step 3: preparing an independent source domain structure which is the same as the target domain structure, knocking the source domain structure under different bolt torque states, and collecting sound data generated by knocking; preprocessing the collected sound data to obtain an auxiliary set A; enhancing the auxiliary set A by using a data enhancement method based on time domain and frequency domain masking to obtain an enhanced auxiliary set A 1; wherein: when the time domain masking data are enhanced, all values of a certain time period of the collected sound signals are set to be zero at random to be used as new data; in frequency masking, carrying out Fourier transform on an acquired sound signal, then selecting a frequency interval outside a central frequency band in the whole frequency band of the sound signal to mask, and carrying out inverse Fourier transform on the masked signal to obtain new data;
Step 4: training the model on the auxiliary set A 1 for a plurality of periods;
in a training period, randomly constructing a plurality of auxiliary support subsets A s and auxiliary test subsets A t from an auxiliary set A 1, wherein the number of categories and samples in the auxiliary support subset A s is the same as that of a support set S, and the number of categories and samples in the auxiliary test subset A t is the same as that of a test set T; a group of A s and A t form an episode, and the small sample learning model built in the step 1 is trained by using the built episodes in sequence;
Reconstructing A s and A t randomly in each training period, and training by using a model which is trained in the previous period; when the last training period is finished, saving the trained small sample learning model for bolt loosening monitoring of the target domain structure;
Step 5: inputting training samples in the support set S into the embedded function in the small sample learning model trained in the step 4, calculating the embedded vector of each training sample, and averaging the embedded vectors of all samples in the same category in the support set S to obtain a prototype vector of the category; and (3) inputting samples in the test set T into the embedded function in the small sample learning model trained in the step (4), calculating an embedded vector of each test sample, calculating the cosine distance between the embedded vector of each test sample and each class prototype vector, classifying according to the cosine distance, and obtaining the class of each test sample, thereby realizing bolt loosening monitoring on the bolt connection structure actually installed on the integral structure.
2. The bolt looseness monitoring method based on small sample learning voice recognition of claim 1, wherein the method comprises the following steps of: the embedding function in the step1 adopts a convolution neural network, a long-term and short-term memory recurrent neural network or a multi-layer feedforward neural network.
3. The bolt looseness monitoring method based on small sample learning voice recognition of claim 2, wherein the method comprises the following steps of: the embedding function in the step 1 adopts a convolutional neural network and comprises 4 convolutional modules, wherein each convolutional module comprises 1 convolutional layer, 1 batch normalization layer, 1 ReLU layer and 1 maximum pooling layer.
4. The bolt looseness monitoring method based on small sample learning voice recognition of claim 1, wherein the method comprises the following steps of: in the step 2 and the step 3, the preprocessing process for the collected sound data comprises the following steps: and normalizing the sound loudness, cutting off silence before and after, and unifying the data length.
5. The bolt looseness monitoring method based on small sample learning voice recognition of claim 1, wherein the method comprises the following steps of: in step 3, the specific process of enhancing the time domain masking data is as follows:
For the preprocessed one-dimensional sound signal, randomly selecting a time domain interval [ t 0,t0 +delta t ], and giving a signal value on the interval to be 0 to obtain new data; wherein the selectable maximum value of Δt is 10% of the signal length and the selectable minimum value is one third of the selectable maximum value.
6. The bolt looseness monitoring method based on small sample learning voice recognition of claim 1, wherein the method comprises the following steps of: in step 3, the specific process of enhancing the frequency domain masking data is as follows:
1) Transforming the filtered one-dimensional sound signal to a frequency domain through FFT;
2) Acquiring a signal main lobe frequency range [ f min,fmax ];
3) Selecting a frequency interval [ f 0,f0 +Deltaf ] randomly from other areas in the frequency range [ f min,fmax ] except for the range [ f cen×95％,fcen multiplied by 105 percent ], and assigning the FFT amplitude in the frequency band interval to be 0; wherein f cen is the center frequency, the selectable maximum value of Δf is 15% of the signal main lobe frequency band, and the selectable minimum value is one third of the selectable maximum value;
4) And converting the signals after frequency domain masking into time domain through IFFT to obtain new data.
7. A storage medium storing a computer-executable program, characterized in that: the computer executable program, when executed, is capable of implementing the method of any of claims 1-6.
8. A portable detection device for bolt looseness monitoring, its characterized in that: the device comprises a control chip, a man-machine interaction interface, a mobile power supply and a pickup mechanism;
the pick-up mechanism is used for collecting bolt knocking sound information in the bolt loosening detection process and transmitting the sound information to the control chip;
The control chip can control the pickup mechanism to collect sound information according to the instruction received by the man-machine interaction interface; the storage medium of claim 7 is preset in the control chip, and can detect the sound information collected by the sound pickup mechanism according to the instruction received by the man-machine interaction interface to obtain a detection result and display the detection result on the man-machine interaction interface;
the mobile power supply supplies power to the control chip, the man-machine interaction interface and the pickup mechanism.
9. The portable detection device for bolt looseness monitoring of claim 8, wherein: the control chip adopts an embedded Linux board card.
10. A portable detection device for bolt looseness monitoring of claim 9, wherein: the embedded Linux board is connected with the mobile power supply and the man-machine interaction interface through the GPIO interface and is connected with the pickup mechanism through the USB interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410222726.3A CN118070162A (en) | 2024-02-28 | 2024-02-28 | Bolt loosening monitoring method based on small sample learning voice recognition and portable detection device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410222726.3A CN118070162A (en) | 2024-02-28 | 2024-02-28 | Bolt loosening monitoring method based on small sample learning voice recognition and portable detection device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118070162A true CN118070162A (en) | 2024-05-24 |
Family
ID=91110827
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410222726.3A Pending CN118070162A (en) | 2024-02-28 | 2024-02-28 | Bolt loosening monitoring method based on small sample learning voice recognition and portable detection device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118070162A (en) |
-
2024
- 2024-02-28 CN CN202410222726.3A patent/CN118070162A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Mohd Ghazali et al. | Vibration analysis for machine monitoring and diagnosis: a systematic review | |
| CN105841797A (en) | Window motor abnormal noise detection method and apparatus based on MFCC and SVM | |
| CN1743839A (en) | Structure defect ultrasonic on-line intelligent identifying system and identifying method | |
| CN112461934B (en) | Aero-engine blade crack source positioning method based on acoustic emission | |
| CN109387565A (en) | A method of brake block internal flaw is detected by analysis voice signal | |
| CN110808068A (en) | Sound detection method, device, equipment and storage medium | |
| CN115932659A (en) | Transformer fault detection method based on voiceprint characteristics | |
| CN108074581A (en) | For the control system of human-computer interaction intelligent terminal | |
| CN101344576A (en) | Judging functional failure of electromechanical by speech recognition technology | |
| Du et al. | An improved prototype network and data augmentation algorithm for few-shot structural health monitoring using guided waves | |
| CN116698986B (en) | Sound absorbing material quality control system and method based on tuned band-pass filter | |
| Qin et al. | Acoustic emission for surface quality monitoring in laser shock peening via dual-feature fusion convolution neural network | |
| CN118070162A (en) | Bolt loosening monitoring method based on small sample learning voice recognition and portable detection device | |
| CN211478951U (en) | Fault diagnosis device and fault diagnosis system | |
| Prawin et al. | A novel Mel-frequency cepstral analysis based damage diagnostic technique using ambient vibration data | |
| CN117191956A (en) | Acoustic emission-based titanium alloy stress corrosion damage classification method and apparatus | |
| Fu et al. | A Mobile Application for Sound Event Detection. | |
| Castillero et al. | Composite leading edge monitoring with a guided wave system | |
| Ebrahimkhanlou et al. | A deep learning-based framework for two-step localization and characterization of acoustic emission sources in metallic panels using only one sensor | |
| Seno et al. | A comparative study of impact localisation in composite structures using neural networks under environmental and operational variations | |
| CN104766610A (en) | Voice recognition system and method based on vibration | |
| Argus et al. | Development of a small-scale and low-cost SHM system for thin-walled CFRP structures based on acoustic emission analysis and neural networks | |
| Sohn et al. | Instantaneous online monitoring of unmanned aerial vehicles without baseline signals | |
| Tossavainen | Sound based fault detection system | |
| CN117647587B (en) | Acoustic emission signal classification method, computer equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |