CN117309446A

CN117309446A - Automatic detection method and system for full-automatic mechanical equipment

Info

Publication number: CN117309446A
Application number: CN202311244414.4A
Authority: CN
Inventors: 吴志华; 熊言山; 刘明东
Original assignee: Anhui Jiadun Automation Equipment Co ltd
Current assignee: Anhui Jiadun Automation Equipment Co ltd
Priority date: 2023-09-25
Filing date: 2023-09-25
Publication date: 2023-12-29

Abstract

The application relates to the field of intelligent detection, and particularly discloses an automatic detection method and an automatic detection system of full-automatic mechanical equipment, which use an artificial intelligence technology based on a deep neural network model to intelligently perform feature coding and extraction on a sound signal of equipment to be detected and a reference sound signal of equipment in normal operation so as to obtain a more accurate classification label for representing whether the equipment is in normal operation. Thus, a scheme for intelligently detecting the running state of the full-automatic mechanical equipment is constructed, and the detection accuracy is improved. Meanwhile, the detection efficiency is improved.

Description

Automatic detection method and system for full-automatic mechanical equipment

Technical Field

The application relates to the field of intelligent detection, and more particularly relates to an automatic detection method and system of full-automatic mechanical equipment.

Background

An automatic detection method of full-automatic mechanical equipment, comprising: 1. visual inspection: the operation state of the mechanical equipment is observed through manual visual or using equipment such as a camera, and whether abnormal conditions such as damage, foreign matters and the like exist is detected. 2. And (3) physical parameter detection: the sensors are used to detect physical parameters of the mechanical device, such as temperature, pressure, vibration, etc., to determine if the device is operating properly. 3. Manual inspection: and dispatching staff regularly to patrol the mechanical equipment, observing the running condition of the equipment, and checking whether faults or anomalies exist. However, these conventional detection methods have some drawbacks: 1. subjectivity: the traditional method relies on manual observation and judgment, is influenced by personnel experience and subjective factors, and is easy to cause misjudgment or missed judgment. 2. The traditional method is usually regular or intermittent detection, cannot monitor the running state of equipment in real time, and can miss the occurrence of abnormal situations.

Thus, an optimized automated inspection scheme for fully automated mechanical equipment is desired.

Disclosure of Invention

The present application has been made in order to solve the above technical problems. The embodiment of the application provides an automatic detection method and an automatic detection system for full-automatic mechanical equipment, which use an artificial intelligence technology based on a deep neural network model to intelligently perform feature coding and extraction on a sound signal of equipment to be detected and a reference sound signal of the equipment when the equipment is in normal operation so as to obtain a more accurate classification label for representing whether the equipment is in normal operation. Thus, a scheme for intelligently detecting the running state of the full-automatic mechanical equipment is constructed, and the detection accuracy is improved. Meanwhile, the detection efficiency is improved.

According to one aspect of the present application, there is provided an automatic detection method of a fully automatic mechanical device, including:

acquiring a sound signal of equipment to be detected and a reference sound signal when the equipment works normally;

performing Fourier transform on the sound signal of the equipment to be detected and the reference sound signal of the equipment in normal operation to obtain a plurality of detection frequency domain statistic values and a plurality of reference frequency domain statistic values;

The detection feature matrix is obtained through a first Clip model by the aid of the plurality of detection frequency domain statistic values and the waveform diagram of the sound signals of the equipment to be detected;

the plurality of reference frequency domain statistic values and the reference sound signals of the equipment in normal operation are passed through a second Clip model to obtain a reference feature matrix;

calculating a differential feature matrix between the detection feature matrix and the reference feature matrix;

carrying out local feature distribution-based density domain probability on the differential feature matrix to obtain an optimized differential feature matrix;

and the optimized differential feature matrix passes through a classifier to obtain a classification result, wherein the classification result indicates whether the equipment operates normally or not.

In the above automatic detection method of a fully automatic mechanical device, the step of obtaining a detection feature matrix by passing the plurality of detection frequency domain statistics values and a waveform diagram of a sound signal of the device to be detected through a first Clip model includes:

inputting the plurality of detected frequency domain statistics into a sequence encoder of the first Clip model to obtain a detected frequency domain statistics feature vector;

inputting a waveform diagram of the sound signal of the equipment to be detected into an image encoder of the first Clip model to obtain a detected image feature vector;

And inputting the detected image feature vector and the detected frequency domain statistical feature vector into a coding optimizer of the first Clip model to obtain the detected feature matrix.

In the above automatic detection method of a fully automatic mechanical device, inputting the plurality of detected frequency domain statistics into a sequence encoder of the first Clip model to obtain a detected frequency domain statistics feature vector, including:

inputting the detection frequency domain statistic values into a first convolution layer of the multi-scale neighborhood feature extraction module to obtain a first-scale detection frequency domain statistic feature vector, wherein the first convolution layer is provided with a first one-dimensional convolution kernel with a first length;

inputting the plurality of detection frequency domain statistics into a second convolution layer of the multi-scale neighborhood feature extraction module to obtain a second scale detection frequency domain statistics feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length;

and cascading the first scale detection frequency domain statistical feature vector and the second scale detection frequency domain statistical feature vector to obtain the detection frequency domain statistical feature vector.

In the above automatic detection method of a fully automatic mechanical device, inputting a waveform diagram of a sound signal of the device to be detected to an image encoder of the first Clip model to obtain a detected image feature vector, including:

Each layer of the image encoder using the first Clip model performs in forward transfer of the layer:

carrying out convolution processing on input data to obtain a convolution characteristic diagram;

carrying out mean pooling based on a local feature matrix on the convolution feature map to obtain a pooled feature map;

non-linear activation is carried out on the pooled feature map so as to obtain an activated feature map;

the output of the last layer of the image encoder of the first Clip model is the detected image feature vector, and the input of the first layer of the image encoder of the first Clip model is a waveform diagram of the sound signal of the equipment to be detected.

In the above automatic detection method of a fully automatic mechanical device, inputting the detected image feature vector and the detected frequency domain statistical feature vector into a code optimizer of the first Clip model to obtain the detected feature matrix, including:

inputting the detected image feature vector and the detected frequency domain statistical feature vector into a coding optimizer of the first Clip model by using the following coding formula to obtain the detected feature matrix;

wherein, the coding formula is:

wherein,a transpose vector representing the detected image feature vector, V ₂ Representing the detection frequency domain statistical feature vector, M representing the detection feature matrix,/for>Representing matrix multiplication.

In the automatic detection method of the fully automatic mechanical equipment, calculating the differential feature matrix between the detection feature matrix and the reference feature matrix includes:

calculating a differential feature matrix between the detection feature matrix and the reference feature matrix with the following differential formula;

wherein, the difference formula is:

wherein M is ₁ Representing the detection feature matrix, M ₂ Representing the reference feature matrix, M _c Representing the differential feature matrix.

In the automatic detection method of the fully automatic mechanical equipment, the performing the density domain probability based on the local feature distribution on the differential feature matrix to obtain an optimized differential feature matrix includes:

performing block segmentation on the differential feature matrix to obtain a plurality of differential molecular block feature matrices; respectively carrying out mean value pooling on the plurality of difference molecular block feature matrixes to obtain a plurality of difference molecular block global semantic feature vectors;

calculating global per-position mean value vectors of the global semantic feature vectors of the plurality of difference sub-blocks to obtain differential global semantic pivot feature vectors;

Calculating cross entropy between each difference sub-block global semantic feature vector and the difference global semantic pivot feature vector in the plurality of difference sub-block global semantic feature vectors to obtain a local feature distribution relative density semantic feature vector composed of a plurality of cross entropy values;

inputting the local feature distribution relative density semantic feature vector into a Softmax activation function to obtain a local feature distribution relative density probabilistic feature vector;

weighting each difference sub-block feature matrix by using the feature value of each position in the local feature distribution relative density probability feature vector to obtain a plurality of weighted difference sub-block feature matrices;

and splicing the plurality of weighted differential sub-block feature matrices to obtain the optimized differential feature matrix.

In the above automatic detection method of a fully automatic mechanical device, the step of passing the optimized differential feature matrix through a classifier to obtain a classification result, where the classification result indicates whether the device operates normally, includes:

expanding the optimized differential feature matrix into classification feature vectors according to row vectors or column vectors;

performing full-connection coding on the classification feature vectors by using a full-connection layer of the classifier to obtain coded classification feature vectors;

And inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.

According to another aspect of the present application, there is also provided an automatic detection system of a fully automatic mechanical device, including:

the signal acquisition module is used for acquiring a sound signal of equipment to be detected and a reference sound signal when the equipment works normally;

the Fourier transform module is used for performing Fourier transform on the sound signal of the equipment to be detected and the reference sound signal of the equipment in normal operation to obtain a plurality of detection frequency domain statistic values and a plurality of reference frequency domain statistic values;

the detection coding module is used for obtaining a detection feature matrix through a first Clip model according to the plurality of detection frequency domain statistic values and the waveform diagram of the sound signals of the equipment to be detected;

the reference coding module is used for enabling the plurality of reference frequency domain statistic values and the reference sound signals of the equipment in normal operation to pass through a second Clip model to obtain a reference feature matrix;

the difference module is used for calculating a difference characteristic matrix between the detection characteristic matrix and the reference characteristic matrix;

the optimization module is used for carrying out local feature distribution-based density domain probability on the differential feature matrix to obtain an optimized differential feature matrix;

And the result generation module is used for enabling the optimized differential feature matrix to pass through a classifier to obtain a classification result, wherein the classification result indicates whether the equipment operates normally or not.

Compared with the prior art, the automatic detection method and system for the full-automatic mechanical equipment provided by the application use the artificial intelligence technology based on the deep neural network model to intelligently perform feature coding and extraction on the sound signals of the equipment to be detected and the reference sound signals of the equipment in normal operation so as to obtain more accurate classification labels for representing whether the equipment operates normally. Thus, a scheme for intelligently detecting the running state of the full-automatic mechanical equipment is constructed, and the detection accuracy is improved. Meanwhile, the detection efficiency is improved.

Drawings

The foregoing and other objects, features and advantages of the present application will become more apparent from the following more particular description of embodiments of the present application, as illustrated in the accompanying drawings. The accompanying drawings are included to provide a further understanding of embodiments of the application and are incorporated in and constitute a part of this specification, illustrate the application and not constitute a limitation to the application. In the drawings, like reference numerals generally refer to like parts or steps.

Fig. 1 illustrates a flow chart of an automatic detection method of a fully automated mechanical device according to an embodiment of the present application.

Fig. 2 illustrates an architecture diagram of an automatic detection method of a fully automated mechanical device according to an embodiment of the present application.

Fig. 3 illustrates a flowchart of passing the plurality of detection frequency domain statistics and a waveform diagram of a sound signal of the device to be detected through a first Clip model to obtain a detection feature matrix in an automatic detection method of a fully automatic mechanical device according to an embodiment of the present application.

Fig. 4 illustrates a block diagram of an automated inspection system of a fully automated mechanical apparatus, according to an embodiment of the present application.

Fig. 5 illustrates a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application and not all of the embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.

Summary of the application

As described above in the background art, the conventional detection method has some drawbacks: 1. subjectivity: the traditional method relies on manual observation and judgment, is influenced by personnel experience and subjective factors, and is easy to cause misjudgment or missed judgment. 2. The traditional method is usually regular or intermittent detection, cannot monitor the running state of equipment in real time, and can miss the occurrence of abnormal situations. Thus, an optimized automated inspection scheme for fully automated mechanical equipment is desired.

Aiming at the technical problems, an optimized automatic detection scheme of full-automatic mechanical equipment is provided, and an artificial intelligence technology based on a deep neural network model is used for intelligently carrying out feature coding and extraction on a sound signal of equipment to be detected and a reference sound signal of equipment in normal operation so as to obtain a more accurate classification label for representing whether the equipment is in normal operation. Thus, a scheme for intelligently detecting the running state of the full-automatic mechanical equipment is constructed, and the detection accuracy is improved. Meanwhile, the detection efficiency is improved.

At present, deep learning and neural networks have been widely used in the fields of computer vision, natural language processing, speech signal processing, and the like. In addition, deep learning and neural networks have also shown levels approaching and even exceeding humans in the fields of image classification, object detection, semantic segmentation, text translation, and the like.

In recent years, deep learning and development of neural networks provide new solutions and schemes for automatic detection schemes of fully automatic mechanical devices.

Specifically, first, a sound signal of the operation of the device to be detected and a reference sound signal of the normal operation of the device are acquired. And then, respectively carrying out Fourier transformation on the sound signal of the equipment to be detected and the reference sound signal of the equipment in normal operation to obtain a plurality of detection frequency domain statistic values and a plurality of reference frequency domain statistic values. Considering that the sound signal is a complex waveform in the time domain, but can be represented as components of different frequencies in the frequency domain. By fourier transformation, the sound signal can be converted into a frequency domain representation, resulting in statistics of the different frequency components. These statistics may provide information about the frequency characteristics of the sound signal, including spectral distribution, frequency intensity, etc., helping to distinguish between different sound patterns. Meanwhile, the operating sound of the mechanical equipment generally has a certain periodicity, such as a cyclical operating sound of the engine. By fourier transformation, the sound signal can be converted into a frequency domain representation and the periodic components highlighted. This helps to capture the natural frequency signature of the device when it is operating normally and compare it with the sound signal of the device to be detected when it is operating. It is also contemplated that the sound signals are often subject to interference from ambient noise, such as background noise, and the like. The fourier transform may separate the noise in the frequency domain so that the effect of the noise in the frequency domain is less. By extracting the frequency domain statistic value, the interference of noise on feature extraction and classification can be reduced, and the detection accuracy is improved.

And then, the detection frequency domain statistic values and the waveform diagram of the sound signal of the equipment to be detected are processed through a first Clip model to obtain a detection feature matrix. And then, the plurality of reference frequency domain statistic values and the reference sound signals during normal operation of the equipment pass through a second Clip model to obtain a reference feature matrix. Both the time domain waveform and the frequency domain statistics of the sound signal contain information about the sound characteristics, but they represent different aspects. The time domain waveform may provide information about the amplitude, waveform shape, and timing characteristics of the sound signal, while the frequency domain statistics reflect the frequency distribution and frequency characteristics of the sound signal. By combining the two, the time-frequency characteristics of the sound signals can be comprehensively considered, and the expression capacity of the characteristics can be improved. A plurality of feature vectors can be generated by encoding a plurality of detected frequency domain statistics and a waveform diagram of the sound signal by the first Clip model. These feature vectors may capture sound features at different levels and angles, increasing the diversity of features. This helps to better describe the acoustic characteristics of the device to be detected and improves the accuracy of subsequent differential analysis and classification. A second Clip model is used for encoding a plurality of reference frequency domain statistic values and reference sound signals, and a model can be established to represent sound characteristics of the normal working state of the equipment. The reference model can capture the sound characteristic distribution and mode of the equipment in the normal working state and serve as a standard for subsequent differential analysis. The sound signal when the device is operating normally has certain inherent characteristics such as frequency distribution, frequency intensity etc. These inherent features can be extracted by the second Clip model encoding and expressed as a reference feature matrix. This facilitates comparing the sound signature of the normal operating condition of the device with the sound signal of the device to be detected to detect an abnormal or faulty condition.

Next, a differential feature matrix between the detection feature matrix and the reference feature matrix is calculated. By calculating the differential feature matrix, the difference between the sound features of the equipment to be detected and the normal working state of the equipment can be quantified. Abnormal or fault conditions typically result in changes in the acoustic signature that may be manifested in the differential signature matrix. By analyzing the differential feature matrix, the abnormal mode of the sound feature can be detected, so as to judge whether the equipment has faults or abnormal conditions.

And then, the differential feature matrix passes through a classifier to obtain a classification result, wherein the classification result indicates whether the equipment operates normally or not. By using a classifier, an automated detection of the operational status of the device can be achieved. The classifier can learn and identify feature patterns in the differential feature matrix that are associated with normal operating conditions and abnormal conditions. By inputting the differential feature matrix into the classifier, whether the running state of the equipment is normal or not can be automatically judged, and manual intervention is not needed. Thus, a scheme for intelligently detecting the running state of the full-automatic mechanical equipment is constructed, and the detection accuracy is improved. Meanwhile, the detection efficiency is improved.

In particular, the differential feature matrix is calculated by calculating the difference between the device working sound to be detected and the device normal working sound. Since the device may have different operating states or abnormal conditions, the difference between the operating sound of the device to be detected and the normal operating sound of the device may spatially exhibit non-uniformity. Different operating conditions or anomalies may result in certain regions of the differential feature matrix having large differences, while other regions may be relatively small. Meanwhile, the differential feature matrix is used as a feature representation of the input classifier, and the spatial heterogeneity and the heterogeneity of the feature distribution may affect the distinguishing capability of the classifier on different classes. If some regions of the differential feature matrix have a large difference in space, while other regions are relatively small, the classifier may be more concerned with regions having large differences, resulting in a class probability domain offset.

Thus, the differential feature matrix has spatial heterogeneity and non-uniformity in its feature distribution, which may be due to differences in feature extraction processes, device operating states, and feature representation capabilities. These factors lead to class probability domain shifts in the classification results obtained by the input classifier, i.e. the classifier has a certain deviation from the probability distribution of different classes. Based on this, the differential feature matrix performs density domain probability based on the local feature distribution.

Specifically, performing density domain probability based on local feature distribution on the differential feature matrix to obtain an optimized differential feature matrix, including: performing block segmentation on the differential feature matrix to obtain a plurality of differential molecular block feature matrices; respectively carrying out mean value pooling on the plurality of difference molecular block feature matrixes to obtain a plurality of difference molecular block global semantic feature vectors; calculating global per-position mean value vectors of the global semantic feature vectors of the plurality of difference sub-blocks to obtain differential global semantic pivot feature vectors; calculating cross entropy between each difference sub-block global semantic feature vector and the difference global semantic pivot feature vector in the plurality of difference sub-block global semantic feature vectors to obtain a local feature distribution relative density semantic feature vector composed of a plurality of cross entropy values; inputting the local feature distribution relative density semantic feature vector into a Softmax activation function to obtain a local feature distribution relative density probabilistic feature vector; weighting each difference sub-block feature matrix by using the feature value of each position in the local feature distribution relative density probability feature vector to obtain a plurality of weighted difference sub-block feature matrices; and splicing the plurality of weighted differential sub-block feature matrices to obtain the optimized differential feature matrix.

The method comprises the steps of carrying out space domain block segmentation on the differential feature matrix to obtain a plurality of differential molecular block feature matrices, carrying out mean value pooling on the plurality of differential molecular block feature matrices to obtain a plurality of differential molecular block global semantic feature vectors, taking a global position-based mean value vector of the plurality of differential molecular block global semantic feature vectors as a class center of feature distribution of the plurality of differential molecular block feature matrices, and further calculating cross entropy between each differential molecular block global semantic feature vector and the differential global semantic pivot feature vector in the plurality of differential molecular block global semantic feature vectors so as to measure feature distribution space consistency and offset of each differential molecular block feature matrix relative to the global class center. And further, probability is carried out on the local feature distribution relative density semantic feature vector formed by the plurality of cross entropy values through a Softmax activation function, and the feature value of each position in the local feature distribution relative density probability feature vector is used for weighting each difference sub-block feature matrix so as to carry out feature distribution correction based on spatial distribution consistency on each local feature matrix of the difference feature matrix, thereby improving the structural rationality and robustness of feature expression of the difference feature matrix.

Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described in detail with reference to the accompanying drawings.

Exemplary method

Fig. 1 is a flowchart of an automatic detection method of a fully automatic mechanical device according to an embodiment of the present application. As shown in fig. 1, a method for automatically detecting a fully automatic mechanical device according to an embodiment of the present application includes: s110, acquiring a sound signal of equipment to be detected and a reference sound signal when the equipment works normally; s120, carrying out Fourier transformation on the sound signal of the equipment to be detected and the reference sound signal of the equipment in normal operation to obtain a plurality of detection frequency domain statistic values and a plurality of reference frequency domain statistic values; s130, passing the plurality of detection frequency domain statistic values and the waveform diagram of the sound signal of the equipment to be detected through a first Clip model to obtain a detection feature matrix; s140, the plurality of reference frequency domain statistic values and the reference sound signals of the equipment in normal operation are passed through a second Clip model to obtain a reference feature matrix; s150, calculating a differential feature matrix between the detection feature matrix and the reference feature matrix; s160, carrying out local feature distribution-based density domain probability on the differential feature matrix to obtain an optimized differential feature matrix; and S170, enabling the optimized differential feature matrix to pass through a classifier to obtain a classification result, wherein the classification result indicates whether the equipment is normal in operation.

Fig. 2 is a block diagram of an automatic detection method of a fully automatic mechanical device according to an embodiment of the present application. In this architecture, as shown in fig. 2, first, a sound signal of the operation of the device to be detected and a reference sound signal of the normal operation of the device are acquired. And then, respectively carrying out Fourier transformation on the sound signal of the equipment to be detected and the reference sound signal of the equipment in normal operation to obtain a plurality of detection frequency domain statistic values and a plurality of reference frequency domain statistic values. And then, the detection frequency domain statistic values and the waveform diagram of the sound signal of the equipment to be detected are processed through a first Clip model to obtain a detection feature matrix. And then, the plurality of reference frequency domain statistic values and the reference sound signals during normal operation of the equipment pass through a second Clip model to obtain a reference feature matrix. Next, a differential feature matrix between the detection feature matrix and the reference feature matrix is calculated. And then, carrying out density domain probability based on local feature distribution on the differential feature matrix to obtain an optimized differential feature matrix. And then, the optimized differential feature matrix passes through a classifier to obtain a classification result, wherein the classification result indicates whether the equipment operates normally or not.

In step S110, a sound signal of the operation of the device to be detected and a reference sound signal of the normal operation of the device are acquired. As described above in the background art, the conventional detection method has some drawbacks: 1. subjectivity: the traditional method relies on manual observation and judgment, is influenced by personnel experience and subjective factors, and is easy to cause misjudgment or missed judgment. 2. The traditional method is usually regular or intermittent detection, cannot monitor the running state of equipment in real time, and can miss the occurrence of abnormal situations. Thus, an optimized automated inspection scheme for fully automated mechanical equipment is desired.

In recent years, deep learning and development of neural networks provide new solutions and schemes for automatic detection schemes of fully automatic mechanical devices. Specifically, first, a sound signal of the operation of the device to be detected and a reference sound signal of the normal operation of the device are acquired.

In step S120, fourier transforms are performed on the sound signal of the device to be detected and the reference sound signal of the device when the device is operating normally, so as to obtain a plurality of detected frequency domain statistics values and a plurality of reference frequency domain statistics values. Considering that the sound signal is a complex waveform in the time domain, but can be represented as components of different frequencies in the frequency domain. By fourier transformation, the sound signal can be converted into a frequency domain representation, resulting in statistics of the different frequency components. These statistics may provide information about the frequency characteristics of the sound signal, including spectral distribution, frequency intensity, etc., helping to distinguish between different sound patterns. Meanwhile, the operating sound of the mechanical equipment generally has a certain periodicity, such as a cyclical operating sound of the engine. By fourier transformation, the sound signal can be converted into a frequency domain representation and the periodic components highlighted. This helps to capture the natural frequency signature of the device when it is operating normally and compare it with the sound signal of the device to be detected when it is operating. It is also contemplated that the sound signals are often subject to interference from ambient noise, such as background noise, and the like. The fourier transform may separate the noise in the frequency domain so that the effect of the noise in the frequency domain is less. By extracting the frequency domain statistic value, the interference of noise on feature extraction and classification can be reduced, and the detection accuracy is improved.

In step S130, the detected frequency domain statistics and the waveform diagram of the sound signal of the to-be-detected device are passed through a first Clip model to obtain a detection feature matrix. Both the time domain waveform and the frequency domain statistics of the sound signal contain information about the sound characteristics, but they represent different aspects. The time domain waveform may provide information about the amplitude, waveform shape, and timing characteristics of the sound signal, while the frequency domain statistics reflect the frequency distribution and frequency characteristics of the sound signal. By combining the two, the time-frequency characteristics of the sound signals can be comprehensively considered, and the expression capacity of the characteristics can be improved. A plurality of feature vectors can be generated by encoding a plurality of detected frequency domain statistics and a waveform diagram of the sound signal by the first Clip model. These feature vectors may capture sound features at different levels and angles, increasing the diversity of features. This helps to better describe the acoustic characteristics of the device to be detected and improves the accuracy of subsequent differential analysis and classification.

Fig. 3 illustrates a flowchart of passing the plurality of detection frequency domain statistics and a waveform diagram of a sound signal of the device to be detected through a first Clip model to obtain a detection feature matrix in an automatic detection method of a fully automatic mechanical device according to an embodiment of the present application. As shown in fig. 3, passing the plurality of detection frequency domain statistics and the waveform diagram of the sound signal of the to-be-detected device through a first Clip model to obtain a detection feature matrix, including: s131, inputting the plurality of detection frequency domain statistic values into a sequence encoder of the first Clip model to obtain a detection frequency domain statistic feature vector; s132, inputting a waveform diagram of the sound signal of the equipment to be detected to an image encoder of the first Clip model to obtain a detected image feature vector; and S133, inputting the detected image feature vector and the detected frequency domain statistical feature vector into a coding optimizer of the first Clip model to obtain the detected feature matrix.

Specifically, in the automatic detection method of the fully automatic mechanical device, inputting the plurality of detected frequency domain statistics into the sequence encoder of the first Clip model to obtain a detected frequency domain statistics feature vector includes: inputting the detection frequency domain statistic values into a first convolution layer of the multi-scale neighborhood feature extraction module to obtain a first-scale detection frequency domain statistic feature vector, wherein the first convolution layer is provided with a first one-dimensional convolution kernel with a first length; inputting the plurality of detection frequency domain statistics into a second convolution layer of the multi-scale neighborhood feature extraction module to obtain a second scale detection frequency domain statistics feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and cascading the first scale detection frequency domain statistical feature vector and the second scale detection frequency domain statistical feature vector to obtain the detection frequency domain statistical feature vector.

Specifically, in the automatic detection method of the fully automatic mechanical device, inputting the waveform diagram of the sound signal of the device to be detected into the image encoder of the first Clip model to obtain the detected image feature vector includes: each layer of the image encoder using the first Clip model performs in forward transfer of the layer: carrying out convolution processing on input data to obtain a convolution characteristic diagram; carrying out mean pooling based on a local feature matrix on the convolution feature map to obtain a pooled feature map; non-linear activation is carried out on the pooled feature map so as to obtain an activated feature map; the output of the last layer of the image encoder of the first Clip model is the detected image feature vector, and the input of the first layer of the image encoder of the first Clip model is a waveform diagram of the sound signal of the equipment to be detected.

Specifically, in the automatic detection method of the fully automatic mechanical device, inputting the detected image feature vector and the detected frequency domain statistical feature vector into the code optimizer of the first Clip model to obtain the detected feature matrix includes: inputting the detected image feature vector and the detected frequency domain statistical feature vector into a coding optimizer of the first Clip model by using the following coding formula to obtain the detected feature matrix; wherein, the coding formula is:

wherein,a transpose vector representing the detected image feature vector, V ₂ Representing the detected frequency domain statistical feature vector, M representing the detected feature momentArray (S)>Representing matrix multiplication.

In step S140, the plurality of reference frequency domain statistics and the reference sound signal during normal operation of the device are passed through a second Clip model to obtain a reference feature matrix. A second Clip model is used for encoding a plurality of reference frequency domain statistic values and reference sound signals, and a model can be established to represent sound characteristics of the normal working state of the equipment. The reference model can capture the sound characteristic distribution and mode of the equipment in the normal working state and serve as a standard for subsequent differential analysis. The sound signal when the device is operating normally has certain inherent characteristics such as frequency distribution, frequency intensity etc. These inherent features can be extracted by the second Clip model encoding and expressed as a reference feature matrix. This facilitates comparing the sound signature of the normal operating condition of the device with the sound signal of the device to be detected to detect an abnormal or faulty condition. Specifically, the processing is performed in accordance with the processing manner of the first Clip model.

In step S150, a differential feature matrix between the detection feature matrix and the reference feature matrix is calculated. By calculating the differential feature matrix, the difference between the sound features of the equipment to be detected and the normal working state of the equipment can be quantified. Abnormal or fault conditions typically result in changes in the acoustic signature that may be manifested in the differential signature matrix. By analyzing the differential feature matrix, the abnormal mode of the sound feature can be detected, so as to judge whether the equipment has faults or abnormal conditions.

Specifically, in the automatic detection method of the fully automatic mechanical device, calculating a differential feature matrix between the detection feature matrix and the reference feature matrix includes: calculating a differential feature matrix between the detection feature matrix and the reference feature matrix with the following differential formula; wherein, the difference formula is:

In step S160, the differential feature matrix is subjected to density domain probability based on local feature distribution to obtain an optimized differential feature matrix. The differential feature matrix is calculated by taking the difference between the working sound of the equipment to be detected and the normal working sound of the equipment into consideration. Since the device may have different operating states or abnormal conditions, the difference between the operating sound of the device to be detected and the normal operating sound of the device may spatially exhibit non-uniformity. Different operating conditions or anomalies may result in certain regions of the differential feature matrix having large differences, while other regions may be relatively small. Meanwhile, the differential feature matrix is used as a feature representation of the input classifier, and the spatial heterogeneity and the heterogeneity of the feature distribution may affect the distinguishing capability of the classifier on different classes. If some regions of the differential feature matrix have a large difference in space, while other regions are relatively small, the classifier may be more concerned with regions having large differences, resulting in a class probability domain offset.

Specifically, in the automatic detection method of the fully automatic mechanical device, the performing, on the differential feature matrix, density domain probability based on local feature distribution to obtain an optimized differential feature matrix includes: performing block segmentation on the differential feature matrix to obtain a plurality of differential molecular block feature matrices; respectively carrying out mean value pooling on the plurality of difference molecular block feature matrixes to obtain a plurality of difference molecular block global semantic feature vectors; calculating global per-position mean value vectors of the global semantic feature vectors of the plurality of difference sub-blocks to obtain differential global semantic pivot feature vectors; calculating cross entropy between each difference sub-block global semantic feature vector and the difference global semantic pivot feature vector in the plurality of difference sub-block global semantic feature vectors to obtain a local feature distribution relative density semantic feature vector composed of a plurality of cross entropy values; inputting the local feature distribution relative density semantic feature vector into a Softmax activation function to obtain a local feature distribution relative density probabilistic feature vector; weighting each difference sub-block feature matrix by using the feature value of each position in the local feature distribution relative density probability feature vector to obtain a plurality of weighted difference sub-block feature matrices; and splicing the plurality of weighted differential sub-block feature matrices to obtain the optimized differential feature matrix.

In step S170, the optimized differential feature matrix is passed through a classifier to obtain a classification result, where the classification result indicates whether the device operates normally. By using a classifier, an automated detection of the operational status of the device can be achieved. The classifier can learn and identify feature patterns in the optimized differential feature matrix that are associated with normal operating conditions and abnormal conditions. By inputting the optimized differential feature matrix into the classifier, whether the running state of the equipment is normal or not can be automatically judged, and manual intervention is not needed. Thus, a scheme for intelligently detecting the running state of the full-automatic mechanical equipment is constructed, and the detection accuracy is improved. Meanwhile, the detection efficiency is improved.

Specifically, in the automatic detection method of the fully automatic mechanical device, the method includes that the optimized differential feature matrix passes through a classifier to obtain a classification result, wherein the classification result indicates whether the device operates normally or not, and the method includes: expanding the optimized differential feature matrix into classification feature vectors according to row vectors or column vectors; performing full-connection coding on the classification feature vectors by using a full-connection layer of the classifier to obtain coded classification feature vectors; and inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.

In summary, the automatic detection method of the full-automatic mechanical equipment according to the embodiment of the application has been elucidated, and the artificial intelligence technology based on the deep neural network model is used for intelligently performing feature coding and extraction on the sound signal of the equipment to be detected and the reference sound signal when the equipment is in normal operation, so as to obtain a more accurate classification label for indicating whether the equipment is in normal operation. Thus, a scheme for intelligently detecting the running state of the full-automatic mechanical equipment is constructed, and the detection accuracy is improved. Meanwhile, the detection efficiency is improved.

Exemplary System

Fig. 4 is a block diagram of an automated inspection system of a fully automated mechanical apparatus according to an embodiment of the present application. As shown in fig. 4, an automatic detection system 100 of a fully automatic mechanical device according to an embodiment of the present application includes: a signal acquisition module 110, configured to acquire a sound signal of the device to be detected when the device is operating and a reference sound signal of the device when the device is operating normally; the fourier transform module 120 is configured to perform fourier transform on the sound signal of the to-be-detected device and the reference sound signal of the device when the device is operating normally, so as to obtain a plurality of detection frequency domain statistics values and a plurality of reference frequency domain statistics values; the detection encoding module 130 is configured to pass the plurality of detection frequency domain statistics values and a waveform diagram of a sound signal of the to-be-detected device through a first Clip model to obtain a detection feature matrix; the reference encoding module 140 is configured to pass the plurality of reference frequency domain statistics and the reference sound signal during normal operation of the device through a second Clip model to obtain a reference feature matrix; a difference module 150, configured to calculate a difference feature matrix between the detection feature matrix and the reference feature matrix; an optimization module 160, configured to perform local feature distribution-based density domain probability on the differential feature matrix to obtain an optimized differential feature matrix; and a result generating module 170, configured to pass the optimized differential feature matrix through a classifier to obtain a classification result, where the classification result indicates whether the device operates normally.

Here, it will be understood by those skilled in the art that the specific functions and operations of the respective units and modules in the automatic inspection system 100 of the fully automatic machine described above have been described in detail in the above description of the automatic inspection method of the fully automatic machine with reference to fig. 1 to 3, and thus, repetitive descriptions thereof will be omitted.

In summary, an automatic detection system of a fully automatic mechanical device according to an embodiment of the present application has been elucidated, which uses an artificial intelligence technology based on a deep neural network model to intelligently perform feature encoding and extraction on a sound signal of a device to be detected and a reference sound signal when the device is operating normally, so as to obtain a more accurate classification tag for indicating whether the device is operating normally. Thus, a scheme for intelligently detecting the running state of the full-automatic mechanical equipment is constructed, and the detection accuracy is improved. Meanwhile, the detection efficiency is improved.

As described above, the automatic detection system 100 of the full-automatic mechanical device according to the embodiment of the present application may be implemented in various terminal devices, such as an automatic detection server of the full-automatic mechanical device, and the like. In one example, the automated inspection system 100 of a fully automated mechanical device according to embodiments of the present application may be integrated into a terminal device as a software module and/or hardware module. For example, the automated inspection system 100 of the fully automated mechanical equipment may be a software module in the operating system of the terminal equipment, or may be an application developed for the terminal equipment; of course, the automated inspection system 100 of the fully automated mechanical equipment may also be one of a plurality of hardware modules of the terminal equipment.

Alternatively, in another example, the automated inspection system 100 of the fully automated mechanical device and the terminal device may be separate devices, and the automated inspection system 100 of the fully automated mechanical device may be connected to the terminal device through a wired and/or wireless network and transmit the interactive information in a agreed data format.

Exemplary electronic device

Next, an electronic device according to an embodiment of the present application is described with reference to fig. 5. Fig. 5 is a block diagram of an electronic device according to an embodiment of the present application. As shown in fig. 5, the electronic device 10 includes one or more processors 11 and a memory 12.

The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM) and/or cache memory (cache), and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer readable storage medium that can be executed by the processor 11 to perform the functions in the automated inspection method of the fully automated mechanical equipment of the various embodiments of the present application described above and/or other desired functions. Various contents such as a sound signal of the operation of the device to be detected and a reference sound signal of the normal operation of the device may also be stored in the computer-readable storage medium.

In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown).

The input means 13 may comprise, for example, a keyboard, a mouse, etc.

The output device 14 can output various information to the outside, including whether the apparatus is operating normally or not, and the like. The output means 14 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, etc.

Of course, only some of the components of the electronic device 10 that are relevant to the present application are shown in fig. 5 for simplicity, components such as buses, input/output interfaces, etc. are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer readable storage Medium

In addition to the methods and apparatus described above, embodiments of the present application may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform steps in the functions of the automatic detection method of a fully automatic mechanical device according to the various embodiments of the present application described in the "exemplary methods" section of the present specification.

The computer program product may write program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer-readable storage medium, having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in the functions of the automatic detection method of a fully automatic mechanical device according to various embodiments of the present application described in the above "exemplary method" section of the present specification.

The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The basic principles of the present application have been described above in connection with specific embodiments, however, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are merely examples and not limiting, and these advantages, benefits, effects, etc. are not to be considered as necessarily possessed by the various embodiments of the present application. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, as the application is not intended to be limited to the details disclosed herein as such.

The block diagrams of the devices, apparatuses, devices, systems referred to in this application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. As will be appreciated by one of skill in the art, the devices, apparatuses, devices, systems may be connected, arranged, configured in any manner. Words such as "including," "comprising," "having," and the like are words of openness and mean "including but not limited to," and are used interchangeably therewith. The terms "or" and "as used herein refer to and are used interchangeably with the term" and/or "unless the context clearly indicates otherwise. The term "such as" as used herein refers to, and is used interchangeably with, the phrase "such as, but not limited to.

It is also noted that in the apparatus, devices and methods of the present application, the components or steps may be disassembled and/or assembled. Such decomposition and/or recombination should be considered as equivalent to the present application.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the application to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.

Claims

1. An automatic detection method of a fully automatic mechanical device, comprising:

2. The automatic detection method of a fully automatic mechanical device according to claim 1, wherein passing the plurality of detected frequency domain statistics and a waveform diagram of a sound signal of the device to be detected through a first Clip model to obtain a detection feature matrix comprises:

3. The automatic detection method of a fully automatic mechanical device according to claim 2, wherein inputting the plurality of detected frequency domain statistics into a sequence encoder of the first Clip model to obtain a detected frequency domain statistics feature vector, comprises:

4. The automatic inspection method of a fully automatic machine according to claim 3, wherein inputting a waveform diagram of a sound signal of the operation of the machine to be inspected into an image encoder of the first Clip model to obtain an inspected image feature vector comprises:

5. The automatic detection method of a fully automatic mechanical device according to claim 4, wherein inputting the detected image feature vector and the detected frequency domain statistical feature vector into a code optimizer of the first Clip model to obtain the detected feature matrix comprises:

wherein, the coding formula is:

6. The method of claim 5, wherein calculating a differential feature matrix between the detection feature matrix and the reference feature matrix comprises:

wherein, the difference formula is:

7. The method of claim 6, wherein performing the local feature distribution-based density domain probability on the differential feature matrix to obtain an optimized differential feature matrix comprises:

8. The method for automatically detecting the full-automatic mechanical equipment according to claim 7, wherein the step of passing the optimized differential feature matrix through a classifier to obtain a classification result, wherein the classification result indicates whether the equipment is operating normally or not, comprises the steps of:

9. An automatic inspection system for a fully automated mechanical device, comprising:

10. The automated inspection system of the fully automated mechanical equipment of claim 6, wherein the differencing module is configured to:

wherein, the difference formula is: