CN115783923A - Elevator fault mode identification system based on big data - Google Patents

Elevator fault mode identification system based on big data Download PDF

Info

Publication number
CN115783923A
CN115783923A CN202211517939.6A CN202211517939A CN115783923A CN 115783923 A CN115783923 A CN 115783923A CN 202211517939 A CN202211517939 A CN 202211517939A CN 115783923 A CN115783923 A CN 115783923A
Authority
CN
China
Prior art keywords
feature
frequency domain
matrix
elevator
vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211517939.6A
Other languages
Chinese (zh)
Other versions
CN115783923B (en
Inventor
张真明
施利平
张玉伟
顾家栋
王唐龙
吴瑶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Volkslift Schindler Elevator Co Ltd
Original Assignee
Volkslift Schindler Elevator Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Volkslift Schindler Elevator Co Ltd filed Critical Volkslift Schindler Elevator Co Ltd
Priority to CN202211517939.6A priority Critical patent/CN115783923B/en
Publication of CN115783923A publication Critical patent/CN115783923A/en
Application granted granted Critical
Publication of CN115783923B publication Critical patent/CN115783923B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
    • Y02B50/00—Energy efficient technologies in elevators, escalators and moving walkways, e.g. energy saving or recuperation technologies

Landscapes

  • Maintenance And Inspection Apparatuses For Elevators (AREA)
  • Indicating And Signalling Devices For Elevators (AREA)

Abstract

The application relates to the field of intelligent recognition, and particularly discloses an elevator fault pattern recognition system based on big data, which extracts comprehensive characteristic association distribution information of a sound signal and a vibration signal of an elevator to be detected on a time domain and a frequency domain respectively by adopting an artificial intelligence detection algorithm based on deep learning, and further fuses the sound characteristic distribution information and the vibration characteristic distribution information of the elevator to be detected so as to recognize the fault pattern of the elevator to be detected. Like this, can discern the judgement to the fault mode of elevator accurately, and then be convenient for maintenance personnel to maintain the elevator, guarantee the normal safe operation of elevator.

Description

Elevator fault mode identification system based on big data
Technical Field
The present application relates to the field of intelligent identification, and more particularly, to an elevator fault pattern identification system based on big data.
Background
With the continuous development of science and technology in China, the elevator is widely applied to our lives. However, various faults of the elevator often occur, which brings inconvenience to people, so that the maintenance of the elevator is very important, and the fault diagnosis of the elevator equipment is an important content of the maintenance of the elevator.
Along with the development of urban construction, the construction rate of the elevator is greatly increased, but because qualified elevator maintenance companies and professional elevator maintenance workers are fewer, the elevator cannot be maintained well, faults occur frequently, and the life and life safety of people are seriously influenced. The elevator has complicated electric mechanical equipment structure, various possible faults and more complicated reasons which can cause the faults, fault symptoms and fault reasons are not in one-to-one correspondence but are influenced in a cross mode, and the operation of the elevator is a gradually changing process from fault-free operation to operation with the faults, so that a maintenance worker can accurately judge the faults of the elevator to bring obstacles.
Therefore, an optimized elevator failure mode identification scheme is desired.
Disclosure of Invention
The present application is proposed to solve the above-mentioned technical problems. The embodiment of the application provides an elevator fault mode identification system based on big data, which extracts comprehensive characteristic associated distribution information of a sound signal and a vibration signal of an elevator to be detected on a time domain and a frequency domain respectively by adopting an artificial intelligence detection algorithm based on deep learning, and further fuses the sound characteristic distribution information and the vibration characteristic distribution information of the elevator to be detected so as to identify the fault mode of the elevator to be detected. Like this, can discern the judgement to the fault mode of elevator accurately, and then be convenient for maintenance personnel to maintain the elevator, guarantee the normal safe operation of elevator.
According to one aspect of the present application, there is provided a big data based elevator failure mode recognition system, comprising:
the fault data acquisition module is used for acquiring a detection vibration signal and a detection sound signal of the elevator to be detected in a preset time period, which are acquired by the vibration sensor and the sound sensor;
a frequency domain feature extraction module, configured to perform fourier transform on the detection vibration signal and the detection sound signal to obtain a plurality of first frequency domain feature statistics and a plurality of second frequency domain feature statistics;
the first joint coding module is used for inputting the detection vibration signal and the plurality of first frequency domain characteristic statistics values into a first Clip model to obtain an optimized vibration characteristic matrix;
the second joint coding module is used for inputting the detection sound signal and the plurality of second frequency domain characteristic statistic values into a second Clip model to obtain an optimized sound characteristic matrix;
the feature fusion module is used for fusing the optimized vibration feature matrix and the optimized sound feature matrix to obtain a classification feature matrix; and
and the elevator fault mode identification module is used for enabling the classification characteristic matrix to pass through a multi-label classifier to obtain a classification result, and the classification result is a probability value that the elevator to be detected has each fault.
In the above big data based elevator failure mode identification system, the first joint encoding module includes: a first frequency domain sequence feature extraction unit, configured to input the multiple first frequency domain feature statistics into a sequence encoder of the first Clip model to obtain a first frequency domain statistical feature vector; a first signal waveform feature extraction unit, configured to input a waveform diagram of the detected vibration signal into an image encoder of the first Clip model to obtain a detected vibration waveform feature vector; and the first joint encoding unit is used for optimizing the feature encoding of the characteristic vector of the detected vibration waveform based on the first frequency domain statistical feature vector to obtain the optimized vibration feature matrix.
In the above elevator failure mode identification system based on big data, the first frequency domain sequence feature extraction unit includes: a first scale feature extraction subunit, configured to pass the first frequency domain feature statistic through a first convolution layer of a trained sequence encoder of a Clip model to obtain a first neighborhood scale first frequency domain statistic feature vector, where the first convolution layer has a first one-dimensional convolution kernel of a first length; a second scale feature extraction subunit, configured to pass the first frequency domain feature statistic through a second convolution layer of a trained sequence encoder of the Clip model to obtain a second neighborhood scale first frequency domain statistic feature vector, where the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and the multi-scale cascading subunit is used for cascading the first neighborhood scale first frequency domain statistical feature vector and the second neighborhood scale first frequency domain statistical feature vector to obtain the first frequency domain statistical feature vector.
In the above big data based elevator failure mode identification system, the first scale feature extraction subunit is further configured to: performing one-dimensional convolution coding on the first frequency domain characteristic statistic value by using a first convolution layer of the trained sequence coder of the Clip model according to the following formula to obtain a first neighborhood scale first frequency domain statistical characteristic vector; wherein the formula is:
Figure BDA0003972532030000031
wherein a is the width of the first convolution kernel in the X direction, F (a) is a first convolution kernel parameter vector, G (X-a) is a local vector matrix operated with a convolution kernel function, w is the size of the first convolution kernel, and X represents the first frequency domain feature statistic value. The second scale feature extraction subunit is further configured to: performing one-dimensional convolution coding on the first frequency domain feature statistic value by using a second convolution layer of the trained sequence encoder of the Clip model according to the following formula to obtain a second neighborhood scale first frequency domain statistic feature vector; wherein the formula is:
Figure BDA0003972532030000032
wherein b is the width of the second convolution kernel in the X direction, F (b) is a second convolution kernel parameter vector, G (X-b) is a local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the first frequency domain feature statistic.
In the above elevator fault pattern recognition system based on big data, the first signal waveform feature extraction unit is further configured to: the layers of the image encoder using the first Clip model perform, in a forward pass of the layers, respectively: performing convolution processing on input data to obtain a convolution characteristic diagram; pooling the convolution feature map based on a feature matrix to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; and outputting the last layer of the image encoder of the first Clip model as the detected vibration waveform characteristic vector, and inputting the first layer of the image encoder of the first Clip model as a waveform diagram of the detected vibration signal.
In the above big data based elevator failure mode identification system, the first joint encoding unit is further configured to: optimizing feature encoding of the probe vibration waveform feature vector based on the first frequency-domain statistical feature vector using a joint encoder of the first Clip model to obtain the optimized vibration feature matrix; wherein the formula is:
Figure BDA0003972532030000033
wherein V s Representing the first frequency-domain statistical feature vector,
Figure BDA0003972532030000034
a transposed vector, V, representing the first frequency domain statistical feature vector b Representing a feature vector, M, of said detected vibration waveform b Representing the matrix of optimized vibration characteristics,
Figure BDA0003972532030000035
representing vector multiplication.
In the above big data based elevator failure mode recognition system, the second joint encoding module includes: a second frequency domain sequence feature extraction unit, configured to input the multiple second frequency domain feature statistics values into a sequence encoder of the second Clip model to obtain a second frequency domain statistical feature vector; a second signal waveform feature extraction unit, configured to input a waveform diagram of the sounding signal into an image encoder of the second Clip model to obtain a sounding waveform feature vector; and the second joint encoding unit is used for optimizing the feature codes of the detection sound waveform feature vector based on the second frequency domain statistical feature vector to obtain the optimized sound feature matrix.
In the above big data based elevator failure mode identification system, the feature fusion module is further configured to: fusing the optimized vibration feature matrix and the optimized sound feature matrix according to the following formula to obtain the classification feature matrix; wherein the formula is:
Figure BDA0003972532030000041
wherein M is 1 And M 2 Respectively, said optimized vibration feature matrix and said optimized sound feature matrix, M c Is the classification feature matrix, reLU (-) represents the ReLU activation function,
Figure BDA0003972532030000042
representing the multiplication of matrices and the division between the numerator matrix and denominator matrix as a division by location of the eigenvalues of the matrices, exp (-) represents the exponential operation of the matrices, which represents the calculation of a natural exponential function value raised to the power of the eigenvalues of each location in the matrix.
In the above elevator failure mode identification system based on big data, the elevator failure mode identification module includes: an expansion unit configured to expand the classification feature matrix into classification feature vectors based on row vectors or column vectors; a full-concatenation encoding unit, configured to perform full-concatenation encoding on the classification feature vector using a plurality of full-concatenation layers of the classifier to obtain an encoded classification feature vector; and the classification result generating unit is used for enabling the coded classification feature vector to pass through a Softmax classification function of the classifier to obtain the classification result.
According to another aspect of the present application, there is provided a big data-based elevator failure mode identification method, including:
acquiring a detection vibration signal and a detection sound signal of the elevator to be detected in a preset time period, which are acquired by a vibration sensor and a sound sensor;
performing Fourier transform on the detection vibration signal and the detection sound signal to obtain a plurality of first frequency domain feature statistics and a plurality of second frequency domain feature statistics;
inputting the detection vibration signal and the plurality of first frequency domain characteristic statistics values into a first Clip model to obtain an optimized vibration characteristic matrix;
inputting the detection sound signal and the plurality of second frequency domain feature statistical values into a second Clip model to obtain an optimized sound feature matrix;
fusing the optimized vibration characteristic matrix and the optimized sound characteristic matrix to obtain a classification characteristic matrix; and
and passing the classification characteristic matrix through a multi-label classifier to obtain a classification result, wherein the classification result is a probability value of each fault of the elevator to be detected.
According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory having stored therein computer program instructions which, when executed by the processor, cause the processor to perform the big data based elevator failure mode identification method as described above.
According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the big data based elevator failure mode identification method as described above.
Compared with the prior art, the elevator fault mode recognition system based on the big data extracts comprehensive characteristic association distribution information of the sound signal and the vibration signal of the elevator to be detected in the time domain and the frequency domain respectively by adopting an artificial intelligence detection algorithm based on deep learning, and further fuses the sound characteristic distribution information and the vibration characteristic distribution information of the elevator to be detected so as to recognize the fault mode of the elevator to be detected. Like this, can discern the judgement to the fault mode of elevator accurately, and then be convenient for maintenance personnel to maintain the elevator, guarantee the normal safe operation of elevator.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally represent like parts or steps.
Fig. 1 is a diagram of an application scenario of a big data based elevator failure mode identification system according to an embodiment of the present application;
fig. 2 is a block diagram of a big data based elevator failure mode identification system according to an embodiment of the present application;
fig. 3 is a system architecture diagram of a big data based elevator failure mode identification system according to an embodiment of the present application;
fig. 4 is a block diagram of a first joint encoding module in a big data based elevator failure mode identification system according to an embodiment of the present application;
fig. 5 is a block diagram of a first frequency-domain series feature extraction unit in a big-data based elevator fault pattern recognition system according to an embodiment of the present application;
fig. 6 is a flow chart of image encoding in a big data based elevator failure mode identification system according to an embodiment of the present application;
fig. 7 is a block diagram of an elevator failure mode identification module in a big data based elevator failure mode identification system according to an embodiment of the present application;
fig. 8 is a flow chart of a big data based elevator failure mode identification method according to an embodiment of the present application;
fig. 9 is a block diagram of an electronic device according to an embodiment of the application.
Detailed Description
Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and that the present application is not limited by the example embodiments described herein.
Summary of the application
As the background art mentioned above, with the continuous development of science and technology in our country, elevators are widely used in our lives. However, various faults of the elevator often occur, which brings inconvenience to people, so that the maintenance of the elevator is very important, and the fault diagnosis of the elevator equipment is an important content of the maintenance of the elevator.
Along with the development of urban construction, the construction rate of the elevator is greatly increased, but due to the fact that qualified elevator maintenance companies and professional elevator maintenance workers are few, the elevator cannot be well maintained, faults occur frequently, and life safety of people are seriously affected. The elevator has complicated electric mechanical equipment structure, various possible faults and more complicated reasons which can cause the faults, fault symptoms and fault reasons are not in one-to-one correspondence but are influenced in a cross mode, and the operation of the elevator is a gradually changing process from fault-free operation to operation with the faults, so that a maintenance worker can accurately judge the faults of the elevator to bring obstacles. Therefore, an optimized elevator failure mode identification scheme is desired.
At present, deep learning and neural networks have been widely applied in the fields of computer vision, natural language processing, speech signal processing, and the like. In addition, deep learning and neural networks also exhibit a level close to or even exceeding that of humans in the fields of image classification, object detection, semantic segmentation, text translation, and the like.
In recent years, deep learning and the development of neural networks provide new solutions and solutions for pattern recognition of elevator faults.
Accordingly, the elevator can be used for fault identification by considering that the elevator generates a fixed vibration mode characteristic in the normal operation process, and the vibration of the elevator generates a sound signal which can reflect part of state characteristic information of the elevator, so that the expression of the vibration characteristic of the elevator is strengthened by the sound characteristic, and the precision of elevator fault diagnosis can be obviously improved. Based on the above, in the technical scheme of the application, the comprehensive characteristic associated distribution information of the sound signal and the vibration signal of the elevator to be detected in the time domain and the frequency domain is respectively extracted by adopting an artificial intelligence detection algorithm based on deep learning, and the sound characteristic distribution information and the vibration characteristic distribution information of the elevator to be detected are further fused, so that the fault mode identification of the elevator to be detected is performed. Like this, can discern the judgement to the fault mode of elevator accurately, and then be convenient for maintenance personnel to maintain the elevator, guarantee the normal safe operation of elevator.
Specifically, in the technical scheme of the application, firstly, a detection vibration signal and a detection sound signal of the elevator to be detected in a preset time period are respectively collected through a vibration sensor and a sound sensor. Then, considering that when the time domain characteristics of the detection vibration signal and the detection sound signal are used for identifying the fault mode of the elevator to be detected, the time domain characteristics contain more environment-caused interference characteristic information, which can cause serious influence on the detection result, and therefore, the frequency domain statistical characteristics of the detection signal are further combined to improve the detection accuracy. That is, specifically, the detection vibration signal and the detection sound signal are respectively fourier-transformed to obtain a plurality of first frequency-domain feature statistical values and a plurality of second frequency-domain feature statistical values.
Then, for vibration feature extraction of the detected vibration signal, a first Clip model comprising a sequence encoder and an image encoder is used to process the detected vibration signal and the plurality of first frequency domain feature statistics values respectively to obtain an optimized vibration feature matrix. Specifically, feature mining is performed on a plurality of first frequency domain feature statistics values of the vibration detection signal by using a multi-scale neighborhood feature extraction module with a sequence encoder of the first Clip model, so as to extract multi-scale neighborhood associated feature information of the frequency domain statistics features of the plurality of vibration detection signals, thereby obtaining a first frequency domain statistical feature vector. And performing feature mining on the oscillogram of the detection vibration signal by using an image encoder of the first Clip model to extract time-domain implicit feature distribution information of the oscillogram of the detection vibration signal. Then, coding optimization expression of image attributes is carried out on time domain implicit feature distribution of the waveform diagram of the detection vibration signal based on the multi-scale neighborhood correlation features of the frequency domain statistical feature values of the detection vibration signal, so that the optimized vibration feature matrix is obtained. Therefore, the obtained optimized vibration characteristic matrix not only contains the frequency domain characteristic content of the detection vibration signal, but also reflects the change rule characteristic of the frequency domain content along with time, and the accuracy of the elevator fault mode is improved.
Similarly, for the sound feature extraction of the detection sound signal, it is considered that the periodic feature information of the detection sound signal and the periodic feature of the detection vibration signal have similar regular distribution, and therefore, in the technical solution of the present application, the Clip model is also used to encode the detection sound signal. Specifically, the detection sound signal and the plurality of second frequency domain feature statistical values are input into a second Clip model to obtain an optimized sound feature matrix, and then image attribute coding optimization is performed on the time domain implicit feature of the waveform diagram of the detection sound signal based on the multi-scale neighborhood correlation feature of the frequency domain statistical feature values of the detection sound signal to obtain the optimized sound feature matrix.
Further, the feature information in the optimized vibration feature matrix and the optimized sound feature matrix is fused to obtain a classification feature matrix, and classification processing is performed in a classifier, so that a classification result for indicating the probability value that the elevator to be detected has each fault can be obtained. Therefore, the fault mode of the elevator can be accurately identified and judged, and the elevator can be conveniently maintained by a maintenance worker.
Particularly, in the technical solution of the present application, when the optimized vibration feature matrix and the optimized sound feature matrix are fused to obtain the classification feature matrix, since the detection sound signal itself has a certain randomness with respect to the detection vibration signal, after the extraction of the signal waveform semantic features associated with the frequency domain statistical features is performed through the CLIP model, there is a negative correlation relationship between corresponding positions between the extracted optimized vibration feature matrix and the optimized sound feature matrix, thereby affecting the fusion effect of bit-by-bit fusion of the optimized vibration feature matrix and the optimized sound feature matrix.
Therefore, the applicant of the present application fuses the optimized vibration feature matrix and the optimized sound feature matrix in a full orthographic nonlinear re-weighting manner, which is expressed as:
Figure BDA0003972532030000081
M 1 and M 2 Respectively, the optimized vibration feature matrix and the optimized sound feature matrix, mc is the classification feature matrix, and the division between the numerator matrix and the denominator matrix is the division by position of the matrix feature values.
Here, the full forward projection nonlinear re-weighting guarantees full positive of the projection by the ReLU function to avoid aggregating negatively correlated information, and at the same time introduces a nonlinear re-weighting mechanism to cluster the eigenvalue distributions of the optimized vibration feature matrix and the optimized sound feature matrix with respect to each other, so that the intrinsic structure of the classification feature matrix can penalize distant connections to strengthen local coupling. In this way, a synergistic effect of spatial feature transformation (feature transform) corresponding to full orthographic re-weighting of the optimized vibration feature matrix and the optimized sound feature matrix in a high-dimensional feature space is achieved, and a fusion effect of the optimized vibration feature matrix and the optimized sound feature matrix is improved. Like this, can discern the judgement to the fault mode of elevator accurately, and then be convenient for maintenance personnel to maintain the elevator, guarantee the normal safe operation of elevator.
Based on this, this application has proposed elevator fault pattern recognition system based on big data, it includes: the fault data acquisition module is used for acquiring a detection vibration signal and a detection sound signal of the elevator to be detected in a preset time period, which are acquired by the vibration sensor and the sound sensor; a frequency domain feature extraction module, configured to perform fourier transform on the detection vibration signal and the detection sound signal to obtain a plurality of first frequency domain feature statistics and a plurality of second frequency domain feature statistics; the first joint coding module is used for inputting the detection vibration signal and the plurality of first frequency domain characteristic statistics values into a first Clip model to obtain an optimized vibration characteristic matrix; the second joint coding module is used for inputting the detection sound signal and the plurality of second frequency domain characteristic statistic values into a second Clip model to obtain an optimized sound characteristic matrix; the feature fusion module is used for fusing the optimized vibration feature matrix and the optimized sound feature matrix to obtain a classification feature matrix; and the elevator fault mode identification module is used for enabling the classification characteristic matrix to pass through a multi-label classifier to obtain a classification result, and the classification result is a probability value of each fault of the elevator to be detected.
Fig. 1 is a view of an application scenario of a big data-based elevator failure mode identification system according to an embodiment of the present application. As shown in fig. 1, in this application scenario, a detection vibration signal (e.g., S1 as illustrated in fig. 1) of an elevator to be detected (e.g., E as illustrated in fig. 1) for a predetermined period of time is collected by a vibration sensor (e.g., V1 as illustrated in fig. 1) and a detection sound signal (e.g., S3 as illustrated in fig. 1) of the elevator to be detected for a predetermined period of time is collected by a sound sensor (e.g., S2 as illustrated in fig. 1). Then, the signals are input into a server (e.g., S in fig. 1) deployed with an elevator fault pattern recognition algorithm based on big data, wherein the server can process the input signals by the elevator fault pattern recognition algorithm based on big data to generate a classification result, and the classification result is a probability value that the elevator to be detected has each fault.
Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.
Exemplary System
Fig. 2 is a block diagram of a big data based elevator failure mode identification system according to an embodiment of the present application. As shown in fig. 2, a big data based elevator failure mode recognition system 300 according to an embodiment of the present application includes: a fault data acquisition module 310; a frequency domain feature extraction module 320; a first joint encoding module 330; a second joint encoding module 340; a feature fusion module; and an elevator failure mode identification module 360.
The fault data acquisition module 310 is configured to acquire a detection vibration signal and a detection sound signal of the elevator to be detected in a predetermined time period, which are acquired by a vibration sensor and a sound sensor; the frequency domain feature extraction module 320 is configured to perform fourier transform on the detection vibration signal and the detection sound signal to obtain a plurality of first frequency domain feature statistics and a plurality of second frequency domain feature statistics; the first joint encoding module 330 is configured to input the detected vibration signal and the plurality of first frequency-domain feature statistics into a first Clip model to obtain an optimized vibration feature matrix; the second joint encoding module 340 is configured to input the probe sound signal and the plurality of second frequency-domain feature statistics into a second Clip model to obtain an optimized sound feature matrix; the feature fusion module 350 is configured to fuse the optimized vibration feature matrix and the optimized sound feature matrix to obtain a classification feature matrix; and the elevator fault mode identification module 360 is configured to pass the classification feature matrix through a multi-label classifier to obtain a classification result, where the classification result is a probability value that the elevator to be detected has each fault.
Fig. 3 is a system architecture diagram of a big data based elevator failure mode identification system according to an embodiment of the present application. As shown in fig. 3, in the network architecture, firstly, the fault data acquisition module 310 acquires a detection vibration signal and a detection sound signal of the elevator to be detected, which are acquired by the vibration sensor and the sound sensor, in a predetermined time period; next, the frequency domain feature extraction module 320 performs fourier transform on the detected vibration signal and the detected sound signal acquired by the fault data acquisition module 310 to obtain a plurality of first frequency domain feature statistics values and a plurality of second frequency domain feature statistics values; the first joint encoding module 330 inputs the detected vibration signal obtained by the fault data acquisition module 310 and the plurality of first frequency domain feature statistical values obtained by the frequency domain feature extraction module 320 into a first Clip model to obtain an optimized vibration feature matrix; meanwhile, the second joint encoding module 340 inputs the detected sound signal obtained by the fault data acquisition module 310 and the plurality of second frequency domain feature statistical values obtained by the frequency domain feature extraction module 320 into a second Clip model to obtain an optimized sound feature matrix; then, the feature fusion module 350 fuses the optimized vibration feature matrix obtained by the first joint encoding module 330 and the optimized sound feature matrix obtained by the second joint encoding module 340 to obtain a classification feature matrix; furthermore, the elevator fault pattern recognition module 360 passes the classification feature matrix through a multi-label classifier to obtain a classification result, where the classification result is a probability value that the elevator to be detected has each fault.
Specifically, during the operation of the elevator fault pattern recognition system 300 based on big data, the fault data acquisition module 310 is configured to acquire the detected vibration signal and the detected sound signal of the elevator to be detected, which are acquired by the vibration sensor and the sound sensor, in a predetermined time period. In consideration of the fact that the elevator generates a fixed vibration mode characteristic in the normal operation process, the vibration characteristic of the elevator can be used for fault identification, and the vibration of the elevator generates a sound signal which can reflect part of state characteristic information of the elevator, so that the expression of the vibration characteristic of the elevator is enhanced by using the sound characteristic, the fault diagnosis accuracy of the elevator can be improved, and therefore in a specific example of the application, the detection vibration signal of the elevator to be detected in a preset time period can be collected by a vibration sensor and the detection sound signal of the elevator to be detected in the preset time period can be collected by a sound sensor.
Specifically, in the operation process of the big data based elevator fault pattern recognition system 300, the frequency domain feature extraction module 320 is configured to perform fourier transform on the detection vibration signal and the detection sound signal to obtain a plurality of first frequency domain feature statistics and a plurality of second frequency domain feature statistics. Considering that when the time domain characteristics of the detection vibration signal and the detection sound signal are used for identifying the fault mode of the elevator to be detected, the time domain characteristics contain more environment-caused interference characteristic information, which can cause serious influence on the detection result, and therefore, the frequency domain statistical characteristics of the detection signal are further combined to improve the detection accuracy. That is, specifically, the detection vibration signal and the detection sound signal are respectively fourier-transformed to obtain a plurality of first frequency-domain feature statistical values and a plurality of second frequency-domain feature statistical values.
Specifically, during the operation of the big data based elevator failure mode identification system 300, the first joint encoding module 330 is configured to input the detected vibration signal and the plurality of first frequency-domain feature statistics into a first Clip model to obtain an optimized vibration feature matrix. That is, for vibration feature extraction of a detected vibration signal, a first Clip model including a sequence encoder and an image encoder is used to process the detected vibration signal and the plurality of first frequency-domain feature statistics values, respectively, to obtain an optimized vibration feature matrix. Specifically, feature mining is performed on a plurality of first frequency domain feature statistics values of the vibration detection signal by using a multi-scale neighborhood feature extraction module with a sequence encoder of the first Clip model, so as to extract multi-scale neighborhood associated feature information of the frequency domain statistics features of the plurality of vibration detection signals, thereby obtaining a first frequency domain statistical feature vector. And performing feature mining on the oscillogram of the detection vibration signal by using an image encoder of the first Clip model to extract time-domain implicit feature distribution information of the oscillogram of the detection vibration signal. Then, coding optimization expression of image attributes is carried out on time domain implicit feature distribution of the waveform diagram of the detection vibration signal based on the multi-scale neighborhood correlation features of the frequency domain statistical feature values of the detection vibration signal, so that the optimized vibration feature matrix is obtained. Therefore, the obtained optimized vibration characteristic matrix not only contains the frequency domain characteristic content of the detection vibration signal, but also reflects the change rule characteristic of the frequency domain content along with time, and the accuracy of the elevator fault mode is improved. More specifically, in a specific example of the present application, the inputting the detected vibration signal and the plurality of first frequency-domain feature statistics into a first Clip model to obtain an optimized vibration feature matrix includes: inputting the plurality of first frequency domain feature statistics into a sequence encoder of the first Clip model to obtain a first frequency domain feature vector, inputting a oscillogram of the detection vibration signal into an image encoder of the first Clip model to obtain a detection vibration waveform feature vector, and optimizing feature coding of the detection vibration waveform feature vector based on the first frequency domain feature vector to obtain the optimized vibration feature matrix.
Fig. 4 is a block diagram of a first joint encoding module in a big data based elevator failure mode identification system according to an embodiment of the present application. As shown in fig. 4, the first joint encoding module 330 includes: a first frequency-domain sequence feature extracting unit 331, configured to input the multiple first frequency-domain feature statistics values into a sequence encoder of the first Clip model to obtain a first frequency-domain statistical feature vector; a first signal waveform feature extraction unit 332, configured to input a waveform diagram of the detected vibration signal into an image encoder of the first Clip model to obtain a detected vibration waveform feature vector; a first joint encoding unit 333, configured to optimize feature encoding of the detected vibration waveform feature vector based on the first frequency-domain statistical feature vector to obtain the optimized vibration feature matrix. The first frequency domain sequence feature extraction unit 331 includes: passing the first frequency domain feature statistic through a first convolution layer of a sequence encoder of a trained Clip model to obtain a first neighborhood scale first frequency domain statistic feature vector, wherein the first convolution layer has a first one-dimensional convolution kernel with a first length; passing the first frequency domain feature statistic through a second convolution layer of a sequence encoder of a trained Clip model to obtain a second neighborhood scale first frequency domain statistic feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and cascading the first neighborhood scale first frequency domain statistical feature vector and the second neighborhood scale first frequency domain statistical feature vector to obtain the first frequency domain statistical feature vector. More specifically, in a specific example of the present application, the first signal waveform feature extraction unit 332 is further configured to: the layers of the image encoder using the first Clip model perform, in a forward pass of the layers, respectively: carrying out convolution processing on input data to obtain a convolution characteristic diagram; pooling the convolution feature map based on a feature matrix to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the first CThe output of the last layer of the image encoder of the lip model is the detected vibration waveform feature vector, and the input of the first layer of the image encoder of the first Clip model is the waveform diagram of the detected vibration signal. More specifically, the first joint encoding unit 333 includes: optimizing feature encoding of the probe vibration waveform feature vector based on the first frequency-domain statistical feature vector using a joint encoder of the first Clip model to obtain the optimized vibration feature matrix; wherein the formula is:
Figure BDA0003972532030000131
wherein V s Representing the first frequency-domain statistical feature vector,
Figure BDA0003972532030000132
a transposed vector, V, representing the first frequency domain statistical feature vector b Representing a feature vector, M, of said detected vibration waveform b A matrix representing the optimized vibration characteristics is generated,
Figure BDA0003972532030000133
representing vector multiplication.
Fig. 5 is a block diagram of a first frequency-domain series feature extraction unit in a big-data based elevator fault pattern recognition system according to an embodiment of the present application. As shown in fig. 5, the first frequency-domain sequence feature extraction unit 331 includes: a first scale feature extraction subunit 3311, configured to pass the first frequency domain feature statistic through a first convolution layer of a trained sequence encoder of a Clip model to obtain a first neighborhood scale first frequency domain statistic feature vector, where the first convolution layer has a first one-dimensional convolution kernel of a first length; a second scale feature extraction subunit 3312, configured to pass the first frequency domain feature statistic through a second convolution layer of a trained sequence encoder of the Clip model to obtain a second neighborhood scale first frequency domain statistic feature vector, where the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and a multi-scale concatenation subunit 3313, configured to concatenate the first neighborhood scale first frequency domain statistical feature vector and the second neighborhood scale first frequency domain statistical feature vector to obtain the first frequency domain statistical feature vector. More specifically, in one specific example, the first scale feature extraction subunit 3311 includes: performing one-dimensional convolution coding on the first frequency domain characteristic statistic value by using a first convolution layer of the trained sequence coder of the Clip model according to the following formula to obtain a first neighborhood scale first frequency domain statistical characteristic vector; wherein the formula is:
Figure BDA0003972532030000134
wherein a is the width of the first convolution kernel in the X direction, F (a) is a first convolution kernel parameter vector, G (X-a) is a local vector matrix operated with a convolution kernel function, w is the size of the first convolution kernel, and X represents the first frequency domain feature statistic value; the second scale feature extraction subunit 3312 includes: performing one-dimensional convolution coding on the first frequency domain feature statistic value by using a second convolution layer of the trained sequence encoder of the Clip model according to the following formula to obtain a second neighborhood scale first frequency domain statistic feature vector; wherein the formula is:
Figure BDA0003972532030000141
wherein b is the width of the second convolution kernel in the X direction, F (b) is a second convolution kernel parameter vector, G (X-b) is a local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the first frequency domain feature statistic. Further, the multi-scale cascading subunit 3313 includes: cascading the first neighborhood scale first frequency domain statistical feature vector and the second neighborhood scale first frequency domain statistical feature vector according to the following formula to obtain the first frequency domain statistical feature vector; wherein the formula is: v c ＝Concat[V 1 ，V 2 ]Wherein V is 1 Represents the aboveFirst neighborhood scale first frequency domain statistical feature vector, V 2 Representing the first frequency domain statistical feature vector, concat [, ] of the second neighborhood scale]Representing a cascade function, V c Representing the first frequency-domain statistical feature vector.
Fig. 6 is a flowchart of image coding in a big data based elevator failure mode identification system according to an embodiment of the present application. As shown in fig. 6, the image encoding process includes: the layers of the image encoder using the first Clip model perform, in a forward pass of the layers, respectively: s210, performing convolution processing on input data to obtain a convolution characteristic diagram; s220, pooling the convolution feature map based on a feature matrix to obtain a pooled feature map; and S230, carrying out nonlinear activation on the pooled feature map to obtain an activated feature map; and outputting the last layer of the image encoder of the first Clip model as the detected vibration waveform characteristic vector, and inputting the first layer of the image encoder of the first Clip model as a waveform diagram of the detected vibration signal.
Specifically, during the operation of the big data based elevator failure mode identification system 300, the second joint encoding module 340 is configured to input the probing sound signal and the plurality of second frequency-domain feature statistics into a second Clip model to obtain an optimized sound feature matrix. It should be understood that, also for the sound feature extraction of the detection sound signal, it is considered that the periodic feature information of the detection sound signal and the periodic feature of the detection vibration signal have similar regular distribution, and therefore, in the technical solution of the present application, a Clip model is also used for encoding the detection sound signal. Specifically, the detection sound signal and the plurality of second frequency domain feature statistics are input into a second Clip model to obtain an optimized sound feature matrix, and then image attribute coding optimization is performed on the time domain implicit features of the detection sound signal oscillogram based on the multi-scale neighborhood correlation features of the frequency domain statistics of the detection sound signal to obtain the optimized sound feature matrix. More specifically, the inputting the probe sound signal and the plurality of second frequency-domain feature statistics into a second Clip model to obtain an optimized sound feature matrix includes: inputting the plurality of second frequency domain feature statistics into a sequence encoder of the second Clip model to obtain a second frequency domain statistical feature vector; inputting the oscillogram of the detection sound signal into an image coder of the second Clip model to obtain a detection sound waveform characteristic vector; and optimizing feature codes of the detection sound waveform feature vector based on the second frequency domain statistical feature vector to obtain the optimized sound feature matrix.
Specifically, during the operation of the big data based elevator fault pattern recognition system 300, the feature fusion module 350 is configured to fuse the optimized vibration feature matrix and the optimized sound feature matrix to obtain a classification feature matrix. Particularly, in the technical solution of the present application, when the optimized vibration feature matrix and the optimized sound feature matrix are fused to obtain the classification feature matrix, since the detection sound signal itself has a certain randomness with respect to the detection vibration signal, after the extraction of the signal waveform semantic features associated with the frequency domain statistical features is performed through the CLIP model, there is a negative correlation relationship between corresponding positions between the extracted optimized vibration feature matrix and the optimized sound feature matrix, thereby affecting the fusion effect of bit-by-bit fusion of the optimized vibration feature matrix and the optimized sound feature matrix. Therefore, the applicant of the present application fuses the optimized vibration feature matrix and the optimized sound feature matrix in a full orthographic nonlinear re-weighting manner, which is expressed as:
Figure BDA0003972532030000151
wherein M is 1 And M 2 Respectively, said optimized vibration feature matrix and said optimized sound feature matrix, M c Is the classification feature matrix, reLU (-) represents the ReLU activation function,
Figure BDA0003972532030000152
representing the multiplication of matrices and the division between the numerator matrix and denominator matrix as a division by location of the eigenvalues of the matrices, exp (-) represents the exponential operation of the matrices, which represents the calculation of a natural exponential function value raised to the power of the eigenvalues of each location in the matrix. Here, the full forward projection nonlinear re-weighting guarantees full positive of the projection by the ReLU function to avoid aggregating negatively correlated information, and at the same time introduces a nonlinear re-weighting mechanism to cluster the eigenvalue distributions of the optimized vibration feature matrix and the optimized sound feature matrix with respect to each other, so that the intrinsic structure of the classification feature matrix can penalize distant connections to strengthen local coupling. In this way, the synergistic effect of the spatial feature transformation (feature transform) corresponding to the full orthographic re-weighting of the optimized vibration feature matrix and the optimized sound feature matrix in the high-dimensional feature space is realized, and the fusion effect of the optimized vibration feature matrix and the optimized sound feature matrix is improved. Like this, can discern the judgement to the fault mode of elevator accurately, and then be convenient for maintenance personnel to maintain the elevator, guarantee the normal safe operation of elevator.
Specifically, in the operation process of the elevator fault pattern recognition system 300 based on big data, the elevator fault pattern recognition module 360 is configured to pass the classification feature matrix through a multi-label classifier to obtain a classification result, where the classification result is a probability value that an elevator to be detected has each fault. It should be understood that the classification feature matrix is classified by the classifier, so that a classification result indicating the probability value that the elevator to be detected has each fault can be obtained. Therefore, the fault mode of the elevator can be accurately identified and judged, and the elevator can be conveniently maintained by a maintenance worker. In a specific example of the present application, the passing the classification feature matrix through a multi-label classifier to obtain a classification result includes: processing the classification feature matrix using the classifier to obtain a classification result with the following formula:
Figure BDA0003972532030000161
where O is the output result matrix, W i And b i Respectively, a weight and a bias matrix corresponding to the ith classification, exp (-) represents an exponential operation of the matrix, and the exponential operation of the matrix represents a natural exponential function value taking the characteristic value of each position in the matrix as a power. In particular, the classifier includes a plurality of fully-connected layers and a Softmax layer cascaded with a last fully-connected layer of the plurality of fully-connected layers. In the classification processing of the classifier, the classification feature matrix is first projected as a vector, for example, in a specific example, the classification feature matrix is expanded as a classification feature vector along a row vector or a column vector; then, carrying out multiple full-connection coding on the classification characteristic vector by using multiple full-connection layers of the classifier to obtain a coding classification characteristic vector; and then, inputting the coding classification feature vector into a Softmax layer of the classifier, namely, classifying the coding classification feature vector by using the Softmax classification function to obtain a classification result, wherein the classification result is a probability value that the elevator to be detected has each fault.
Fig. 7 is a block diagram of an elevator fault pattern recognition module in a big data based elevator fault pattern recognition system according to an embodiment of the present application. As shown in fig. 7, the elevator failure mode recognition module 360 includes: an unfolding unit 361, configured to unfold the classification feature matrix into a classification feature vector based on a row vector or a column vector; a full-concatenation encoding unit 362 for performing full-concatenation encoding on the classification feature vector using a plurality of full-concatenation layers of the classifier to obtain an encoded classification feature vector; and a classification result generating unit 363, configured to pass the encoded classification feature vector through a Softmax classification function of the classifier to obtain the classification result.
In summary, the elevator fault pattern recognition system 300 based on big data according to the embodiment of the present application is illustrated, which extracts the comprehensive characteristic association distribution information of the sound signal and the vibration signal of the elevator to be detected in the time domain and the frequency domain by using the artificial intelligence detection algorithm based on deep learning, and further fuses the sound characteristic distribution information and the vibration characteristic distribution information of the elevator to be detected, so as to perform the fault pattern recognition of the elevator to be detected. Like this, can discern the judgement to the fault mode of elevator accurately, and then be convenient for maintenance personnel to maintain the elevator, guarantee the normal safe operation of elevator.
As described above, the big data based elevator failure mode recognition system according to the embodiment of the present application can be implemented in various terminal devices. In one example, the big data based elevator failure mode identification system 300 according to an embodiment of the present application may be integrated into a terminal device as one software module and/or hardware module. For example, the big data based elevator failure mode identification system 300 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the big data based elevator failure mode identification system 300 could equally be one of the numerous hardware modules of the terminal device.
Alternatively, in another example, the big data based elevator failure mode recognition system 300 and the terminal device may also be separate devices, and the big data based elevator failure mode recognition system 300 may be connected to the terminal device through a wired and/or wireless network and transmit interactive information according to an agreed data format.
Exemplary method
Fig. 8 is a flowchart of a big data based elevator failure mode identification method according to an embodiment of the present application. As shown in fig. 8, the big data based elevator fault pattern recognition method according to the embodiment of the present application includes the steps of: s110, acquiring a detection vibration signal and a detection sound signal of the elevator to be detected in a preset time period, which are acquired by a vibration sensor and a sound sensor; s120, performing Fourier transform on the detection vibration signal and the detection sound signal to obtain a plurality of first frequency domain characteristic statistical values and a plurality of second frequency domain characteristic statistical values; s130, inputting the detection vibration signal and the plurality of first frequency domain characteristic statistical values into a first Clip model to obtain an optimized vibration characteristic matrix; s140, inputting the detection sound signal and the plurality of second frequency domain feature statistical values into a second Clip model to obtain an optimized sound feature matrix; s150, fusing the optimized vibration characteristic matrix and the optimized sound characteristic matrix to obtain a classification characteristic matrix; and S160, the classification characteristic matrix is processed by a multi-label classifier to obtain a classification result, and the classification result is the probability value of each fault of the elevator to be detected.
In one example, in the above elevator fault pattern recognition method based on big data, the step S130 includes: inputting the plurality of first frequency domain feature statistics into a sequence encoder of the first Clip model to obtain a first frequency domain statistical feature vector; inputting the oscillogram of the detection vibration signal into an image encoder of the first Clip model to obtain a detection vibration waveform feature vector; optimizing feature codes of the detected vibration waveform feature vector based on the first frequency domain statistical feature vector to obtain the optimized vibration feature matrix. Wherein the inputting the plurality of first frequency-domain feature statistics into the sequence encoder of the first Clip model to obtain a first frequency-domain statistical feature vector comprises: obtaining a first neighborhood scale first frequency domain statistical feature vector by passing the first frequency domain feature statistical value through a first convolution layer of a sequence encoder of a trained Clip model, wherein the first convolution layer has a first one-dimensional convolution kernel with a first length; passing the first frequency domain feature statistic through a second convolution layer of a sequence encoder of a trained Clip model to obtain a second neighborhood scale first frequency domain statistic feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and cascading the first neighborhood scale first frequency domain statistical feature vector and the second neighborhood scale first frequency domain statistical feature vector to obtain the first frequency domain statistical feature vector. More specifically, the inputting the plurality of first frequency-domain feature statistics into the sequence encoder of the first Clip model to obtain a first frequency-domain statistical feature vector includes: performing one-dimensional convolution coding on the first frequency domain characteristic statistic value by using a first convolution layer of the trained sequence coder of the Clip model according to the following formula to obtain a first neighborhood scale first frequency domain statistical characteristic vector; wherein the formula is:
Figure BDA0003972532030000181
wherein a is the width of the first convolution kernel in the X direction, F (a) is a first convolution kernel parameter vector, G (X-a) is a local vector matrix operated with a convolution kernel function, w is the size of the first convolution kernel, and X represents the first frequency domain feature statistic value; and the step of obtaining a second neighborhood scale first frequency domain statistical feature vector by passing the first frequency domain feature statistical value through a second convolution layer of a trained sequence encoder of the Clip model comprises the following steps: performing one-dimensional convolution coding on the first frequency domain feature statistic value by using a second convolution layer of the trained sequence encoder of the Clip model according to the following formula to obtain a second neighborhood scale first frequency domain statistic feature vector; wherein the formula is:
Figure BDA0003972532030000182
wherein b is the width of the second convolution kernel in the X direction, F (b) is a second convolution kernel parameter vector, G (X-b) is a local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the first frequency domain feature statistic. More specifically, the inputting the waveform diagram of the detected vibration signal into the image encoder of the first Clip model to obtain the detected vibration waveform feature vector includes: the layers of the image encoder using the first Clip model perform, in a forward pass of the layers, respectively: carrying out convolution processing on input data to obtain a convolution characteristic diagram; pooling the convolution feature map based on a feature matrix to obtain a pooled feature map; and, applying a non-linear excitation to the pooled feature mapObtaining an activation characteristic diagram; and outputting the last layer of the image encoder of the first Clip model as the detected vibration waveform characteristic vector, and inputting the first layer of the image encoder of the first Clip model as a waveform diagram of the detected vibration signal. The optimizing the feature coding of the detected vibration waveform feature vector based on the first frequency domain statistical feature vector to obtain the optimized vibration feature matrix includes: optimizing feature encoding of the probe vibration waveform feature vector based on the first frequency-domain statistical feature vector using a joint encoder of the first Clip model to obtain the optimized vibration feature matrix; wherein the formula is:
Figure BDA0003972532030000191
wherein V s Representing the first frequency-domain statistical feature vector,
Figure BDA0003972532030000192
a transposed vector, V, representing the first frequency domain statistical feature vector b Representing a feature vector, M, of said detected vibration waveform b Representing the matrix of optimized vibration characteristics,
Figure BDA0003972532030000193
representing vector multiplication.
In one example, in the above elevator fault pattern recognition method based on big data, the step S140 includes: inputting the plurality of second frequency domain feature statistics into a sequence encoder of the second Clip model to obtain a second frequency domain statistical feature vector; inputting the oscillogram of the detection sound signal into an image encoder of the second Clip model to obtain a detection sound waveform characteristic vector; and optimizing feature codes of the feature vectors of the detection sound waveform based on the second frequency domain statistical feature vector to obtain the optimized sound feature matrix.
In one example, in the above elevator fault pattern recognition method based on big data, the step S150 includes: fusing the optimized vibration feature matrix and the optimized sound feature matrix according to the following formula to obtain the classification feature matrix; wherein the formula is:
Figure BDA0003972532030000194
wherein M is 1 And M 2 Respectively, said optimized vibration characteristic matrix and said optimized sound characteristic matrix, M c Is the classification feature matrix, reLU (-) represents the ReLU activation function,
Figure BDA0003972532030000195
representing the multiplication of matrices and the division between the numerator matrix and denominator matrix as a division by location of the eigenvalues of the matrices, exp (-) represents the exponential operation of the matrices, which represents the calculation of a natural exponential function value raised to the power of the eigenvalues of each location in the matrix.
In one example, in the above elevator fault pattern recognition method based on big data, the step S160 includes: expanding the classification feature matrix into a classification feature vector based on a row vector or a column vector; performing full-join encoding on the classification feature vectors using a plurality of full-join layers of the classifier to obtain encoded classification feature vectors; and passing the encoding classification feature vector through a Softmax classification function of the classifier to obtain the classification result.
In summary, the elevator fault pattern recognition method based on big data according to the embodiment of the application is clarified, and the fault pattern recognition of the elevator to be detected is performed by respectively extracting comprehensive characteristic associated distribution information of the sound signal and the vibration signal of the elevator to be detected in the time domain and the frequency domain by adopting an artificial intelligence detection algorithm based on deep learning, and further fusing the sound characteristic distribution information and the vibration characteristic distribution information of the elevator to be detected. Like this, can discern the judgement to the fault mode of elevator accurately, and then be convenient for maintenance personnel to maintain the elevator, guarantee the normal safe operation of elevator.
Exemplary electronic device
Next, an electronic apparatus according to an embodiment of the present application is described with reference to fig. 9.
FIG. 9 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.
As shown in fig. 9, the electronic device 10 includes one or more processors 11 and memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer readable storage medium that processor 11 may execute to implement the functions in the big data based elevator fault pattern identification system of the various embodiments of the present application described above and/or other desired functions. Various contents such as an optimized vibration feature matrix may also be stored in the computer-readable storage medium.
In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The input device 13 may include, for example, a keyboard, a mouse, and the like.
The output device 14 can output various information including the classification result to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present application are shown in fig. 9, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.
Exemplary computer program product and computer-readable storage Medium
In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the functions in the big data based elevator failure mode identification method according to the various embodiments of the present application described in the "exemplary systems" section above of this specification.
The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps in the functions in the big data based elevator failure mode identification method according to various embodiments of the present application described in the above-mentioned "exemplary systems" section of this specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.
The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by one skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims (9)

1. An elevator fault pattern recognition system based on big data, comprising:
the fault data acquisition module is used for acquiring a detection vibration signal and a detection sound signal of the elevator to be detected in a preset time period, which are acquired by the vibration sensor and the sound sensor;
a frequency domain feature extraction module, configured to perform fourier transform on the detection vibration signal and the detection sound signal to obtain a plurality of first frequency domain feature statistics and a plurality of second frequency domain feature statistics;
the first joint coding module is used for inputting the detection vibration signal and the plurality of first frequency domain characteristic statistics values into a first Clip model to obtain an optimized vibration characteristic matrix;
the second joint coding module is used for inputting the detection sound signal and the plurality of second frequency domain characteristic statistic values into a second Clip model to obtain an optimized sound characteristic matrix;
the feature fusion module is used for fusing the optimized vibration feature matrix and the optimized sound feature matrix to obtain a classification feature matrix; and
and the elevator fault mode identification module is used for enabling the classification characteristic matrix to pass through a multi-label classifier to obtain a classification result, and the classification result is a probability value that the elevator to be detected has each fault.
2. The big data based elevator failure mode identification system of claim 1, wherein the first joint encoding module comprises:
a first frequency domain sequence feature extraction unit, configured to input the multiple first frequency domain feature statistics values into a sequence encoder of the first Clip model to obtain a first frequency domain statistical feature vector;
a first signal waveform feature extraction unit, configured to input a waveform diagram of the detected vibration signal into an image encoder of the first Clip model to obtain a detected vibration waveform feature vector;
and the first joint encoding unit is used for optimizing the feature encoding of the characteristic vector of the detected vibration waveform based on the first frequency domain statistical feature vector to obtain the optimized vibration feature matrix.
3. The big data based elevator fault pattern recognition system of claim 2, wherein the first frequency domain series feature extraction unit comprises:
a first scale feature extraction subunit, configured to pass the first frequency domain feature statistic through a first convolution layer of a trained sequence encoder of a Clip model to obtain a first neighborhood scale first frequency domain statistic feature vector, where the first convolution layer has a first one-dimensional convolution kernel of a first length;
a second scale feature extraction subunit, configured to pass the first frequency domain feature statistic through a second convolution layer of a trained sequence encoder of the Clip model to obtain a second neighborhood scale first frequency domain statistic feature vector, where the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and
and the multi-scale cascading subunit is used for cascading the first neighborhood scale first frequency domain statistical feature vector and the second neighborhood scale first frequency domain statistical feature vector to obtain the first frequency domain statistical feature vector.
4. The big data based elevator failure mode identification system of claim 3,
the first scale feature extraction subunit is further configured to: performing one-dimensional convolution coding on the first frequency domain characteristic statistic value by using a first convolution layer of the trained sequence coder of the Clip model according to the following formula to obtain a first neighborhood scale first frequency domain statistical characteristic vector;
wherein the formula is:
Figure FDA0003972532020000021
wherein a is the width of the first convolution kernel in the X direction, F (a) is a first convolution kernel parameter vector, G (X-a) is a local vector matrix operated with a convolution kernel function, w is the size of the first convolution kernel, and X represents the first frequency domain feature statistic value.
The second scale feature extraction subunit is further configured to: performing one-dimensional convolution encoding on the first frequency domain feature statistics by using a second convolution layer of the sequence encoder of the trained Clip model according to the following formula to obtain the second neighborhood scale first frequency domain statistical feature vector;
wherein the formula is:
Figure FDA0003972532020000022
wherein b is the width of the second convolution kernel in the X direction, F (b) is a second convolution kernel parameter vector, G (X-b) is a local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the first frequency domain feature statistic.
5. The big data based elevator fault pattern recognition system of claim 4, wherein the first signal waveform feature extraction unit is further configured to: the layers of the image encoder using the first Clip model perform, in a forward pass of the layers, respectively:
performing convolution processing on input data to obtain a convolution characteristic diagram;
pooling the convolution feature map based on a feature matrix to obtain a pooled feature map; and
performing nonlinear activation on the pooled feature map to obtain an activated feature map;
and outputting the last layer of the image encoder of the first Clip model as the detected vibration waveform characteristic vector, and inputting the first layer of the image encoder of the first Clip model as a waveform diagram of the detected vibration signal.
6. The big data based elevator failure mode identification system of claim 5, wherein the first joint encoding unit is further configured to: optimizing feature encoding of the probe vibration waveform feature vector based on the first frequency-domain statistical feature vector using a joint encoder of the first Clip model to obtain the optimized vibration feature matrix;
wherein the formula is:
Figure FDA0003972532020000031
wherein V s Representing the first frequency-domain statistical feature vector,
Figure FDA0003972532020000032
a transposed vector, V, representing the first frequency domain statistical feature vector b Representing a feature vector, M, of said detected vibration waveform b Representing the matrix of optimized vibration characteristics,
Figure FDA0003972532020000033
representing vector multiplication.
7. The big data based elevator failure mode identification system of claim 6, wherein the second joint encoding module comprises:
a second frequency domain sequence feature extraction unit, configured to input the multiple second frequency domain feature statistics values into a sequence encoder of the second Clip model to obtain a second frequency domain statistical feature vector;
a second signal waveform feature extraction unit, configured to input a waveform diagram of the sounding signal into an image encoder of the second Clip model to obtain a sounding waveform feature vector;
and the second joint encoding unit is used for optimizing the feature encoding of the detection sound waveform feature vector based on the second frequency domain statistical feature vector to obtain the optimized sound feature matrix.
8. The big data based elevator failure mode identification system of claim 7, wherein the feature fusion module is further to: fusing the optimized vibration feature matrix and the optimized sound feature matrix according to the following formula to obtain the classification feature matrix;
wherein the formula is:
Figure FDA0003972532020000034
wherein M is 1 And M 2 Respectively, said optimized vibration feature matrix and said optimized sound feature matrix, M c Is the classification feature matrix, reLU (-) represents the ReLU activation function,
Figure FDA0003972532020000035
representing the multiplication of matrices and the division between the numerator matrix and denominator matrix as a division by location of the eigenvalues of the matrices, exp (-) represents the exponential operation of the matrices, which represents the calculation of a natural exponential function value raised to the power of the eigenvalues of each location in the matrix.
9. The big data based elevator fault pattern recognition system of claim 8, wherein the elevator fault pattern recognition module comprises:
an expansion unit configured to expand the classification feature matrix into classification feature vectors based on row vectors or column vectors;
a full-concatenation encoding unit, configured to perform full-concatenation encoding on the classification feature vector using a plurality of full-concatenation layers of the classifier to obtain an encoded classification feature vector; and
a classification result generating unit, configured to pass the encoded classification feature vector through a Softmax classification function of the classifier to obtain the classification result.
CN202211517939.6A 2022-11-30 2022-11-30 Elevator fault mode identification system based on big data Active CN115783923B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211517939.6A CN115783923B (en) 2022-11-30 2022-11-30 Elevator fault mode identification system based on big data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211517939.6A CN115783923B (en) 2022-11-30 2022-11-30 Elevator fault mode identification system based on big data

Publications (2)

Publication Number Publication Date
CN115783923A true CN115783923A (en) 2023-03-14
CN115783923B CN115783923B (en) 2023-08-01

Family

ID=85443546

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211517939.6A Active CN115783923B (en) 2022-11-30 2022-11-30 Elevator fault mode identification system based on big data

Country Status (1)

Country Link
CN (1) CN115783923B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116101864A (en) * 2023-04-12 2023-05-12 中国矿业大学 Elevator door system fault diagnosis method and system based on voice recognition technology
CN116819379A (en) * 2023-04-23 2023-09-29 东莞市冠锦电子科技有限公司 Fault monitoring system and method for power adapter
CN117252926A (en) * 2023-11-20 2023-12-19 南昌工控机器人有限公司 Mobile phone shell auxiliary material intelligent assembly control system based on visual positioning
CN117361256A (en) * 2023-10-10 2024-01-09 广东全联富士电梯有限公司 Elevator safety management method and system based on artificial intelligence
CN117494056A (en) * 2023-12-29 2024-02-02 长春黄金设计院有限公司 Equipment fault early warning system and method based on big data technology

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012184115A (en) * 2012-06-01 2012-09-27 Hitachi Building Systems Co Ltd Fault diagnosis system of elevator
CN111179964A (en) * 2019-12-27 2020-05-19 烟台艾迪恩信息科技有限公司 Elevator landing door state identification method based on sound frequency domain features
CN114330430A (en) * 2021-12-22 2022-04-12 浙江工业大学 Elevator fault judgment method and system based on big data characteristic analysis
CN114782882A (en) * 2022-06-23 2022-07-22 杭州电子科技大学 Video target behavior abnormity detection method and system based on multi-mode feature fusion
CN115169505A (en) * 2022-09-06 2022-10-11 杭州浅水数字技术有限公司 Early warning method and early warning system for mechanical fault of special equipment moving part
CN115159285A (en) * 2022-07-01 2022-10-11 重庆大学 Elevator state monitoring method and elevator state monitoring device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012184115A (en) * 2012-06-01 2012-09-27 Hitachi Building Systems Co Ltd Fault diagnosis system of elevator
CN111179964A (en) * 2019-12-27 2020-05-19 烟台艾迪恩信息科技有限公司 Elevator landing door state identification method based on sound frequency domain features
CN114330430A (en) * 2021-12-22 2022-04-12 浙江工业大学 Elevator fault judgment method and system based on big data characteristic analysis
CN114782882A (en) * 2022-06-23 2022-07-22 杭州电子科技大学 Video target behavior abnormity detection method and system based on multi-mode feature fusion
CN115159285A (en) * 2022-07-01 2022-10-11 重庆大学 Elevator state monitoring method and elevator state monitoring device
CN115169505A (en) * 2022-09-06 2022-10-11 杭州浅水数字技术有限公司 Early warning method and early warning system for mechanical fault of special equipment moving part

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116101864A (en) * 2023-04-12 2023-05-12 中国矿业大学 Elevator door system fault diagnosis method and system based on voice recognition technology
CN116101864B (en) * 2023-04-12 2024-02-02 中国矿业大学 Elevator door system fault diagnosis method and system based on voice recognition technology
CN116819379A (en) * 2023-04-23 2023-09-29 东莞市冠锦电子科技有限公司 Fault monitoring system and method for power adapter
CN117361256A (en) * 2023-10-10 2024-01-09 广东全联富士电梯有限公司 Elevator safety management method and system based on artificial intelligence
CN117361256B (en) * 2023-10-10 2024-03-12 广东全联富士电梯有限公司 Elevator safety management method and system based on artificial intelligence
CN117252926A (en) * 2023-11-20 2023-12-19 南昌工控机器人有限公司 Mobile phone shell auxiliary material intelligent assembly control system based on visual positioning
CN117252926B (en) * 2023-11-20 2024-02-02 南昌工控机器人有限公司 Mobile phone shell auxiliary material intelligent assembly control system based on visual positioning
CN117494056A (en) * 2023-12-29 2024-02-02 长春黄金设计院有限公司 Equipment fault early warning system and method based on big data technology
CN117494056B (en) * 2023-12-29 2024-03-22 长春黄金设计院有限公司 Equipment fault early warning system and method based on big data technology

Also Published As

Publication number Publication date
CN115783923B (en) 2023-08-01

Similar Documents

Publication Publication Date Title
CN115783923B (en) Elevator fault mode identification system based on big data
CN115203380B (en) Text processing system and method based on multi-mode data fusion
CN109034368B (en) DNN-based complex equipment multiple fault diagnosis method
US11520993B2 (en) Word-overlap-based clustering cross-modal retrieval
CN115796173A (en) Data processing method and system for supervision submission requirements
CN115759658B (en) Enterprise energy consumption data management system suitable for smart city
CN115471216B (en) Data management method of intelligent laboratory management platform
CN114387567B (en) Video data processing method and device, electronic equipment and storage medium
CN116015837A (en) Intrusion detection method and system for computer network information security
CN114724386B (en) Short-time traffic flow prediction method and system under intelligent traffic and electronic equipment
CN112287672A (en) Text intention recognition method and device, electronic equipment and storage medium
CN115186774B (en) Intelligent cable stripping equipment and method thereof
CN117077075A (en) Water quality monitoring system and method for environmental protection
CN116759053A (en) Medical system prevention and control method and system based on Internet of things system
CN117115581A (en) Intelligent misoperation early warning method and system based on multi-mode deep learning
CN116866054A (en) Public information safety monitoring system and method thereof
CN116247824B (en) Control method and system for power equipment
CN112800217A (en) Vector relevance matrix-based intelligent assessment method for government affair transaction processing report
CN112465805A (en) Neural network training method for quality detection of steel bar stamping and bending
CN116124448A (en) Fault diagnosis system and method for wind power gear box
CN112836752A (en) Intelligent sampling parameter control method based on feature map fusion of depth values
KR102215259B1 (en) Method of analyzing relationships of words or documents by subject and device implementing the same
CN111199170B (en) Formula file identification method and device, electronic equipment and storage medium
CN113392278A (en) Industrial internet-based equipment liquid pipeline flow detection method
CN112434973A (en) Key area safety control index evaluation method based on one-dimensional and two-dimensional convolution

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant