TWI762089B - Machine condition detection system and machine condition detection method - Google Patents

Abstract

A machine condition detection system and a machine condition detection method are provided. The machine condition detection system includes an audio sensor module and a processing module. The audio sensing module receives an audio signal in the field. The processing module converts the audio signal into an audio spectrogram, extracts a plurality of first features from the audio spectrogram, and recognizes that the at least one machine is in one of a plurality of operating conditions according to the plurality of first features.

Description

Machine condition detection system and machine condition detection method

The present invention relates to a machine condition detection system and a machine condition detection method, and more particularly, to a machine condition detection system and a machine condition detection method for judging the machine condition by audio signals.

Each machine operates according to an operating program. Generally speaking, based on operating procedures, the machine is usually in a normal operating condition. However, in order to prevent losses or industrial safety accidents arising from machine abnormalities, a machine state monitoring or detection mechanism is required. Therefore, how to implement a monitoring or detection mechanism for accurately judging the running state of the machine is one of the subjects that those skilled in the art have been working on.

The invention provides a machine condition detection system and a machine condition detection method capable of accurately judging the operation condition of the machine.

The machine condition detection system of the present invention is suitable for detecting the operation condition of at least one machine in the field. The machine condition detection system includes an audio sensing module and a processing module. The audio sensing module is configured to continuously receive audio signals within the field. The processing module is coupled to the audio sensing module. The processing module is configured to convert the audio signal into an audio spectrogram corresponding to a unit time interval, extract a plurality of first features in the audio spectrogram, and identify the at least one machine according to the plurality of first features The station is in one of several health states.

The machine condition detection method of the present invention is suitable for detecting the operation condition of at least one machine in the field. The machine condition detection method includes: continuously receiving audio signals in the field; converting the audio signals into audio spectrograms corresponding to unit time intervals; extracting a plurality of first features in the audio spectrograms; and a first feature to identify that the at least one machine is in one of a plurality of operating conditions.

Based on the above, the machine condition detection system and the machine condition detection method of the present invention identify that the machine is in one of a plurality of operating conditions according to the audio spectrogram associated with the audio signal. In this way, the machine state detection system and the machine state detection method can accurately determine the operation state of the machine by using the audio signal.

In order to make the above-mentioned features and advantages of the present invention more obvious and easy to understand, the following embodiments are given and described in detail with the accompanying drawings as follows.

Some embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Element symbols quoted in the following description will be regarded as the same or similar elements when the same element symbols appear in different drawings. These examples are only a part of the invention and do not disclose all possible embodiments of the invention. Rather, these embodiments are merely exemplary of apparatus and methods within the scope of the present invention.

Please refer to FIG. 1 . FIG. 1 is a schematic diagram of a machine condition detection system according to a first embodiment of the present invention. In this embodiment, the machine condition detection system 100 can be adapted to detect the operation conditions of the machines in the field F. As shown in FIG. For convenience of description, one machine M is taken as an example in the field F of this embodiment. In some embodiments, the number of machines in field F may be multiple. The present invention is not limited to the number of machines in the field F. In this embodiment, the field F may be at least a part of any indoor space. The machine M can be any manufacturing equipment or inspection equipment.

In this embodiment, the machine condition detection system 100 includes an audio sensing module 110 and a processing module 120 . The audio sensing module 110 continuously receives the audio signal AS in the field F. In this embodiment, the audio sensing module 110 continuously receives the audio signal AS from any position in the field F. As shown in FIG. In this embodiment, the audio sensing module 110 may include at least one microphone. In order to be able to receive the audio signal AS from the machine M, at least one microphone of the audio sensing module 110 can be disposed at a position close to the machine M. In some embodiments, at least one of the at least one microphone of the audio sensing module 110 may also be disposed close to the machine M.

In this embodiment, the audio signal AS may be a signal conforming to MP3, WMA, and WAV file formats.

In this embodiment, the processing module 120 is coupled to the audio sensing module 110 . The processing module 120 converts the audio signal AS into an audio spectrogram SP. In this embodiment, the processing module 120 may, for example, perform Fourier transform on the audio signal AS to generate the audio spectrogram SP. The audio spectrogram SP corresponds to a specific unit time interval. The processing module 120 may perform a short-time Fourier transform (STFT) on the audio signal AS to generate an audio spectrogram SP. In this embodiment, the processing module 120 captures the first feature F1 in the audio spectrogram SP and identifies that the machine M is in one of a plurality of operating conditions according to the first feature F1. In this embodiment, the number of the first features F1 captured by the processing module 120 may be one or more.

For example, the multiple operating conditions of the machine M include normal operating conditions and abnormal operating conditions. When the processing module 120 recognizes that the first feature F1 conforms to the audio feature of the normal operating state of the machine M, the processing module 120 recognizes that the machine M is in the normal operating state. On the other hand, when the processing module 120 recognizes that the first feature F1 conforms to the audio feature of the abnormal operating condition of the machine M (for example, an abnormal collision sound or a part falling sound), the processing module 120 will recognize The ejector M is in an abnormal operating state. In this embodiment, the processing module 120 identifies one of the multiple operating conditions of the machine M by means of artificial intelligence (Artificial Intelligence, AI).

It is worth mentioning here that the processing module 120 identifies that the machine M is in one of a plurality of operating states according to the audio spectrogram SP associated with the audio signal AS. In this way, the machine state detection system 100 can accurately and instantly determine the operation state of the machine M by the audio signal AS.

In this embodiment, the processing module 120 is, for example, a central processing unit (Central Processing Unit, CPU), or other programmable general-purpose or special-purpose microprocessors (Microprocessors), digital signal processors (Digital Signal Processors) Processor, DSP), Programmable Controller, Application Specific Integrated Circuits (ASIC), Programmable Logic Device (PLD), or other similar devices or combinations of these devices, which can carry enter and execute computer programs. The processing module 120 may be installed in a server, a backend host or an electronic device. The above-mentioned server, backend host or electronic device may be arranged outside the field F. In some embodiments, the above-mentioned server, backend host or electronic device may be arranged in the field F. In some embodiments, the processing module 120 may be integrated with the audio sensing module 110 in the same device.

In this embodiment, the audio sensitivity of the audio sensing module 110 is -50 decibels (dB) to -44 decibels, and the sampling frequency of the audio sensing module 110 is greater than 20000 Hz. That is to say, the audio sensitivity of the microphone of the audio sensing module 110 is -50 decibels to -44 decibels, and the sampling frequency of the microphone is greater than 20000 hertz (Hz). In this way, the present embodiment can improve the first feature resolution of the audio spectrogram SP.

To further illustrate the audio spectrogram, please refer to FIG. 1 and FIG. 2 at the same time. FIG. 2 is a schematic diagram of an audio spectrogram according to an embodiment of the present invention. In this embodiment, FIG. 2 illustrates audio spectrograms SP1 to SP7 corresponding to different operating conditions. The audio spectrogram SP1 is the spectrogram of the background sound. The background sound is the sound when the machine M is in normal operation. The audio spectrogram SP2 is the spectrogram of the human voice. The audio spectrogram SP3 is the spectrogram of the sound produced by the plastic friction of the machine M. The audio spectrogram SP4 is a spectrogram of the fan sound from the machine M. The audio spectrogram SP5 comes from the spectrogram of the sound produced by the metal impact of the machine M. The audio spectrogram SP6 is a spectrogram of the sound produced by the metal falling from the machine M. The audio spectrogram SP7 is a spectrogram of the sound produced by the impact of the wafer from the machine M (the machine M is, for example, a semiconductor process equipment).

In this embodiment, the vertical axes of the audio spectrograms SP1 to SP7 are represented as audio frequencies. The audio frequency can be represented by, for example, an index (the invention is not limited to this). The horizontal axis of the audio spectrograms SP1 to SP7 is represented as a time interval. The time interval is determined in relation to an identifiable time that can represent all of the above operating conditions. For example, the time interval may be 3 seconds (the invention is not limited to this). That is to say, the audio signal AS within 3 seconds is sufficient to reflect the operating status of the machine M. Therefore, the audio spectrograms SP1 to SP7 are, for example, intensity panes (window slots) in the last 3 seconds. The colors or grayscales in the audio spectrogram SP1~SP7 correspond to the intensity of the audio and time points. The audio spectrograms SP1 to SP7 each include audio intensity distributions corresponding to multiple time points and multiple frequencies in a unit time interval. The audio intensity distribution of the audio spectrograms SP1~SP7 is the feature corresponding to the audio spectrogram SP1~SP7.

In this embodiment, the processing module 120 continuously converts the audio signal AS in the last 3 seconds into the audio spectrogram SP, and extracts the first feature F1 in the audio spectrogram SP according to the audio intensity distribution. For example, when the first feature F1 is determined to conform to the feature of the audio spectrogram SP1, it means that in the last 3 seconds, there is only background sound in the field F. Therefore, the processing module 120 recognizes that the machine M is in normal operation, and provides relevant information that the machine M is in normal operation. For another example, when the first feature F1 is determined to conform to the features of the audio spectrogram SP6, the processing module 120 determines that a metal collision occurs in the machine M, and provides a warning message that a metal collision occurs in the machine M.

For example, in this embodiment, the processing module 120 captures a plurality of parts of a plurality of audio intensity distributions in the audio spectrogram SP, and converts the plurality of parts of the plurality of audio intensity distributions to generate a plurality of audio frequency intensity distributions. A first feature F1. After the processing module 120 captures the plurality of audio intensity distributions, for example, a convolutional neural network (Convolutional Neural Networks, CNN) model (not limited in the present invention) can be used to analyze the plurality of audio intensity distributions. The audio intensity distribution is converted to obtain a plurality of first features F1. The above-mentioned multiple partial regions are, for example, multiple preset regions of interest (regions of interest) in the audio spectrogram SP or partial regions with preset interest intensity distributions. Therefore, the number of the first features F1 of the present embodiment can be set.

Please refer to FIG. 1 and FIG. 3 at the same time. FIG. 3 is a flowchart of a method for detecting a machine condition according to a first embodiment of the present invention. The machine condition detection method of this embodiment can be applied to the machine condition detection system 100 . In step S110, the audio signal AS in the field F is continuously received. In this embodiment, step S110 may be performed by the audio sensing module 110 . In step S120, the received audio signal AS is converted into an audio spectrogram SP. In step S130, the first feature F1 in the audio spectrogram SP is extracted. In step S140, according to the first feature F1, it is identified that the machine is in one of a plurality of operating conditions. In this embodiment, steps S120 to S140 may be performed by the processing module 120 . The implementation details of steps S110 to S140 in this embodiment can be sufficiently taught from the various embodiments in FIG. 1 and FIG. 2 , and therefore will not be repeated here.

Please refer to FIG. 2 and FIG. 4 at the same time. FIG. 4 is a schematic diagram of a machine condition detection system according to a second embodiment of the present invention. In this embodiment, the machine condition detection system 200 includes an audio sensing module 210 , a processing module 220 and a learning module 230 . The cooperative operation of the audio sensing module 210 and the processing module 220 in this embodiment can be sufficiently taught from the various embodiments in FIG. 1 to FIG. 3 , so it will not be repeated here. In this embodiment, the learning module 230 is coupled to the processing module 220 . The learning module 230 collects sample audio signals SAS_1 to SAS_7 and a plurality of operating conditions. The learning module 230 captures the second features F2_1 ˜ F2_7 in each sample audio signal SAS_1 ˜SAS_7 . For example, the learning module 230 will capture the second feature F2_1 in the sample audio signal SAS_1, capture the second feature F2_2 in the sample audio signal SAS_2, and so on.

In this embodiment, the learning module 230 further converts the sample audio signals SAS_1 to SAS_7 into audio spectrograms. For example, the learning module 230 converts the sample audio signal SAS_1 into an audio spectrogram SP1, converts the sample audio signal SAS_2 into an audio spectrogram SP2, and so on. Therefore, the learning module 230 can extract the second feature F2_1 according to the audio intensity distribution of the audio spectrogram SP1, extract the second feature F2_2 from the audio intensity distribution of the audio spectrogram SP2, and so on. For example, the learning module 230 can convert the parts of the audio intensity distribution of the audio spectrogram SP1 into a plurality of second features F2_1, and convert the parts of the audio intensity distribution of the audio spectrogram SP2 into a plurality of second features F2_1. The second feature F2_2, and so on.

In this embodiment, the learning module 230 further performs deep learning according to the second features F2_1 to F2_7 in the sample audio signals SAS_1 to SAS_7, so that the sample audio signals SAS_1 to SAS_7 respectively correspond to the received operating conditions. one of them. In this embodiment, the deep learning learning model may be a long short-term memory (Long Short-Term Memory, LSTM) model, a recurrent neural network (Recurrent Neural Networks, RNN) model, a convolutional neural network model, a deep neural network Road (Deep Neural Networks, DNN) model, or Region-based Convolutional Neural Networks (Region-based Convolutional Neural Networks, R-CNN) model.

After deep learning is performed, the learning module 230 corresponds the second feature F2_1 to the background sound. The learning module 230 corresponds the second feature F2_2 to the human voice. The learning module 230 corresponds the second feature F2_3 to the sound generated by the plastic friction from the machine (the machine M shown in FIG. 1 ). The learning module 230 corresponds the second feature F2_4 to the fan sound from the machine. The learning module 230 corresponds the second feature F2_5 to the sound produced by the metal impact from the machine. The learning module 230 corresponds the second feature F2_6 to the sound produced by the metal falling from the machine. In addition, the learning module 230 also corresponds the second feature F2_7 to the sound generated by the chip impact from the machine. Therefore, the second features F2_1 to F2_7 correspond to different operating conditions, respectively.

In this embodiment, the learning module 230 provides the second features F2_1 to F2_7 to the processing module 220 . In addition, for example, according to actual usage requirements, the second features F2_1 , F2_3 , and F2_4 can be set as the normal operating conditions of the machine. The second features F2_5 to F2_7 may be set as abnormal operating conditions of the machine.

It is worth mentioning here that, according to actual usage requirements, the sample audio signals SAS_1 to SAS_7 and a plurality of operating conditions can be provided to the learning module 230 . In this way, the learning module 230 can respectively correspond the second features F2_1 to F2_7 of the sample audio signals SAS_1 to SAS_7 provided by the user to different operating conditions of the machine, so as to realize the customization of the second features F2_1 to F2_7 set up.

In this embodiment, the processing module 220 further compares the first feature F1 with the received second features F2_1 ˜ F2_7 to determine whether the type of the audio signal AS conforms to one of the sample audio signals SAS_1 ˜ SAS_7 One category. For example, when the processing module 220 determines that the first feature F1 is similar to or equal to the second feature F2_1, the processing module 220 determines that the audio signal AS matches the sample audio signal SAS_1, and recognizes that the machine is in a normal operating state, and Provides information that the machine is in normal operating condition. For another example, when the processing module 220 determines that the first feature F1 is similar to or equal to the second feature F2_7, the processing module 220 determines that the audio signal AS matches the sample audio signal SAS_7, and determines that the machine has a chip impact, and provides Corresponds to the warning message of wafer collision in the machine. In this embodiment, the warning message may include at least one of warning light, warning sound and warning text.

Please refer to FIG. 4 and FIG. 5 at the same time. FIG. 5 is a flowchart of a machine condition detection method according to a second embodiment of the present invention. The machine condition detection method of this embodiment can be applied to the machine condition detection system 200 . In step S210, sample audio signals SAS_1 to SAS_7 are collected. In step S220, the second features F2_1-F2_7 in each sample audio signal SAS_1-SAS_7 are extracted. In step S230 , deep learning is performed according to the second features F2_1 ˜ F2_7 , so that the sample audio signals SAS_1 ˜ SAS_7 respectively correspond to one of the plurality of operating conditions. In this embodiment, steps S210 to S230 may be performed by the learning module 230 . In step S240 , the audio signal AS in the field (eg, field F shown in FIG. 1 ) is continuously received. In this embodiment, steps S210 to S230 may be performed by the audio sensing module 210 . In step S250, the received audio signal AS is converted into an audio spectrogram SP. In step S260, the first feature F1 in the audio spectrogram SP is extracted. In this embodiment, steps S250 and S260 may be performed by the processing module 220 . In step S270, according to the first feature F1, it is identified that the machine is in one of a plurality of operating conditions. Further, in this embodiment, the processing module 220 compares the first feature F1 with the received second features F2_1˜F2_7 to determine whether the type of the audio signal AS conforms to the sample audio signals SAS_1˜SAS_7 One of the categories, and then identify the operating status of the machine. In step S280, when the machine is identified as being in an abnormal state, the processing module 220 provides a warning message corresponding to the abnormal state.

In this embodiment, step S230 may be performed before step S240. Step S230 may also be performed before step S270.

To sum up, the machine condition detection system and the machine condition detection method of the present invention identify that the machine is in one of a plurality of operating conditions according to the audio spectrogram associated with the audio signal. In this way, the machine state detection system and the machine state detection method can accurately determine the operation state of the machine by using the audio signal. In addition, the present invention can respectively correspond to a plurality of second features of a plurality of sample audio signals provided by a user to different operating conditions of the machine, thereby realizing customized settings.

Although the present invention has been disclosed above by the embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, The protection scope of the present invention shall be determined by the scope of the appended patent application.

100, 200: Machine condition detection system

110, 210: Audio sensing module

120, 220: Processing module

230: Learning Mods

AS: audio signal

F: Field

F1: First feature

F2_1~F2_7: Second feature

M: machine

S110~S140: Steps

S210~S280: Steps

SAS_1~SAS_7: Sample audio signal

SP, SP1~SP7: Audio Spectrogram

FIG. 1 is a schematic diagram of a machine condition detection system according to a first embodiment of the present invention. FIG. 2 is a schematic diagram of an audio spectrogram according to an embodiment of the present invention. FIG. 3 is a flowchart of a method for detecting a machine condition according to the first embodiment of the present invention. FIG. 4 is a schematic diagram of a machine condition detection system according to a second embodiment of the present invention. FIG. 5 is a flowchart of a method for detecting a machine condition according to a second embodiment of the present invention.

100: Machine condition detection system

110: Audio sensing module

120: Processing modules

AS: audio signal

F: Field

F1: First feature

M: machine

SP: Audio Spectrogram

Claims (14)

A machine condition detection system, suitable for detecting the operation condition of at least one machine in a field, wherein the machine condition detection system comprises: an audio sensing module configured to continuously receive a an audio signal; and a processing module, coupled to the audio sensing module, configured to: convert the audio signal into an audio spectrogram corresponding to a unit time interval; a first feature, and according to the first features to distinguish which one of a plurality of sound types the audio signal corresponds to, wherein the sound types include: the normal running background sound of the at least one machine, the at least one sound A plastic friction sound in a machine, a metal impact sound in the at least one machine, a metal falling sound in the at least one machine, and a chip impact sound in the at least one machine; and when judging the at least one machine When the sound type of the at least one machine does not belong to the normal operating background sound, it is determined according to the sound type of the at least one machine which type of abnormality one of the multiple operating conditions during the operation of the at least one machine corresponds to situation. The machine condition detection system of claim 1, wherein an audio sensitivity of the audio sensing module is -50 dB to -44 dB, and a sampling frequency of the audio sensing module is greater than 20000 Hz. The machine condition detection system according to claim 1, wherein the processing module identifies one of the operating conditions of the at least one machine by means of artificial intelligence. The machine condition detection system of claim 1, wherein: the audio spectrogram includes an audio intensity distribution corresponding to a plurality of time points and a plurality of frequencies in the unit time interval, and the processing module is further configured to The first features are extracted according to portions of the audio intensity distribution. The machine condition detection system of claim 1, further comprising: a learning module, coupled to the processing module, configured to collect a plurality of sample audio signals and the operating conditions, and capture each of the samples A plurality of second features in the audio signal, and deep learning is performed according to the second features in each of the sample audio signals, so that each of the sample audio signals corresponds to one of the operating conditions respectively. The machine condition detection system as claimed in claim 5, wherein the processing module is further configured to compare the first features with the second features to determine whether the type of the audio signal conforms to the samples One of the types of audio signals. The machine condition detection system of claim 1, wherein when at least one of the at least one machine is identified as one of a plurality of abnormal conditions in a plurality of operating conditions, the processing module provides a corresponding Warning message. A machine condition detection method, suitable for detecting the operation condition of at least one machine in a field, wherein the machine condition detection method comprises: continuously receiving an audio signal in the field; converting the audio signal into a corresponding an audio spectrogram in a unit time interval; extracting a plurality of first features in the audio spectrogram, and distinguishing which of a plurality of sound types the audio signal corresponds to according to the first features, The sound types include: the normal running background sound of the at least one machine, the plastic friction sound in the at least one machine, the metal impact sound in the at least one machine, and the metal falling sound in the at least one machine , the chip impact sound in the at least one machine; and when judging that the sound type of the at least one machine does not belong to the normal operation background sound, identify the at least one machine according to the sound type of the at least one machine. One of the multiple operating conditions in an operation corresponds to what type of abnormal condition. The device condition detection method according to claim 8, wherein the step of continuously receiving the audio signal in the field comprises: continuously receiving the audio signal in the field based on an audio sensitivity and a sampling frequency, Wherein the audio sensitivity is -50 decibels to -44 decibels, and the sampling frequency is greater than 20000 Hz. The machine condition detection method according to claim 8, wherein when it is judged that the sound type of the at least one machine does not belong to the normal running background sound, according to the The step of identifying which type of abnormal condition one of the operating conditions of the at least one machine during the operation of the at least one machine corresponds to the sound type of the at least one machine comprises: identifying the at least one machine by means of artificial intelligence. One of the operating conditions of a machine corresponds to what type of abnormal condition. The device condition detection method according to claim 8, wherein the audio spectrogram includes an audio intensity distribution corresponding to a plurality of time points and a plurality of frequencies in the unit time interval, wherein the audio spectrogram in the audio spectrogram is extracted The step of the first features includes: extracting the first features according to a plurality of parts of the audio intensity distribution. The machine condition detection method according to claim 8, further comprising: collecting a plurality of sample audio signals and the operating conditions; capturing a plurality of second features in each of the sample audio signals; and according to each of the samples Deep learning is performed on the second features in the audio signal, so that each of the sample audio signals corresponds to one of the operating conditions, respectively. The machine condition detection method as claimed in claim 12, further comprising: comparing the first features with the second features to determine whether the type of the audio signal conforms to one of the sample audio signals category. The machine condition detection method according to claim 12, further comprising: when at least one of the at least one machine is identified as one of a plurality of abnormal conditions among a plurality of operating conditions, providing a corresponding warning message .

Images