CN112733588A - Machine running state detection method and device and electronic equipment - Google Patents
Machine running state detection method and device and electronic equipment Download PDFInfo
- Publication number
- CN112733588A CN112733588A CN202010809719.5A CN202010809719A CN112733588A CN 112733588 A CN112733588 A CN 112733588A CN 202010809719 A CN202010809719 A CN 202010809719A CN 112733588 A CN112733588 A CN 112733588A
- Authority
- CN
- China
- Prior art keywords
- result
- target machine
- state
- running
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 51
- 238000000034 method Methods 0.000 claims abstract description 68
- 230000004927 fusion Effects 0.000 claims abstract description 41
- 230000008569 process Effects 0.000 claims abstract description 26
- 238000003062 neural network model Methods 0.000 claims description 34
- 230000002159 abnormal effect Effects 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 19
- 238000004364 calculation method Methods 0.000 claims description 17
- 238000004422 calculation algorithm Methods 0.000 claims description 13
- 238000003860 storage Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 239000003245 coal Substances 0.000 description 137
- 238000005065 mining Methods 0.000 description 104
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 238000005520 cutting process Methods 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000005315 distribution function Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
- G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The invention provides a method and a device for detecting the running state of a machine and electronic equipment, and relates to the technical field of fault detection. The machine running state detection method comprises the following steps: acquiring audio data in the running process of a target machine, and determining the running state of the target machine based on the audio data to obtain a first result; acquiring an operation image of the target machine, and identifying the operation state of the target machine based on the operation image to obtain a second result; and performing data fusion on the first result and the second result to obtain a state detection result of the target machine. The invention can save labor cost and improve the accuracy of state detection of the target machine.
Description
Technical Field
The invention relates to the technical field of fault detection, in particular to a method and a device for detecting a machine running state and electronic equipment.
Background
Along with the development of science and technology, the application scene of machine is more and more extensive, in order to guarantee the normal operating of machine, is very necessary to the monitoring of machine running state to in time maintain when the machine breaks down, when guaranteeing production efficiency, avoid the emergence of major accident. Taking a coal mining machine in a coal mining site as an example, abnormal coal mining states such as an idle state or an idle running state of the coal mining machine may occur during the coal mining process of the coal mining machine, and the current detection mode of the operation state of the coal mining machine mainly adopts manual monitoring of the operation state of the coal mining machine or collection of images of the coal mining machine, and the operation state of the coal mining machine is judged according to the images. However, the manual monitoring of the operation state of the coal mining machine consumes labor cost, and the problem of low accuracy may exist in the judgment of the state of the coal mining machine only by means of the coal mining machine image. Therefore, the existing machine fault detection method has the problems of consuming labor cost or low state detection accuracy.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a method and an apparatus for detecting a machine operating state, and an electronic device, which can save labor cost and improve accuracy of detecting a target machine state.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, an embodiment of the present invention provides a method for detecting a machine operating state, including: acquiring audio data in the running process of a target machine, and determining the running state of the target machine based on the audio data to obtain a first result; acquiring an operation image of the target machine, and identifying the operation state of the target machine based on the operation image to obtain a second result; and performing data fusion on the first result and the second result to obtain a state detection result of the target machine.
Further, an embodiment of the present invention provides a first possible implementation manner of the first aspect, where the step of performing data fusion on the first result and the second result to obtain a state detection result of the target machine includes: and performing fusion calculation on the first result and the second result based on a D-S evidence theory algorithm to obtain a state detection result of the target machine.
Further, an embodiment of the present invention provides a second possible implementation manner of the first aspect, where the step of performing fusion calculation on the first result and the second result based on a D-S evidence theory algorithm to obtain the state detection result of the target machine includes: and determining a state detection result of the target machine based on the first result, the second result and a fusion calculation formula.
Further, an embodiment of the present invention provides a third possible implementation manner of the first aspect, where the target machine includes a coal mining machine, and the operation state includes a normal operation state, a shutdown state, and an abnormal operation state; the fusion calculation formula is:
wherein m (A) is the result of the state detection, m1(A1) As the first result, m2(A2) For the second result, the first result and the second result comprise probability values of the shearer in the respective operating states, A1To determine events of the operating state of the target machine based on the audio data, A2K is a collision factor for identifying an event of the operational state of the target machine based on the operational image.
Further, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where the step of obtaining audio data in an operation process of a target machine, determining an operation state of the target machine based on the audio data, and obtaining a first result includes: collecting audio data in the running process of a target machine, and determining a spectrogram generated by the running of the target machine based on the audio data; inputting the spectrogram into a first neural network model obtained by pre-training to obtain a first result of the target machine; the first neural network model is obtained by training based on a spectrogram sample marked with an operation state.
Further, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, wherein the step of determining, based on the audio data, a spectrogram generated by the target machine in operation includes: dividing the audio data into audio segments with preset duration; and denoising the audio segments, and converting the audio segments into corresponding spectrograms by using an audio data processing library.
Further, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, wherein the step of identifying the operating state of the target machine based on the operating image to obtain a second result includes: inputting the running image into a pre-trained second neural network model to obtain a second result of the target machine; the second neural network model is obtained by training based on a running image sample marked with a running state; or acquiring parameter information related to the running state of the target machine according to the running image, and determining the running state of the target machine based on the parameter information to obtain a second result.
In a second aspect, an embodiment of the present invention further provides a device for detecting an operating state of a machine, including: the state determining module is used for acquiring audio data of a target machine, determining the running state of the target machine based on the audio data and obtaining a first result; the state identification module is used for acquiring the running image of the target machine, identifying the running state of the target machine based on the running image and obtaining a second result; and the data fusion module is used for carrying out data fusion on the first result and the second result to obtain a state detection result of the target machine.
In a third aspect, an embodiment of the present invention provides an electronic device, including: the device comprises a sound acquisition device, an image acquisition device, a processor and a storage device; the image acquisition device is used for acquiring audio data in the running process of the target machine; the sound acquisition device is used for acquiring an operation image of the target machine; the storage means has stored thereon a computer program which, when executed by the processor, performs the method of any of the first aspects.
In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the method in any one of the above first aspects.
The embodiment of the invention provides a machine running state detection method, a device and electronic equipment, wherein in the method, audio data in the running process of a target machine is obtained firstly, and the running state of the target machine is determined based on the audio data to obtain a first result; then acquiring an operation image of the target machine, and identifying the operation state of the target machine based on the operation image to obtain a second result; and finally, performing data fusion on the first result and the second result to obtain a state detection result of the target machine. The state detection result of the target machine can be automatically obtained by performing data fusion on the first result obtained based on the audio data of the target machine operation and the second result obtained based on the operation image of the target machine, so that the labor cost is saved, and the accuracy of state detection of the target machine is improved by comprehensively judging the operation state based on the sound and the image of the target machine operation.
Additional features and advantages of embodiments of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of the invention as set forth above.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flow chart of a method for detecting a machine operating state according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating confidence intervals of a D-S evidence theory provided by an embodiment of the present invention;
fig. 3 shows a normal coal cutting spectrum diagram of a coal mining machine according to an embodiment of the present invention;
fig. 4 shows a frequency spectrum diagram of an empty cutter of a coal mining machine according to an embodiment of the invention;
FIG. 5 illustrates a shearer baseplate spectrogram provided by embodiments of the present invention;
fig. 6 shows a flow chart of detecting an operation state of a coal mining machine according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of a device for detecting an operating condition of a machine according to an embodiment of the present invention;
fig. 8 shows a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
The first embodiment is as follows:
the embodiment provides a method for detecting a machine operating state, which may be applied to an electronic device, where the electronic device may include a sound collection device and an image collection device, and referring to a flowchart of the method for detecting a machine operating state shown in fig. 1, the method mainly includes the following steps S102 to S106:
step S102, audio data in the running process of the target machine is obtained, the running state of the target machine is determined based on the audio data, and a first result is obtained.
The method comprises the steps that sound in the running process of a target machine is collected in real time or at preset time intervals through a sound collection device (such as a sound recorder or a high-definition explosion-proof camera with a sound recording function) to obtain audio data of the running of the target machine, the generated audio data are different due to different sounds of the machine in different running states, and the running state of the target machine can be judged according to the audio data of the running process of the target machine.
And step S104, acquiring the running image of the target machine, identifying the running state of the target machine based on the running image, and obtaining a second result.
The method comprises the steps of acquiring images of a target machine in the running process in real time or at preset time intervals through an image acquisition device while acquiring audio data, or acquiring video data of the target machine in the running process through the image acquisition device to obtain running images of the target machine, and identifying the running state of the target machine according to the running images of the target machine, such as the position change condition of coal blocks in the running images of the coal mining machine, so as to determine the running state of the coal mining machine.
And S106, performing data fusion on the first result and the second result to obtain a state detection result of the target machine.
The first result and the second result (the first result and the second result are results obtained by detecting the target device in the same time period) are subjected to data fusion, for example, the first result and the second result may be subjected to weighted average calculation, and the state detection result of the target device may be obtained by comprehensively considering the first result obtained by sound judgment of the target device and the second result obtained by image judgment of the target device.
According to the machine running state detection method provided by the embodiment, the first result obtained based on the audio data of the running of the target machine and the second result obtained based on the running image of the target machine are subjected to data fusion, so that the state detection result of the target machine can be automatically obtained, the labor cost is saved, the running state is comprehensively judged based on the sound and the image of the running of the target machine, and the accuracy of the state detection of the target machine is improved.
In order to accurately obtain the state detection result of the target machine, the present embodiment provides a specific implementation manner of performing data fusion on the first result and the second result to obtain the state detection result of the target machine: and performing fusion calculation on the first result and the second result based on a D-S evidence theory algorithm to obtain a state detection result of the target machine. And performing fusion judgment on the first result and the second result by adopting a D-S evidence theory algorithm, and calculating to obtain a final fusion result. The basic principle of the D-S evidence theory algorithm is as follows: the possible values of the random variable X form the basic framework U, i.e. the set of all values. The values are mutually incompatible and independent, and can be called as a recognition frame of the target X. Power set 2 of UUThe power series with the number of elements in U being N is obtained to be 2N. Let U be an identification frame, the number of elements in U be N, and its power set be 2U. For 2UAny subset a of (a), called proposition a. Basic probability distribution function m is at 2UDefined above, take the value [0,1]M (a) represents the confidence that evidence holds for proposition a, satisfying the following rules:
let U be an identification frame, the number of elements in U be N, and its power set be 2U. In 2UThe above defined belief function Bel takes the value [0, 1%](ii) a In 2UDefining a likelihood function Pl above, and taking the value of [0, 1%]. To 2UAny proposition a in (1), satisfies:
referring to a D-S evidence theory confidence interval diagram as shown in FIG. 2, for any proposition A, Pl (A)A)≥Bel(A)，[Bel(A),Pl(A)]A trust interval referred to as a. The reliability function and the likelihood function respectively depict the upper limit and the lower limit of the reliability of the proposition A.
When proposition a is supported by a plurality of mutually independent evidence sources (such as identifying the operating state of a target machine based on audio data, identifying the operating state of the target machine based on an operating image), evidence theory uses a combination rule to calculate a basic probability distribution value BPA. For a BPA function of n pieces of evidence, the combination rule is:
wherein the content of the first and second substances,
setting the proposition A as the state detection result of the target machine (the proposition A has two mutually independent evidence sources: the running state m of the target machine identified based on the audio data1(A1) And the running state m of the target machine is obtained based on the running image recognition2(A2) The target machine comprises a coal mining machine, and the running state comprises a normal running state, a shutdown state and an abnormal running state; and determining the state detection result of the target machine based on the first result, the second result and the fusion calculation formula. The above fusion calculation formula is:
wherein m (A) is a state detection result, m1(A1) As a first result, m2(A2) The second result comprises probability values of the coal mining machine in each operating state, A1For determining events of the operating state of the target machine on the basis of the audio data, A2To identify an event of the operational state of the target machine based on the operational image, k is a collision factor,
when k is 0, represents m1(A1) And m2(A2) The two evidences are completely conflicted, the denominator is 0, the synthesis rule is meaningless, and when k is not equal to 0, the Dempster synthesis rule can be used for carrying out data fusion on probability distribution functions of proposition A1 and proposition A2. Through the fusion calculation mode, the final operation state of the target machine can be accurately calculated based on the first result obtained by utilizing the audio data identification and the second result obtained by utilizing the operation image identification, and the accuracy of machine fault detection is improved.
In order to improve the accuracy of the first result, the embodiment provides an implementation manner of obtaining the audio data in the operation process of the target device, and determining the operation state of the target device based on the audio data to obtain the first result, which may be specifically executed with reference to the following steps (1) to (2):
step (1): and acquiring audio data in the running process of the target machine, and determining a spectrogram generated by the running of the target machine based on the audio data.
A sound acquisition device, such as a recording device like a microphone or a camera with a sound acquisition function, is arranged beside the target machine, and clear machine sound in the running process of the target machine is acquired in real time through the sound acquisition device to obtain audio data of the target machine. The method comprises the steps of dividing audio data into audio segments with preset duration, carrying out denoising processing on each audio segment, and converting each audio segment into a corresponding spectrogram by using an audio data processing library. The method comprises the steps of averagely dividing collected audio data into a plurality of audio segments with the same time period (preset time duration, such as any value between 4s and 10 s), denoising each audio segment, and converting each denoised audio segment into a spectrogram by using an audio data processing library (such as a Librosa audio processing library).
Step (2): and inputting the spectrogram into a first neural network model obtained by pre-training to obtain a first result of the target machine.
The neural network model may be a convolutional neural network, and since the spectrograms generated by the target machine in different working states have differences in frequency, strength, and the like, the obtained spectrograms are input into a first neural network model obtained by pre-training, and a state recognition result corresponding to the input spectrograms can be obtained and recorded as a first result. In practical applications, the target machine may be a coal mining machine, the operating states include a normal operating state, a shutdown state and an abnormal operating state, wherein the abnormal operating state includes an empty cutter running state and a floor cutting state, and the first result may be presented in the form of an array including probabilities that the spectrogram is each operating state, such as the first result may be [0.9, 0.02, 0.08], wherein 0.9 is the probability that the coal mining machine is in the normal operating state, 0.02 is the probability that the coal mining machine is in the shutdown state, and 0.08 is the probability that the coal mining machine is in the abnormal operating state.
The first neural network model is obtained by training based on a spectrogram sample marked with an operating state, and a plurality of spectrograms marked with different operating states are input into the first neural network model for training until a preset iteration number is reached, so that the trained first neural network model is obtained. In order to improve the identification accuracy of the first neural network model, the spectrogram sample may include spectrograms of the coal mining machine in each operating state, see the normal coal cutting spectrogram of the coal mining machine shown in fig. 3, the spectrograms of the coal mining machine in the normal operating state shown in fig. 3, and the spectrograms of the coal mining machine in the idle running knife spectrogram shown in fig. 4, the spectrograms of the coal mining machine in the idle running knife spectrogram in the abnormal state shown in fig. 4, see the spectrograms of the cutting floor of the coal mining machine shown in fig. 5, and the spectrograms of the coal mining machine in the abnormal cutting floor shown in fig. 5, as can be seen from fig. 3 to 5, the frequencies and the intensities of the spectrograms of the coal mining machine in each operating state are different, and the first neural network model may determine the operating state corresponding to the spectrograms according to the sound vibration frequencies and the vibration intensities in the spectrograms.
Each spectrogram in the spectrogram sample is obtained by converting an audio segment with a preset time length, and the lengths of the audio segments are the same, and when the lengths of the audio segments are inconsistent, the recognition accuracy is easily caused to be low in later spectrogram recognition, so that spectrograms used in the training and recognition processes of the neural network model are obtained by converting the audio segments with the same length, and the length of the audio segment can be 4s, for example.
In order to improve the accuracy of the second result, the present embodiment provides two implementation manners for obtaining the second result based on the operation state of the target machine identified by the operation image, and the implementation manners may specifically refer to the following implementation manners:
the first implementation mode comprises the following steps: and inputting the running image into a second neural network model obtained by pre-training to obtain a second result of the target machine. The image acquisition device is used for acquiring the running image of the target machine, and can be arranged right above or right in front of the target machine so as to shoot the working process of the target machine. Because the positions of the parts of the target machine in different running states are different and the positions or the quantity of the materials on the target machine in different running states are different, the running image of the target machine is input into a second neural network model obtained by pre-training, and the running state of the target machine can be identified and obtained and recorded as a second result. The second result can also be presented in the form of an array, wherein the array comprises probability values of the operation images input into the second neural network model as the operation states.
The second neural network model can identify the coordinate positions of the two rollers and the coal breakage image in the running image, and obtain a second result according to the coordinate positions of the two rollers and the coal breakage image. For example, when the second neural network model identifies that two rollers of the coal mining machine in the running image are not in the same horizontal line and the image within the preset distance range from the rollers contains the coal dropping image, the probability value corresponding to the normal running state in the obtained second result is the maximum, namely the coal mining machine is in the normal running state; because the coal-dropping position of the coal mining machine can be changed during the operation, the operation image input into the second neural network model can be one operation image or a plurality of continuous frame operation images, if the second neural network identifies that two rollers of the coal mining machine are not positioned on the same horizontal line in the adjacent first preset number of frame images, and the images within the preset distance range from the rollers contain the operation images of the coal-dropping images, the coal mining machine is determined to be in a normal operation state; when the second neural network model identifies that two rollers of the coal mining machine are positioned on the same horizontal line in the adjacent second preset number of running images and keep the positions unchanged, the probability value corresponding to the shutdown state in the obtained second result is the maximum, namely the coal mining machine is in the shutdown state; when the second neural network model identifies that two rollers of the coal mining machine are positioned on the same horizontal line and have position changes in the adjacent third preset number of running images, the probability value corresponding to the abnormal running state in the obtained second result is the maximum, namely the coal mining machine is in the abnormal running state.
The second neural network model is obtained by training based on running image samples marked with running states, and in order to improve the accuracy of the second neural network model, the running image samples comprise running image samples in each running state. The method comprises the steps of marking a running image sample of a target machine, marking the position of a part or the position or the quantity of materials on the target machine in the running image, marking the running state corresponding to the running image, inputting a plurality of marked running image samples into a second neural network model, and training the second neural network model based on the plurality of running image samples until a preset iteration number is reached to obtain a trained second neural network model.
The second embodiment: and acquiring parameter information related to the running state of the target machine according to the running image, and determining the running state of the target machine based on the parameter information to obtain a second result. In this embodiment, the operation image includes a plurality of continuous frame operation images, the target machine is a coal mining machine, and the parameter information related to the operation state of the target machine includes position information of two rollers of the coal mining machine and an image within a preset distance range from the rollers. The operation states of the coal mining machine comprise a normal operation state, a stop state, an idle running state and an idle running state. And obtaining the running state of the target machine according to the position information of the two rollers of the coal mining machine in the running image and whether the image within a preset distance range from the rollers has a coal falling image.
Specifically, the positions of two rollers in the running image of the coal mining machine can be identified to obtain the position information of the two rollers, and the running state of the coal mining machine is determined according to the position information of the two rollers in the multiple continuous frame images and whether the image within a preset distance range from the upper roller contains information such as a coal breakage image.
And when the two rollers of the coal mining machine in the first frame image are not positioned on the same horizontal line, the position of the upper roller of the coal mining machine in the second frame image is different from the position of the upper roller of the coal mining machine in the first frame image, and the image in the second frame image within a preset distance range from the upper roller contains a coal breakage image, determining that the coal mining machine is in a normal operation state. In practical applications, the position information corresponding to each roller in the running image can be identified by a convolutional neural network method, wherein the position information comprises a center coordinate, a height and a width corresponding to each roller position. And when the absolute difference value between the vertical coordinates in the central coordinates of the two rollers is larger than the preset percentage value of the height, determining that the two rollers are not in the same horizontal line, otherwise, determining that the two rollers are in the same horizontal line. Whether the images within the preset distance range from the upper roller contain the coal breakage images or not is identified by utilizing a pre-constructed convolutional neural network model, wherein the pre-constructed convolutional neural network model is a convolutional neural network model obtained after a first sample and a second sample are trained together, and the first sample is an image sample containing the coal breakage images within the preset distance range of the upper roller; the second sample is a sample which does not contain the coal breakage image within the preset distance range of the upper roller, and the images within the preset distance range of the upper roller are identified by preset identification in the first sample and the second sample.
When two rollers of the coal mining machine in the first frame image are not in the same horizontal line, the position of the upper roller of the coal mining machine in the second frame image is different from the position of the upper roller of the coal mining machine in the first frame image, and an image in the second frame image within a preset distance range from the upper roller does not contain a coal falling image, recording a first moment, and identifying images after the second frame image in a frame-by-frame mode within the first preset time by taking the first moment as a start, and determining that the coal mining machine is in a normal operation state when the image within the preset distance range from the upper roller contains the coal falling image. Or when the two rollers of the coal mining machine in the first frame image are not in the same horizontal line, the position of the upper roller of the coal mining machine in the second frame image is the same as the position of the upper roller of the coal mining machine in the first frame image, and the image contained in the second frame image within the preset distance range from the upper roller contains the coal breakage image, determining that the coal mining machine is in a normal operation state.
When the two rollers are determined to be in the same horizontal line according to the position information of the two rollers of the coal mining machine in the first frame image, recording a second moment, identifying whether the position of the upper roller of the coal mining machine in the 2+ i frame image is the same as the position of the upper roller in the 1+ i frame image or not within a second preset time in a frame-by-frame mode by taking the second moment as a starting point, and indirectly determining that the coal mining machine is in a normal running state or in an idle running state, wherein i is a positive integer, i is sequentially subjected to progressive value taking, and the initial value is 1.
When two rollers of the coal mining machine in the first frame image are not in the same horizontal line, the position of the upper roller of the coal mining machine in the second frame image is different from the position of the upper roller of the coal mining machine in the first frame image, the images in the 2+ i frame image after the first moment are identified in a frame-by-frame mode within a preset distance range from the upper roller without coal drop images, and the frame-by-frame identification time exceeds the first preset time, the coal mining machine is determined to be in an idle running state, or when the position information of the two rollers of the coal mining machine in the first frame image in two adjacent frame images determines that the two rollers are in the same horizontal line, the position of the two rollers of the coal mining machine in the image after the second moment is identified in a frame-by-frame mode, the frame-by-frame identification time exceeds the second preset time, and the position of the two rollers of the coal mining machine in the last frame image within the second preset time is determined to be different from the position of the two rollers of the coal mining machine, and determining that the coal mining machine is in an idle running state.
The parameter information also comprises coal block images on the scraper conveyor. When the two rollers of the coal mining machine corresponding to the first frame image are determined not to be in the same horizontal line according to the position information of the two rollers of the coal mining machine in the two adjacent frame images, the position of the upper roller of the coal mining machine corresponding to the second frame image in the two adjacent frame images is the same as the position of the upper roller of the coal mining machine in the first frame image, and no coal falling image is identified in the image within a preset distance range from the upper roller in the 2+ i frame image, if the coal blocks on the scraper conveyor are determined not to move according to the coal block images on the scraper conveyor included in the 2+ i frame image, the coal mining machine is determined to be in an abnormal halt state. Or if the coal on the scraper conveyor is determined to move according to the coal image on the scraper conveyor contained in the 2+ i frame image, recording a third moment; identifying images containing the coal block images on the scraper conveyor in a frame-by-frame mode within third preset time from a third moment, and if the coal blocks on the scraper conveyor are determined not to move, determining that the coal mining machine is in an abnormal shutdown state; or if the time of frame-by-frame identification exceeds the third preset time and the coal block on the scraper conveyor contained in the 2+ j frame image is determined to move from the third moment, determining that the coal mining machine is in an idle running state, wherein j is a positive integer greater than or equal to i, j is sequentially subjected to progressive value taking, and the initial value is 1.
The parameter information also comprises a coal block image on the scraper conveyor, when the two rollers of the coal mining machine in the first frame image are not positioned on the same horizontal line, the positions of the two rollers of the coal mining machine in the image after the second moment are identified to be positioned on the same horizontal line in a frame-by-frame mode, the frame-by-frame identification time exceeds a second preset time, and the positions of the two rollers of the coal mining machine in the last frame image in the second preset time are determined to be the same as the positions of the two rollers of the coal mining machine in the previous frame image; and if the coal mining machine is determined to be in a normal shutdown state when the coal on the scraper conveyor contained in the last frame image in the second preset time is determined not to move according to the coal images on the scraper conveyor contained in the last frame image in the second preset time and the coal images on the scraper conveyor contained in the last frame image in the second preset time. And if the coal on the scraper conveyor contained in the last frame image in the second preset time is determined to move according to the coal block image on the scraper conveyor contained in the last frame image in the second preset time and the coal block image on the scraper conveyor contained in the last frame image in the second preset time, recording the fourth time, identifying that the coal on the scraper conveyor contained in the image after the fourth time does not move in a frame-by-frame mode, and determining that the coal mining machine is in an idle state when the frame-by-frame identified time exceeds the third preset time.
According to the machine running state detection method provided by the embodiment, the running state of the target machine can be detected in real time by collecting and identifying the sound generated by the running of the field machine in real time and collecting and identifying the sound generated by the field machine in real time, so that related personnel can be timely notified to maintain the equipment when the machine breaks down, the labor cost is reduced, the problem that the machine cannot be timely found to be abnormal due to insufficient experience of workers on the production field is avoided, and the accuracy of machine running state detection is improved.
Example two:
for the method for detecting the operating state of the machine provided in the second embodiment, an example of performing fault detection on the coal mining machine by using the method for detecting the operating state of the machine provided in the second embodiment of the present invention is provided, referring to the flowchart for detecting the operating state of the coal mining machine shown in fig. 6, the following steps S502 to S510 may be specifically referred to for execution:
step S502: and acquiring a working image in the coal mining process of the coal mining machine, and identifying parameter information related to the running state of the coal mining machine based on the working image.
Step S504: and determining the operation state of the coal mining machine according to the parameter information related to the operation state of the coal mining machine to obtain a first result.
Step S506: and acquiring audio data in the coal mining process of the coal mining machine, and identifying the working state of the coal mining machine according to a voice recognition algorithm to obtain a second result.
Step S508: and performing data fusion on the first result and the second result according to a preset data fusion algorithm to obtain the final running state of the coal mining machine.
The preset data fusion algorithm can be a weighted average algorithm and endows corresponding weights to the first result and the second result, and the fusion calculation can be carried out on the first result and the second result based on a D-S evidence theory algorithm to obtain the final running state of the coal mining machine.
Step S510: and when the final running state of the coal mining machine is an abnormal running state, sending an alarm signal to prompt a worker to carry out fault maintenance.
The abnormal operation states comprise abnormal operation states of an empty running cutter, a cutting bottom plate and the like of the coal mining machine. By carrying out safety early warning monitoring on the coal mining machine, when the running state of the coal mining machine is found to be an abnormal running state, an alarm signal is sent out, the running fault of the coal mining machine can be automatically and timely found, and the production efficiency is ensured.
Example three:
for the method for detecting the operating state of the machine provided in the second embodiment, the embodiment of the present invention provides a device for detecting the operating state of the machine, referring to a schematic structural diagram of the device for detecting the operating state of the machine shown in fig. 7, the device includes the following modules:
and the state determining module 61 is configured to acquire audio data of the target machine, determine an operating state of the target machine based on the audio data, and obtain a first result.
And the state identification module 62 is configured to acquire an operation image of the target machine, identify an operation state of the target machine based on the operation image, and obtain a second result.
And the data fusion module 63 is configured to perform data fusion on the first result and the second result to obtain a state detection result of the target machine.
According to the device for detecting the running state of the machine, the first result obtained based on the running audio data of the target machine and the second result obtained based on the running image of the target machine are subjected to data fusion, the state detection result of the target machine can be automatically obtained, the labor cost is saved, the running state is comprehensively judged through the sound and the image based on the running of the target machine, and the accuracy of the state detection of the target machine is improved.
In an embodiment, the data fusion module 63 is further configured to perform fusion calculation on the first result and the second result based on a D-S evidence theory algorithm to obtain a state detection result of the target machine.
In an embodiment, the data fusion module 63 is further configured to determine a status detection result of the target machine based on the first result, the second result and the fusion calculation formula.
In one embodiment, the target machine comprises a coal mining machine, and the operating state comprises a normal operating state, a shutdown state and an abnormal operating state; the fusion calculation formula is:
wherein m (A) is a state detection result, m1(A1) As a first result, m2(A2) The second result comprises probability values of the coal mining machine in each operating state, A1For determining events of the operating state of the target machine on the basis of the audio data, A2To identify an event of the running state of the target machine based on the running image, k is a collision factor.
In an embodiment, the state determining module 61 is further configured to collect audio data during the operation of the target machine, and determine a spectrogram generated by the operation of the target machine based on the audio data; inputting the spectrogram into a first neural network model obtained by pre-training to obtain a first result of a target machine; the first neural network model is obtained by training based on a spectrogram sample marked with an operation state.
In an embodiment, the audio data is divided into audio segments of a preset duration; and denoising each audio segment, and converting each audio segment into a corresponding spectrogram by using an audio data processing library.
In an embodiment, the state recognition module 62 is further configured to input the running image into a second neural network model obtained through pre-training, so as to obtain a second result of the target machine; the second neural network model is obtained by training based on a running image sample marked with a running state; or acquiring parameter information related to the running state of the target machine according to the running image, and determining the running state of the target machine based on the parameter information to obtain a second result.
The above-mentioned machine running state detection device that this embodiment provided, through the sound and the discernment of gathering the operation of on-the-spot machine in real time, and gather the sound and the discernment of the machine scene in real time, the running state that can the real-time detection target machine, so that can in time inform relevant personnel to carry out the equipment maintenance and overhaul when the machine breaks down, the human cost has been reduced, avoided simultaneously because of the not enough problem that causes in time to discover the machine anomaly of production site staff experience, the accuracy that the machine running state detected has been promoted.
The device provided by the embodiment has the same implementation principle and technical effect as the foregoing embodiment, and for the sake of brief description, reference may be made to the corresponding contents in the foregoing method embodiment for the portion of the embodiment of the device that is not mentioned.
Example four:
an embodiment of the present invention provides an electronic device, as shown in a schematic structural diagram of the electronic device shown in fig. 8, the electronic device includes a sound acquisition device (not shown in the figure), an image acquisition device (not shown in the figure), a processor 71, and a memory 72, where the image acquisition device is used to acquire audio data during an operation process of a target machine, the sound acquisition device is used to acquire an operation image of the target machine, a computer program that can be executed on the processor is stored in the memory, and the processor implements the steps of the method provided in the foregoing embodiment when executing the computer program.
Referring to fig. 8, the electronic device further includes: a bus 74 and a communication interface 73, and the processor 71, the communication interface 73 and the memory 72 are connected by the bus 74. The processor 71 is arranged to execute executable modules, such as computer programs, stored in the memory 72.
The Memory 72 may include a high-speed Random Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 73 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used.
The bus 74 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 8, but that does not indicate only one bus or one type of bus.
The memory 72 is configured to store a program, and the processor 71 executes the program after receiving an execution instruction, and the method executed by the apparatus defined by the flow process disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 71, or implemented by the processor 71.
The processor 71 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 71. The Processor 71 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like. The device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory 72, and the processor 71 reads the information in the memory 72 and performs the steps of the above method in combination with hardware thereof.
Example five:
embodiments of the present invention provide a computer-readable medium, wherein the computer-readable medium stores computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the method of the above-mentioned embodiments.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the system described above may refer to the corresponding process in the foregoing embodiments, and is not described herein again.
The method and the device for detecting the machine running state and the computer program product of the electronic device provided by the embodiments of the present invention include a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the method described in the foregoing method embodiments, and specific implementation may refer to the method embodiments, and will not be described herein again.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A method for detecting a machine operating state, comprising:
acquiring audio data in the running process of a target machine, and determining the running state of the target machine based on the audio data to obtain a first result;
acquiring an operation image of the target machine, and identifying the operation state of the target machine based on the operation image to obtain a second result;
and performing data fusion on the first result and the second result to obtain a state detection result of the target machine.
2. The method of claim 1, wherein the step of fusing the first result and the second result to obtain the status detection result of the target machine comprises:
and performing fusion calculation on the first result and the second result based on a D-S evidence theory algorithm to obtain a state detection result of the target machine.
3. The method according to claim 2, wherein the step of performing fusion calculation on the first result and the second result based on the D-S evidence theory algorithm to obtain the state detection result of the target machine comprises:
and determining a state detection result of the target machine based on the first result, the second result and a fusion calculation formula.
4. The method of claim 3, wherein the target machine comprises a shearer loader, the operating conditions include a normal operating condition, a shutdown condition, and an abnormal operating condition; the fusion calculation formula is:
wherein m (A) is the result of the state detection, m1(A1) As the first result, m2(A2) For the second result, the first result and the second result comprise probability values of the shearer in the respective operating states, A1To determine events of the operating state of the target machine based on the audio data, A2K is a collision factor for identifying an event of the operational state of the target machine based on the operational image.
5. The method of claim 1, wherein the step of obtaining audio data of the target machine during operation, determining the operation status of the target machine based on the audio data, and obtaining the first result comprises:
collecting audio data in the running process of a target machine, and determining a spectrogram generated by the running of the target machine based on the audio data;
inputting the spectrogram into a first neural network model obtained by pre-training to obtain a first result of the target machine; the first neural network model is obtained by training based on a spectrogram sample marked with an operation state.
6. The method of claim 5, wherein the step of determining a spectrogram produced by operation of the target machine based on the audio data comprises:
dividing the audio data into audio segments with preset duration;
and denoising the audio segments, and converting the audio segments into corresponding spectrograms by using an audio data processing library.
7. The method of claim 1, wherein the step of identifying the operational status of the target machine based on the operational image to obtain a second result comprises:
inputting the running image into a pre-trained second neural network model to obtain a second result of the target machine; the second neural network model is obtained by training based on a running image sample marked with a running state;
alternatively, the first and second electrodes may be,
and acquiring parameter information related to the running state of the target machine according to the running image, and determining the running state of the target machine based on the parameter information to obtain a second result.
8. A machine operation state detection device, characterized by comprising:
the state determining module is used for acquiring audio data of a target machine, determining the running state of the target machine based on the audio data and obtaining a first result;
the state identification module is used for acquiring the running image of the target machine, identifying the running state of the target machine based on the running image and obtaining a second result;
and the data fusion module is used for carrying out data fusion on the first result and the second result to obtain a state detection result of the target machine.
9. An electronic device, comprising: the device comprises a sound acquisition device, an image acquisition device, a processor and a storage device;
the image acquisition device is used for acquiring audio data in the running process of the target machine;
the sound acquisition device is used for acquiring an operation image of the target machine;
the storage device has stored thereon a computer program which, when executed by the processor, performs the method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of the claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010809719.5A CN112733588A (en) | 2020-08-13 | 2020-08-13 | Machine running state detection method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010809719.5A CN112733588A (en) | 2020-08-13 | 2020-08-13 | Machine running state detection method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112733588A true CN112733588A (en) | 2021-04-30 |
Family
ID=75597144
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010809719.5A Pending CN112733588A (en) | 2020-08-13 | 2020-08-13 | Machine running state detection method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112733588A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113376457A (en) * | 2021-05-10 | 2021-09-10 | 首钢京唐钢铁联合有限责任公司 | Equipment operation state detection method, system, device, equipment and medium |
| CN113641544A (en) * | 2021-08-23 | 2021-11-12 | 北京百度网讯科技有限公司 | Method, apparatus, device, medium and product for detecting application status |
| CN113743355A (en) * | 2021-09-15 | 2021-12-03 | 中国南方电网有限责任公司超高压输电公司大理局 | Method, device and system for checking state of switch device and computer equipment |
| CN114964467A (en) * | 2022-03-21 | 2022-08-30 | 珠海格力电器股份有限公司 | Equipment detection method and device, sound collection equipment and storage medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102528225A (en) * | 2012-02-09 | 2012-07-04 | 上海市特种设备监督检验技术研究院 | Sound signal transduction and prediction method of GTAW (gas tungsten arc welding) welding fusion penetration state |
| CN106826846A (en) * | 2017-01-06 | 2017-06-13 | 南京赫曼机器人自动化有限公司 | The intellect service robot and method driven based on abnormal sound and image event |
| CN107272607A (en) * | 2017-05-11 | 2017-10-20 | 上海斐讯数据通信技术有限公司 | A kind of intelligent home control system and method |
| CN108830457A (en) * | 2018-05-14 | 2018-11-16 | 国网辽宁省电力有限公司 | Operation of power networks monitoring and control panorama management system based on artificial intelligence |
| CN109669087A (en) * | 2019-01-31 | 2019-04-23 | 国网河南省电力公司 | A kind of method for diagnosing fault of power transformer based on Multi-source Information Fusion |
| CN109903392A (en) * | 2017-12-11 | 2019-06-18 | 北京京东尚科信息技术有限公司 | Augmented reality method and apparatus |
| CN110206748A (en) * | 2019-06-12 | 2019-09-06 | 精英数智科技股份有限公司 | A kind of methods, devices and systems of monitoring lock between air and electricity experiment |
| CN110400448A (en) * | 2019-07-29 | 2019-11-01 | 精英数智科技股份有限公司 | A kind of coalcutter operating status judgment method, system and storage medium |
| CN110556129A (en) * | 2019-09-09 | 2019-12-10 | 北京大学深圳研究生院 | Bimodal emotion recognition model training method and bimodal emotion recognition method |
| CN110659676A (en) * | 2019-09-09 | 2020-01-07 | 卓尔智联(武汉)研究院有限公司 | Information processing method, device and storage medium |
| CN110867196A (en) * | 2019-12-03 | 2020-03-06 | 桂林理工大学 | Machine equipment state monitoring system based on deep learning and voice recognition |
| CN111199257A (en) * | 2020-01-10 | 2020-05-26 | 中国铁道科学研究院集团有限公司电子计算技术研究所 | Fault diagnosis method and device for high-speed rail driving equipment |
| CN111362089A (en) * | 2020-04-30 | 2020-07-03 | 郑州大学 | Method and system for identifying entering of electric vehicle into lift car through artificial intelligence |
-
2020
- 2020-08-13 CN CN202010809719.5A patent/CN112733588A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102528225A (en) * | 2012-02-09 | 2012-07-04 | 上海市特种设备监督检验技术研究院 | Sound signal transduction and prediction method of GTAW (gas tungsten arc welding) welding fusion penetration state |
| CN106826846A (en) * | 2017-01-06 | 2017-06-13 | 南京赫曼机器人自动化有限公司 | The intellect service robot and method driven based on abnormal sound and image event |
| CN107272607A (en) * | 2017-05-11 | 2017-10-20 | 上海斐讯数据通信技术有限公司 | A kind of intelligent home control system and method |
| CN109903392A (en) * | 2017-12-11 | 2019-06-18 | 北京京东尚科信息技术有限公司 | Augmented reality method and apparatus |
| CN108830457A (en) * | 2018-05-14 | 2018-11-16 | 国网辽宁省电力有限公司 | Operation of power networks monitoring and control panorama management system based on artificial intelligence |
| CN109669087A (en) * | 2019-01-31 | 2019-04-23 | 国网河南省电力公司 | A kind of method for diagnosing fault of power transformer based on Multi-source Information Fusion |
| CN110206748A (en) * | 2019-06-12 | 2019-09-06 | 精英数智科技股份有限公司 | A kind of methods, devices and systems of monitoring lock between air and electricity experiment |
| CN110400448A (en) * | 2019-07-29 | 2019-11-01 | 精英数智科技股份有限公司 | A kind of coalcutter operating status judgment method, system and storage medium |
| CN110556129A (en) * | 2019-09-09 | 2019-12-10 | 北京大学深圳研究生院 | Bimodal emotion recognition model training method and bimodal emotion recognition method |
| CN110659676A (en) * | 2019-09-09 | 2020-01-07 | 卓尔智联(武汉)研究院有限公司 | Information processing method, device and storage medium |
| CN110867196A (en) * | 2019-12-03 | 2020-03-06 | 桂林理工大学 | Machine equipment state monitoring system based on deep learning and voice recognition |
| CN111199257A (en) * | 2020-01-10 | 2020-05-26 | 中国铁道科学研究院集团有限公司电子计算技术研究所 | Fault diagnosis method and device for high-speed rail driving equipment |
| CN111362089A (en) * | 2020-04-30 | 2020-07-03 | 郑州大学 | Method and system for identifying entering of electric vehicle into lift car through artificial intelligence |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113376457A (en) * | 2021-05-10 | 2021-09-10 | 首钢京唐钢铁联合有限责任公司 | Equipment operation state detection method, system, device, equipment and medium |
| CN113641544A (en) * | 2021-08-23 | 2021-11-12 | 北京百度网讯科技有限公司 | Method, apparatus, device, medium and product for detecting application status |
| CN113641544B (en) * | 2021-08-23 | 2024-04-26 | 北京百度网讯科技有限公司 | Method, apparatus, device, medium and product for detecting application state |
| CN113743355A (en) * | 2021-09-15 | 2021-12-03 | 中国南方电网有限责任公司超高压输电公司大理局 | Method, device and system for checking state of switch device and computer equipment |
| CN113743355B (en) * | 2021-09-15 | 2024-01-09 | 中国南方电网有限责任公司超高压输电公司大理局 | Switch device state checking method, device, system and computer equipment |
| CN114964467A (en) * | 2022-03-21 | 2022-08-30 | 珠海格力电器股份有限公司 | Equipment detection method and device, sound collection equipment and storage medium |
| CN114964467B (en) * | 2022-03-21 | 2023-01-13 | 珠海格力电器股份有限公司 | Equipment detection method and device, sound collection equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112733588A (en) | Machine running state detection method and device and electronic equipment | |
| CN110191094B (en) | Abnormal data monitoring method and device, storage medium and terminal | |
| CN108965826B (en) | Monitoring method, monitoring device, processing equipment and storage medium | |
| CN109981328B (en) | Fault early warning method and device | |
| US7467067B2 (en) | Self-learning integrity management system and related methods | |
| CN112712113B (en) | Alarm method, device and computer system based on index | |
| US20230005176A1 (en) | Throwing position acquisition method and apparatus, computer device and storage medium | |
| CN110609780B (en) | Data monitoring method and device, electronic equipment and storage medium | |
| CN111275951A (en) | Information processing method, device and equipment and computer storage medium | |
| CN111753875A (en) | Power information system operation trend analysis method and device and storage medium | |
| CN108306997B (en) | Domain name resolution monitoring method and device | |
| CN116302809A (en) | Edge end data analysis and calculation device | |
| CN115794532A (en) | Multi-index intelligent dynamic threshold monitoring method and system | |
| CN113420800A (en) | Data anomaly detection method and device | |
| CN114285612B (en) | Method, system, device, equipment and medium for detecting abnormal data | |
| CN117421692A (en) | Garbage illegal delivery identification method, device and equipment for garbage delivery station | |
| CN112862220A (en) | Equipment life cycle prediction method and device and related equipment | |
| CN109714196B (en) | Data monitoring method and platform | |
| US20220343113A1 (en) | Automatic model reconstruction method and automatic model reconstruction system for component recognition model | |
| CN113535697B (en) | Climbing frame data cleaning method, climbing frame control device and storage medium | |
| CN113958463A (en) | Online monitoring method, system and device for fan blade | |
| CN112486773A (en) | Data processing method, device and system | |
| CN111258866A (en) | Computer performance prediction method, device, equipment and readable storage medium | |
| CN114387540A (en) | Deployment and control supervision method, device and system | |
| CN118051863B (en) | Health data acquisition system and method based on digital metering technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |