KR20200128239A - Mechanical diagnostic system based on image learning and method for mechanical diagnosis using the same - Google Patents

Abstract

In a machine diagnosis system through image learning and a machine diagnosis method using the same, the machine diagnosis system includes a photographing unit, a selection unit, a learning unit, and a determination unit. The photographing unit continuously photographs images of an object. The selection unit includes a pre-classification unit selecting images of a predetermined section to be learned from the continuously photographed images, and a determination criterion input unit providing a determination criterion for a normal state or an abnormal state of the object. The learning unit learns a state of the object based on the determination criterion with respect to the images of the selected predetermined section. The determination unit determines the state of the object from the continuously photographed images using the learned result.

Description

Machine diagnosis system through image learning and machine diagnosis method using the same {MECHANICAL DIAGNOSTIC SYSTEM BASED ON IMAGE LEARNING AND METHOD FOR MECHANICAL DIAGNOSIS USING THE SAME}

The present invention relates to a machine diagnosis system through image learning and a machine diagnosis method using the same, and more particularly, to a normal state of an object such as a mechanical device by performing learning based on an image captured by a photographing device such as a camera. The present invention relates to a machine diagnosis system through image learning capable of performing diagnosis by performing a judgment on an abnormal state, and a machine diagnosis method using the same.

A number of studies are being conducted on the technology to measure the motion of an object by using the captured image, and its application range is expanding with the advancement of high-speed shooting techniques and image processing technologies.

Recently, MIT CSAIL (Computer Science and Artificial Intelligence Laboratory) published a study to amplify the fine motion of an image without inputting prior information such as frequency of an object using deep learning, and RDI Technologies amplifies and visualizes the vibration behavior. Developed and released the equipment. These techniques can be utilized in a technique for determining and diagnosing changes in macroscopic movements with human eyes by amplifying and visualizing microscopic mechanical vibrations of a rotating machine such as a pump into large motions.

However, in the case of the above-described visualization technology, the final reading of an image is an area of an expert who still has physical knowledge and experience of an object, and there is an uncertainty in which the reading result may differ from person to person.

Meanwhile, as a technology related to a method or apparatus for diagnosing an object based on a deep learning technology, Korean Patent Registration No. 10-1818394 extracts a feature based on data acquired from a facility to generate a feature map image, and Disclosing a technique for diagnosing the state of equipment by performing learning.

In addition, Korean Patent Registration No. 10-0199105 is a method to find out the cause of an abnormality by analyzing a vibration signal from a rotating body such as a motor or an engine. Disclosing technology to predict.

As described above, the technology for diagnosing the state of an object by learning based on the current deep learning technology is information processed through a separate signal processing step, and the effectiveness of learning by using the processed information. Although can be improved, since a signal processing step for information processing is required, data processing and learning time increases, and there is a problem that the versatility is not high because there is a limitation applied only to the state of the object or the state to be diagnosed.

Korean Patent Registration No. 10-1818394 Korean Patent Registration No. 10-0199105

Accordingly, the technical problem of the present invention is conceived in this respect, and an object of the present invention is to learn by using the original image measured by the photographing device and not image-processed for a dynamic system including a moving or vibrating mechanical device. Because it is possible to determine the normal state or abnormal state of the object by using the device, it is possible to easily monitor the vibration of the device at all times by omitting a separate data processing step, and to improve the usability of the machine through image learning. It is to provide a diagnostic system.

Another object of the present invention is to provide a machine diagnosis method using the machine diagnosis system.

A machine diagnosis system according to an embodiment for realizing the object of the present invention includes a photographing unit, a selection unit, a learning unit, and a determination unit. The photographing unit continuously photographs images of the object. The selection unit includes a pre-classification unit that selects images of a predetermined section to be learned from among the continuously photographed images, and a determination criterion input unit that provides a determination criterion for a normal state or an abnormal state of the object. The learning unit learns the state of the object based on the determination criterion with respect to the images of the selected predetermined section. The determination unit determines the state of the object from the consecutively photographed images using the learned result.

In an embodiment, the learning unit may learn the state of the object using a convolution neural network (CNN).

In one embodiment, information input for providing information on whether the learning unit learns with a 2D convolutional neural network, or learns with a 2D convolutional neural network using multiple frames or a 3D convolutional neural network to the photographing unit May contain more wealth.

In one embodiment, the photographing unit, when the learning unit learns with a two-dimensional convolutional neural network, photographs the object at a fixed position and angle, and the learning unit learns with a two-dimensional convolutional neural network using multiple frames, or 3 When learning with a dimensional convolutional neural network, the object may be photographed at a changed position or angle.

In one embodiment, the determination unit comprises a first determination unit that determines a normal state or an abnormal state of the object based on images of the object continuously photographed at the fixed position and angle, and is independent of the position or angle. It may include a second determination unit that determines a normal state or an abnormal state of the object based on images of the object continuously photographed over time.

In one embodiment, the object may be a vibration structure that vibrates in a fixed state.

In an embodiment, the criterion for determining the state of the object may be provided as a threshold for the vibration range of the vibration structure.

In an embodiment, the learning unit may learn a state of the object based on whether a difference between consecutively photographed images of the object falls within a range of the threshold value.

In one embodiment, the preceding classification unit provides images of the remaining sections except for images of a predetermined section to be learned from among the consecutively photographed images to the determination section, and the determination section includes images of the remaining sections. The state of the object may be determined.

In the machine diagnosis method according to an embodiment for realizing another object of the present invention described above, images of an object are continuously photographed. A criterion for determining a normal state or an abnormal state of the subject is provided. Among the continuously photographed images, images of a predetermined section to be learned are selected. With respect to the images of the selected predetermined section, the state of the object is learned based on the determination criterion. Using the learned result, the state of the object is determined from the continuously photographed images.

In one embodiment, prior to the step of capturing an image of the object, as a feature of the space to be photographed, whether the learning unit learns with a 2D convolutional neural network or learns with a 2D convolutional neural network using multiple frames Or, the step of receiving information on whether to learn with a 3D convolutional neural network may be further included.

In one embodiment, in the step of capturing an image of the object, when the learning unit learns with a 2D convolutional neural network, the learning unit captures the object at a fixed position and angle, and the learning unit uses a 2D convolutional neural network. When learning or learning with a 3D convolutional neural network, the object can be photographed at a changed position or angle.

According to embodiments of the present invention, for a dynamic system including a moving or vibrating mechanical device, a separate image processing is not performed on consecutive images of an object captured by the photographing unit, and an original image Because it is possible to perform judgment on the state of the object, that is, a normal state or an abnormal state by learning them as it is, it is possible to quickly and immediately process vibrations on the object, such as a dynamic system including the mechanical device, so that constant monitoring is performed. Can be done, and usability can be improved.

That is, since separate original image processing is not required, it is possible to monitor in real time whether an object is in a normal state, such as a dynamic system including a moving or vibrating mechanical device, even through a general-purpose image capturing device such as CCTV.

In particular, the learning is performed using a convolutional neural network, and by performing learning through the neural network to determine whether the vibration of the vibrating structure vibrating in a fixed state is in a normal state, diagnosis of the vibrating structure through neural network learning Can be performed accurately and effectively.

On the other hand, learning through the neural network may be divided into two-dimensional or three-dimensional learning, and in particular, when performing three-dimensional learning (or two-dimensional learning using multiple frames), it is irrelevant to the position or angle of the photographing unit. Since it can be determined by learning whether or not the vibration is in a normal state through the original image taken, real-time monitoring can be performed using cameras such as CCTV installed or moved in various locations for actual machine diagnosis.

In addition, among consecutively photographed images, images of a predetermined section are selected for learning and learning is performed, and images of the remaining sections other than the section may be provided to determine the state of the object. Even if separate image data is not provided, it is possible to learn and judge successive photographed images for the current state of the object, thereby improving the efficiency of machine diagnosis and enabling real-time monitoring.

1 is a schematic diagram showing a machine diagnosis system according to an embodiment of the present invention.
FIG. 2 is a schematic diagram illustrating an example of learning with a two-dimensional convolutional neural network (CNN) in the learning unit of FIG. 1.
FIG. 3A is a schematic diagram showing an example of learning with a two-dimensional CNN using a plurality of frames in the learning unit of FIG. 1, and FIG. 3B is a schematic diagram showing an example of learning with a three-dimensional CNN in the learning unit of FIG. 1.
FIG. 4A is a schematic diagram showing an example of a structure in which the learning unit of FIG. 1 performs learning, and FIG. 4B is a schematic diagram showing an example of a feature map in the structure of FIG. 4A.
FIG. 5A illustrates an example of an image photographed with respect to an object by the photographing unit of FIG. 1, and FIG. 5B illustrates an example of differences between images of the object photographed in FIG. 5A.
FIG. 6A illustrates another example of an image photographed with respect to an object by the photographing unit of FIG. 1, and FIG. 6B illustrates another example of differences between images of the object photographed in FIG. 6A.
7 is a graph showing an example of analyzing the frequency of the captured images of FIG. 6A, and FIG. 8 is a graph showing the determination result by the determination unit of FIG. 1 for each analyzed example of FIG. 7. .
9 is a flowchart illustrating a machine diagnosis method using the machine diagnosis system of FIG. 1.

The present invention will be described in detail in the text, since various modifications can be made and various forms can be obtained. However, this is not intended to limit the present invention to a specific form of disclosure, it is to be understood as including all changes, equivalents, or substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals have been used for similar elements. Terms such as first and second may be used to describe various components, but the components should not be limited by the terms.

These terms are used only for the purpose of distinguishing one component from another component. The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise.

In the present application, terms such as "comprise" or "consist of" are intended to designate the presence of features, numbers, steps, actions, elements, parts, or a combination thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance the possibility of being added.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

1 is a schematic diagram showing a machine diagnosis system according to an embodiment of the present invention. FIG. 2 is a schematic diagram illustrating an example of learning with a two-dimensional convolutional neural network (CNN) in the learning unit of FIG. 1. FIG. 3A is a schematic diagram showing an example of learning with a two-dimensional CNN using a plurality of frames in the learning unit of FIG. 1, and FIG. 3B is a schematic diagram showing an example of learning with a three-dimensional CNN in the learning unit of FIG. 1. FIG. 4A is a schematic diagram showing an example of a structure in which the learning unit of FIG. 1 performs learning, and FIG. 4B is a schematic diagram showing an example of a feature map in the structure of FIG. 4A. FIG. 5A illustrates an example of an image photographed with respect to an object by the photographing unit of FIG. 1, and FIG. 5B illustrates an example of differences between images of the object photographed in FIG. 5A. FIG. 6A illustrates another example of an image photographed with respect to an object by the photographing unit of FIG. 1, and FIG. 6B illustrates another example of differences between images of the object photographed in FIG. 6A.

First, referring to FIG. 1, the machine diagnosis system 10 according to the present embodiment provides a convolution neural network (CNN) for an object 100, which is a vibration structure that vibrates in a fixed state such as a pump or a motor. ) To perform learning, and to determine whether the vibration state of the object 100 is a normal state or an abnormal state based on this, and diagnose the object 100.

More specifically, the machine diagnosis system 10 includes a photographing unit 200, an information input unit 300, a selection unit 400, a learning unit 500, and a determination unit 600.

The photographing unit 200 photographs an image of the object 100 and may be a device capable of photographing, such as a camera or a CCTV.

The photographing unit 200 photographs continuous images of the object 100 at predetermined time intervals, and the photographed continuous photographed images are provided to the selection unit 400.

In this embodiment, the object 100 is a vibrating structure that vibrates as described above, and when the vibrating structure vibrates, the continuous photographed images captured by the photographing unit 200 are regions or parts where vibration is performed. Will change in

That is, as a result of comparing two consecutive images taken consecutively with each other, a difference occurs between the two consecutive images centering on a part where the vibration structure vibrates. That is, in the present embodiment, the learning unit 500 may perform learning based on the difference between the original image itself of the two consecutive images.

In this regard, referring to FIG. 5A, as an example of the object 100 by the photographing unit 200, an image 110 photographed of a vibrating structure in which a cantilever 101 is fixed on a vibrator Is shown.

That is, the photographing unit 200 acquires a continuous photographing image of the vibrating structure including the cantilever 101 as shown in FIG. 5A through continuous photographing of the object 100.

In this case, referring to FIG. 5B regarding the difference between the images of the object 100 photographed continuously, since the vibration structure is vibrating around the cantilever 101, only the structure centered on the cantilever 101 is It can be seen that it emerges from the image difference.

As another example of the object 100, referring to FIG. 6A, as another example of the object 100 by the photographing unit 200, an image 111 photographed with respect to a fixed vibration structure such as a pump 102 ) Is shown.

That is, the photographing unit 200 acquires a continuous photographing image of the pump 102 as shown in FIG. 6A through continuous photographing of the object 100.

In this case, referring to FIG. 6B regarding the difference between the images of the object 100 photographed continuously, the pump 102 is vibrating as a whole, so each part or element on which the vibration is performed is You can see that it emerges from the difference.

As described above, the learning unit 500 performs learning based on the difference between the original image itself of these two consecutive images, which will be described later, but the difference between the consecutive images, that is, the object manifested from the image difference Based on whether the difference between is within a predetermined range, it is learned whether there is a normal state or an abnormal state.

On the other hand, the photographing unit 200 may photograph the object 100 at a fixed position and angle according to the type of learning in the learning unit 500. As the position or angle changes, the object 100 ) Can also be taken.

That is, the learning unit 500 to be described later performs learning on the continuous photographed images captured by the photographing unit 200 using a so-called convolution neural network (CNN), which is a two-dimensional CNN. Alternatively, it can be learned with a two-dimensional CNN or a three-dimensional CNN using a plurality of frames.

When the learning unit 500 performs learning by using a 2D CNN, the learning unit 500 performs learning by extracting spatial features of the captured images, and the space of the captured images is It should remain the same.

Accordingly, if the learning unit 500 performs learning using a 2D CNN, the photographing unit 200 continuously photographs the object 100 at a fixed position and angle where the position and angle are not changed. Is to be performed, and the continuous images captured in this way are provided to the selection unit 400.

On the contrary, if the learning unit 500 performs learning using a two-dimensional CNN or a three-dimensional CNN using a plurality of frames, the learning unit 500 features time and frequency characteristics of the captured images. By extracting and classifying them, and performing learning, the space of the captured images does not need to be kept the same.

Thus, if the learning unit 500 performs learning using a two-dimensional CNN or a three-dimensional CNN using a plurality of frames, the photographing unit 200 continuously moves the object 100 while the position or angle is changed. Capturing can be performed, and continuous images captured in this way are provided to the selection unit 400.

When the learning unit 500 performs learning using the convolutional neural network, the information input unit 300 determines whether to learn with a two-dimensional CNN, or otherwise, a two-dimensional CNN or a three-dimensional CNN using a plurality of frames. Information on whether to learn is provided to the photographing unit 200.

Thus, the photographing unit 200 receives information on the type of learning performed by the learning unit 500 provided from the information input unit 300, and based on this, the photographing of the object 100 is performed. Select a method, that is, whether to perform photographing at a fixed position and angle, or to perform photographing while changing the position or angle.

In contrast, the information input unit 300 does not receive information on the type of learning performed by the learning unit 500, and information on a method of photographing from the photographing unit 200 to the information input unit 300 You can also provide

That is, if the photographing unit 200 can perform photographing of the object 100 only when the position and angle are fixed, corresponding information is provided to the information input unit 300 and the learning unit 500 By utilizing in, the learning unit 500 may perform learning with a 2D CNN.

Similarly, if the photographing unit 200 can perform photographing of the object 100 even when the position or angle is not fixed, corresponding information is provided to the information input unit 300 and the learning unit ( 500), the learning unit 500 may perform learning and learning using a 2D CNN or a 3D CNN using a plurality of frames.

As described above, the photographing unit 200 continuously photographs images of the object 100 and provides the photographed images to the selection unit 400.

The selection unit 400 includes a prior classification unit 410 and a determination criterion input unit 420.

The preceding classification unit 410 selects images to be learned from among consecutive images of the object 100 continuously photographed by the photographing unit 200, and the learning unit 500 Provided as

In this case, since the photographed images are consecutive images photographed at regular time intervals, the preceding classification unit 410 selects a time of a predetermined interval, and the continuous images photographed in the selected interval are subject to learning. Selected images are selected and provided to the learning unit 500.

The section selected by the preceding classification unit 410 may be set in various ways according to the user's selection, but must be set as a minimum section for the learning unit 500 to perform learning.

Meanwhile, images photographed in the section other than the section selected by the preceding classification unit 410 are provided to the determination unit 600.

That is, in the case of the present embodiment, images of a predetermined section among consecutively photographed images of the object 100 are subject to learning by the learning unit 500, and an image of a section that is not selected as an object of learning. Are provided to the determination unit 600, and finally become data for determining whether a state such as vibration of the object 100 is normal or abnormal.

As described above, by selecting an arbitrary section for continuous images photographed by the photographing unit 200 and performing learning, and determining the state of the object for the remaining sections, preparation of data for separate learning is performed. Since it can be omitted, learning and status determination can be performed in real time while continuously monitoring the object, so that the usability and utility of the machine diagnosis system can be improved.

In contrast, in the photographing unit 200, images subject to learning in the learning unit 500 may be separately photographed and provided to the learning unit 500, and the determination unit 600 determines Images that are the target of may be separately photographed and provided to the determination unit 600.

The determination criterion input unit 420 provides a criterion for determining a normal state or an abnormal state of the object 100, and a criterion may be set and provided by a user.

Unlike this, although not shown, the determination criterion input unit 420, based on information on the type of an object previously determined through a separate database, and a criterion in which risk occurs due to vibration in the object, the A criterion for determining a normal state or an abnormal state of the object 100 may be set by itself.

If the object 100 is a vibration structure that vibrates in a fixed state as illustrated above, the determination criterion provided to the determination criterion input unit 420 or set by the determination criterion input unit 420 is the vibration It may be a threshold for the vibration range of the structure.

That is, based on the threshold value of the vibration range of the vibration structure, if the threshold value is exceeded, the object 100 corresponds to an abnormal state, and if the object 100 vibrates within the threshold value, the object 100 Is corresponding to the normal state, and provides information on such a threshold to the learning unit 500.

The learning unit 500 selects the object 100 based on the determination criterion with respect to consecutive images of a predetermined section selected by the preceding classification unit 410 among consecutive images captured by the photographing unit 200. ) To perform learning.

In this case, through the preceding classification unit 410, consecutive images of a predetermined section may be provided, and as described above, consecutive images separately photographed for learning may be provided to the learning unit 500. .

The learning unit 500 performs learning on the provided consecutively photographed images. In this case, the learning performance method uses a convolutional neural network as described above.

That is, the learning unit 500, based on the provided determination criterion, for example, a threshold value for a vibration range, whether the object 100 corresponds to a normal state based on the continuously photographed images, Or, they learn whether they fall into an abnormal state.

Regarding the learning method of the learning unit 500, the learning unit 500 performs learning using a neural network, and a detailed internal learning algorithm is schematically illustrated in FIGS. 4A and 4B.

That is, in the learning unit 500 in the present embodiment, in performing learning using a neural network, as shown in FIG. 4A, when images for a plurality of consecutive objects 100 are provided, based on this, While changing the size of the image, a plurality of neural network circuits are performed to classify whether the state of the object 100 is in a normal state or an abnormal state to perform learning.

Through the learning performed by the learning unit 500, as shown in FIG. 4B, the image of the object 100 is gradually learned to be concrete, so that the learning of the overall shape of the object 100 is performed. Based on this, learning is performed to the extent that it is possible to judge the state of the object 100.

Meanwhile, in the case of the present embodiment, the learning unit 500 learns the vibration state of the object based on the original images. As described above, by comparing two consecutive images taken continuously, the vibration structure is vibrated. By doing so, learning can be performed by checking the part where the difference between the images occurs.

That is, through the difference between the images, the learning unit 500 may learn whether or not the range of the vibration of such a difference is within the range of the threshold value that is the provided criterion.

The steady state learning unit 510 may learn images in which a difference between the consecutively photographed images falls within a range of the threshold value, and learn about the normal state of the object 100.

Likewise, the abnormal state learning unit 520 may learn images in which the difference between the consecutively photographed images exceeds the range of the threshold value to perform learning on the abnormal state of the object 100.

On the other hand, it has been described that the learning unit 500 can learn with a 2D CNN, or otherwise, learn with a 2D CNN or a 3D CNN using multiple frames.

That is, referring to FIG. 2, when the learning unit 500 performs learning with a 2D CNN, the photographing unit 200 captures an image 110 of the object 100 at a fixed position and angle. The learning unit 500 is continuously photographed, and the learning unit 500 performs learning as described above through the continuous images 110 photographed in this way, and the state of the object 100 through the determination unit 600 to be described later. Judgment can be carried out.

When learning is performed with such a two-dimensional CNN, the learning unit 500 performs learning by extracting spatial features of the captured image, and for this purpose, the position and angle must be fixed.

In this case, the images of the object 100 that are the object of the state determination are also images captured at a fixed position and angle.

In contrast, referring to FIG. 3A, when the learning unit 500 performs learning with a 2D CNN using a plurality of frames, the photographing unit 200 includes a plurality of frames corresponding to various positions or angles. ), the image 120 of the object 100 is continuously photographed, and the learning unit 500, through the continuous images 120 photographed in this way, performs learning as described above, and will be described later. The determination unit 600 may determine the state of the object 100.

In this case, the images of the object 100 that are the object of the state determination are also images captured on a plurality of frames corresponding to various positions or angles.

Furthermore, referring to FIG. 3B, when the learning unit 500 performs learning with a 3D CNN, the photographing unit 200 includes images of the object 100 at various positions or angles in the 3D space ( 130) is continuously photographed, and the learning unit 500 performs learning as described above through the continuous images 130 photographed in this way, and the object 100 through the determination unit 600 to be described later. ) Can be performed.

In this case, the images of the object 100, which are the object of the state determination, are also images photographed at various positions or angles in the 3D space.

When learning is performed using a two-dimensional CNN or a three-dimensional CNN using a plurality of frames, the learning unit 500 performs learning by extracting a time or frequency feature of the photographed image. Their position or angle can be varied in various ways.

The determination unit 600 uses the result learned by the learning unit 500 to determine the object 100 from consecutive images of the object 100 selected and provided through the preceding classification unit 410. ).

Specifically, the determination unit 600 includes a first determination unit 610 and a second determination unit 620, wherein the first determination unit 610 is the object continuously photographed at the fixed position and angle. A normal state or an abnormal state of the object 100 is determined based on the images of 100, and the second determination unit 620 determines the object 100 continuously photographed over time, regardless of the position or angle. A normal state or an abnormal state of the object 100 is determined based on the images of ).

That is, when the learning unit 500 learns through a 2D CNN, the first determination unit 610 determines a normal state or an abnormal state of the object 100 based on the learned result, When the learning unit 500 learns through a two-dimensional CNN or a three-dimensional CNN using a plurality of frames, based on the learned result in the second determination unit 620, the normal state of the object 100 or Determine the abnormal state.

Of course, in the present embodiment, the determination unit 600 is divided into the first determination unit 610 and the second determination unit 620, and the subject of the determination is changed according to the learning method. The determination unit 600 may determine a normal state or an abnormal state of the object 100 in consideration of a learning method in one determination unit without being classified as described above.

Meanwhile, in the present embodiment, the determination unit 600 is a separate configuration from the learning unit 500, and the determination unit 600 determines the object 100 based on the result learned by the learning unit 500. ), but the determination unit 600 is formed integrally with the learning unit 500, and the object is based on images provided with the learning unit 500 The function of determining the state of (100) may be performed at the same time.

7 is a graph showing an example of analyzing the frequency of the captured images of FIG. 6A, and FIG. 8 is a graph showing the determination result by the determination unit of FIG. 1 for each analyzed example of FIG. 7. .

These are graphs of experimental results for verifying the accuracy of machine diagnosis through the machine diagnosis system 10 in this embodiment.

That is, FIGS. 7 and 8 show nine different vibrations with respect to the vibration state of the pump 102 when the object 100 is a fixed vibration structure such as the pump 102 as described with reference to FIG. 6A. The learning unit 500 performs learning based on images of the state (in this case, the images are images taken by the photographing unit at a fixed position and angle), and the pump (in this case, the determination unit 600) The result of determining whether the vibration state of 102) is a normal state or an abnormal state is shown.

However, in order to evaluate the accuracy of the determination by the determination unit 600, a separate frequency analysis was performed on the images for the nine different vibration states.

That is, referring to FIG. 7, as a result of frequency analysis for 9 different vibration states of the pump 102, only Case 1 and Case 9 are in a predetermined range, that is, a steady state that falls within the range of the threshold value described above. It was found to be, and the remaining Cases 2 to 8 were analyzed to be out of the range of the threshold value.

This frequency analysis result and the result of learning about the vibration state of the pump 102 using the machine diagnosis system 10 according to the present embodiment and the determination of the vibration state are compared, as shown in FIG. , The determination unit 600 determined that Case 1 and Case 2 were in a normal state, and Case 2 to Case 8 determined that it was an abnormal state.

As described above, based on images of nine different vibration states of the pump 102, the machine diagnostic system 10 judged the vibration state in the same manner as the frequency analysis result.

[Table 1] Diagnosis accuracy results using a machine diagnosis system

In addition, [Table 1] is a result of training and testing using an image of 120 frames per second (fps), and one image data is 1080*1920*3.

As confirmed through [Table 1], learning is performed using the machine diagnosis system 10 of the present embodiment, and based on this, a diagnosis of the vibration state of a vibration structure such as the pump 102 is performed. When doing so, it can be confirmed that the diagnosis accuracy of whether the vibration state is a normal state or an abnormal state is 100% without error.

Hereinafter, a machine diagnosis method using the machine diagnosis system 10 described with reference to FIG. 1 will be described with reference to the drawings. In addition, in the machine diagnosis method to be described below, since the configuration or function of each component of the machine diagnosis system 10 is substantially the same, the description will focus on the diagnosis step, but the configuration or function of the same component as previously described. A redundant description of is omitted.

9 is a flowchart illustrating a machine diagnosis method using the machine diagnosis system of FIG. 1.

Referring to FIG. 9, in the machine diagnosis method, first, the information input unit 300 provides information on spatial features to the photographing unit 200, and the photographing unit 200 provides information on the spatial characteristics. Information about is received (step S10).

In this case, the spatial feature is a spatial feature in which the object 100 photographed by the photographing unit 200 is located, and whether the learning unit 500 learns with a two-dimensional convolutional neural network or a plurality of It corresponds to information on whether to learn with a 2D convolutional neural network using frames or a 3D convolutional neural network.

Further, depending on the embodiment, without being provided with information on the spatial characteristics through the information input unit 300, the photographing unit 200 can shoot only in a state in which the position and angle are fixed. Information on whether this is possible can be provided to the information input unit 300 in reverse, and the information on the spatial features provided as such is provided to the learning unit 500, so that a method of performing learning can be selected. As described above.

Thereafter, the photographing unit 200 continuously photographs images of the object 100 (step S20).

In this case, when the learning unit 500 learns with a two-dimensional convolutional neural network, the photographing unit 200 photographs the object 100 at a fixed position and angle, and selects the photographed images. Provided at 400.

In contrast, when the learning unit 500 learns with a two-dimensional convolutional neural network or a three-dimensional convolutional neural network, the photographing unit 200 photographs the object 100 at a changed position or angle, The photographed images are provided to the selection unit 400.

Thereafter, the determination criterion input unit 420 of the selection unit 400 determines whether the determination criterion of the object 100, that is, the vibration state of the object 100 corresponds to a normal state or an abnormal state. The criterion for determining is provided (step S30).

In this case, the provided determination criterion may be determined by a user's input, and may be determined and provided through a separate database.

In addition, the determination criterion may be, for example, a threshold for a vibration range of the vibration structure in the case of a vibration structure that vibrates while the object 100 is fixed.

Thereafter, the preceding classification unit 410, among consecutively photographed images of the object 100 provided from the photographing unit 200, the predetermined section of the learning unit 500 Images are selected and provided to the learning unit 500 (step S40).

In this case, the selected section may be set to contain a minimum amount of information for performing learning in the learning unit 500, and among the consecutively photographed images, a predetermined section selected by the preceding classification unit 410 Images of other sections, excluding the images of sections, are provided to the determination unit 600 (step S60).

In contrast, in the photographing unit 200, continuous images for performing learning in the learning unit 500 may be separately photographed, and the continuous photographed images taken in this way may be provided to the learning unit 500. have.

Meanwhile, if the photographing unit 200 separately photographs images for learning and provides them to the learning unit, images for determining the state of the object 100 are also photographed separately, and the determination unit 600 It may be provided as (step S60).

On the other hand, the learning unit 500 learns whether the state of the object 100 is a normal state or an abnormal state based on the continuously photographed images to be learned (step S50).

In this case, the criterion of learning in the normal state or the abnormal state is applied to the determination criterion input through the determination criterion input unit 420.

The learning unit 500 may perform learning with a 2D convolutional neural network. Alternatively, learning may be performed with a 2D convolutional neural network using multiple frames or a 3D convolutional neural network.

This method of performing the learning unit may be selected in consideration of whether or not the photographing of the object can be performed only at a fixed position and angle in which the photographing of the photographing unit 200 is fixed. Unlike this, the information inputting unit 300 As described above, it may be selected based on the information on the spatial feature input through.

Further, a method of performing learning using a convolutional neural network in the learning unit 500 is the same as described above.

Thus, when the learning unit 500 completes learning whether the vibration state of the object 100 is in a normal state or an abnormal state, the determination unit 600 The photographed images are provided (step S60), and a normal state or an abnormal state of the object 100 is determined from the continuously photographed images based on the spatial feature (step S70).

In this case, the determination unit 600 includes a first determination unit 610 and a second determination unit 620, and in the first determination unit 610, the learning unit 500 is In the case of performing learning using a neural network, a normal state or an abnormal state of the object 100 is determined based on images of the object 100 continuously photographed at a fixed position and angle.

In contrast, in the second determination unit 620, regardless of a position or angle, when the learning unit 500 performs learning using a two-dimensional convolutional neural network or a three-dimensional convolutional neural network using multiple frames. A normal state or an abnormal state of the object 100 is determined based on images of the object 100 continuously photographed over time.

As described above, through the machine diagnosis method, learning about the vibration state of the vibrating structure is performed based on the original image of a vibrating structure such as a pump that vibrates in a fixed state, based on this. , It is possible to determine whether the vibrating structure vibrates in a range of a normal state or a range of an abnormal state.

According to the embodiments of the present invention as described above, for a dynamic system including a moving or vibrating mechanical device, a separate image processing is not performed on consecutive images of an object captured by the photographing unit. , As the original images can be learned as it is to determine the state of the object, that is, a normal state or an abnormal state, it is possible to quickly and immediately process the vibration of the object, such as a dynamic system including the mechanical device, Monitoring can be performed and usability can be improved.

That is, since separate original image processing is not required, it is possible to monitor in real time whether an object is in a normal state, such as a dynamic system including a moving or vibrating mechanical device, even through a general-purpose image capturing device such as CCTV.

In particular, the learning is performed using a convolutional neural network, and by performing learning through the neural network to determine whether the vibration of the vibrating structure vibrating in a fixed state is in a normal state, diagnosis of the vibrating structure through neural network learning Can be performed accurately and effectively.

On the other hand, learning through the neural network may be divided into two-dimensional or three-dimensional learning, and in particular, when performing three-dimensional learning (or two-dimensional learning using multiple frames), it is irrelevant to the position or angle of the photographing unit. Since it can be determined by learning whether or not the vibration is in a normal state through the original image taken, real-time monitoring can be performed using cameras such as CCTV installed or moved in various locations for actual machine diagnosis.

In addition, among consecutively photographed images, images of a predetermined section are selected for learning and learning is performed, and images of the remaining sections other than the section may be provided to determine the state of the object. Even if separate image data is not provided, it is possible to learn and judge successive photographed images for the current state of the object, thereby improving the efficiency of machine diagnosis and enabling real-time monitoring.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will be able to variously modify and change the present invention without departing from the spirit and scope of the present invention described in the following claims. You will understand that you can

10: mechanical diagnosis system 100: object
200: photographing unit 300: information input unit
400: selection unit 410: preceding classification unit
420: judgment criteria input unit 500: learning unit
510: steady state learning unit 520: abnormal state learning unit
600: judgment unit 610: first judgment unit
620: second judgment unit

Claims (12)

A photographing unit that continuously photographs images of the object;
A selection unit including a pre-classifying unit for selecting images of a predetermined section to be learned from among the continuously photographed images, and a determination criterion input unit for providing a determination criterion for a normal state or an abnormal state of the object;
A learning unit that learns the state of the object based on the determination criterion for the images of the selected predetermined section; And
A machine diagnosis system including a determination unit that determines a state of the object from the continuously photographed images using the learned result. The method of claim 1, wherein the learning unit,
A machine diagnostic system comprising learning the state of the object using a convolution neural network (CNN). The method of claim 2,
The machine further comprising an information input unit for providing information on whether the learning unit is to learn with a 2D convolutional neural network, a 2D convolutional neural network using multiple frames, or a 3D convolutional neural network to the photographing unit Diagnostic system. The method of claim 3, wherein the photographing unit,
When the learning unit learns with a 2D convolutional neural network, photographing the object at a fixed position and angle,
When the learning unit learns with a two-dimensional convolutional neural network using a plurality of frames or a three-dimensional convolutional neural network, the learning unit photographs the object at a changed position or angle. The method of claim 4, wherein the determination unit,
A first determination unit determining a normal state or an abnormal state of the object based on images of the object continuously photographed at the fixed position and angle; And
And a second determination unit that determines a normal state or an abnormal state of the object based on images of the object continuously photographed over time regardless of the position or angle. The method of claim 1, wherein the subject,
Machine diagnosis system, characterized in that the vibration structure vibrating in a fixed state. The method of claim 6, wherein the criteria for determining the state of the object are:
Machine diagnosis system, characterized in that provided as a threshold for the vibration range of the vibration structure. The method of claim 7, wherein the learning unit,
A machine diagnostic system, characterized in that the state of the object is learned based on whether a difference between the consecutively photographed images of the object falls within a range of the threshold value. The method of claim 1,
The preceding classification unit provides, to the determination unit, images of the remaining sections excluding images of a predetermined section to be learned from among the continuously photographed images,
And the determination unit determines a state of the object from images of the remaining section. Continuously photographing an image of the object;
Receiving a criterion for determining a normal state or an abnormal state of the subject;
Selecting images of a predetermined section to be learned from among the continuously photographed images;
Learning a state of the object based on the determination criterion with respect to the images in the selected predetermined section; And
And determining a state of the object from the continuously photographed images using the learned result. The method of claim 10, prior to the step of taking an image of the object,
The step of receiving information on whether the learning unit learns with a two-dimensional convolutional neural network, learning with a two-dimensional convolutional neural network using multiple frames, or learning with a three-dimensional convolutional neural network, as a feature of the space to be photographed. Machine diagnostic method further comprising. The method of claim 11, wherein in the step of taking an image of the object,
When the learning unit learns with a 2D convolutional neural network, photographing the object at a fixed position and angle,
When the learning unit learns with a two-dimensional convolutional neural network or a three-dimensional convolutional neural network, the object is photographed at a changed position or angle.

Images