KR102346676B1 - Method for creating damage figure using the deep learning-based damage image classification of facility - Google Patents

Abstract

The present invention relates to a method for generating a damage figure using deep learning-based facility damage image classification, which comprises: (a) augmenting, by an image data augmentation module, the number of acquired image data of a facility; (b) training, by a map learning module, a convolutional neural network (CNN) by using damage image data and non-damage image data, which complete labeling to designate the presence or absence of damage by class and pixel, as training data; (c) automatically classifying, by a damaged image classifier which is a CNN completing map learning, the damaged images by using test image data or arbitrary image data to acquire a damage figure as input data; (d) pre-processing, by a damage image data pre-processing module, the data classified as the damage image, which is output data through the damaged image classifier, to efficiently detect a damage area; (e) detecting, by a damage area detection module, the damage image area to increase accuracy of the damage figure and extract a damage shape pattern for the damage image completing the pre-processing operation; and (f) automatically generating, by a damage figure generating module, the damage figure by repeatedly using a feature vector and short line segment for the damage. Accordingly, since a deep learning technology is applied to detailed inspection of facilities, a method that has been carried out by manpower for a long time is replaced, and thus damage can be recognized objectively, promptly, and automatically.

Description

{Method for creating damage figure using the deep learning-based damage image classification of facility}

The present invention relates to a damage level generation method using deep learning-based facility damage image classification. It relates to a damage level generation method using deep learning-based facility damage image classification that enables damage recognition with

The advancement of domestic construction technology and the development of 4th industrial technology are gradually mass-producing high-rise, large-scale, and complex of facilities including buildings. In contrast, the current detailed inspection of buildings mainly relies on the investigator's visual observation, handwriting method, and partial photography, making it difficult to evaluate objectively and quantitatively, and it is impossible to observe the building at all times, and the results are managed in the form of a report. As a result, there is a problem that the utilization of information is low.

In this regard, artificial intelligence (AI), a rapidly evolving 4th industrial technology, especially deep learning technology, has been introduced in the detailed inspection of facilities including buildings, and the current method, which has been carried out for a long time centered on human resources, is innovative and reliable. We need to build this excellent system.

Existing studies have been conducted to automatically perform damage recognition on buildings, but the main focus is on the detection of specific damage (cracks) and the comparison of the performance of the deep learning network. In another previous study, there were many difficulties in accumulating damaged image data according to budget and period, and close-up and high-resolution image data were insufficient due to hardware problems. As a result, open datasets were learned. According to the Korea Institute of Construction Technology, the understanding of measures to secure image big data of facilities and the maintenance and management of actual facilities and regulatory procedures is poor, and it is unlikely that a condition assessment based on damage assessment will be developed. In particular, compared to studies on large structures such as tunnels, there is a lack of research that has progressed to the stage of practical use of AI-based precision inspection and diagnosis technology for buildings.

Therefore, in order to enable deep learning-based detailed inspection of buildings in the actual field, it is necessary not only to develop mechanical specifications for acquiring images of damage to buildings, but also to develop technical developments for acquiring high-quality data and to understand the status and methods of building maintenance.

Accordingly, the present invention intends to develop a technology for objectively, quickly, and quantitatively evaluating the damage of a facility based on artificial intelligence technology during a precise inspection of the facility. Through this, real-time monitoring of facilities is possible, various safety accidents and disasters can be prevented, and an efficient safety management system can be established.

Meanwhile, in the prior art, as disclosed in Korean Patent Publication No. 10-2095751 (announced on April 01, 2020), installing an acceleration measuring unit in a structure; obtaining a structure acceleration response measured by the acceleration measurement unit; training an artificial intelligence model using the structure acceleration response; generating a predictive modeling unit using the learning result of the artificial intelligence model; predicting a structure acceleration response using the predictive modeling unit; and determining whether the structure is damaged; in which the training of the artificial intelligence model includes defining the structure acceleration responses of the first measurement unit and the second measurement unit measured in the previous time step as input data, and then It is characterized in that an artificial intelligence model is trained by defining the structure acceleration response of the second measurement unit measured in time step as output data, but this also has limitations in objectivity, rationality and accuracy.

Republic of Korea Patent Publication No. 10-2095751 (Announcement on April 01, 2020, title of invention: Structure damage detection system and method)

The present invention has been devised to solve the above problems, and an object of the present invention is to introduce artificial intelligence (AI), in particular, deep learning technology, which is a rapidly evolving 4th industrial technology in the detailed inspection of facilities, for a long time. The purpose of this project is to provide a damage level generation method using deep learning-based facility damage image classification that enables objective, rapid and automatic damage recognition in the method that has been conducted based on time and manpower.

Another object of the present invention is to enable rational evaluation of safety ratings of facilities through objective and efficient condition evaluation of facilities.

Another object of the present invention is to generate an accurate degree of damage by applying damage area detection to the damaged image that has undergone deep learning-based damage image classification once more.

In order to achieve the above object, the present invention comprises the steps of: (a) augmenting the number of image data (damaged image data and non-damaged image data) of the acquired facility by the image data augmentation module; (b) learning, by a supervised learning module, a Convolutional Neural Network (CNN) using the damaged image data and the undamaged image data that have undergone labeling to designate the presence or absence of damage for each class and pixel as training data; (c) automatically classifying the damaged image by using the damaged image classifier, which is a CNN supervised learning, the test image data or arbitrary image data to obtain a degree of damage as input data; (d) pre-processing, by the damaged image data pre-processing module, the data classified as the damaged image as output data through the damaged image classifier to efficiently detect the damaged area; (e) detecting, by the damaged area detection module, the image damaged area in order to increase the accuracy of the damage degree and extract the damage shape pattern for the damaged image on which the pre-processing operation has been completed, and (f) the damage level generating module It is characterized in that it consists of a step of automatically generating a degree of damage by repeatedly using a feature vector and a short line segment for this damage.

In addition, the augmentation in step (a) of the present invention uses a flip technique, a translate technique, a rotate technique, and a color modification technique.

In addition, for the damage to be found in the learning process of step (b) of the present invention, the minimum image size to perform damage detection, the color type to be binarized, the number of supervised learning training times, the brightness and contrast change values of each image, and damage A boundary value is used as a parameter to distinguish an image without damage from an image with damage.

In addition, the pre-processing in step (d) of the present invention includes the steps of averaging the pixel brightness with an adjacent region in the image using a mean filter on the image, and using a median filter on the image Compensating for image sharpness and reducing noise by using the average filter, and image binarization step of removing image noise and speeding up operation in image processing are sequentially performed.

In addition, in the step (e) of the present invention, the detection of the damaged area includes a segmentation step of classifying unnecessary data rather than damage to detect non-linear and diverse damage of the facility, and damage from the separated shape. To extract, feature values including important features for each shape are calculated, and feature points are extracted through Fisher's linear discriminant, and the feature points using a support vector machine (SVM) Steps of determining the feature vector for non-damage and the feature vector for damage are sequentially performed.

In addition, in the present invention, the segmentation is the following equation,

(여기서, T는 출력 영상, I는 입력 영상(전처리된 손상영상), S는 연산에 포함된 주위픽셀에 대한 선형 구성요소(0도,45도,90도,135도 방향))을 이용하고, 상기 특징값들은 이심률(분리된 영역과 동일한 2차 모멘트를 갖는 타원의 이심률), 상기 타원 면적으로 나눈 분리된 영역의 면적, 고형비(볼록 껍질(convex hull) 면적 대비 Contour면적의 비율) 및 수직-수평 픽셀 좌표간 상관계수의 절대값(

(여기서, x와 y는 영상의 손상영역에 속하는 요소의 픽셀값,

와

는 손상영역에 속하는 모든 요소의 평균값))이다.

(where T is the output image, I is the input image (preprocessed damage image), and S is the linear component (0 degree, 45 degree, 90 degree, 135 degree direction) of the surrounding pixels included in the calculation) , the characteristic values are the eccentricity (the eccentricity of an ellipse having the same second moment as the separated region), the area of the isolated region divided by the ellipse area, the solid ratio (the ratio of the contour area to the convex hull area) and Absolute value of correlation coefficient between vertical-horizontal pixel coordinates (

(where x and y are pixel values of elements belonging to the damaged area of the image,

Wow

is the average value of all elements belonging to the damaged area)).

In addition, the generation of damage in the step (f) of the present invention is the following equation,

(여기서, J^m은 스케일 m의 선분에 대한 손상도, S_min은 선분의 최소크기, C^k는 손상에 대한 특징 벡터인 k를 선분으로 사용하여 생성된 손상영상, u와 v는 손상도 영상의 픽셀좌표, [] 는 올림함수)을 이용하되, 상기 손상영역 검출에서의 손상에 대한 특징 벡터를 활용하고 상기 수학식에 의한 각각의 픽셀좌표에 대한 짧은 선분인 손상도의 조합으로 이루어진다.

(where J ^m is the damage degree for the line segment of scale m, S _min is the minimum size of the line segment, C ^k is the damage image generated using k, a feature vector for damage, as the line segment, and u and v are the damage degree images. pixel coordinates, [] is a rounding function), but using a feature vector for damage in the detection of the damage area, and a combination of degree of damage that is a short line segment for each pixel coordinate according to the above equation.

As seen above, the damage generation method using deep learning-based facility damage image classification, which is the present invention, introduces deep learning technology to a detailed facility inspection, and the method that has been carried out for a long time centered on manpower objectively, quickly and automatically recognizes damage. The effect of creating an accurate degree of damage by applying the detection of the damaged area once more to the damaged image that has undergone deep learning-based damage image classification. there is

1 is a diagram showing an overall flow chart of a damage level generation method using deep learning-based facility damage image classification according to the present invention.
2 is a view showing an embodiment of image data augmentation in the present invention.
3 is a view showing an embodiment labeled with a damaged area class in the present invention.
4 is a diagram illustrating an embodiment of data classified as an injured image by an injured image classifier in the present invention.
5 is a view showing an embodiment of an image after binarization of a pre-processing operation is performed in the present invention.
6 is a view showing an embodiment of the damaged area detected from the pre-processed damaged image in the present invention.
7 is a view showing an embodiment of the damage generated in the present invention (損傷圖).
8 is a block diagram showing an embodiment of a device related to a damage degree generation method using deep learning-based facility damage image classification according to the present invention.

A preferred embodiment of the present invention configured as described above will be described in detail with reference to the accompanying drawings. It should be noted that the accompanying drawings and the description referring to them are exemplified so that those of ordinary skill in the art can easily understand the present invention, and are not intended to limit the spirit and scope of the present invention. will have to

8 is a block diagram showing an embodiment of a device related to a damage level generating method using deep learning-based facility damage image classification according to the present invention, and the damage level generating device 10 using damage image classification is facility precision By introducing deep learning technology to inspection, the method that has been carried out for a long time centered on manpower enables objective, rapid and automatic damage recognition, and objective and efficient facility condition evaluation to enable reasonable safety rating of facilities to be calculated. An image data augmentation module 11 that augments the number of image data of a facility, a supervised learning module that trains a CNN (Convolutional Neural Network) by using the labeled damaged image data and the non-injured image data as training data (12), damage image data pre-processing module (13) that pre-processes data classified as damage image, which is output data through the damage image classifier, for efficient damage area detection It includes a damage area detection module 14 that detects an image damage area for increasing the accuracy of the image and extracts a damage shape pattern, and a damage level generation module 15 that automatically generates a damage level (appearance survey network) of the image. . That is, the image data augmentation module 11, the supervised learning module 12, the damaged image data preprocessing module 13, the damaged area detection module 14 and the damage level generating module 15 allow the present invention to be performed on a computer. As a technical means for this, the image data augmentation unit, the supervised learning unit, the damaged image data preprocessor, the damaged area detection unit, and the damage level generator may be respectively named.

On the other hand, the damaged image classifier refers to a CNN on which supervised learning has been completed, and is not separately shown in FIG. 8 .

The damage level generating device 10 using the damage image classification is a server, desktop, laptop or portable terminal, etc., and includes storing software for performing damage generation using deep learning-based facility damage image classification.

In addition, it is preferable that the data and training data calculated or input/output in the damage level generating device 10 using the damage image classification be stored in a separate storage device 20, but the damage level using the damage image classification is generated Device 10 may also include storage device 20 .

Training data, which will be described later, required for the present invention is RGB image data. Such image data includes the position and optical information of the image, and the method of acquiring image data is preferably using a drone, but is not limited thereto. Image data may also be applicable.

The images acquired from such drones, that is, UAVs, are cheaper than satellite and aerial images, have a faster shooting cycle, and have high resolution. rich and accurate analysis can be performed.

When using the drone in the present invention, it is preferable to adopt the autonomous flight method rather than manual flight by manpower. (flight controller) is a control device that allows the drone to fly stably in the air. It controls the drone by combining sensor information such as gyro, accelerometer, and GPS to give a signal to the speed controller.

Here, the technology that allows the user to upload a mission and perform the desired mission without manual operation is called automatic flight navigation technology. A setting method is applied, which is a method of automatically flying after recording the route the drone wants to fly on the map on an imaging display connected by a remote control.

The general route point setting method is suitable for flight to a wide area where it is difficult to access mainly by manpower or to apply 3D topographic information surveying, etc., but in structure measurement, the flight stability and accuracy of the flight path are low, so it cannot be directly applied. The route point setting method using the displayed longitude and latitude coordinate system is applied. In addition, the waypoint-based flight control that sets the drone's flight path can enable accurate and stable autonomous flight.

Mobile positioning is a series of surveying processes in which the mobile station obtains the positioning results in real time using the correction value for the carrier phase of the reference station that has precise location information. In this case, the ground station method with a short base distance is advantageous in terms of accuracy if a method by a ground station that installs a GPS on the ground and transmits this calibration information to a mobile station is used for the calibration method.

The GPS receives radio waves from three artificial satellites and calculates the position on the earth by trilateration. Depending on the position of the satellite, an error of about 3-10m may occur, and even if flight control is performed according to this reference information, 1- An error of 4 m can be expected to occur additionally.

It is desirable to set the return to home error of the drone within 4m, which is the minimum summation error, and the real time kinematic (RTK) that can improve the satellite reception rate for the flying-chain drone, By employing a payload, the precise location of the drone can be estimated.

In the present invention, the facilities to be photographed are buildings, tunnels, bridges, embankments, dams, etc., and the size of the image capturing section of these facilities is preferably at least 140 cm. point) error should be set to within 70cm (half of the size of the imaging section) so that there is no missing part after overlapping shooting.

The present invention consists of a process as shown in FIG. 1, which will be described in detail below.

First, the acquired limited image data can be divided into damaged image data and non-injured image data, and is used as training data, which is input data, in a supervised learning process to be described later. Deep using the image data (ie, training data) In the learning environment, the number of image data can be increased by using image augmentation (S10), which has a positive effect on the improvement of deep learning learning performance. On the other hand, since the pattern of damage may be different for each facility to be photographed, it is preferable that such training data is provided in a dataset for each facility.

When training an artificial neural network, it is always necessary to pay attention to overfitting. This overfitting often occurs when the number of data is insufficient. One way to solve this problem is to increase the number of image data by arbitrarily manipulating the image data. This is to solve the overfitting problem by augmenting the image data that randomly cuts data using the image augmentation technology, increases the virtual number of image data through vertical inversion, random contrast, and color change.

In the present invention, the following four methods of Flip, Translate, Rotate, and Color modification (FIG. 2A is the original image before augmentation) are used to augment the image data, which is the training data. Through this process, the total number of training images is about 8. It can be increased by ~10 times.

① Flip technique: Invert the original image left and right or up and down (Fig. 2b)

② Translate technique: The original image is slightly moved in various directions (Fig. 2c)

③ Rotate technique: Rotate the original image at various angles (Fig. 2d)

④ Color modification technique: Change the color of the original image (Fig. 2e)

Accordingly, the image data augmentation module 11 augments the number of acquired image data (damaged image data and non-damaged image data) of the facility. This augmentation process is coded with an algorithm through a programming language to finally be performed by a computer. In other words, such a program is stored in the damage level generating device 10 or the storage device 20, and the image data augmentation module 11 is input and stored in the damage level generating device 10 or the storage device 20. By using the image data of the facility and the program, the number of image data of the facility is increased.

On the other hand, to label the damaged area of the augmented damaged image data (see FIG. 3 ), after a labeling operation of grouping pixel values of the same class, a data set is constructed through a process of converting a file into an xml format. That is, after specifying the class and the presence or absence of damage for each pixel, a grouped labeling image file is extracted. Since this labeling operation must be accurate, it is preferable to do it manually in the present invention. Here, the class refers to a category that contains information on the material (concrete, loess, glass, etc.) and damage (crack, rebar exposure, peeling, peeling, leaking, whitening, etc.).

In this way, the damaged area labeled damage image data is used as input data in the supervised learning step to be described later.

Next, using a deep learning-based platform, we want to find by learning a number of sample images (that is, damaged image data and uninjured image data labeled as input data) in which various types of damage are detected through supervised learning. The following four characteristic values (main parameters for supervised learning) for the damage done are sufficient data to detect various damage in the input data (ie, training data) so that they can be utilized in the learning process (S20).

① Feature size/color: Minimum image size to perform damage detection and color type to be binarized

② Number of learning (count epochs): Number of supervised learning training

③ Luminance/contrast: Changes in brightness and contrast of each image

④ Threshold: A threshold value for distinguishing images without damage and images with damage. The closer to 1, the higher the probability of failure.

In the present invention, it is preferable to perform the supervised learning by learning and using VGGNet, which is a kind of convolutional neural network (CNN), which is a deep learning technology, and finally to perform damage level generation. The CNN is a type of multi-layer feed-forward artificial neural network used to analyze visual images, and is classified as a deep neural network in deep learning. Also known as a neural network (SIANN), it has applications in image and video recognition, recommendation systems, image classification, medical image analysis, and natural language processing.

In this way, the supervised learning CNN can operate as a damaged image classifier in the present invention, and various types using test image data or arbitrary image data to obtain a degree of damage by the damaged image classifier as input data If there is damage, it is automatically classified as a damaged image (S30, see FIG. 4).

Accordingly, the supervised learning module 12 learns a CNN (Convolutional Neural Network) by using the damaged image data and the undamaged image data that have undergone labeling to designate the presence or absence of damage for each class and pixel as training data. A CNN that has been trained can act as a damaged image classifier, and this learning process and damaged image classification of arbitrary image data is a program coded with an algorithm through a programming language to finally be performed by a computer, that is, such a program is stored in the damage level generating device 10 or storage device 20, including the CNN, so that the supervised learning module 12 and the damage image classifier are input and stored in the damage level generating device 10 or the storage device 20 Supervised learning and damage image classification are made using training data, arbitrary image data, and the program.

Next, before the image damaged area detection step to be described later, the data classified as the damaged image, which is output data through the damaged image classifier, is preprocessed in a form suitable for use in the image damaged area detection step (S40).

The image data pre-processing is performed sequentially in three steps as follows.

Step 1 is to average the pixel brightness with an adjacent area in the image using a mean filter on the image (a method of taking the average using the noise characteristic that there is a large brightness distribution between the current pixel and neighboring pixels) As a result, it reduces noise (Gaussian noise, uniform noise), but the disadvantage is that the edge, which is the damaged area, is also treated as noise and the image becomes blurry.

Step 2 is to use a median filter on the image to compensate for the image sharpness that has been degraded by the average filter and to reduce noise. see. On average, the calculation time is 5 times longer than the calculation time of the average filter.

Step 3 is image binarization, which can be implemented with the Otsu algorithm, in order to remove image noise and ultimately speed up the operation, which is the most important in image processing (see FIG. 5 ).

Accordingly, the damaged image data pre-processing module 13 pre-processes the data classified as the damaged image, which is the output data through the damaged image classifier, for efficient detection of the damaged area. It is a program coded with an algorithm through ), the pre-processing is performed using the data classified as the damaged image and stored in the program.

Next, for the damaged image on which the pre-processing operation has been completed, the image damaged area detection is performed, which is an operation for increasing the accuracy of the damage degree and extracting a shape pattern similar to the damage (S50, see FIG. 6). Here, the damage is, as described above, cracks, rebar exposure, peeling, peeling, water leakage, whitening, and the like.

The detection of the image damaged area is sequentially performed in the following three processes.

① Segmentation technique: In order to detect damage, a damage-like shape is primarily detected from the damage image that has been pre-processed.

The purpose of this separation technique is to increase the analysis efficiency by classifying unnecessary data that is not clearly damaged in the image. let there be

The morphology technique detects damage having various directions, and the following Equation 1 can be applied.

(Where T is the output image, I is the input image (preprocessed damage image), and S is the linear component of the surrounding pixels included in the operation (0 degree, 45 degree, 90 degree, 135 degree direction))

By using the nonlinear filter as in Equation 1, the entire damage in the facility to be photographed can be directly extracted, unlike the limited contour extraction technique that recognizes and processes damage as an outline.

② Feature extraction technique: important feature points that can be recognized as damage among the shapes detected by the separation technique are extracted.

After separating the damaged shape in the above separation technique, the feature points defined by the following feature values are extracted from the separated shape. In order to derive four distinctive features of the damage shape pattern that meet the purpose of detection, analysis of a number of features is carried out.

In the present invention, after selecting four features suitable for damage analysis, the following feature values (values of the four features) for extracting damage by finally applying Fisher's linear discriminant at a significance level of 95% ) is applied. That is, when four feature values are calculated and input as input values to Fisher's linear discriminant, the feature points are extracted.

i) eccentricity: the eccentricity of an ellipse with the same second moment as the separated region

ii) the area of the separated region divided by the area of the ellipse in i)

iii) solidity ratio: ratio of the area of the contour to the area of the convex hull (Equation 2)

iv) Absolute value of correlation coefficient between vertical-horizontal pixel coordinates (Equation 3)

와

는 손상영역에 속하는 모든 요소의 평균값)(where x and y are pixel values of elements belonging to the damaged area of the image,

Wow

is the average value of all elements belonging to the damaged area)

③ Decision making technique: For learning and evaluation of a classifier for classifying damage, a feature set consisting of n1 non-damaged feature vectors and n2 damaged feature vectors from the extracted feature points. make up In this process, the intact feature vector is separated from the preprocessed damaged image.

About 60% of the feature vector set from the configured feature set is used for learning the damage classifier, and the remaining feature vectors can be used for the damage classification performance evaluation experiment.

Preferably, the damage classifier uses a support vector machine (SVM) using a cubic polynomial kernel, and when the feature point data for learning and evaluation is insufficient, manually separate the secured actual damage and use it Therefore, it is possible to develop and apply an algorithm to learn damage by randomly regenerating it.

That is, by using the damage classifier, SVM, it is possible to accurately detect a damaged area (determining a feature vector for non-damage and a feature vector for damage) based on the feature points, and the feature vector for damage is a damage to be described later. It is used for Tao (損傷圖).

Accordingly, the damaged area detection module 14 performs image damage area detection to increase the accuracy of the damage degree and extract a damage shape pattern for the damaged image on which the pre-processing operation has been completed. It is a program coded with an algorithm through a program language to be performed by a computer as The area detection module 14 detects the damaged area using the program input and stored in the damage level generating device 10 or the storage device 20 .

Lastly, in the damage level generation step, the damage area detection process is followed to generate the damage level by repeatedly using short line segments of various sizes to automatically generate the damage level (appearance survey network) of the image. make it (S60).

Short line segments of various sizes are used according to the location according to the characteristics such as cracks, which are linear damages, or white spots, which are area-type damages, in the damage diagram (exterior survey network diagram) generated through this process.

Equation 4 below is an equation for generating damage (appearance survey network).

(where J ^m is the damage degree for the line segment of scale m, S _min is the minimum size of the line segment, C ^k is the damage image generated using k, a feature vector for damage, as the line segment, and u and v are the damage degree images. pixel coordinates, [] is a rounding function)

That is, the degree of damage (Equation 4) for each pixel coordinate is gathered to generate a degree of damage as shown in FIG. 7, and the degree of damage shown in FIG. 7 is formed by combining the short line segments of the above formula (4).

Accordingly, the damage level generating module 15 generates a degree of damage by repeatedly using short line segments of various sizes to automatically generate a degree of damage (appearance survey network) of an image using the feature vector for the damage. Such damage level generation is a program coded with an algorithm through a program language to be finally performed by a computer, that is, such a program is stored in the damage level generating device 10 or the storage device 20 while including the above Equation 4 Thus, the damage level generating module 15 generates a degree of damage by using the feature vector for the damage input and stored in the damage level generating device 10 or the storage device 20 and the program.

10: Damage level generating device using damage image classification
11: Image data augmentation module
12: Supervised Learning Module
13: Damaged image data pre-processing module
14: damage area detection module
15: damage generation module
20: storage device

Claims (7)

(a) augmenting the number of image data (damaged image data and non-damaged image data) of the acquired facility by the image data augmentation module 11 (S10);
(b) the supervised learning module 12 learning the CNN (Convolutional Neural Network) by using the damaged image data and the undamaged image data that have undergone labeling to designate the presence or absence of damage for each class and pixel as training data (S20) and ;
(c) automatically classifying the damaged image using the test image data or arbitrary image data for which the supervised learning is completed CNN damage classifier to obtain the degree of damage as input data (S30);
(d) pre-processing, by the damaged image data pre-processing module 13, the data classified as the damaged image, which is the output data through the damaged image classifier, in order to efficiently detect the damaged area (S40);
(e) detecting, by the damaged area detection module 14, the image damaged area in order to increase the accuracy of the degree of damage and extract the damage shape pattern for the damaged image on which the pre-processing operation has been completed (S50), and
(f) the damage level generating module 15 automatically generates a degree of damage by repeatedly using the feature vector and short line segment for damage (S60),
In the step (f), the generation of damage is the following equation,

(where J ^m is the damage degree for the line segment of scale m, S _min is the minimum size of the line segment, C ^k is the damage image generated using k, a feature vector for damage, as the line segment, and u and v are the damage degree images. It is characterized in that by using the pixel coordinates of , [] is a rounding function), the feature vector for damage in the detection of the damage area is used, and the damage degree is a combination of short line segments for each pixel coordinate according to the above equation. A damage level generation method using deep learning-based facility damage image classification.
The method of claim 1,
In step (a), the augmentation is a damage generation method using deep learning-based facility damage image classification, characterized in that it uses a flip technique, a translate technique, a rotate technique, and a color modification technique.
The method of claim 1,
For the damage to be found in the learning process of step (b),
The minimum image size for damage detection, the color type to be binarized, the number of supervised learning training, the brightness and contrast change values of each image, and the boundary value for distinguishing an image without damage and an image with damage as parameters. Damage level generation method using deep learning-based facility damage image classification, characterized in that.
The method of claim 1,
The pretreatment process in step (d) is,
A step of averaging the pixel brightness with an adjacent region in the image using a mean filter on the image A method for generating damage using deep learning-based facility damage image classification, characterized in that the step of reducing the image noise and image binarization step of removing image noise and speeding up the operation speed in image processing are sequentially performed .
The method of claim 1,
The detection of the damaged area in step (e) is,
A segmentation step to classify unnecessary data rather than damage to detect non-linear and diverse damage to the facility;
In order to extract damage from the separated shape, calculating feature values including important features for each shape and extracting the feature points through Fisher's linear discriminant, and
A deep learning-based facility damage image, characterized in that the step of determining the feature points as a feature vector for non-damage and a feature vector for damage by using a support vector machine (SVM) is sequentially performed Damage generation method using classification.
6. The method of claim 5,
The segmentation is the following equation,

(where T is the output image, I is the input image (preprocessed damage image), and S is the linear component (0 degree, 45 degree, 90 degree, 135 degree direction) of the surrounding pixels included in the calculation) ,
The characteristic values are the eccentricity (the eccentricity of an ellipse having the same second moment as the isolated region), the area of the isolated region divided by the ellipse area, the solid ratio (the ratio of the contour area to the convex hull area), and the perpendicular -Absolute value of correlation coefficient between horizontal pixel coordinates (

(where x and y are pixel values of elements belonging to the damaged area of the image,

Wow

is the average value of all elements belonging to the damaged area)), a method of generating damage using deep learning-based facility damage image classification.
delete

Images