KR102559586B1 - Structural appearance inspection system and method using artificial intelligence - Google Patents

Abstract

The present invention discloses a structure exterior inspection system and method using artificial intelligence. In the present invention, when a damage inspection image is generated by photographing a damaged area with a camera to inspect damage to a structure, the type of damage is classified by applying a pre-learned artificial intelligence classification engine to the damage inspection image. In addition, by using an AI damage detection engine pre-learned from damage images corresponding to each damage type for each damage type, the damage inspection image is analyzed to detect the type and size of the damage, thereby significantly reducing the manpower, time, and cost required for damage inspection and analysis, enabling efficient damage inspection. In addition, according to the present invention, when an inspector captures a damage inspection image, a damage identification code whose size has been determined in advance is attached to the side of the damaged part and guided to take the damaged image. When a damage inspection image including the damage identification code is generated, parameters such as pixel resolution of a camera are calculated using the size of the damage identification code, thereby enabling more accurate damage analysis.

Description

Structure appearance inspection system and method using artificial intelligence {Structural appearance inspection system and method using artificial intelligence}

The present invention relates to a structure exterior inspection system and method, and more particularly to a structure exterior inspection system and method using artificial intelligence.

Recently, due to the aging of social infrastructure, interest in safety inspection and maintenance is rapidly increasing. In particular, in the case of concrete structures, in order to suppress durability deterioration due to cracking, it is necessary to limit the crack width requiring repair by setting an allowable crack width.

As such, it is important to perform maintenance at the right place in a timely manner by more effectively performing exterior safety inspections such as crack width in order to manage life span and secure safety of social infrastructure. However, most of the existing structure exterior inspection technologies rely on manpower, and the inspector approaches the target structure, inspects the surface using a crack diameter or displacement gauge, and records the measured values on a drawing to create a visual survey network. There is an inconvenience.

In addition, in the case of high-altitude areas, there are cases where drones or digital cameras are used, but they are limited to acquiring images of the surface by taking pictures with cameras close to the surface to be inspected. Therefore, in order to detect damage from the surface image of a structure photographed in the field to create a visual survey network map, the inspector must check all the images taken one by one.

In addition, even if damage is detected from a photographed image, it is impossible to quantify the damage because the length or width of the damage can be calculated only in pixel units of the image, and when there are many photographed images, it is difficult to link the design drawings to the photographed points. Visual inspection using a simple photographing technique has limitations.

The problem to be solved by the present invention is to overcome the manpower and time limitations of the exterior safety inspection of existing structures, thereby reducing the enormous manpower, time, and cost required for existing inspections and investigations to effectively perform exterior inspections of structures. It is to provide a structure exterior inspection system and method using artificial intelligence.

In order to solve the above problems, a method for inspecting the external appearance of a structure according to a preferred embodiment of the present invention is a method for inspecting the external appearance of a structure performed by an inspector terminal and a structure external inspection device, comprising: (a) the inspector terminal photographing a damaged area of a structure, acquiring a damage inspection image and transmitting the image to the structure external diminishing device; (b) classifying, by the structure exterior inspection device, the type of damage by applying the damage inspection image to a pre-learned artificial intelligence classification engine; (c) obtaining the shape and size of the damage by applying the damage inspection image to a pre-learned artificial intelligence damage detection engine according to the type of damage, by the apparatus for inspecting the external appearance of the structure; and (d) overlaying and storing the shape and size of the detected damage on the damage inspection image, by the apparatus for inspecting the appearance of the structure.

In addition, in the step (a), (a2) when a damage identification code having a predefined size is attached to the damaged area of the structure and then activating the camera, calculating an effective target shooting distance using the hardware parameters of the inspector's terminal, calculating an actual shooting distance using the image of the damage identification code input through the camera and the hardware parameters, guiding the inspector so that the actual shooting distance is less than or equal to the effective target shooting distance, and acquiring the damage inspection image when the actual shooting distance becomes less than or equal to the effective target shooting distance I can do it.

In addition, the step (a) may further include, prior to the step (a2), (a1), when a camera is activated after a damage identification code is attached to the damaged area of the structure, recognizing a damage management number from the damage identification code attached to the inspector's terminal, receiving structure identification information and the location of the damaged area on the structure drawing from the inspector, and transmitting the information to the structure exterior inspection device, and the structure exterior inspection device storing the damage management number, the structure identification information, and the location of the damaged area in association with each other.

In addition, the step (c) may include: (c1) performing preprocessing by removing noise from the damage inspection image and adjusting brightness; (c2) detecting the damage identification code in the damage detection image, binarizing the damage identification code, and counting pixels corresponding to the damage identification code to calculate a pixel resolution of the image compared to the physical size; (c3) detecting a damaged area by performing segmentation on the preprocessed image using the artificial intelligence damage detection engine; (c4) generating a binary image of the damaged area by extracting and binarizing the damaged area from the damage inspection image; and (c5) applying a morphology algorithm to the binarized image of the damaged region to obtain a shape of the damage and a size of the damage including at least one of the length, width, and width of the damage.

In addition, in the step (c5), a Euclidean distance transformation image is obtained by applying a Euclidean distance transformation algorithm to the damaged area binarization image, a skeletonization algorithm is applied to the damaged area binarization image to obtain a skeleton image, pixel values of the Euclidean distance transformation image and the skeletonization image are multiplied by pixel values corresponding to each other and a scaling factor is multiplied to generate a crack damage image, and a crack damage image is generated using the binarization image, the skeleton image, and the crack damage image. You can get length, width and area.

In addition, in the step (c5), the extent of the damage is obtained using the number of pixels indicated as the damaged area in the binary image, the length of the damage is obtained using the number of pixels indicated as the damaged area in the skeleton image, and the width of the damage can be obtained using the maximum pixel value in the crack damage image.

In the step (d), the damage image and the size of the damage may be overlaid and stored on the damage inspection image.

Meanwhile, a computer program according to another preferred embodiment of the present invention for solving the above problems is stored in a non-temporary storage medium, and is executed in a computer including a processor to perform the above structure external inspection method.

On the other hand, an apparatus for external appearance inspection of a structure according to another preferred embodiment of the present invention for solving the above problems is an apparatus for inspecting external appearance of a structure including a processor and a memory for storing predetermined instructions, wherein the processor executing the instructions stored in the memory performs the steps of (a) receiving a damage inspection image generated by photographing a damaged area of a structure from an inspector terminal; (b) classifying the type of damage by applying the damage inspection image to a pre-learned artificial intelligence classification engine; (c) obtaining the shape and size of the damage by applying the damage inspection image to an artificial intelligence damage detection engine trained in advance according to the type of damage; and (d) overlaying and storing the shape and size of the detected damage on the damage inspection image.

In addition, in the step (a), (a2) when a damage identification code having a predefined size is attached to the damaged area of the structure and then activating the camera, calculating an effective target shooting distance using the hardware parameters of the inspector's terminal, calculating an actual shooting distance using the image of the damage identification code input through the camera and the hardware parameters, guiding the inspector so that the actual shooting distance is less than or equal to the effective target shooting distance, and acquiring the damage inspection image when the actual shooting distance becomes less than or equal to the effective target shooting distance You can.

In addition, before the step (a2), the processor may further perform (a1), when a damage identification code is attached to the damaged area of the structure and the camera is activated, recognizes a damage management number from the damage identification code attached to the inspector's terminal, receives structure identification information and the location of the damaged area on the structural drawing of the structure from the inspector, and transmits the input to the structure exterior inspection device, and the structure exterior inspection device stores the damage management number, the structure identification information, and the location of the damaged area in association with each other.

In addition, the step (c) may include: (c1) performing preprocessing by removing noise from the damage inspection image and adjusting brightness; (c2) detecting the damage identification code in the damage detection image, binarizing the damage identification code, and counting pixels corresponding to the damage identification code to calculate a pixel resolution of the image compared to the physical size; (c3) detecting a damaged area by performing segmentation on the preprocessed image using the artificial intelligence damage detection engine; (c4) generating a binary image of the damaged area by extracting and binarizing the damaged area from the damage inspection image; and (c5) applying a morphology algorithm to the binarized image of the damaged region to obtain a shape of the damage and a size of the damage including at least one of the length, width, and width of the damage.

In addition, in the step (c5), a Euclidean distance transformation image is obtained by applying a Euclidean distance transformation algorithm to the damaged area binarization image, a skeletonization algorithm is applied to the damaged area binarization image to obtain a skeleton image, pixel values of the Euclidean distance transformation image and the skeletonization image are multiplied by pixel values corresponding to each other and a scaling factor is multiplied to generate a crack damage image, and a crack damage image is generated using the binarization image, the skeleton image, and the crack damage image. You can get length, width and area.

In addition, in the step (c5), the extent of the damage is obtained using the number of pixels indicated as the damaged area in the binary image, the length of the damage is obtained using the number of pixels indicated as the damaged area in the skeleton image, and the width of the damage can be obtained using the maximum pixel value in the crack damage image.

In the step (d), the damage image and the size of the damage may be overlaid and stored on the damage inspection image.

In the present invention, when a damage inspection image is generated by photographing a damaged area with a camera to inspect damage to a structure, the type of damage is classified by applying a pre-learned artificial intelligence classification engine to the damage inspection image. In addition, by using an AI damage detection engine pre-learned from damage images corresponding to each damage type for each damage type, the damage inspection image is analyzed to detect the type and size of the damage, thereby significantly reducing the manpower, time, and cost required for damage inspection and analysis, enabling efficient damage inspection.

In addition, according to the present invention, when an inspector captures a damage inspection image, a damage identification code whose size has been determined in advance is attached to the side of the damaged part and guided to take the damaged image. When a damage inspection image including the damage identification code is generated, parameters such as pixel resolution of a camera are calculated using the size of the damage identification code, thereby enabling more accurate damage analysis.

1 is a diagram showing the overall configuration of a structure exterior inspection system using artificial intelligence according to a preferred embodiment of the present invention.
2 is a diagram showing detailed hardware configurations of an inspector's terminal and a structure external inspection device according to a preferred embodiment of the present invention.
3 is a flowchart illustrating a method for inspecting the exterior of a structure using artificial intelligence according to a preferred embodiment of the present invention.
4A is a view for explaining an example of selecting a structure to be inspected for exterior appearance, and FIG. 4B is a view for explaining an example for selecting a position of a damaged region in a structure to be inspected for exterior appearance.
5A and 5B are diagrams illustrating an example of capturing a damaged area and a damage identification code in an inspector terminal.
6 is a diagram illustrating a method of learning an artificial intelligence model according to a preferred embodiment of the present invention.
7 is a diagram explaining the structure of a U-Net architecture.
8 is a diagram illustrating an example of detecting damage using an artificial intelligence model according to a preferred embodiment of the present invention.
9 is a diagram illustrating a process of obtaining parameters of a damaged area using a morphology algorithm according to a preferred embodiment of the present invention.
FIG. 10 is a diagram showing examples of damage inspection images in which a damaged area is overlaid according to a preferred embodiment of the present invention.
11 is a diagram illustrating an example of damage management data stored in a memory and displayed on an inspector terminal and a manager terminal according to a preferred embodiment of the present invention.
12 is a diagram showing an example of a report provided by the structure external inspection device.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

Here, the above objects, features and advantages of the present invention will become more apparent through the following detailed description in conjunction with the accompanying drawings. However, the present invention can apply various changes and can have various embodiments, hereinafter, specific embodiments will be illustrated in the drawings and described in detail.

Like reference numerals designate essentially like elements throughout the specification. In addition, components having the same function within the scope of the same idea appearing in the drawings of each embodiment are described using the same reference numerals.

When it is said that a certain part "includes" a certain component throughout the specification, it means that it may further include other components without excluding other components unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software.

If it is determined that a detailed description of a known function or configuration related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, numbers (eg, 1st, 2nd, etc.) used in the description process of this specification are only identifiers for distinguishing one component from another component.

1 is a diagram showing the overall configuration of a structure exterior inspection system using artificial intelligence according to a preferred embodiment of the present invention.

Referring to FIG. 1 , a structure exterior inspection system using artificial intelligence (hereinafter referred to as “structure exterior inspection system”) according to a preferred embodiment of the present invention includes an inspector terminal 100 and a structure exterior inspection device 200, and may further include a manager terminal 300 that is connected to the structure exterior inspection device 200 and manages it.

The inspector terminal 100 photographs the damaged area of the structure and transmits the damage inspection image to the structure exterior inspection device 200 through a wired or wireless communication network. At this time, the inspector terminal 100 may be implemented as a mobile phone, smart pad, tablet PC, etc. having a camera 110, and an application program having a camera 110 and performing functions of the present invention described later may be installed.

Meanwhile, briefly reviewing the operation of the inspector's terminal 100, when an inspector attaches a damage identification code that is physically manufactured so that it can be attached to a structure around the damaged area of the structure to be inspected, and executes the program of the present invention installed in the inspector's terminal 100, the camera 110 included in the inspector's terminal 100 is activated to photograph the damaged area of the structure and the damage identification code.

Then, the inspector terminal 100 calculates the effective target shooting distance using the hardware specifications (parameters) of the inspector terminal 100, and calculates the actual shooting distance using the image of the damage identification code input through the camera 110 and the hardware parameters. At this time, if the actual shooting distance is longer than the effective target shooting distance, the inspector is guided to take a closer shot, and when the actual shooting distance is less than the effective target shooting distance, a damage inspection image is taken and transmitted to the structure appearance inspection device 200.

In addition, the inspector terminal 100 recognizes the damage management number from the damage identification code, receives identification information of the structure to be inspected and location information of the damaged area within the structure to be inspected from the inspector, and transmits the structure exterior inspection device 200 together with the damage management number, allowing the structure exterior inspection device 200 to link and store these information.

The apparatus 200 for inspecting the appearance of a structure has an artificial intelligence classification engine learned in advance to classify the type of damage formed on a structure from a damage inspection image, and according to the type of damage classified by the artificial intelligence classification engine, the type and size of damage are learned and equipped with an artificial intelligence damage detection engine learned in advance to detect.

In addition, when a damage inspection image is received from the inspector terminal 100, the structure exterior inspection apparatus 200 applies the damage inspection image to an artificial intelligence classification engine to identify the type of damage (concrete wall damage, asphalt floor damage, externally exposed steel structure damage, etc.), and if the type of damage is identified, the damage inspection image is applied to the AI damage detection engine pre-learned from the damage images of the corresponding type to detect the location and size of the damage.

The structure exterior inspection apparatus 200 may overlay and store the shape and size of the detected damage on the damage inspection image, and display them according to users' requests. The damage inspection image and information on the location and size of damage are stored in association with the damage management number. Afterwards, when the exterior inspection is performed again on the structure, the damage inspection image additionally obtained by performing the above process and the location and size of the damage are additionally stored in association with the damage management number. Therefore, inspectors and managers can track the change in shape and size of damage over time by inquiring the damage management number, enabling efficient management.

2 is a diagram showing detailed hardware configurations of an inspector terminal 100 and a structure external inspection device 200 according to a preferred embodiment of the present invention, and FIG. 3 is a flowchart illustrating a structure external inspection method using artificial intelligence according to a preferred embodiment of the present invention.

With further reference to FIGS. 2 and 3 , a structure exterior inspection system using artificial intelligence and a structure exterior inspection method using artificial intelligence according to a preferred embodiment of the present invention will be described.

First, referring to FIG. 2 , the inspector terminal 100 according to a preferred embodiment of the present invention includes a camera 110, a processor 120, a memory 130, an input unit 140, an output unit 160, and a communication module 150.

The memory 130 according to a preferred embodiment of the present invention may store instructions executable by the processor 120 and application programs executed by the processor 120, and may also store input/output data. Examples of the memory 130 include a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a read-only memory (ROM), and a random access memory (RAM).

The processor according to a preferred embodiment of the present invention may be implemented as a CPU (Central Processing Unit) or a device similar thereto (eg, MPU (Micro Processing Unit), MCU (Micro Control Unit), etc.), and executes instructions stored in the memory 130 or an application program, thereby performing each step of the process performed in the inspector terminal 100 among the processes performed in the method for inspecting the exterior of a structure described below with reference to FIG. 3 .

The input unit 140 and the output unit 160 are integrally implemented as a touch screen, and display information output from the processor 120 to the user, receive setting information and selection information from the user, and output the information to the processor 120.

In addition, the input unit 140 is implemented as a typical input means such as a mouse and a keyboard, and may receive setting information and selection information from the user and output them to the processor 120, and the output unit 160 is implemented as a monitor and the like to display data generated by the processor 120 to the user.

The communication module 150 communicates with the structure exterior inspection device 200 in a wired or wireless communication method. As a wired communication method used by the communication module 150, LAN (Local Area Network) or USB (Universal Serial Bus) communication is a representative example, and other methods are also possible.

In addition, in the case of a wireless type, a wireless personal area network (WPAN) communication method such as Bluetooth or Zigbee may be mainly used. However, since the wireless communication protocol is not limited thereto, the first wireless communication module may use a wireless local area network (WLAN)-based communication method such as Wi-Fi or other known communication methods.

Meanwhile, the apparatus 200 for inspecting the external appearance of a structure includes a communication module 220, a processor 210, and a memory 230.

The memory 230 according to a preferred embodiment of the present invention may store instructions executable by the processor 210 and programs executed by the processor 210, and may also store input/output data. Examples of the memory 230 include a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a read-only memory (ROM), and a random access memory (RAM). The memory 230 may be alternatively operated as a web storage or cloud server that performs the function of a storage medium on the Internet.

The processor 210 according to a preferred embodiment of the present invention may be implemented as a Central Processing Unit (CPU) or a device similar thereto (eg, a Micro Processing Unit (MPU), Micro Control Unit (MCU), etc.), and executes instructions or programs stored in the memory 230, and performs each step of the processes performed by the apparatus 200 for inspecting structure appearance among the processes included in the method for inspecting structure appearance to be described later with reference to FIG. 3 .

Hereinafter, with further reference to FIG. 3, the function of the structure exterior inspection system using artificial intelligence and each step of the structure exterior inspection method using artificial intelligence will be described.

First, the inspector attaches a damage identification code of a predefined size around the damaged area of the structure in order to inspect the appearance of the structure according to a preferred embodiment of the present invention, and executes the application program of the present invention installed on the inspector terminal 100 (S311). Here, the damage identification code may be implemented as a QR code or barcode of a predefined size to which an adhesive is applied to enable physical adhesion to the wall surface.

Thereafter, the inspector terminal 100 accesses the structure exterior inspection device 200 through a wired/wireless communication network, logs in by inputting an ID/password, selects identification information (eg, project name) of a structure to be inspected (see FIG. 4A), selects a location of a damaged area on the structural drawing corresponding to the identification information of the selected structure to be inspected (see FIG. 4B), and transmits the selected information to the structure exterior inspection device 200 (S313).

After that, the inspector terminal 100 activates the camera 110 to photograph the damaged part, recognizes the damage management number from the damage identification code included in the photographed image, and transmits it to the structure exterior inspection device 200 (S315). The structure exterior inspection device 200 stores the damage management number, structure identification information, and damage location in association with each other (S317). The damage management number recognized from the attached damage identification code is maintained in the structure exterior inspection device 200 without information linked to each other, and then functions as identification information indicating the structure selected by the inspector and the location of the structure's damage from step S317 onwards.

Next, the application program of the present invention executed in the inspector terminal 100 calculates an effective target shooting distance using the hardware parameters of the inspector terminal 100, activates the camera 110 of the inspector terminal 100, and calculates the actual shooting distance using the image of the damage identification code input through the camera 110 and the hardware parameters, as shown in FIG. 5A (S321).

In addition, the inspector terminal 100 guides the inspector on the shooting distance so that the actual shooting distance is less than or equal to the effective target shooting distance, and when the actual shooting distance becomes less than or equal to the effective target shooting distance, as shown in FIG.

Here, the effective target shooting distance and the actual shooting distance are calculated to adjust the shooting distance so that the damage to be photographed matches the minimum pixel resolution detectable by the inspector terminal 100 .

The effective target shooting distance (working distance, WD _I , Working Distance) is calculated according to Equation 1 below using the resolution of the camera 110, the size of detection damage, the size of the sensor, and the safety factor.

In Equation 1, each parameter is defined as shown in Table 1 below.

The constants described in the calculation formulas in Table 1 are based on the hardware configuration applied to the inspector terminal 100 according to the preferred embodiment of the present invention, and are changed accordingly when the hardware configuration of the inspector terminal 100 is changed.

On the other hand, the actual shooting distance (WD _A , Working Distance) is calculated according to Equation 2 below, and compared with the effective target shooting distance calculated according to Equation 1 above, so that the photographer can adjust the shooting distance himself. A guide (eg, “shoot closer”) is provided (see FIG. 5A ).

As described in Table 1, the pixel resolution (R) in Equation 2 is the actual length (M _L ) of the damage identification code recognized on the camera 110 and the damage identification code (damage mark) in the camera 110. It is calculated using the number of captured pixels (M _p ).

At this time, the inspector terminal 100 recognizes the shape of the damage from the image of the damaged area input from the camera 110, and provides a horizontal/vertical shooting guide for angles in the XZ and YZ directions so that the damage inspection image of the surface to be photographed is not distorted (see FIG. 5A).

At this time, the inspector terminal (tablet PC) 100 measures the horizontal/vertical angle with respect to the wall surface using the 6-axis sensor included therein, and guides the horizontal/vertical angle to be photographed within a range of 2 degrees.

Meanwhile, the processor 210 of the structure damage inspection apparatus 200 receiving the damage inspection image classifies the type of damage by applying the damage inspection image to a previously learned artificial intelligence classification engine (S331). The present invention learns and uses two types of deep learning-based artificial intelligence models. One is an artificial intelligence damage classification engine that classifies the type of damage included in a damage inspection image, and the other is an artificial intelligence damage detection engine that is selectively applied to each type of damage and detects the location and size of damage.

The artificial intelligence damage classification engine applies images containing various types of damage (eg, concrete wall damage, asphalt floor damage, externally exposed steel structure damage, etc.) in advance to an input terminal, and applies information on the type of corresponding damage to an output terminal to perform supervised learning, thereby learning to identify the type of damage included in the input damage inspection image.

In addition, the deep learning-based artificial intelligence damage detection engine uses damage images corresponding to each type of damage to detect a damaged area in the input damage inspection image, and learns to detect the shape and size (width, length, width, etc.) of the damage.

The artificial intelligence damage detection engine used in a preferred embodiment of the present invention is an image individually built according to each classification (concrete wall, asphalt floor, externally exposed steel structure, etc.), as shown in FIG.

Since U-Net is an artificial intelligence model widely known in the technical field of the present invention, briefly described below, U-Net is an end-to-end Fully-Convolutional Network proposed for segmentation. By using FCN's "skip architecture" concept, a method of combining shallow-layer feature maps with deep-layer feature maps is applied. It is characterized in that it can include both local and global information by combining these two layers that extract .

Referring to FIG. 7, the U-Net architecture consists of a contracting path and an expanding path, in which a network for obtaining overall context information of an image and a network for accurate localization are configured in a symmetrical form.

Contracting Path repeats 3x3 convolutions twice without padding, and the activation function uses ReLU, 2x2 max-pooling (stride: 2), and has an architecture that doubles the number of channels per down-sampling.

Expanding Path expands the feature map with an operation opposite to that of the Contracting Path. It repeats 2x2 convolution ("up-convolution") and 3x3 convolutions twice without padding, and halves the number of channels for each up-sampling through Up-Conv. The activation function is ReLU, and the up-convized feature map is concatenated with the feature map in which the border of the contracting path is cropped, and a 1x1 convolution operation is performed on the last layer.

The output value of the network is predicted by Softmax in units of pixels, and the predicted value of pixel x for the final feature map (channel k) is It is calculated as

Here, a(x) is the activation of pixel x, and K is the number of classes.

If U-Net is used, excellent performance can be expected in various segmentation problems by using data augmentation even with a small amount of training data, and accurate localization is possible while using context information well. In addition, since the place where verification is finished is skipped and new verification is performed from the next patch, the end-to-end structure has the advantage of fast inference speed.

Referring back to FIG. 3 , in step S331, the processor 210 of the structure damage inspection device 200 classifies the type of damage by applying the damage inspection image to the artificial intelligence classification engine learned in advance, and then performs preprocessing (S333).

In the preprocessing operation, the processor 210 removes objects (screw marks, pipes, exposed rebars, letters, etc.) unnecessary for damage detection from the damage inspection image using a previously created object detection model and adjusts the brightness.

After that, the processor 210 detects a predefined damage identification code in the preprocessed damage inspection image, binarizes the detected damage identification code, counts pixels corresponding to the damage identification code in the damage inspection image, and calculates the pixel resolution (R) of the damage inspection image compared to the physical size (S335).

Here, since the size of the damage identification code is a predefined value and the resolution of the damage inspection image is also a value that can be known by being provided by the inspector terminal 100 or set by the inspector in advance, the processor 210 can calculate the pixel resolution (R) according to R = M _L / M _P described in Table 1 above.

Then, the processor 210 detects a damaged area by performing segmentation on the preprocessed damage inspection image using an AI damage detection engine (S341).

However, as shown in FIG. 8 , since an overestimated background area including actual damage exists in the damaged area detected by segmentation, the processor 210 removes this overestimated area using a morphology algorithm.

To this end, the processor 210 first extracts an area corresponding to the detected damaged area from the original damage inspection image, and generates a binary image C ^B corresponding to the damaged area using the extracted image (S343).

Thereafter, the processor 210 extracts a damaged image of the damaged area by applying a morphology algorithm to the binary image C ^B , and extracts parameters (width, length, area, etc.) of the damaged area from the extracted damaged image (S351 to S355).

The morphology algorithm is an algorithm used to detect an object in an image and clarify the structure of the object in the image. In a preferred embodiment of the present invention, the processor 210 generates a damaged image (V ^F ) through a Euclidean distance transform (EDT) and a skeletonization process for the binary image (C ^B ), extracts a damage shape from the damaged image, and calculates parameters (width, length, area, etc.) of the corresponding damage.

Referring to FIG. 9, in more detail, the processor 210 obtains a Euclidean distance conversion image (C E ) by applying a binary image ( ^CB ) to one of the Euclidean functions of open CV library functions, and obtains a skeleton image ( ^{CS ) by applying a binary image (CB )} to a well-known bone strength function as one of open CV library functions (S351), and obtains a Euclidean distance conversion image ( ^{CE ) and a skeleton image (C E )} . The pixel values of the corresponding pixels of ^S ) are multiplied with each other, and the damaged image V ^F is obtained by multiplying this by a scaling factor (S353).

Skeletonization function is a well-known function among open CV's morphology library functions, and outputs a skeletonization image ( ^CS ) composed only of the central skeleton of a binary image (CB ⁾ . However, the skeleton image ( ^CS ) shows only the central point of the damage and the width of the damage is unknown.

Accordingly, the processor 210 obtains a Euclidean distance conversion image (C ^E ) by applying a binary image ( ^CB ) to a well-known Euclidean function among morphology library functions of open CV, and generates a damage image (V ^F ) by multiplying pixels corresponding to each other of the skeletonized image ( ^CS ) and the Euclidean distance conversion image ( ^CE ), and multiplying by a scaling factor. At this time, the Euclidean distance conversion image C ^E , as shown in FIG. 9 , for each pixel corresponding to the damage (eg, pixels whose pixel value is set to 1), pixels other than the nearest damage (eg, pixels whose pixel value is set to 0) and the shortest distance to the pixel can be obtained and assigned to the corresponding pixel. The value of each pixel in the damage image (V ^F ) represents the width of crack damage centered on that pixel.

Meanwhile, the processor 210 calculates a damage parameter defining damage using the binary image C ^B , the Euclidean distance conversion image C ^E , the skeleton image C ^S , and the crack damage image V ^F described above (S361).

An example of how the processor 210 obtains the damage parameters (length, width, and width of the crack) is as shown in Table 2 below.

In Table 2, Ps represents the number of pixels having a pixel value of 1 in the skeletonized image ^CS , and P _A represents the number of pixels having a pixel value of 1 in the binary image C ^B .

Accordingly, the processor 210 multiplies the pixel resolution by the number of pixels indicated as the damaged area in the binary image, and divides the preset constant to obtain the area of the damage. In addition, the length of the damage is obtained by multiplying the pixel resolution by the number of pixels marked as the damaged area in the skeleton image and dividing by a preset constant. Also, the damage width is obtained using the maximum pixel value in the crack damage image.

The processor 210 generates an overlay image by overlapping and displaying the damaged area on the damage inspection image, which is the original image, according to the damage parameters obtained as described above, and stores it in the memory in association with the damage management number (S363). (See FIG. 10)

Then, when the inspector terminal 100 or the manager terminal 300 transmits the damage management number and requests corresponding exterior inspection information, the processor 210 of the structure exterior inspection device 200 provides damage inspection information including damage location information stored in association with the received damage management number and a damage inspection image in which a damage area according to a damage parameter is overlaid to the inspector terminal 100 or the manager terminal 300 (S370).

11 is a diagram illustrating an example of damage management data stored in a memory and displayed on the inspector terminal 100 and the manager terminal 300 according to a preferred embodiment of the present invention.

On the other hand, according to the request of the manager terminal 300 or the inspector terminal 100, the processor 210 of the structure external inspection device 200 converts data stored in association with a specific damage management number into a report format and outputs it on a display, or generates and provides a file format that can be read by a computer. 12 shows an example of a report provided by the structure external inspection device 200 .

The structure external inspection method according to the preferred embodiment of the present invention described so far may be implemented as a computer executable command and a computer program stored in a non-temporary storage medium.

The storage medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable storage media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. In addition, the computer-readable storage medium is distributed in computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner.

So far, the present invention has been looked at with respect to its preferred embodiments. Those skilled in the art to which the present invention pertains will be able to understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent scope will be construed as being included in the present invention.

100: inspector terminal
110: camera 120: processor
130: memory 140: input unit
150: communication module 160: output unit
200: Structure appearance inspection device
210: processor 220: communication module
230: memory
300: manager terminal

Claims (15)

A structure exterior inspection method performed at an inspector terminal and a structure exterior inspection device,
(a) in a state where the inspector terminal has a damage identification code of a predefined size attached around the damaged area of the structure, photographing the damaged area of the structure to include the damage identification code, obtaining a damage inspection image and transmitting to the structure exterior inspection device;
(b) classifying, by the structure exterior inspection device, the type of damage by applying the damage inspection image to a pre-learned artificial intelligence classification engine;
(c) obtaining the shape and size of the damage by applying the damage inspection image to a pre-learned artificial intelligence damage detection engine according to the type of damage, by the apparatus for inspecting the external appearance of the structure; and
(d) the structure appearance inspection device overlaying and storing the shape and size of the detected damage on the damage inspection image;

The step (c) is
(c1) performing preprocessing by removing noise from the damage inspection image and adjusting brightness;
(c2) detecting the damage identification code in the damage detection image, binarizing the damage identification code, and counting pixels corresponding to the damage identification code to calculate a pixel resolution of the image relative to the physical size;
(c3) detecting a damaged area by performing segmentation on the preprocessed image using the artificial intelligence damage detection engine;
(c4) generating a binary image of the damaged area by extracting and binarizing the damaged area from the damage inspection image; and
(c5) confirming the shape of the damaged area by applying a morphology algorithm to the binarized image of the damaged area, and using the number of pixels of the damaged area and the pixel resolution, the length, width, and width of the damage. Obtaining a size of the damage including at least one; a structure appearance inspection method comprising a. The method of claim 1, wherein step (a) is
(A2) When the damage identification code of the size defined in the dictionary is attached around the damaged area of the structure, the camera is activated, calculating the effective target shooting distance using the hardware parameter of the inspection terminal, and using the image of the damage identification code input through the camera and the hardware parameter to calculate the actual shooting distance. The actual shooting distance is guided to the inspector so that the actual shooting distance is below the effective target shooting distance. The method of claim 2, wherein step (a) is performed before step (a2),
(a1) After a damage identification code is attached to a damaged area of a structure and a camera is activated, the inspector's terminal recognizes a damage management number from the attached damage identification code, receives structure identification information and a location of a damaged area on a structural drawing of the structure from the inspector, transmits the input to the structure exterior inspection device, and the structure exterior inspection device further comprises the step of storing the damage management number, the structure identification information, and the location of the damaged area in association with each other. delete The method of claim 1, wherein step (c5)
A Euclidean distance transformation image is obtained by applying a Euclidean distance conversion algorithm to the damaged area binarization image, a skeletonization image is obtained by applying a skeletonization algorithm to the damaged area binarization image, pixel values of each pixel corresponding to each other of the Euclidean distance conversion image and the skeletonization image are multiplied and a crack damage image is generated by multiplying a scaling factor to obtain the length, width, and width of the damage using the binarized image, the skeletonization image, and the crack damage image A method for inspecting the exterior of a structure characterized by The method of claim 5, wherein step (c5)
The extent of the damage is obtained using the number of pixels indicated as the damaged area in the binarized image, the length of the damage is obtained using the number of pixels indicated as the damaged area in the skeletonized image, and the maximum pixel value in the crack damage image. Structure appearance inspection method, characterized in that for acquiring the width of the damage. delete A computer program that is stored in a non-transitory storage medium and executed on a computer including a processor to perform the method for inspecting the appearance of a structure according to any one of claims 1, 2, 3, 5, and 6.
A structure appearance inspection device including a processor and a memory for storing predetermined instructions,
The processor executing the instructions stored in the memory
(a) receiving, from an inspector terminal, a damage inspection image generated by photographing a damaged area of a structure to include the damage identification code in a state where a damage identification code having a predefined size is attached around the damaged area of the structure;
(b) classifying the type of damage by applying the damage inspection image to a pre-learned artificial intelligence classification engine;
(c) obtaining the shape and size of the damage by applying the damage inspection image to an artificial intelligence damage detection engine trained in advance according to the type of damage; and
(d) overlaying and storing the shape and size of the detected damage on the damage inspection image;

The step (c) is
(c1) performing preprocessing by removing noise from the damage inspection image and adjusting brightness;
(c2) detecting the damage identification code in the damage detection image, binarizing the damage identification code, and counting pixels corresponding to the damage identification code to calculate a pixel resolution of the image relative to the physical size;
(c3) detecting a damaged area by performing segmentation on the preprocessed image using the artificial intelligence damage detection engine;
(c4) generating a binary image of the damaged area by extracting and binarizing the damaged area from the damage inspection image; and
(c5) confirming the shape of the damaged area by applying a morphology algorithm to the binarized image of the damaged area, and using the number of pixels of the damaged area and the pixel resolution to obtain a size of the damage including at least one of the length, width, and width of the damage; Structure appearance inspection device, characterized in that it comprises a. 10. The method of claim 9, wherein step (a) is
(A2) When the damage identification code of the size defined in the dictionary is attached around the damaged area of the structure, the camera is activated, calculating the effective target shooting distance using the hardware parameter of the inspection terminal, and using the image of the damage identification code input through the camera and the hardware parameter to calculate the actual shooting distance. The actual shooting distance is guided to the inspector so that the actual shooting distance is below the effective target shooting distance. 11. The method of claim 10, wherein the processor,
Before step (a2),
(a1) After the damage identification code is attached to the damaged area of the structure and the camera is activated, the inspector's terminal recognizes the damage management number from the attached damage identification code, receives structure identification information and the location of the damaged area on the structural drawing of the structure from the inspector, transmits the input to the structure exterior inspection device, and the structure exterior inspection device further performs the step of storing the damage management number, the structure identification information, and the location of the damaged area in association with each other. delete 10. The method of claim 9, wherein step (c5)
A Euclidean distance transformation image is obtained by applying a Euclidean distance conversion algorithm to the damaged area binarization image, a skeletonization image is obtained by applying a skeletonization algorithm to the damaged area binarization image, pixel values of each pixel corresponding to each other of the Euclidean distance conversion image and the skeletonization image are multiplied and a crack damage image is generated by multiplying a scaling factor to obtain the length, width, and width of the damage using the binarized image, the skeletonization image, and the crack damage image Structure appearance inspection device characterized by. 14. The method of claim 13, wherein step (c5)
The extent of the damage is obtained using the number of pixels indicated as the damaged area in the binarized image, the length of the damage is obtained using the number of pixels indicated as the damaged area in the skeletal image, and the maximum pixel value in the crack damage image.
delete

Images