KR20200013148A - Method, system and computer program for providing defect analysis service of concrete structure - Google Patents

Abstract

The present invention relates to a method, system and computer program for providing a concrete structure defect analysis service. The method for providing a concrete structure defect analysis service includes the following steps: receiving a concrete structure image from a first terminal as a query image; extracting at least one defective area estimated to have a crack or defect from the query image by applying the query image to a pre-learned defective area detection model; creating defect information by identifying a cracked or defective part of the defective area by pixel; and transmitting the defect information to at least one second terminal. The pre-learned defective area detection model is learned based on a plurality of concrete structure images, a bounding box including a cracked or defective part in the concrete structure images and a label indicating the characteristics of the bounding box. According to the present invention, since an artificial neural network improved more than before is used, a crack and a defect in a concrete structure can be more quickly and accurately extracted, and various defects such as delamination, exfoliation, leakage, rebar exposure and the like can be extracted.

Description

METHOOD, SYSTEM AND COMPUTER PROGRAM FOR PROVIDING DEFECT ANALYSIS SERVICE OF CONCRETE STRUCTURE}

The present invention relates to a method, a system and a computer program for providing a concrete structure defect analysis service, and more particularly, to a method and a system capable of automatically extracting and analyzing defects and damages of a concrete structure based on a cloud.

Due to the explosion of old facilities built during the high growth period, the importance of strengthening safety management is increasing, and recently, the paradigm of the construction industry is changing rapidly with the focus on facility safety and maintenance.

In particular, the collapse of construction and civil engineering facilities can cause great casualties. In order to detect the collapse of construction and civil engineering facilities in advance and to ensure safety, structural defects such as cracks, damages, and breakdowns of facilities are periodically investigated and The importance of pre-assessment of evaluation is emerging.

In particular, concrete structures that make up most of construction and civil engineering facilities have structural defects represented by cracks due to material characteristics, defects in construction, transient loads, and external environment. Increasing and subsequent cracking can significantly reduce structural safety, usability and durability resulting in collapse of the structure.

In order to solve this problem, conventionally, the appearance status including defect information such as cracks of facilities is temporarily recorded depending on the naked eye of the inspector, and the safety diagnosis of the concrete structure is then made by manually drawing the exterior inspection network through the visit. This was done, and the repair method and timing were decided based on this. Evaluating the condition of the facility by visual inspection takes a lot of time and manpower, there is a problem that the objectivity and reliability of the survey data is not guaranteed.

In order to solve the problem of the condition evaluation of the facility by visual inspection, image processing technology has been used, but crack detection and crack information is recognized by recognizing crack-like features (construction joints, joints, spider webs, intaglio, etc.) as cracks. The problem has been raised that the extraction accuracy is poor and still requires further expert review.

In recent years, attempts to recognize cracks using artificial neural networks have been made at home and abroad, such as Korean Patent No. 1772916, but there is a problem in that it cannot reflect various environments of actual facilities due to limited learning image data. In addition, in order to increase the speed of crack recognition processing, an excellent computing environment is required. Therefore, the application and use of the technology is not achieved.

In addition, damage to the structure due to various defects such as peeling, dropping, leaking, rebar exposure in addition to the crack defects, the service that can easily analyze various types of defects is required.

An object of the present invention is to provide a method for quickly extracting defects such as cracking, peeling, and rebar exposure of a concrete structure by using an artificial neural network improved compared to the conventional art.

In addition, the present invention provides a cloud computing-based analysis system, a software and system for easily verifying the analysis results of the image and evaluate the state of the structure by a plurality of subjects simply by transmitting the structure image collected in various places Another aim is to enable significant reductions in the cost of introduction.

In order to achieve the above object, the present invention provides a method for providing a cloud-based concrete structure defect analysis service, the method comprising: receiving a concrete structure image as a query image from a first terminal, a defect region previously learned from the query image Extracting one or more defect regions in which the crack or defect is estimated to be present in the query image, and generating defect information by identifying, in a pixel unit, a portion in which the crack or defect has occurred in the query image; And transmitting the defect information to one or more second terminals, wherein the defect area detection model includes a plurality of concrete structure images, a bounding box including a portion where cracks or defects are generated in the concrete structure image, and the Use labels that indicate the characteristics of the bounding box. It is characterized by learning.

According to the present invention as described above, using the artificial neural network improved compared to the conventional, it is possible to extract the crack defects of the concrete structure more quickly and accurately, it is possible to extract various defects, such as peeling, peeling, leaks, rebar exposure.

In addition, according to the present invention, a plurality of subjects can easily check the analysis result of the image only by transmitting the facility survey image collected in various places, each management agent is a software and system for determining the crack defect of the structure The cost of introduction can be greatly reduced.

1 is a view showing a cloud-based concrete structure defect analysis service providing system according to an embodiment of the present invention,
2 is a block diagram illustrating a server according to an embodiment of the present invention;
3 is a block diagram illustrating a defect information generation unit according to an embodiment of the present invention;
4 is a flowchart illustrating a cloud-based concrete structure defect analysis service providing method according to an embodiment of the present invention;
5 is a flowchart illustrating a method for generating defect information according to an embodiment of the present invention;
6 is a flowchart illustrating a structure of a defect area detection model according to an embodiment of the present invention;
7 is a view for explaining a method for generating defect information according to an embodiment of the present invention;
8 to 10 are diagrams for describing input data and output data of a defect information generation service according to an embodiment of the present invention.

The above objects, features, and advantages will be described in detail with reference to the accompanying drawings, whereby those skilled in the art may easily implement the technical idea of the present invention. In describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings are used to indicate the same or similar components, all combinations described in the specification and claims may be combined in any way. And unless stated otherwise, reference to the singular may include one or more, and reference to the singular may also include the plural expression.

1 is a view showing a cloud-based concrete structure defect analysis service providing system according to an embodiment of the present invention. Referring to FIG. 1, the concrete structure defect analysis service providing system according to an exemplary embodiment of the present invention includes a terminal 100 and a server 200.

The terminal 100 is an electronic device that collects and transmits a concrete structure image to a server. The terminal 100 includes a wired and wireless communication module for transmitting and receiving data through a wired and wireless network, and a computer for controlling the collection, transmission, editing, and display of images. It includes a processor. Examples of the terminal 100 may include a personal computer, a smartphone, and a tablet PC 100a that transmits a concrete structure image photographed from the outside to the server 200.

In addition, the terminal 100 may be a computing device including a camera module. Other examples of the terminal 100 may include a drone, a survey robot, a CCTV, a tunnel scanning equipment, a sewer pipe survey equipment, a general camera, etc., which directly acquires an image of a concrete structure and transmits it to the server 200.

The server 200 receives an analysis target image (query image) from the first terminals 100a, 100b, and 100c, and transmits a result of analyzing the query image to the second terminal 100d. Here, the first terminal and the second terminal may be the same terminal or may be a different terminal.

As an example of the case where the terminal transmitting the query image and the subject receiving the defect information are the same, the first terminal 100a transmits the concrete structure image stored in the first terminal 100a to the server 200 as the query image. The server 200 may receive an analysis result (defect information) of the query image.

As an example of the case where the first terminal and the second terminal are different, when the first terminal 100b is a facility checking apparatus such as a drone, the drone 100b may display a server in real time or at a time point at which the shooting ends. And transmit to 200. When the analysis of the received query image is completed, the server 200 may transmit the analysis result to a terminal (second terminal) of one or more facility inspection agencies that are registered.

The transmission of the analysis result may be provided by a push method, and when the second terminal accesses a web server to request an analysis result, the server may be provided by transmitting the analysis result in response to the request.

The concrete structure defect analysis service of the present invention may be a service provided as a cloud (Software as a Service), so even if the concrete structure defect analysis application is not installed and / or stored in the terminal, the terminal in the web browser The service provided by the server 200 may be accessed through the Internet.

Therefore, it can be understood that the concrete structure defect analysis service providing computer program according to an embodiment of the present invention is an application program that can be accessed, managed, and / or used on a network basis. In addition, according to the present invention, since the terminal 100, which manages activities in the central server 200, can access the application program through the web, the use of the service is limited by the computing environment, place, and type of the terminal. There is an advantage that is not.

Hereinafter, the operation of the server 200 in the concrete structure defect analysis service providing system according to an embodiment of the present invention will be described in more detail with reference to FIGS. 2 to 5.

2, the server 200 may include a communication unit 210, an image processor 230, a defect area extractor 250, a defect information generator 270, and a storage unit 290.

The communication unit 210 includes a communication module for transmitting and receiving data with the terminal 100 through a wired or wireless network. The communication unit 210 may receive the concrete structure image as a query image from the first terminal and transmit defect information to the second terminal. As described above, the first terminal and the second terminal may be the same terminal, but may be different from each other, the communication unit 210 of the present invention is not limited by the data transmission and reception protocol and communication method. The query image may be a video including a plurality of frames, and may be a single or a plurality of 2D / 3D images of the concrete structure. In addition, the query image may be a part or the whole of the concrete structure. When the query image is a video, the query image may be transmitted to the image processor 230 and may be converted into a single image generated using a plurality of frames.

The image processor 230 extracts a plurality of images from the concrete structure image received from at least one first terminal, and generates a deployment image of the concrete structure by joining the plurality of images. When the query image received by the communication unit 210 is a video or a plurality of images, the image processor 230 may generate a single image using the video or the plurality of images. Therefore, if the query image is a two-dimensional single image, the communication unit 210 may transfer the received single image to the defect region extraction unit 250 directly without transferring the received single image to the image processing unit 230.

For example, suppose that a drone photographed a concrete pier in the air and photographed a portion of the concrete pier using a scanning device from the ground in order to analyze a defect of the concrete pier and transmitted it to the server 200. The image processor 230 may extract a frame necessary for generating a developed image of the piers from the captured image received from the drone and the captured image received from the scanning device. Then, the extracted frame may be bonded to generate a developed image of the piers. As another embodiment, as shown in FIG. 9, the image processor 230 may generate an original image development degree B by correcting, editing, and bonding a plurality of structure photographed images A captured by a high definition digital camera. .

In more detail, when the image processor 230 receives a plurality of captured images, the image processor 230 generates the original image development degree by correcting the images, editing the images by matching the corrected images, and extracting the images and joining the extracted images. can do. The image processor 230 may extract the images after synchronizing the start and end frames of the plurality of images.

When generating the original image development degree using the video, the image processor 230 synchronizes the frames of the video, adjusts the brightness and color of the original image (a), and corrects the distortion of the image (c). Afterwards, the moving image may be cut (d), the image may be extracted, the file may be divided, and the image may be joined to generate an original image development degree. Since the image processor 230 has different blurring and time intervals depending on the speed of the moving image photographing vehicle, the image processor 230 may extract a large number of images after cutting the joined image in the narrowest possible range. In addition, it is possible to automatically join the extracted image to the final image form through feature point matching after cutting out the portion where there is no camera distortion in the image. Feature point matching may be performed by extracting a common feature point between multiple images using a feature operator, for example, SIFT, SURF, ORB, or by synthesizing a linked image by comparing similarities of feature amounts between feature points on an image.

In the above-described image registration, the image processor 230 may detect an image feature point in each frame and match the image by matching the feature points. In an embodiment of the present invention, the image processing speeds up parallel computing. It can be processed faster using cloud computing features that utilize algorithms.

The image processor 230 transfers the generated development image to the defect area extraction unit 250 so that the defect area extraction unit 250 may extract the defect area from the development image.

The defect region extractor 250 may apply one or more defect regions to which a crack or defect is estimated in the query image by applying the query image to a previously trained defect region detection model. If the image input to the defect area extractor 250 is a developed image generated by joining a plurality of images by the image processor 230, the defect area extractor 250 may apply the developed image to the defect area detection model. Can be.

The defect area detection model of the defect area extracting unit 250 is trained by using a plurality of concrete structure images, a bounding box including a portion in which a crack or a defect is generated in the concrete structure image, and a label representing characteristics of the bounding box as data sets. Can be. The data set is divided into training data and verification data for verifying convergence and evaluating accuracy performance. As will be described later, the defective area detection model according to an embodiment of the present invention includes a plurality of neural networks according to each function, but the entire model can be collectively trained with the above data set.

More specifically, the defect region detection model according to an embodiment of the present invention is a convolutional neural network that generates a feature map of features in a query image, and an area for extracting a proposed region from the feature map. It may include a region proposal network, and a region of interest pooling layer (RoI pooling layer) for extracting features of the region corresponding to the proposal region from the feature map.

Referring to FIG. 6, which illustrates a structure of the defect area detection model of the present invention, the defect area extractor 250 may include a feature extractor 253, an area suggestor 255, and a classifier 257. have.

In the feature extractor 253, the query image 11 is input to a convolutional neural network (CNN) of the defect area detection model, and the composite product neural network 13 is configured to input the query image 11. The feature map may be extracted to generate a feature map 15 of features in the query image. The feature map (or feature map) refers to a stage in which a feature for class classification is expressed in a form immediately before classifying a class of each object (crack or defect) in the composite product neural network.

The convolutional neural network 13 may include a plurality of convolutional layers and a plurality of subsampling layers. The composite product layer is a layer that performs a convolution on the input image, and the subsampling layer is a layer that locally extracts the maximum value of the input image and maps it to the 2D image. Subsampling may be performed.

The feature map 15 generated by the feature extractor 253 is input to a region proposal network 23 of the region suggestor 255. The area suggestion network 23 is a network that extracts the proposed area from the feature map, in which the 3 x 3 sliding window moves across the feature map 15 at the last layer of the composite product neural network 13 and lowers it. Mapping in dimensions, we create several possible regions based on k fixed-rate anchors for each sliding window position (this is called a 'bounding box').

The area suggestion network 23 may output whether an object (crack or defect) is included in each bounding box, and what coordinates of the bounding box are. That is, the area suggestion network 23 proposes an area in which the existence of an object is high in the feature map. If the “objectivity” score of the bounding box is higher than a specific threshold, the coordinates of the bounding box may be proposed as the object candidate area. The result of the region proposal network 23 including the plurality of object candidate regions may be referred to as a region proposal 25.

The classification unit 157 extracts a feature corresponding to each object candidate region from the feature map 15 using the ROI pooling layer 31 (33).

The feature extracted by the region of interest integration layer 31 is a normalized feature for discriminating an object. The region of interest integration layer 31 is configured to generate object candidate regions of various sizes proposed through the region proposal network 23. Down sampling for input to the Fully-Connected layer 35.

The feature of the object candidate region extracted from the region of interest integration layer 31 may be classified into crack / non-crack, defect (leakage, corrosion, etc.) / Defect in the fully connected layer 35.

The defect area detection model of the present invention includes a plurality of concrete surface images, a bounding box including a portion where cracks or defects occur within the concrete surface image, and a label indicating the characteristics of the bounding box (cracking, non-cracking, defects, defects, End-to-end training using leakage, corrosion, etc.) is possible. That is, in learning a defect area detection model that is a deep neural network, it is possible to collectively train the entire network by replacing a plurality of neural network learnings according to each function, thereby minimizing memory load.

Furthermore, the defect area detection model of the present invention can greatly improve the defect recognition processing speed than the conventional method of detecting cracks on the concrete surface using only the synthetic neural network (CNN), and in addition to cracking defects of the concrete structure, It has the advantage that it can be applied to various types of defects and damages such as dropout, leakage and rebar exposure.

Defect area detection model according to an embodiment of the present invention can be learned using the mini batch gradient descent (MBGD) in order to optimally recognize the crack defects of the concrete structure. The minibatch gradient method is a kind of gradient descent that randomly extracts a certain size data set and learns it. The gradient descent method is an optimization technique that adjusts variables in a direction in which a set error function decreases, and is used to update weights of neural networks during learning.

When the defect area extraction unit 250 extracts a defect area from the query image and identifies a kind of a defect of each defect area, the extracted defect area is transferred to the defect information generation unit 270.

The defect information generator 270 generates defect information by identifying, in a pixel unit, a portion where a crack or a defect is generated in the defect area. The defect information is a crack or defect that is analyzed by identifying, in a pixel unit, a portion in which a crack or a defect is generated from an image (image) corresponding to the defect region, targeting the defect region extracted by the defect region extraction unit 250 of the present invention. Includes information such as the direction, width, length, area, location of the.

For example, since the cracks are characterized by dark, thin and long lines, the defect information generation unit 270 distinguishes the cracked areas by pixels in the image extracted as the defect area using an image processing technique. Crack information can be extracted and crack information can be analyzed to generate crack information.

The defect information may be in the form of a query image (output image) in which a defect object is displayed, a visual inspection network and crack / defect defect data (aggregate table), and provide qualitative and / or qualitative analysis results of the query image. . This will be described later in more detail with reference to FIGS. 8 to 10.

More specifically, referring to FIG. 3, the defect information generator 270 according to an embodiment of the present invention may include a preprocessor 271, a binarizer 273, a morphology unit 275, and a noise remover 277. It may include an analysis unit 279.

The preprocessing unit 271 may generate the preprocessed defect region 40 by improving the image quality of the defect region 38 extracted by the defect region extraction unit 250 using a digital image processing technique. As an example of the preprocessing technique, brightness histogram smoothing and sharpening techniques may be used.

The brightness histogram smoothing technique resets the brightness value by normalizing the cumulative frequency of brightness values of the target image, and improves the visibility of defects by uniformly correcting the brightness distribution.

The sharpening technique is an image processing technique that enhances image sharpness by emphasizing details corresponding to high frequencies, thereby improving defect information generation performance. In the sharpening technique, the contour of an object determined to be a defect in the crack defect area 38 may be emphasized by performing a convolution operation on a predetermined sharpening kernel.

The binarization unit 273 may binarize the defect area image by comparing the pixel value included in the defect area 38 with a preset reference value and mapping the pixel value to any one of the preset binary values according to the comparison result. . If the defect area 38 is image-processed to improve the quality in the preprocessor 271, the binarization unit 273 may perform binarization using the image of the preprocessed fungal defect area 40 as an input value. That is, the embodiment performed by each configuration of the defect information generation unit in the present specification may be understood to be applicable to both the defect area 38 or the preprocessed defect area 40.

For example, in the above-described embodiment of FIG. 6, the defect region extractor 250 extracts and recommends the defect region 38, which is estimated to have a crack, like a red box. The defect information generator 270 may compare each pixel value of the crack defect area 38 or the preprocessed crack defect area 40 with a preset reference value. Given the nature of the crack defects, which are characterized by dark, thin and long lines, the reference value may be a specific brightness value that can classify the brightness of the pixel into two groups. The binarization unit 273 may binarize the defective area image by mapping the pixel value to 1 when the brightness of the pixel is darker than the reference value and mapping the pixel value to 0 when the brightness of the pixel is brighter than the reference value. An example of binarized crack defect area 42 is shown in FIG. 7.

The morphology unit 275 may perform a morphology operation on the binarized image. The morphology operation process deals with images in terms of morphology, and includes erosion operations on binary images, dilation operations on binary images, opening operations, and closing operations.

The erosion operation is to remove one or two pixels of noise and can be used to thin the object area (the crack in the example of FIG. 7) and to remove the noise. The inflation operation is to fill the hole inside the object and can be used to inflate the object area.

The open operation is to perform an expansion operation after the erosion operation, and the close operation is to perform an erosion operation after the expansion operation. Open operations can be used to remove noise. For example, in the binarized image 42 of FIG. 7, there are a number of black-colored points around the center thin line, which are not cracks, and thus are likely to correspond to noise. The morphology unit 275 performs the above-described operation so that a meaningful object can be highlighted in the image.

The noise removing unit 277 specifies cracks and removes noise from the morphologically computed image. Noise removal may be performed in various ways. For example, the noise removing unit 277 may recognize and remove the noise when the length and / or area of the object is less than a preset value. In another embodiment, the noise remover 277 may classify the noise using a simple neural network. As a result, as shown in FIG. 7 (46), it is possible to obtain a crack defect image in which cracks are specified and noise is removed.

The analysis unit 279 may determine at least one of the direction, width, or length of the crack or the defect corresponding to the defect object in the defect area by using the defect object (the blue object of the 46) specified by the noise removing unit 277. By measuring, defect information can be generated. For example, in the defect area image 40 illustrated in FIG. 7, since only a portion corresponding to the blue line is specified as the crack defect object, the analyzer 279 determines the direction, width, and width of the pixel corresponding to the crack defect object. The length and the like can be measured.

That is, conventionally, since the CNN is applied to the entire image to classify defects / defects in the area corresponding to the sliding window, it takes a long time, but according to the present invention, the area where the defects are estimated to exist at a high speed Since only the extracted region is extracted and the extracted region is processed by image analysis, the overall speed and accuracy of crack and defect extraction are greatly increased.

The storage unit 290 is a query image received from a plurality of terminals, the deployment image of the concrete structure generated by the image processing unit 230, a feature map generated by the defect area extraction unit 250, defect information generation unit 270 The generated defect information is stored in the database. The present invention provides a service based on cloud computing, and the storage unit 290 may not be included in the server 200 but may be separately installed outside the server 200.

Hereinafter, a method of providing a cloud-based concrete structure defect analysis service according to an exemplary embodiment will be described with reference to FIGS. 4 to 5.

Referring to FIG. 4, the server may receive a concrete structure image as a query image from one or more first terminals (S100). The server may extract a plurality of images from the received one or more query images, and may generate a deployment image of the concrete structure by joining the plurality of images (S200). Then, the query image is applied to the previously trained defect region detection model to extract one or more defect regions estimated to have cracks or defects in the query image (S300). When the server extracts the defect area, the server may identify a part where a crack or a defect has occurred in the defect area in pixel units to generate defect information (S400), and transmit the defect information to one or more second terminals (S500). When the server receives a defect information request including pre-registered identification information from any second terminal in step 500, the server may transmit defect information corresponding to the defect information request to the second terminal, and receive and defect the defect information request. The transmission of information can be via the web.

For example, suppose that user A accesses a concrete structure defect analysis service provided through the web. User A can access the server through the ID and password already registered in the server, and upload the analysis target image to the server (S100). The server may analyze the uploaded analysis target image through steps 200 to 400 and store a result of defect analysis on the analysis target image in a database used by the server.

When the generation of the defect information is completed, the server may notify the user A of the generation of the defect information through the push notification, and the user A may access the service server through the Internet and check the defect information. In step 500, the server may immediately transmit defect information generation to the terminal B registered as the user A's.

Meanwhile, in operation 400, the server may perform preprocessing to improve image quality of the defective region extracted in operation 300 (S410). For example, the server may perform brightness histogram smoothing, sharpening, and the like, to uniformly correct the intensity distribution of the defect area to increase the visibility of the defect, or to enhance the sharpness of the image by emphasizing the portion corresponding to the high frequency. Step 410 may be omitted as necessary.

Next, the server may compare the pixel value included in the defective area with a preset reference value and perform binarization by mapping the pixel value to any one of the preset binary values according to the comparison result (S430). If step 410 is performed, the server performs binarization on the preprocessed image. If step 410 is omitted, the server may binarize the extracted defect region image.

Next, the server may process a morphology operation on the binarized image (S450). The server may identify the defect object by removing noise from the morphologically computed image (S470), and generate defect information by measuring at least one of a direction, a width, or a length of a crack or a defect corresponding to the defect object in the defect area. There is (S490).

Hereinafter, an embodiment of input data input to the concrete structure defect analysis service and output data output as a defect analysis result will be described with reference to FIGS. 8 to 10.

Referring to FIG. 8, an analysis target image, such as 1000 of FIG. 8, may be input to the concrete structure defect analysis service providing server 200 according to an exemplary embodiment. The analysis target image may be a single image or may be a video including a plurality of images and / or a plurality of consecutive image frames. When the server 200 receives the analysis object image 1000, the server 200 may output analysis results 1200, 1400, and 1600 of the analysis object image 100 according to the above-described concrete structure defect analysis. The analysis results 1200, 1400, 1600 may be transmitted to a terminal that has previously registered or a terminal that has transmitted the analysis target image 1000.

The analysis result 1 1200 illustrated in FIG. 8 outputs defect information generated on the basis of the bonded single image by combining the analysis target image including a plurality of images into a single image representing the entire concrete structure in the server 200. An example of an image.

Referring to FIG. 10 in which the analysis result 1 (1200) is enlarged and displayed, in the analysis result 1 (1200) including the concrete structure image, the crack defect objects are light blue (less than 0.1 mm), green (0.1 mm to 0.2 mm), Blue (0.2mm ~ 0.3), yellow (0.3mm ~ 0.5mm), red (0.5mm or more) and crack repair can be indicated by dotted lines. It is obvious to those skilled in the art that the display range, display color, etc. of the crack defect object may vary depending on the setting. On the other hand, as shown in the legend of the analysis result 1 (1200), the analysis results may be exposed to the rebar exposure, material separation, leakage, white tae, peeling, peeling, repair parts, reticular cracks, condensation.

Analysis result 2 (1400) is an example of an appearance survey network including defect information on an image to be analyzed. Appearance survey network is a visual representation of the type and location of the defects examined during inspection or diagnosis in the design and / or abbreviated drawings. Appearance survey network 1400 standardized symbols, letters, etc. through the safety inspection and detailed safety diagnosis detailed instructions of the building can be used.

The analysis result 3 1600 is an aggregate table quantitatively displaying defect data according to the analysis result of the analysis target image, and may provide a defect analysis result as shown in Table 1 below. The aggregate table 1600 may include types of defects such as cracks, leaks, breaks and defects, material degradation, and drainage, and may include characteristic information for each type. For example, in the case of a crack defect, the number of cracks for each grade of the crack may be displayed according to whether the crack is longitudinal, transverse, quadrilateral, or lattice.In the case of a leak defect, whether the ceiling is leaked or the wall is leaked. The type of leak may be displayed, such as a floor leak or a construction joint leak.

Defect grades may be classified according to preset criteria, and thickness, length, area, etc. may be used as reference indicators.

Concrete structure defect analysis service according to an embodiment of the present invention is provided through the cloud computing, the web, the user does not need to purchase or install an application separately to use the service. In addition, according to the present invention, since the reliability of the defect area detection model is continuously increased by using the collected query image as learning data, the crack width and the like can be calculated with a very high accuracy of 0.2 mm or less, and when uploading a concrete structure image, the image It takes a very short time of about 10 minutes to extract and analyze the cracks or defects in the bar, which has the advantage that the user can use the service in near real time.

The server may charge a service fee to the terminal according to generation and provision of defect information.

Some embodiments omitted in the present specification may be equally applicable to the same subject matter. In addition, the above-described present invention can be variously substituted, modified and changed without departing from the spirit of the present invention for those skilled in the art to which the present invention belongs, the above-described embodiment and the accompanying It is not limited by the drawings.

100: terminal
200: server

Claims (10)

In the server provides a cloud-based concrete structure defect analysis service,
Receiving the concrete structure image as a query image from the first terminal;
Applying the query image to a previously trained defect region detection model to extract one or more defect regions in which the crack or defect is estimated to exist in the query image;
Generating defect information by identifying, in a pixel unit, a portion where a crack or a defect has occurred in the defect area;
Transmitting the defect information to one or more second terminals,
The defect area detection model is trained using a plurality of concrete structure images, a bounding box including a portion where cracks or defects are generated in the concrete structure image, and a label representing characteristics of the bounding box. To provide concrete structural defect analysis services
The method of claim 1,
The defective area detection model
A convolutional neural network for generating a feature map of features in the query image; A region proposal network identifying an object candidate region in the feature map; And an ROI pooling layer that extracts a feature of the object candidate region from the feature map. Cloud-based extraction, including a fully-connected layer that classifies the features of the extracted object candidate region, extracts a defect region from the query image, and identifies a defect type of an object included in the defect region. How to provide concrete structure defect analysis service.
The method of claim 1,
When receiving the query image,
Extracting a plurality of images from the query image;
Bonding the plurality of images to generate a deployment image of the concrete structure;
The defect region extraction step
Applying the developed image to the defect area detection model;
Cloud-based concrete structure defect analysis service providing method.
The method of claim 1,
Generating the defect information
A binarization step of comparing a pixel value included in the defective area with a preset reference value and mapping the pixel value to any one of a preset binary value according to a comparison result;
A morphology step of morphologically processing the binarized image;
Removing noise from the morphologically processed image to specify a defect object;
And generating defect information by measuring at least one of a direction, a width, or a length of a crack or a defect corresponding to the defect object in the defect area.
The method of claim 1,
The terminal is a cloud-based concrete structure defect analysis service providing method, characterized in that the terminal comprising a camera module.
In the cloud-based concrete structure defect analysis service providing system,
A first terminal collecting the concrete structure image and transmitting the image to the server;
When the concrete structure image is received from the first terminal as a query image, the server comprises a server for generating defect information by analyzing the query image,
The server is
A communication unit which receives the query image from the first terminal and transmits the defect information to a second terminal;
A defect region extraction unit which applies the query image to a previously trained defect region detection model and extracts one or more defect regions in which the crack or defect is estimated to exist in the query image;
A defect information generation unit for generating defect information by identifying a crack or a defect in the defect area in units of pixels,
The defect area detection model is trained using a plurality of concrete surface images, a bounding box including a crack or defect portion in the concrete surface image, and a label representing characteristics of the bounding box. System for providing concrete structural defect analysis services.
The method of claim 6,
The defective area detection model
A convolutional neural network for generating a feature map of features in the query image; A region proposal network identifying an object candidate region in the feature map; And an ROI pooling layer that extracts a feature of the object candidate region from the feature map. A cloud-based method for extracting a defect region from the query image and identifying a defect type of an object included in the defect region, including a fully-connected layer that classifies features of the extracted object candidate region. Concrete structure defect analysis service provision system.
The method of claim 6,
The defect information generation unit
A binarizer for comparing the pixel value included in the defective area with a preset reference value and mapping the pixel value to any one of a preset binary value according to a comparison result;
A morphology unit configured to perform a morphology operation on the binarized image;
A noise removing unit for removing a noise from the morphologically computed image to specify a defect object;
The cloud-based concrete structure defect analysis service providing system including an analysis unit for generating defect image information by measuring at least one of the direction, width or length of the crack or the defect corresponding to the defect object in the defect area.
The method of claim 6,
The server is
The image processing unit may further include extracting a plurality of images from the query image and joining the plurality of images to generate a developed image of the concrete structure.
The defect area extraction unit is a cloud-based concrete structure defect analysis service providing system applying the deployment image to the defect area detection model.
The method of claim 6,
The cloud-based concrete structure defect analysis service providing system, characterized in that the first terminal comprises a camera module.

Images