KR20230149121A - Method and device for determining construction progress through classification based on deep learning - Google Patents

Abstract

The present invention relates to a method for determining construction progress. The deep learning-based construction progress determination method performed on the server according to the present invention includes the steps of receiving a construction site image; Performing pixel-level classification on the construction site image using a learned construction site classification model; and calculating construction progress through the ratio of pixels for each category. According to the present invention, objective indicators can be calculated through a learned neural network and construction progress can be judged more conveniently.

Description

{Method and device for determining construction progress through classification based on deep learning}

The present invention relates to a method for determining construction progress. More specifically, the present invention relates to a method for determining progress using images of a construction site.

With the development of artificial intelligence technology, artificial intelligence is being applied to various technical fields.

In particular, various algorithms based on deep learning are being developed to track objects through mathematical operations using the pixel values of the input image and classify the tracked objects, and perform convolution operations on feature values defined as matrices. CNN (Convolution Neural Network) neural network models, which are combined with multiple layers that perform, are optimized and used according to the domain to which they are applied.

The program used for process management at construction sites establishes a process plan for each work unit (activity) at the beginning of the site and reflects the process performance by manually entering the performance by quantity and period as the construction progresses. The process performance reflection method has a problem in that the actual process progress status is not automatically reflected in the performance.

In order to collect and reflect the performance of each sector, a separate person is needed to understand the field process as a whole and a dedicated manpower to run the program.

In addition, this method of performance collection is difficult to reflect immediately and has low accuracy, so it is rarely used in actual process management and is limited to a forced assortment due to the request of the ordering party.

Therefore, the prior patent (Korea Registered Publication No. 10-1988350) compares the construction daily report with the planned quantity of the construction daily report through AI analysis and then assigns process weights to over/under input, process delay, etc. to ensure optimal process management and input quantity. We propose an automatic on-site process management system through AI analysis of inspection and construction daily reports using a new virtual process module that can be used for prediction.

However, the prior patent also deals with indirect process management based on volume, so there is a need to propose a direct method to monitor actual process results.

The purpose of the present invention is to propose a method for managing construction progress through objective indicators.

The purpose of the present invention is to determine construction progress using structure information obtained from construction site images.

The purpose of the present invention is to solve the lack of consistency in judgment of progress depending on individual judgment through analysis using a learned neural network model.

In addition, the purpose of the present invention is to reduce resources due to human judgment of the construction site and prevent safety accidents.

A deep learning-based construction progress determination method performed on a server according to the present invention to solve the above technical problem includes receiving an image of a construction site; Performing pixel-level classification on the construction site image using a learned construction site classification model; and calculating construction progress through the ratio of pixels for each category.

The above classification includes vertical reinforcing bars or floor reinforcing bars at construction sites.

The above classification includes food in construction sites.

The above classification includes exterior wall formwork or wall formwork within the construction site.

The construction site classification model includes an encoder that extracts a feature map by performing a convolution operation between pixels in the input construction site image, and a decoder that upsamples the extracted feature map and outputs a classification result for each pixel.

The encoder includes a first network that extracts a first feature map by performing convolution from the construction site image; And a second network that extracts a plurality of feature maps by calculating pixel-to-pixel convolution of the first feature map according to a plurality of ratio parameters, and extracts a second feature map by merging the plurality of feature maps.

Preferably, the decoder upsamples the third feature map obtained by reducing and merging the dimensions of the output of the first network and the second feature map and outputs a classification result for each pixel.

It is desirable to divide the construction site image into a plurality of partial images and perform pixel-level classification for each divided partial image.

In the step of calculating the construction progress, it is desirable to calculate the progress using the progress and elapsed period of the previous construction site image for the construction site image input in time series.

The construction site image is an image taken from a plurality of angles, and the step of calculating the construction progress preferably calculates the progress using the ratio of pixels of the construction site image to the plurality of angles.

A deep learning-based construction progress determination device according to the present invention to solve the above technical problem includes a communication unit that receives construction site images; One or more processors that determine construction progress from the construction site image; and an input/output unit, wherein one or more processors receive a construction site image, perform pixel-level classification on the construction site image using a learned construction site classification model, and provide a ratio of pixels for each classification. It is desirable to calculate construction progress through .

According to the present invention, objective indicators can be calculated through a learned neural network and construction progress can be judged more conveniently.

In addition, according to the present invention, errors due to time differences in on-site verification can be reduced by simultaneously determining the progress status from all images taken of the construction site.

Additionally, the present invention can provide users with intuitive decision information by providing images of construction progress.

Additionally, by using images from various angles, more accurate progress can be determined efficiently.

Figure 1 is an exemplary diagram showing a construction progress determination system according to an embodiment of the present invention.
Figure 2 is a flowchart showing construction progress judgment according to an embodiment of the present invention.
Figure 3 is a diagram showing the configuration of a construction progress judgment model according to an embodiment of the present invention.
Figure 4 is a flowchart showing construction progress judgment according to an embodiment of the present invention.
Figure 5 is a diagram showing the detailed configuration of a construction progress judgment model according to an embodiment of the present invention.
Figure 6 is a diagram showing the filter configuration of a construction progress judgment model according to an embodiment of the present invention.
7 to 11 are diagrams showing classification results of a construction progress judgment model according to an embodiment of the present invention.
Figure 12 is a diagram showing a method of learning a construction progress judgment model according to an embodiment of the present invention.
Figures 13 and 14 are diagrams showing the configuration of a construction progress judgment model according to an additional embodiment of the present invention.
Figure 15 is an exemplary diagram showing the implementation of a server in the form of a computing device according to one embodiment of the present invention.

The following merely illustrates the principles of the invention. Therefore, a person skilled in the art can invent various devices that embody the principles of the invention and are included in the concept and scope of the invention, although not clearly described or shown herein. In addition, all conditional terms and embodiments listed in this specification are, in principle, clearly intended only for the purpose of ensuring that the inventive concept is understood, and should be understood as not limiting to the specifically listed embodiments and states. .

The above-mentioned purpose, features and advantages will become clearer through the following detailed description in relation to the attached drawings, and accordingly, those skilled in the art in the technical field to which the invention pertains will be able to easily implement the technical idea of the invention. .

Additionally, when describing the invention, if it is determined that a detailed description of known technology related to the invention may unnecessarily obscure the gist of the invention, the detailed description will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

1 is an exemplary diagram showing the structure of a device that performs a method for determining construction progress according to an embodiment of the present invention.

Referring to FIG. 1, the construction progress determination server 1000 according to this embodiment inputs the site image 50 acquired from the construction site into the learned neural network model 100 and classifies the site image 50 (60). ) can be used to judge the progress of construction.

The construction progress determination server 1000 according to this embodiment can collect site images 50 acquired from various angles in time series and determine progress by identifying changes in works within the images.

At a building construction site, various structures are formed depending on the purpose, so the progress of construction can be judged through the installation status of the structures.

Hereinafter, five structures typically detected in this example will be described.

For example, a vertical reinforcing bar can be used to create a frame by erecting reinforcing bars vertically inside a joint or on a wall plate to vertically connect adjacent upper and lower wall plates within the same plane.

Marking is a line marking on the surface of the shape, size, or location for jointing, custom processing, or attachment of members. It can indicate the position of the seam or the position of the finished surface on walls, floors, pillars, etc. .

Exterior wall formwork is a temporary structure that pours unhardened concrete into a concrete structure in a certain shape or size, curing and supporting it until it reaches the desired strength, and serves as a formwork.

Bottom reinforcement is a reinforcing bar structure formed on the floor for horizontal support and can be expressed as slab reinforcement.

Wall-form is a form used to make concrete walls and is also expressed as a retaining wall form.

The above structures are expressed in images taken of the construction site as they are directly installed at the construction site. Therefore, the method for determining construction progress according to this embodiment detects five essential structures using a neural network learned from the site image 50, and The overall construction progress is judged based on the progress status of the relevant structure.

Furthermore, the five structures detected in this embodiment are just examples, and necessary structures can be added or selected depending on the construction site, and the types of structures detected through the neural network model are not limited to this.

Hereinafter, the method for determining construction progress according to this embodiment will be described in more detail with reference to FIG. 2.

First, in this embodiment, the server 1000 receives the construction site image 50 (S100).

The construction site image 50 may be obtained from CCTV installed at the site or may be captured through a camera mounted on an unmanned aerial vehicle such as a drone.

The construction site image 50 may be an image taken of a large area to determine the progress of the overall construction, and may be taken at an angle that makes it easy to identify depending on the characteristics of the structure.

Next, classification of objects included in the construction site image 50 is performed using the learned construction site classification model 100 (S200).

Specifically, in this embodiment, objects included in the construction site image 50 can be classified on a pixel basis.

Specifically, referring to FIG. 3, in this embodiment, the construction site classification model 100 performs a convolution operation between pixels in the input construction site image 50 to extract a feature map and upsamples the extracted feature map. Thus, it may be configured to include a decoder that outputs classification results for each pixel.

The specific operation of the construction site classification model 100 according to this embodiment may be performed in the same steps as shown in FIG. 4.

Referring to FIG. 4, the encoder of the construction site classification model 100 can extract a feature map in a form in which the characteristics of objects to be identified are emphasized from the input construction site image 50 (S210).

Therefore, the encoder may be configured to include convolution layers that perform convolution operations to extract feature maps. Filters in the convolution layer can output the output by emphasizing the values of pixels including the structure by performing a pixel-level convolution operation on the input construction site image 50.

The feature map with the above features highlighted is restored to the original size image through the decoder's upsampling process, and during this process, the pixel-by-pixel classification result is output in the form of an image (S220).

At this time, the encoder may be configured with a plurality of network structures to extract global features to search for structures formed throughout the construction site image 50.

Referring to Figure 5, the encoder 110 performs convolution from the construction site image 50 to extract the first feature map 52 and the first network 112 and the first feature map 52 at a plurality of ratios. It includes a second network 114 that extracts a group of multiple feature maps 54 by calculating convolution between pixels according to parameters and extracts a second feature map 56 that merges the multiple feature maps 54. It can be configured as follows.

That is, the first network 112 can generate a compressed image by emphasizing features of the input construction site image 50 through internal convolution layers. The first network 112 generates a first feature map 52 in which the features of each structure are emphasized as an intermediate output, but generates the first feature map 52 so that features can be emphasized while partially preserving the location information of the objects. Pass it to an additional convolutional network.

Subsequently, the second network 114 may generate the second feature map 56 using the intermediate output.

The second network 114 also performs a convolution operation between pixels in the first feature map 52 through a convolution operation, but in this embodiment, a convolution operation is performed according to various filters in parallel with respect to the first feature map 52. Multiple feature maps can be obtained simultaneously by performing a solution operation.

Additionally, the filter at this time may have a specified interval as a ratio parameter in order to extract more global features from a limited number of parameters.

Specifically, referring to FIG. 6, a convolution operation can be performed on an area wider than the number of parameters according to the determined ratio parameter (R).

For example, if R = 1, a convolution operation is performed on an area of the same size with 9 parameters in a 3x3 filter with no gap between parameters in the filter, while if R = 2, the parameters in the filter are A convolution operation can be performed with an interval of 1 pixel, and therefore features for 5x5 can be extracted over a wider area.

In other words, the receptive field of the pixels after the convolution operation can be increased by adjusting the spacing between parameters without changing the number of parameters, and through this, feature maps that observe more global features can be output through a more simplified operation. You can.

In addition, the filters for the convolution operation of the first network 112 can also perform a convolution operation according to the rate parameter that adjusts the interval as described above.

Next, the second network performs a process of merging multiple feature maps generated through parallel operations according to various ratio parameters into one. Furthermore, in order to adjust the size of the deepened dimension after merging, a layer that performs 1x1 convolution can be additionally configured and the size of the feature map for input to the decoder 120 can be adjusted.

The decoder 120 may generate a classification result 60 for each pixel of the original size construction site image 50 by upsampling the merged second feature map 56 of the second network 120. However, in order to determine the progress of a construction site, more accurate settings of the boundaries of structures are necessary, and therefore the decoder 120 can use additional information in addition to the second special map 56 for upsampling.

That is, in this embodiment, decoding can be performed using the intermediate feature map 53 of the same size during the upsampling process of the second feature map as the output of the first network 1120 of the encoder 110.

Referring to FIG. 5, the decoder 120 uses the intermediate feature map 53, which is the output of the first network, along with the second feature map 56, which is the intermediate output of the second network 120, as a variable for upsampling. Take advantage of it.

At this time, the intermediate feature map 53 can be used for restoration by using a feature map of the same size as the feature map in the upsampling process using the intermediate output in addition to the final output of the first network 110.

Finally, the decoder 120 merges the intermediate feature map and a plurality of feature maps through parallel operation, then upsamples the third feature map 62 obtained by merging the dimensionally reduced second feature map and outputs classification results for each pixel. .

Referring to FIGS. 7 to 11 , for example, as shown in FIG. 7 , an output 60 may be generated from the construction site image 50 by classifying the area containing vertical reinforcing bars into pixel units 60a.

FIG. 8 is an output image 60 from which pixels 60b including indentations in the construction site image 50 are extracted. Figure 9 illustrates the classification result (60c) of pixels containing external wall formwork, and Figure 10 illustrates the result (60d) of classifying areas containing floor reinforcement into pixels.

In the case of Figure 11, pixels 60e including wall formwork can be output.

At this time, the construction site classification model 100 according to this embodiment outputs classification results for structures in pixel units as shown, and the overall output result is determined pixel by pixel, but the classification results are output to have a continuous form. It can be.

For example, in the case of the floor rebar in FIG. 10, instead of accurately classifying the checkerboard-shaped area of the rebar in the actual construction site, it is learned to output the continuous shape of the floor surface where the rebar is wired as the classification result.

Through this, classification results that are not affected by occlusion by other objects in the image are output when identifying the progress status of the entire construction site.

That is, the construction site classification model 100 may perform learning by configuring learning data to enable output in area units that can determine the progress of the construction site instead of the classification results of actual pixels during the learning process.

Referring to FIG. 12, the training data used in this embodiment directly uses an image 70 in which pixels containing structures within the image are labeled as continuous areas for learning.

In other words, calculate the difference in pixel classification results between the output result 60 of the model 100 for the construction site image 50 and the pixel of the labeled image 70, and backpropagate the error in the direction of minimizing the difference. By doing so, parameters within the encoder and decoder can be learned.

Through the above process, the construction site classification model 100 can output classification results in the form of a continuous area according to the structure.

Next, the method for determining construction progress according to this embodiment calculates construction progress through the ratio of pixels for each classification (S300).

In other words, the number of pixels occupied by the structures is calculated as a ratio compared to the size of all pixels, and the progress of construction is judged through changes in the ratio. Therefore, in this embodiment, the change in ratio can be calculated by using the construction site image 50 acquired from a camera with the same angle of view in time series.

At this time, the adequacy of construction progress according to the elapsed period can be determined by comparing the interval between image acquisition times and whether construction progress is progressing.

Furthermore, in the case of the construction site image 50 in this embodiment, it is also possible to make a more accurate judgment of progress through judgment of the three-dimensional area.

Referring to FIG. 13, it is also possible to judge the construction progress by three-dimensionally reconstructing the same construction site using site images 50a and 50b taken from different directions.

For example, in the case of the first camera, the progress of the vertical rebar in the height direction can be well judged by taking pictures from a side angle, and in the case of the second camera, the width of the bottom of the vertical rebar can be well judged by taking pictures of the floor surface.

Through this, it is possible to judge construction progress more accurately by considering each position of the pixel classified as a structure in the output results (60-1, 2) and calculating it compared to the entire three-dimensional space.

Furthermore, in this embodiment, in addition to outputting the classification results for the entire image, the construction site classification model 100 can also output the results of searching each unit area for the image on the large screen using a sliding window.

Referring to FIG. 13, it is also possible to output a classification result (61) of a unit area (51) within a sliding window of a designated size for the construction site image (50) and determine the overall progress by integrating the classification results. Furthermore, in the case of a sliding window, it is possible to allow some overlap and perform more accurate boundary setting of structures by classifying pixels through the average of the overlapping areas.

Hereinafter, a detailed hardware implementation of the server 1000 that performs the construction site progress determination method will be described with reference to FIG. 14.

Referring to FIG. 14, in some embodiments of the present invention, the server 1000 may be implemented in the form of a computing device. One or more of the modules constituting the Saba 1000 are implemented on a general-purpose computing processor and thus include a processor 1010, an input/output I/O 1020, a memory 1030, and an interface. It may include (1040), storage (1050), and bus (1060). The processor 1010, input/output I/O 1020, memory device 1030, and/or interface 1040 may be coupled to each other through a bus 1060. The bus 1060 corresponds to a path through which data moves.

Specifically, the processor 1010 includes a Central Processing Unit (CPU), Micro Processor Unit (MPU), Micro Controller Unit (MCU), Graphic Processing Unit (GPU), microprocessor, digital signal processor, microcontroller, and application processor (AP). , application processor) and logic elements capable of performing similar functions.

The input/output I/O device 1020 may include at least one of a keypad, keyboard, touch screen, and display device. The memory device 206 may store data and/or programs.

The interface 1040 may perform a function of transmitting data to or receiving data from a communication network. Interface 1040 may be wired or wireless. For example, the interface 1040 may include an antenna or a wired or wireless transceiver as a communication unit. Although not shown, the memory device 1030 is an operating memory for improving the operation of the processor 1010 and may further include high-speed DRAM and/or SRAM.

Internal storage 1050 stores programming and data configuration that provides the functionality of some or all modules described herein. For example, it may include logic to perform selected aspects of the fraud detection method described above.

The memory device 1030 loads a program or application with a set of instructions including each step of performing the above-described fraud detection method stored in the storage 1050 and allows the processor to perform each step.

As above, according to the present invention, objective indicators can be calculated through a learned neural network and construction progress can be judged more conveniently.

In addition, according to the present invention, errors due to time differences in on-site verification can be reduced by simultaneously determining the progress status from all images taken of the construction site.

Additionally, the present invention can provide users with intuitive decision information by providing images of construction progress.

Additionally, by using images from various angles, more accurate progress can be determined efficiently.

Furthermore, various embodiments described herein may be implemented in a recording medium readable by a computer or similar device, for example, using software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions. In some cases, as described herein, The described embodiments may be implemented as a control module itself.

According to software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. Software code can be implemented as a software application written in an appropriate programming language. The software code may be stored in a memory module and executed by a control module.

The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications, changes, and substitutions can be made by those skilled in the art without departing from the essential characteristics of the present invention. will be.

Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments and the attached drawings. . The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

Claims (16)

In the deep learning-based construction progress judgment method performed on the server,
Step of receiving construction site images;
Performing pixel-level classification on the construction site image using a learned construction site classification model; and
A method for determining construction progress, comprising the step of calculating construction progress through the ratio of pixels for each classification. According to claim 1,
The above classification is a construction progress judgment method characterized in that it includes vertical reinforcing bars or floor reinforcing bars within the construction site. According to claim 1,
The above classification is a construction progress judgment method characterized in that it includes food in the construction site. According to claim 1,
The above classification is a construction progress judgment method characterized in that it includes exterior wall formwork or wall formwork within the construction site. According to claim 1,
The construction site classification model includes an encoder that extracts a feature map by performing a convolution operation between pixels in the input construction site image, and
A method for determining construction progress, comprising a decoder that upsamples the extracted feature map and outputs a classification result for each pixel. According to claim 5,
The encoder is a first network that extracts a first feature map by performing convolution from the construction site image, and
Characterized by comprising a second network that extracts a plurality of feature maps by calculating inter-pixel convolution of the first feature map according to a plurality of ratio parameters, and extracts a second feature map by merging the plurality of feature maps. How to judge construction progress. According to claim 6,
The decoder is a method for determining construction progress, characterized in that the output of the first network and the third feature map obtained by reducing and merging the dimensions of the second feature map are upsampled and output a classification result for each pixel. According to claim 5,
A method for determining construction progress, characterized by dividing the construction site image into a plurality of partial images and performing pixel-level classification for each divided partial image. According to claim 1,
The step of calculating the construction progress is a fraud detection method characterized in that the progress is calculated using the progress and elapsed period of the previous construction site image for the construction site image input in time series. According to claim 1,
The construction site image is an image taken from multiple angles,
The step of calculating the construction progress is a fraud detection method characterized in that the progress is calculated using the ratio of pixels of the construction site image to the plurality of angles. a communications department that receives construction site images;
One or more processors that determine construction progress from the construction site image; and
Includes an input/output unit,
Fraud One or more processors,
Receive images of the construction site as input,
Perform pixel-level classification on the construction site image using the learned construction site classification model, and
A construction progress judgment device that calculates construction progress through the ratio of pixels for each category. According to claim 11,
The construction site classification model includes an encoder that extracts a feature map by performing a convolution operation between pixels in the input construction site image, and
A construction progress determination device comprising a decoder that upsamples the extracted feature map and outputs a classification result for each pixel. According to claim 12,
The encoder is a first network that extracts a first feature map by performing convolution from the construction site image, and
Characterized by comprising a second network that extracts a plurality of feature maps by calculating inter-pixel convolution of the first feature map according to a plurality of ratio parameters, and extracts a second feature map by merging the plurality of feature maps. A construction progress judgment device. According to claim 13,
The decoder is a construction progress judgment device characterized in that the output of the first network and the third feature map obtained by reducing and merging the dimensions of the second feature map are upsampled to output a classification result for each pixel. A computer-readable recording medium storing a program for performing the method for detecting fraud according to any one of claims 1 to 10. A program stored in a computer-readable recording medium that performs the method for detecting fraud according to any one of claims 1 to 10.

Images