KR20230015834A - System and method for ground surface information in construction sites - Google Patents

Abstract

Disclosed are a system and method for providing ground surface information in a construction site. The system for providing ground surface information in a construction site, according to an embodiment of the present invention, comprises: an unmanned aerial vehicle for capturing an on-site image from the air in a construction site; and a ground surface classification apparatus for classifying ground surface information by using the on-site image captured by the unmanned aerial vehicle. The ground surface classification apparatus comprises: an image input unit for receiving an input of the on-site image from the unmanned aerial vehicle; a grid division unit for dividing the input on-site image into a grid of a preset site; a classification unit for classifying the divided grid into a ground surface and a non-ground surface and extracting an object in the grid classified as the non-ground surface; a class defining unit for defining a class of the grid according to an operation result of the classification unit; a class information generation unit for generating class information according to the defined class; and a user interface for providing the generated class information to a user. According to the present invention, the efficiency of construction progress can be improved.

Description

Construction site ground surface information provision system and method {SYSTEM AND METHOD FOR GROUND SURFACE INFORMATION IN CONSTRUCTION SITES}

The present invention relates to a system and method for providing ground surface information at a construction site, and more particularly, to a system and method for providing ground surface information at a construction site that automatically provides ground surface information requiring management from field images captured by an unmanned aerial vehicle. it's about

In order to successfully carry out a construction project, it is very important to quickly check and manage surface information. More specifically, factors that have a harmful effect during construction, such as vegetation, puddles, and large rocks, must be identified in advance, removed, and managed.

In this way, in order for the construction to proceed smoothly, it is very important to secure ground surface information before starting construction. Currently, the manager directly tours the site and checks the surface condition with the naked eye.

In a relatively small construction site, it may not be a big problem for a manager to tour the site to check the surface condition, but in a large construction site, it is not easy for a manager to tour the entire site to obtain surface information. no.

In particular, due to the nature of road construction sites distributed over a wide area, there is a problem in that it takes a lot of time and money for a manager to check surface information while touring the entire site.

Korean Registered Patent No. 10-1884920 (published on July 27, 2018)

In order to solve the above problems, the technical problem to be achieved by the present invention is to divide field images taken by unmanned aerial vehicles into grids, extract objects on the ground surface that need to be managed in grid units, and provide information to improve construction progress. The purpose of this study is to propose a system and method for providing ground surface information at a construction site so that influencing factors can be identified quickly and accurately.

The problems of the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

As a means for solving the above-described technical problem, the system for providing ground surface information of a construction site according to an embodiment of the present invention uses an unmanned aerial vehicle for photographing field images from the air at a construction site, and a field image captured by an unmanned aerial vehicle. and a ground surface classification device for classifying ground surface information, wherein the ground surface classification device includes an image input unit for receiving the field image from an unmanned aerial vehicle, a grid partitioning unit for partitioning the input field image into a grid of a predetermined size, and a partitioned grid Classification unit that classifies objects into either ground surface or non-terrestrial surface and extracts objects from the grid classified as non-surface, class definition unit that defines the class of the grid according to the operation result of the classification unit, and class information according to the defined class. It includes a class information generation unit that generates class information, and a user interface unit that provides the generated class information to a user.

Preferably, the grid partitioning unit may partition the field image into a 5m×5m grid.

Preferably, the classification unit may include a ground surface classification unit for classifying grids into land surfaces and non-land surfaces, and an object classification unit for classifying objects within grids classified as non-land surfaces.

Also preferably, the ground surface classification unit is composed of a ResNet-101 network operating by the first data set that has been scaled, and the object classifier is ResNet-101 that operates by the second data set that has been scaled. can be made into a network.

Also preferably, the class information generator may generate class information including defined classes and location information of grids including the classes.

Meanwhile, a method for providing ground surface information of a construction site according to another embodiment of the present invention includes the steps of receiving an input image of a construction site taken from the air from an unmanned aerial vehicle, and dividing the received image of the site into a grid having a preset size. , classifying the partitioned grid into one of ground surface and non-terrestrial surface, extracting objects from the grid classified as non-terrestrial surface, defining the class of the grid according to the classification and extraction result, according to the defined class Generating class information, and providing the generated class information to a user.

Preferably, in the partitioning step, the field image may be partitioned into a 5m×5m grid.

Also preferably, the classifying step is performed by a ResNet-101 network operated by the first previously contracted dataset, and the extracting step is performed by the ResNet-101 network operated by the second previously built-up dataset. It can be configured as a Net-101 network.

Also preferably, the generating step may generate class information including defined classes and location information of grids including the classes.

According to the present invention, the efficiency of construction progress is improved and the time and effort due to manual work is improved by automatically identifying the factors affecting the construction on the ground surface using field images taken by unmanned aerial vehicles in a very wide construction site. There is an effect of providing a system and method for providing ground surface information at a construction site that can be reduced.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

The accompanying drawings are intended to explain the contents of the present invention in more detail to those skilled in the art, but the technical spirit of the present invention is not limited thereto.
1 is a network configuration diagram of a system for providing ground surface information at a construction site according to a preferred embodiment of the present invention;
2 is a block diagram of a land surface classifier according to a preferred embodiment of the present invention;
3 is a view for explaining the operation of the classification unit of the ground surface classification device according to a preferred embodiment of the present invention;
4 is a diagram for explaining grids and classes generated during the operation of the ground surface classifier according to a preferred embodiment of the present invention; and
5 is a flowchart illustrating a method of providing ground surface information of a construction site according to a preferred embodiment of the present invention.

The above objects, other objects, features and advantages of the present invention will be easily understood through the following preferred embodiments in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosed content will be thorough and complete and the spirit of the present invention will be sufficiently conveyed to those skilled in the art.

In this specification, when an element is referred to as being on another element, it means that it may be directly formed on the other element or a third element may be interposed therebetween. Also, in the drawings, the thickness of components is exaggerated for effective description of technical content.

In this specification, when terms such as first and second are used to describe components, these components should not be limited by these terms. These terms are only used to distinguish one component from another. Embodiments described and illustrated herein also include complementary embodiments thereof.

Also, when it is said that a first element (or component) operates or is executed on (ON) a second element (or component), the first element (or component) means that the second element (or component) It should be understood that it is operated or executed in an environment in which it is operated or executed, or operated or executed through direct or indirect interaction with the second element (or component).

Where an element, component, device, or system is referred to as including a component consisting of a program or software, even if not explicitly stated otherwise, that element, component, device, or system means that the program or software executes or operates. It should be understood that it includes hardware (eg, memory, CPU, etc.) or other programs or software (eg, operating system or driver required to drive hardware) required to do so.

In addition, it should be understood that, unless otherwise specified, the element (or component) may be implemented in any form of software, hardware, or both software and hardware.

In addition, terms used in this specification are for describing embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. The terms 'comprises' and/or 'comprising' used in the specification do not exclude the presence or addition of one or more other elements.

1 is a network configuration diagram of a system for providing ground surface information at a construction site according to a preferred embodiment of the present invention.

Referring to FIG. 1 , a system for providing ground surface information at a construction site according to a preferred embodiment of the present invention includes an unmanned aerial vehicle 100 capable of communicating with each other and a ground surface classification device 200 .

An unmanned aerial vehicle (UAV) (100) is a collective term for an air vehicle that flies automatically or semi-automatically according to a pre-programmed route by radio control from the ground without a pilot on board. It is called.

In the ground surface information providing system of the present construction site, the unmanned aerial vehicle 100 captures a field image from the air of the construction site and transmits the captured field image to the ground surface classification device 200. The unmanned aerial vehicle 100 has high mobility and is particularly suitable for taking on-site images of road construction sites distributed over a wide area.

The ground surface classification device 200 classifies ground surface information using field images captured by the unmanned aerial vehicle 100, generates the classified result in the form of information considering user convenience, and provides it to the user (or site manager). . The ground surface classification device 200 will be described in more detail in FIG. 2 to be described later.

2 is a block diagram of a land surface classifier according to a preferred embodiment of the present invention.

Referring to FIG. 2 , the land surface classification apparatus 200 according to a preferred embodiment of the present invention includes an image input unit 210, a grid partition unit 220, a classification unit 230, a class definition unit 240, and class information generation. It includes a unit 250, a user interface unit 260, a storage unit 270, and a control unit 280.

The image input unit 210 receives a field image captured by the unmanned aerial vehicle 100 through network communication with the unmanned aerial vehicle 100 . To this end, the image input unit 210 supports a network interface between the ground surface classification device 200 and the unmanned aerial vehicle 100. You can communicate through communication.

The grid division unit 220 divides the field image input through the image input unit 210 into a plurality of grids having a preset size. By partitioning a grid in an image in which various objects are mixed, the number of object types (objects) included in one grid, which is an operation unit of the classification unit 230 to be described later, can be reduced. This is very effective in improving the performance of classification models. The grid partitioned by the grid dividing unit 220 will be described in more detail in FIG. 4 to be described later.

The classification unit 230 firstly classifies the grid partitioned by the grid partitioning unit 220 into one of a land surface and a non-terrestrial surface, and secondarily extracts an object from the grid only for the grid classified as a non-terrestrial surface. Here, the object may include a puddle, a rock, vegetation, and a protective net.

The classification unit 230 includes a ground surface classification unit 232 and an object classification unit 234 . The ground surface classification unit 232 classifies each grid into one of a ground surface and a non-terrestrial surface through each grid image. In addition, the object classification unit 234 extracts objects from images within the grid only for grids classified as non-terrestrial surfaces by the ground surface classification unit 232 . The operation of the classification unit 230 will be described in detail with reference to FIG. 3D to be described later.

The class definition unit 240 defines a class in each grid according to the operation result of the classification unit 230 . A class is defined as a ground surface class for a grid classified as a ground surface, and a class is defined as an object name extracted from each grid for a grid classified as a non-ground surface.

The class information generation unit 250 generates class information for the purpose of providing to the user. Here, the class information may include a class defined for each grid and location information of a grid including the class.

The user interface unit 260 supports an interface between a user and the ground surface classification device 200, and can provide the user with the class information generated by the class information generation unit 250, and provide predetermined information by user manipulation. can be input.

The storage unit 270 stores all information necessary for the operation of the ground surface classification device 200. For example, the storage unit 270 may store field images transmitted from the unmanned aerial vehicle 100 and may store class information generated by the class information generation unit 250 . Also, the storage unit 270 may store data sets necessary for the operation of the classification unit 230, that is, a first data set and a second data set.

The control unit 280 controls the overall operation of the land surface classification device 200. That is, the control unit 280 includes the image input unit 210, the grid division unit 220, the classification unit 230, the class definition unit 240, the class information generation unit 250, the user interface unit 260, and the storage Signal input/output between units 270 is controlled.

3 is a diagram for explaining the operation of the classification unit of the land surface classification apparatus according to a preferred embodiment of the present invention.

The classification unit 230 analyzes each of the cropped images divided into a plurality of grids and classifies them into ground surfaces and non-terrestrial surfaces. The classification unit 230 firstly classifies grids into ground surfaces and non-terrestrial surfaces, and secondarily extracts objects only for grids classified as non-terrestrial surfaces. To this end, the classification unit 230 includes a ground surface classification unit 232 and an object classification unit 234 .

The ground surface classification unit 232 may classify the grid into a land surface and a non-terrestrial surface through the ResNet-101 network operated by the first data set (DataSet) of the instrument axis. The criterion for classifying the ground surface and non-terrestrial surface in the ground surface classification unit 232 is to classify the grid as a ground surface when the result of analyzing the image of the grid consists only of soil and stones of 60 cm or less, and classify the grid as a non-terrestrial surface otherwise. .

The object classification unit 234 extracts an object from an image in a grid classified as a non-terrestrial surface by the ground surface classification unit 232 through the ResNet-101 network operated by the second datanet. Here, the objects may include vegetation, rocks, puddles, and protection nets. The criteria for defining objects in the object classification unit 234 are shown in Table 1 below.

object Justice Vegetation Includes vegetation such as trees, grass, and bushes rock Contains rocks over 60 cm in diameter pool Contains the water standing on the surface of the earth protective net Including multiple protection nets such as anti-fall nets and slope protectors

As a result, the grid-divided images are firstly divided into land surfaces or non-terrestrial surfaces by the ground surface classification unit 232, and secondarily divided into ground surface, vegetation, rocks, puddles, and separated by a protective net.

As described above, the ground surface classification unit 232 and the object classification unit 234 are composed of the Resnet-101 network operated by the first and second data sets, respectively. The ResNet-101 network is a 101-layer convolutional neural network that can load a pretrained version of a neural network trained on over 1 million images from the ImageNet database.

In this embodiment, by designing the deep learning model architecture using the LesNet-101 network, the classification model is continuously trained based on the previously built-up training dataset, so that classification is performed whenever the classification unit 230 operates. The function of unit 230 can be continuously improved.

Therefore, in order to efficiently classify grid-based images in the classifier 230, it is necessary to construct a first dataset and a second dataset for learning the Lesnet-101 network.

To this end, the first and second datasets were constructed by taking field images of actual construction sites. The result of confirming the performance evaluation of the classification model according to this is shown in Table 2.

Classification class accuracy
(Precision) recall
(Recall) F1 Score ground surface ground surface 0.99 0.97 0.98 non-terrestrial surface 0.97 0.99 0.98 non-terrestrial surface pool 0.82 0.27 0.40 Vegetation 0.94 0.89 0.91 rock 0.82 0.96 0.89 protective net 0.89 0.73 0.80

Referring to Table 2, as a result of evaluating the performance of the classification model by constructing a dataset based on actual field images, it can be seen that most classes show compliant performance. However, in the case of the puddle class, the shape, form, and color are very diverse, and the performance of the classification model is somewhat degraded due to characteristics such as light reflection of water. can

4 is a diagram for explaining grids and classes generated during the operation of the ground surface classifier according to a preferred embodiment of the present invention.

When a field image is input from the unmanned aerial vehicle 100, the field image input by the grid partitioning unit 220 is partitioned into a grid having a predetermined size. The grid partitioning unit 220 may partition the grid in an n×n size, where n may be 5 m. That is, the grid dividing unit 220 may partition the grid in a square shape. In addition, the size of the grid partitioned by the grid partitioning unit 220 may be changed in consideration of the type of site, the area of the site, and the like.

When the grid division of field images is completed by the grid division unit 220, the images are input to the classification unit 230 in grid units. The classification unit 230 includes a ground surface classification unit 232 and an object classification unit 234. The ground surface classification unit 232 operates first, and the object classification unit (using the operation result of the ground surface classification unit 232) 234) works.

The ground surface classification unit 232 classifies each of the images divided into grid units as shown in A into a ground surface or a non-terrestrial surface by the Resnet-101 network operation based on the first dataset.

Thereafter, the object classification unit 234 classifies each of the grids classified as non-surfaces in A into objects by the Resnet-101 network operation operated by the second dataset, and classifies them into puddles, vegetation, and rocks as in B. It is classified as one of , , and protective nets.

Thereafter, the class definition unit 240 defines a class for each grid using the operation results of the ground surface classification unit 232 and the object classification unit 234. Here, class definition corresponds to an operation of performing data labeling. Each grid is defined by the class definition unit 240 as one of ground surface, puddle, vegetation, rock, and protection net. The result finally output from the classification unit 230 corresponds to B.

5 is a flowchart illustrating a method of providing ground surface information of a construction site according to a preferred embodiment of the present invention.

Here, with reference to FIGS. 1 to 5, a method for providing ground surface information of a construction site according to a preferred embodiment of the present invention will be described.

The unmanned aerial vehicle 100 takes a field image while flying within a preset flight range. The field image captured by the unmanned aerial vehicle 100 is transmitted to the ground surface classification device 200 through a predetermined network, and the field image captured by the unmanned air vehicle 100 is transmitted through the image input unit 210 in the ground surface classification device 200. is input (S300).

The field image received through the image input unit 210 is partitioned into a grid having a preset size by the grid partitioning unit 220 (S310). Grids may vary in size and number depending on site conditions and types of sites.

When the grid is partitioned by the grid partitioning unit 220 , the grid unit image is input to the classifying unit 230 . The ground surface classifying unit 232 classifies each grid into a land surface or a non-terrestrial surface (S320), and the object classifying unit 234 extracts an object from a grid classified as a non-terrestrial surface among the grids (S330).

When object extraction is completed for all grids, the class definition unit 240 defines a class for each grid (S340), and the class information generation unit 250 determines the class according to the class generated by the class definition unit 240. Class information is generated (S350). Class information is information generated for the purpose of providing to users, and may include classes defined in each grid and location information of grids including the classes.

The class information generated by the class information generation unit 250 is provided to the user through the user interface unit 260 (S360).

By this operation, when a field image taken through an unmanned aerial vehicle 100 such as a drone is taken at a wide construction site, in particular, a road construction site that is widely distributed, the ground surface classification device 200 divides the grid, and the grid Each object is extracted and class information is provided to the user. As a result, the user can easily grasp the types and positions of objects that have a harmful effect on the construction.

Those skilled in the art to which the present invention pertains will understand that the present invention may be embodied in other specific forms without changing its technical spirit or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

100: unmanned aerial vehicle 200: ground classification device
210: video input unit 220: grid division unit
230: classification unit 232: ground surface classification unit
234: object classification unit 240: class definition unit
250: class information generation unit 260: user interface unit
270: storage unit 270: control unit

Claims (9)

An unmanned aerial vehicle that takes on-site images from the air at a construction site;
A ground surface classification device for classifying ground surface information using the field image captured by the unmanned aerial vehicle; includes,
The ground surface classification device,
an image input unit receiving the field image from the unmanned aerial vehicle;
a grid division unit dividing the received field image into a grid having a preset size;
a classification unit that classifies the partitioned grid into one of a ground surface and a non-terrestrial surface, and extracts an object from the grid classified as a non-terrestrial surface;
a class definition unit defining a class of the grid according to an operation result of the classification unit;
a class information generating unit generating class information according to the defined class; and
A system for providing ground surface information at a construction site, comprising: a user interface unit for providing the generated class information to a user.
According to claim 1,
The grid partitioning unit divides the field image into a 5m×5m grid.
According to claim 1,
The classification unit,
a ground surface classification unit that classifies the grid into the land surface and the non-terrestrial surface; and
A ground surface information providing system for a construction site comprising: an object classification unit for classifying objects within the grid classified as the non-surface.
According to claim 1,
The ground surface classification unit is composed of a ResNet-101 network operated by a first data set that has been scaled,
The object classification unit is composed of a Resnet-101 network operated by the second data set that has been built-up.
According to claim 1,
The system for providing ground surface information at a construction site, characterized in that the class information generating unit generates class information including the defined class and location information of a grid including the class.
Receiving an input field image taken from an aerial view of a construction site from an unmanned aerial vehicle;
Dividing the input field image into a grid having a preset size;
Classifying the partitioned grid into one of a ground surface and a non-terrestrial surface:
extracting an object from the grid classified as the non-surface;
defining a class of the grid according to a result of the classification and extraction;
generating class information according to the defined class; and
A method for providing ground surface information at a construction site, comprising: providing the generated class information to a user.
According to claim 6,
In the step of partitioning, the ground surface information providing method of the construction site, characterized in that for partitioning the site image into a grid of 5m × 5m.
According to claim 6,
The classification step is operated by the ResNet-101 network operated by the first data set that has been built up,
The method of providing ground surface information at a construction site, characterized in that the extracting step is composed of a ResNet-101 network operated by the second data set that has been built-up.
According to claim 6,
The generating step comprises generating class information including the defined class and location information of a grid including the class.

Images