KR102252845B1 - Method, device and system for analyzing concrete surface through machine vision and 3d profile - Google Patents

Abstract

The present invention relates to a method, device, and system for analyzing a concrete surface through a machine vision and a three-dimensional profile. In accordance with an embodiment of the present invention, the method for analyzing the concrete surface through the machine vision and the three-dimensional profile, which is executed by the system, comprises: a step of acquiring three-dimensional data on the concrete surface through lidar, and acquiring two-dimensional data on the concrete surface through a camera; a step of separating a union area of the two-dimensional data and the three-dimensional data, and extracting a first data integrating the two-dimensional data and the three-dimensional data; a step of encoding the first data and generating a first input signal; a step of inputting the first input signal into a first artificial neural network, and acquiring a first output signal based on the result of the input into the first artificial neural network; and a step of generating a first classification result on the concrete surface based on the first output signal. The present invention aims to provide a method, device, and system for analyzing a concrete surface through a machine vision and a three-dimensional profile, which are able to effectively improve the performance in analyzing the concrete surface.

Description

Concrete surface analysis method, device and system through machine vision and 3D profile {METHOD, DEVICE AND SYSTEM FOR ANALYZING CONCRETE SURFACE THROUGH MACHINE VISION AND 3D PROFILE}

The following examples relate to a technique for analyzing a concrete surface.

Damaged concrete structures must be repaired and reinforced appropriately according to the damage information to restore their original functions, and repairs and reinforcements are required in terms of preventive maintenance to enhance the original performance of the concrete structure.

Accordingly, researches on the repair, reinforcement method, and material of concrete structures are being conducted steadily, but there are many problems as the technology for pretreatment of the repair and reinforcement method has not been accumulated.

Until now, rather than systematic technology development, technical analysis of raw materials and repair materials has been conducted, but technology development for pre-construction treatment is required throughout maintenance and repair work.

In the case of repair and reinforcement methods for repairing and reinforcing concrete cross-sectional structures, it is necessary to remove surface concrete with deteriorated durability, and by causing cracks in the surrounding concrete where performance is not unintentionally degraded due to the impact that appears during this process, It may cause damage and cause a problem of re-falling out of the area even after repair.

Accordingly, there is an increasing need for research on the pretreatment process throughout the entire concrete repair and reinforcement process, and there is an increasing demand for a method to check the presence of cracks and even quality control in the pretreatment process.

According to an embodiment, the purpose of the present invention is to provide a method of extracting first data in which 2D data and 3D data are merged through an apparatus, and generating a first classification result for a concrete surface using the first data. do.

In addition, according to an embodiment, providing a method for generating a second classification result for a concrete surface by using the second data by extracting second data from which the damaged area is deleted from the first data through a server. It is for that purpose.

In addition, according to an embodiment, it is an object of the present invention to provide a method of setting a final classification result for a concrete surface by using the first classification result and the second classification result.

The object of the present invention is not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood from the following description.

According to an embodiment, in a method of analyzing a concrete surface through a machine vision and a 3D profile performed by a system, 3D data on the concrete surface is obtained through a lidar, and the concrete surface is obtained through a camera. Obtaining 2D data; Separating the union region of the 2D data and the 3D data, and extracting first data obtained by merging the 2D data and the 3D data; Encoding the first data to generate a first input signal; Inputting the first input signal to a first artificial neural network in a device, and obtaining a first output signal based on a result of the input of the first artificial neural network; And generating a first classification result for the concrete surface based on the first output signal. A method for analyzing a concrete surface through machine vision and a 3D profile is provided.

The method for analyzing a concrete surface through the machine vision and 3D profile includes: transmitting the first data and the first classification result to a server; Analyzing the first data transmitted to the server to detect cracks occurring on the concrete surface; Identifying a crack generated on the concrete surface by region, and dividing a normal region in which a crack is detected below a first reference value and a damaged region in which a crack is detected above the first reference value; Extracting second data from which the damaged area is deleted from the first data; Encoding the second data to generate a second input signal; Inputting the second input signal to a second artificial neural network in the server, and obtaining a second output signal based on a result of the input of the second artificial neural network; Generating a second classification result for the concrete surface based on the second output signal; And when the first classification result and the second classification result are the same, setting one of the first classification result and the second classification result as a final classification result for the concrete surface.

The method of analyzing the concrete surface through the machine vision and the 3D profile includes: analyzing the concrete surface by performing image processing and pattern recognition on the second data according to the final classification result; Obtaining a result of the operator's judgment on the final classification result; If the determination result is collected above a second reference value, transmitting the collected determination results to a cloud connected to the server; Processing the second artificial neural network to perform additional learning based on the determination results; And comparing the performance of the learning result through the second artificial neural network with the learning result through the first artificial neural network, and when the learning result through the second artificial neural network is improved by more than a third reference value, the second It may further include updating the deep learning model of the first artificial neural network through an artificial neural network.

The concrete surface analysis method through the machine vision and the 3D profile, when the first classification result and the second classification result are different, the third data registered as the representative image of the first classification and the representative of the second classification Obtaining fourth data registered as an image; Analyzing a first degree of agreement by comparing the first data and the third data, and analyzing a second degree of agreement by comparing the second data and the fourth data; Setting the first classification result as a final classification result for the concrete surface when it is determined that the first degree of agreement is greater than the second degree of agreement; And setting the second classification result as a final classification result for the concrete surface when it is determined that the second degree of agreement is greater than the first degree of agreement.

The step of dividing the normal region and the damaged region may include detecting a pressure greater than a preset pressure reference value by a crack recognition means for detecting the magnitude of the pressure applied to the concrete surface for each region, a pressure greater than the pressure reference value is detected. Acquiring pressure sensing data for coordinate values of sensors located in the designated area; And dividing a region at a position corresponding to the coordinate value into the normal region, wherein the obtaining of the pressure sensing data comprises: a first sensor that is any one of a plurality of sensors included in the crack recognition means In the case where a pressure greater than the pressure reference value is sensed and a pressure greater than the pressure reference value is sensed in all of the surrounding sensors in contact with the first sensor, pressure detection data for the coordinate value of the first sensor is transmitted to the first sensor. Obtaining from a sensor; And even if the first sensor detects a pressure greater than the pressure reference value, when at least one of the surrounding sensors in contact with the first sensor does not detect a pressure greater than the pressure reference value, the coordinate value of the first sensor. It may include controlling so that pressure sensing data for is not obtained.

According to an embodiment, in order to prepare an objectification standard of CSP (Concrete Surface Profile) for pretreatment of concrete maintenance, establish a classification standard for CSP, and effectively improve performance in analyzing the surface of concrete, cracking of concrete It has the effect of checking the presence or absence.

Meanwhile, the effects according to the embodiments are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those of ordinary skill in the relevant technical field from the following description.

1 is a diagram schematically showing the configuration of a system for analyzing a concrete surface according to an embodiment.
FIG. 2 is a diagram illustrating a classification criterion for a Concrete Surface Profile (CSP) according to an embodiment.
3 is a diagram illustrating a process of generating data on a concrete surface according to an exemplary embodiment.
4 is a view for explaining a process of analyzing a concrete surface corresponding to CSP 3 according to an embodiment.
5 is a view for explaining a process of analyzing a concrete surface corresponding to CSP 5 according to an embodiment.
6 is a view for explaining a process of analyzing a concrete surface corresponding to CSP 7 according to an embodiment.
7 is a diagram illustrating a process of learning data on a concrete surface according to an exemplary embodiment.
8 is a flowchart illustrating a process of generating a first analysis result according to an exemplary embodiment.
9 is a flowchart illustrating a process of generating a second analysis result according to an exemplary embodiment.
10 is a flowchart illustrating a process of performing additional learning on a second artificial neural network according to an embodiment.
11 is a flowchart illustrating a process of updating a deep learning model of a first artificial neural network according to an embodiment.
12 is a flowchart illustrating a process of setting a final classification result according to an embodiment.
13 is a diagram for describing a process of acquiring pressure sensing data for crack recognition according to an exemplary embodiment.
14 is a diagram illustrating an artificial neural network according to an embodiment.
15 is a diagram illustrating a method of learning an artificial neural network according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It is to be understood that all changes, equivalents, or substitutes to the embodiments are included in the scope of the rights.

Specific structural or functional descriptions of the embodiments are disclosed for the purpose of illustration only, and may be changed and implemented in various forms. Accordingly, the embodiments are not limited to a specific disclosure form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical idea.

Although terms such as first or second may be used to describe various components, these terms should be interpreted only for the purpose of distinguishing one component from other components. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

When a component is referred to as being "connected" to another component, it is to be understood that it may be directly connected or connected to the other component, but other components may exist in the middle.

The terms used in the examples are used for illustrative purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in the present application. Does not.

In addition, in the description with reference to the accompanying drawings, the same reference numerals are assigned to the same components regardless of the reference numerals, and redundant descriptions thereof will be omitted. In describing the embodiments, when it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter of the embodiment, a detailed description thereof will be omitted.

The embodiments may be implemented in various types of products such as a personal computer, a laptop computer, a tablet computer, a smart phone, a television, a smart home appliance, an intelligent vehicle, a kiosk, and a wearable device.

1 is a diagram schematically showing the configuration of a system for analyzing a concrete surface according to an embodiment.

Referring to FIG. 1, a system according to an embodiment may include a device 100 and a server 200 capable of communicating with each other through a communication network.

First, the communication network may be composed of wireless networks such as wireless LANs, CDMA, Bluetooth, and satellite communication, and may be implemented in various forms so that communication between a server and a server and communication between a server and a terminal are performed.

The device 100 may be implemented as a computing device having a communication function, for example, a mobile phone, a desktop PC, a laptop PC, a tablet PC, a smartphone, etc., but is not limited thereto, and an external server It may be implemented with various types of communication devices that can be connected with.

The apparatus 100 may analyze an image of the concrete surface, classify which classification the image belongs to, and obtain a first classification result for the concrete surface. Here, the first classification result may represent any one of 10 steps classified according to the classification criteria of CSP (Concrete Surface Profile). A detailed description of the CSP classification criteria will be described later with reference to FIG. 2.

The device 100 may include a communication unit 110, a lidar unit 120, a camera unit 130, a crack recognition unit 140, and a first processor 150, and may further include a memory 160 or separately It may be connected to the implemented memory 160.

The communication unit 110 may transmit and receive data through wireless communication with the server 200.

Specifically, the communication unit 110 may provide a transmission/reception signal between the device 100 and the server 200 in the form of packet data by interworking with a communication network, and transmits a data request to the server 200 and transmits a data request in response thereto. It can play a role of receiving.

The lidar unit 120 may generate 3D data on the concrete surface through a lidar sensor. Here, the 3D data is a 3D image of the concrete surface.

The camera unit 130 may generate 2D data on the concrete surface through camera photographing. Here, the 2D data is a 2D image of the concrete surface.

The crack recognition means 140 may include a plurality of pressure sensors, and may detect the magnitude of the pressure applied to the concrete for each area. A detailed description related to pressure sensing will be described later with reference to FIG. 13.

The first processor 150 is a kind of central processing unit and may control the entire process of obtaining the first classification result. The first processor 150 may perform a process of obtaining a first classification result by controlling the operation of the device 100. A detailed description of the process of obtaining the first classification result will be described later with reference to FIG. 8.

A program for performing a method of obtaining a first classification result may be recorded in the memory 160. In addition, a function of temporarily or permanently storing data processed by the first processor 150 may be performed.

According to an embodiment, the first processor 150 may execute a program and control the device 100. The code of a program executed by the first processor 150 may be stored in the memory 160.

The device 100 may be used to train a first artificial neural network to be described later or to use the learned first artificial neural network. The memory 160 may include a learning-in-progress or learned first artificial neural network. The first processor 150 may include artificial intelligence, and may learn or execute an algorithm of the first artificial neural network stored in the memory 160.

The server 200 may analyze the image of the concrete surface, classify which classification the image belongs to, and obtain a second classification result for the concrete surface. Here, the second classification result may represent any one of 10 steps classified according to the classification criteria of CSP (Concrete Surface Profile). A detailed description of the CSP classification criteria will be described later with reference to FIG. 2.

The server 200 may include a communication unit 210, a data buffer 220, a load balancer 230, and a second processor 240, and may further include a database 250 or a separately implemented database 250 and Can be connected.

The communication unit 210 may transmit and receive data through wired or wireless communication with the device 100.

Specifically, the communication unit 210 may provide a transmission/reception signal between the server 200 and the device 100 in the form of packet data by interworking with the communication network, and transmits a data request to the device 100 and transmits a data request in response thereto. It can play a role of receiving.

The data buffer 220 may serve to temporarily store data received from the communication unit 210.

The load balancer 230 may distribute data stored in the data buffer 220 through a load balancing algorithm.

The second processor 240 is a kind of central processing unit and may control the entire process of obtaining the second classification result. The second processor 240 may perform a process of obtaining a second classification result by controlling the operation of the server 200. A detailed description of the process of obtaining the second classification result will be described later with reference to FIG. 9.

In the database 250, a program for performing a method of obtaining a second classification result may be recorded. In addition, a function of temporarily or permanently storing data processed by the second processor 240 may be performed.

According to an embodiment, the second processor 240 may execute a program and control the server 200. The code of a program executed by the second processor 240 may be stored in the database 250.

The server 200 may be used to train a second artificial neural network to be described later or to use the learned second artificial neural network. The database 250 may include a second artificial neural network that is being trained or trained. The second processor 240 may include artificial intelligence, and may learn or execute an algorithm of the second artificial neural network stored in the database 250.

FIG. 2 is a diagram illustrating a classification criterion for a Concrete Surface Profile (CSP) according to an embodiment.

According to one embodiment, the concrete structure needs to be repaired and reinforced according to the degree of damage.When the repair and reinforcement is performed, the concrete surface can be scratched and repaved to some extent with a grinder, and this action is referred to as a maintenance pretreatment step. . At this time, the CSP (Concrete Surface Profile) can be calculated as a high dimension or a low dimension depending on how deeply the scratch is made in this pretreatment process.

CSP is a method of measuring the degree of scratching on the surface, and it is divided into 10 steps according to the surface condition, and can be classified from CSP 1 to 10. The CSP may have a very flat surface at a lower level, and a very rough surface at a higher level.

In the memory 160 of the device 100 and the database 250 of the server 200, a step-by-step image of the CSP shown in FIG. 2 may be stored.

Specifically, CSP 1 shown in (a) of FIG. 2 is a representative image showing an acid-etched state. CSP 2 shown in (b) of FIG. 2 is a representative image showing a grinding state. CSP 3 shown in (c) of FIG. 2 is a representative image showing a light shotblast state. CSP 4 shown in (d) of FIG. 2 is a representative image showing a light scarification state. CSP 5 shown in (e) of FIG. 2 is a representative image showing a medium shotblast state. CSP 6 shown in (f) of FIG. 2 is a representative image showing a medium scarification state. CSP 7 shown in (g) of FIG. 2 is a representative image showing a heavy abrasive blast state. CSP 8 shown in (h) of FIG. 2 is a representative image showing a scabbled state. CSP 9 shown in (i) of FIG. 2 is a representative image showing a heavy scarification-rotomilled state. CSP 10 shown in Figure 2 (j) is a representative image showing a handheld concrete breaker followed by abrasive blasting state.

The apparatus 100 may compare the analysis target data with the step-by-step image of the CSP stored in the memory 160 to obtain a first analysis result indicating which step the analysis target data corresponds to.

Likewise, the server 200 may obtain a second analysis result indicating which stage the analysis target data corresponds to by comparing the analysis target data with the step-by-step image of the CSP stored in the database 250.

According to an embodiment, the system may automate the inspection of the concrete surface, and may analyze the concrete surface by analyzing the possibility based on the pattern of the data.

The system may include a classifier using machine vision and 3D line-type laser profiling data, and may use pattern recognition technology using 3D point conversion sensor data based on artificial intelligence. Surface inspection can be performed through calibration.

The system can classify 3D displacement sensor data based on artificial intelligence. At this time, a CSV file is provided as the data of the 3D displacement sensor and can be expressed in a 2D format. Since the expressed chart forms a pattern of a specific distribution, it is possible to classify the surface through this.

The system can use a technology that predicts and classifies using a Linear Regressor model to classify the distribution pattern of sensor data.

The device 100 may include a laptop, amplifier, controller, laser sensor, and the like.

When the concrete is positioned at the bottom of the device 100, the concrete surface can be photographed through a lidar and a camera on the upper part of the concrete.

The device 100 may acquire data on the concrete surface.

The apparatus 100 may analyze data on the concrete surface and classify which classification the concrete surface corresponds to on a chart.

3 is a diagram illustrating a process of generating data on a concrete surface according to an exemplary embodiment.

As shown in (a) of FIG. 3, a lidar sensor is installed on the top of the device 100.

As shown in (b) of FIG. 3, the concrete to be analyzed may be disposed under the lidar sensor.

When the data on the concrete surface is acquired by the lidar sensor, as shown in FIG. 3(c), the lidar sensor may move to the side when the measurement is finished, providing a space in which concrete can be removed. .

According to an embodiment, the apparatus 100 may analyze data obtained from a lidar sensor and analyze which of the 10 steps the concrete surface corresponds to.

4 is a view for explaining a process of analyzing a concrete surface corresponding to CSP 3 according to an embodiment.

As shown on the left in (a) of FIG. 4, the device 100 may acquire a first image photographing a concrete surface through a camera, a lidar sensor, etc., and the acquired first image is shown in FIG. 4. As shown on the right in (a), it may be displayed on the screen of the device 100.

As shown in (b) of FIG. 4, the device 100 may analyze and organize characteristics of each column of sensor data measuring the concrete surface based on the first image.

As shown in (c) of FIG. 4, the device 100 extracts only a specific part of the sensor data measuring the concrete surface, and through this, it can be analyzed that the concrete surface corresponds to CSP 3.

5 is a view for explaining a process of analyzing a concrete surface corresponding to CSP 5 according to an embodiment.

As shown on the left in (a) of FIG. 5, the device 100 may acquire a second image photographing the concrete surface through a camera or a lidar sensor, and the acquired second image is shown in FIG. 5. As shown on the right in (a), it may be displayed on the screen of the device 100.

As shown in (b) of FIG. 5, the device 100 may analyze and organize characteristics of each column of sensor data measuring the concrete surface based on the second image.

As shown in (c) of FIG. 5, the device 100 extracts only a specific part of the sensor data measuring the concrete surface, and through this, it can be analyzed that the concrete surface corresponds to CSP 5.

6 is a view for explaining a process of analyzing a concrete surface corresponding to CSP 7 according to an embodiment.

As shown on the left in (a) of FIG. 6, the device 100 may acquire a third image photographing the concrete surface through a camera or a lidar sensor, and the acquired third image is shown in FIG. 6. As shown on the right in (a), it may be displayed on the screen of the device 100.

As shown in (b) of FIG. 6, the device 100 may analyze and organize characteristics of each column of sensor data measuring the concrete surface based on the third image.

As shown in (c) of FIG. 6, the device 100 extracts only a specific part of the sensor data measuring the concrete surface, and through this, it can be analyzed that the concrete surface corresponds to CSP 7.

In the process of measuring data to acquire each of the first image, the second image, and the third image, in the process of measuring the data, the distance between the road specimen and the sensor may be constant. The height can be constant.

Referring to FIGS. 4B, 5B and 6B, as a result of analyzing the internal structure of the data, the measured value -999.999 has a very different value from other measured values,- By judging the values of 999.999 as unnecessary data, you can proceed with analysis after removing all columns in which the data exists.

7 is a diagram illustrating a process of learning data on a concrete surface according to an exemplary embodiment.

According to an embodiment, the system may integrate and perform the process shown in FIG. 7, and the device 100 and the server 200 may separately perform the process shown in FIG. 7.

As shown in (a) of FIG. 7, the system may generate one input image by loading a plurality of images.

As shown in (b) of FIG. 7, the system may perform learning through an input image.

As shown in (c) of FIG. 7, an output result for an input value may be obtained.

According to an embodiment, the size of an image used in a convolutional neural network (CNN) is (300×300) pixels, but the system can import the entire CSV file and extract only the area of the (300×300) pixel portion and apply it to the CNN. At this time, when generating an image of (300×300) pixels from CSV data in advance, a lot of hardware resources and memory space are required, so once the CSV file is loaded, only the start points (x, y) are randomly generated, and the ( Only the pixel area of x ~ x+300, y ~ y+300) can be imported and used for learning. This has the same effect as learning new data for each learning because the starting point is randomly generated each time.

As shown in (d) of FIG. 7, when the learning is completed, the system can acquire and analyze the learning result, and as a result value in the last Dense Layer, the values of CSP 3 Probability, CSP 5 Probability, and CSP 7 Probability Is displayed, and the largest value among the three values can be used as a label for the image to be analyzed.

8 is a flowchart illustrating a process of generating a first analysis result according to an exemplary embodiment.

Referring to FIG. 8, first, in step S801, the device 100 may acquire 3D data on the concrete surface through the lidar. Here, the 3D data is a 3D image of the concrete surface.

In step S802, the device 100 may acquire 2D data on the concrete surface through the camera. Here, the 2D data is a 2D image of the concrete surface.

In operation S803, the apparatus 100 may extract first data obtained by merging the 2D data and the 3D data by separating the union area of the 2D data and the 3D data.

Specifically, the device 100 may compare 2D data and 3D data to identify a union area that overlaps each other, separate the union area from 2D data, separate the union area from 3D data, and merge the separated union area Thus, the first data can be extracted. Here, the first data may be composed of 4 channels, 3 channels may be 2D data indicating RGB values, and 1 channel may be data indicating 3D depth values.

The device 100 may generate a first input signal by encoding the first data.

Specifically, the device 100 may generate a first input signal by encoding a pixel of the first data into color information. The color information may include RGB color information, brightness information, saturation information, and depth information, but is not limited thereto. The apparatus 100 may convert color information into a numerical value, and may encode the first data in the form of a data sheet including this value.

The device 100 may input the first input signal to a first artificial neural network that has been learned in advance in the device 100.

The first artificial neural network according to an embodiment is composed of a feature extraction neural network and a classification neural network, and the feature extraction neural network sequentially stacks an input signal by a convolutional layer and a pooling layer. The convolution layer includes convolution operations, convolution filters, and activation functions. The calculation of the convolution filter is adjusted according to the size of the matrix of the target input, but generally a 9X9 matrix is used. The activation function generally uses a ReLU function, a sigmoid function, and a tanh function, but is not limited thereto. The pooling layer is a layer that reduces the size of an input matrix, and uses a method of extracting a representative value by grouping pixels in a specific area. In general, an average value or a maximum value is often used for the operation of the pooling layer, but is not limited thereto. This operation is performed using a square matrix, usually a 9X9 matrix. The convolutional layer and the pooling layer are alternately iterated until the corresponding input is small enough while maintaining the difference.

According to an embodiment, the classification neural network has a hidden layer and an output layer. The classification neural network of the first artificial neural network for CSP classification of the concrete surface is composed of 5 or less hidden layers, and may include a total of 50 or less hidden layer nodes. The activation function of the hidden layer uses a ReLU function, a sigmoid function, and a tanh function, but is not limited thereto. There is a total of 1 output layer node of the classification neural network, and the output value for the classification of the concrete surface can be output to the output layer node. A detailed description of the first artificial neural network will be described later with reference to FIG. 14.

The device 100 may obtain a first output signal based on a result of the input of the first artificial neural network.

In step S804, the device 100 may generate a first classification result for the concrete surface based on the first output signal. Here, the first classification result may include information on which stage of the CSP the concrete surface is classified.

For example, as a result of checking the output value of the first output signal, when the output value is 1, the device 100 generates a first classification result as the concrete surface corresponds to CSP 1, and when the output value is 2, the concrete surface It is possible to generate a first classification result as corresponding to this CSP 2.

9 is a flowchart illustrating a process of generating a second analysis result according to an exemplary embodiment.

Referring to FIG. 9, first in step S901, when a first classification result is generated through analysis of the first data, the device 100 may transmit the first data and the first classification result to the server 200.

In step S902, the server 200 may analyze the first data received from the device 100 to detect a crack generated on the concrete surface. In the case of crack detection, only the part that is confirmed to be larger than a certain size through image analysis can be detected as a crack generated on the concrete surface.

In step S903, the server 200 may identify cracks generated on the concrete surface for each area, thereby distinguishing between the normal area and the damaged area.

Specifically, the server 200 divides the first data into a plurality of areas, such as a first area and a second area, and can check how many cracks have been detected for each area, and a crack is detected below the first reference value. The damaged area may be divided into a normal area, and an area in which a crack is detected above the first reference value may be divided into a damaged area. In this case, the first reference value may be set differently according to the exemplary embodiment.

In step S904, the server 200 may extract second data from which the damaged area is deleted from the first data.

For example, the image in the first data is composed of a first area, a second area, and a third area. The first area is divided into a damaged area, and the second area and the third area are divided into normal areas. In this case, the server 200 may extract an image including only the second region and the third region as second data.

The server 200 may generate a second input signal by encoding the second data.

Specifically, the server 200 may generate a second input signal by encoding a pixel of the second data into color information. The color information may include RGB color information, brightness information, saturation information, and depth information, but is not limited thereto. The server 200 may convert the color information into a numerical value, and may encode the second data in the form of a data sheet including this value.

The server 200 may input the second input signal to a second artificial neural network that has been learned in advance in the server 200.

The second artificial neural network according to an embodiment is composed of a feature extraction neural network and a classification neural network, and the feature extraction neural network sequentially stacks an input signal with a convolutional layer and a pooling layer. The convolution layer includes convolution operations, convolution filters, and activation functions. The calculation of the convolution filter is adjusted according to the size of the matrix of the target input, but generally a 9X9 matrix is used. The activation function generally uses a ReLU function, a sigmoid function, and a tanh function, but is not limited thereto. The pooling layer is a layer that reduces the size of an input matrix, and uses a method of extracting a representative value by grouping pixels in a specific area. In general, an average value or a maximum value is often used for the operation of the pooling layer, but is not limited thereto. This operation is performed using a square matrix, usually a 9X9 matrix. The convolutional layer and the pooling layer are alternately iterated until the corresponding input is small enough while maintaining the difference.

According to an embodiment, the classification neural network has a hidden layer and an output layer. The classification neural network of the second artificial neural network for CSP classification of the concrete surface is composed of 5 or less hidden layers, and may include a total of 50 or less hidden layer nodes. The activation function of the hidden layer uses a ReLU function, a sigmoid function, and a tanh function, but is not limited thereto. There is a total of 1 output layer node of the classification neural network, and the output value for the classification of the concrete surface can be output to the output layer node. A detailed description of the second artificial neural network will be described later with reference to FIG. 14.

The server 200 may obtain a second output signal based on a result of the input of the second artificial neural network.

In step S905, the server 200 may generate a second classification result for the concrete surface based on the second output signal. Here, the second classification result may include information on which stage of the CSP the concrete surface is classified.

For example, the server 200 checks the output value of the second output signal, and when the output value is 1, generates a second classification result as the concrete surface corresponds to CSP 1, and when the output value is 2, the concrete surface It is possible to generate a second classification result that corresponds to this CSP 2.

The server 200 may set a final classification result for the concrete surface based on the first classification result and the second classification result.

For example, when the first classification result and the second classification result are the same, the server 200 may set any one of the first classification result and the second classification result as the final classification result for the concrete surface. A detailed description of the content for setting the final classification result will be described later with reference to FIG. 12.

10 is a flowchart illustrating a process of performing additional learning on a second artificial neural network according to an embodiment.

Referring to FIG. 10, first in step S1001, the server 200 may check a final classification result set based on the first classification result and the second classification result.

In step S1002, the server 200 may analyze the concrete surface by performing image processing and pattern recognition on the second data according to the final classification result.

For example, in the database 250 of the server 200, an algorithm for image processing and pattern recognition optimized for each CSP step is stored. If the concrete surface is classified as CSP 1 according to the final classification result, , The server 200 can perform image processing and pattern recognition on the second data using the CSP 1 dedicated algorithm, and when the concrete surface is classified as CSP 2 according to the final classification result, the server 200 2 Using a dedicated algorithm, image processing and pattern recognition may be performed on the second data. That is, image processing and pattern recognition for the second data may be performed in different ways according to the CSP step identified through the final classification result.

In step S1003, the server 200 may obtain a result of the operator's determination on the final classification result.

Specifically, the server 200 may transmit the final classification result to the worker terminal, and may receive and obtain a determination result for the final classification result from the worker terminal. In this case, the operator's determination result may include information indicating either success or failure.

In step S1004, the server 200 may check whether the determination result obtained from the worker terminal is collected above the second reference value. Here, the second reference value may be set differently according to embodiments.

If it is confirmed in step S1004 that the determination result is not collected above the second reference value, the process returns to step S1003, and the server 200 may obtain the determination result of the operator again.

When it is confirmed in step S1004 that the determination result is collected above the second reference value, in step S1005, the server 200 may transmit the determination results to a cloud connected to the server 200. In this case, the server 200 may be implemented as a cloud server.

In step S1006, the server 200 may process additional learning on the second artificial neural network in the server 200 based on the determination results. In this case, the server 200 may perform additional learning so that the second artificial neural network is optimized by dividing data in which the determination result is a success and data that is a failure.

11 is a flowchart illustrating a process of updating a deep learning model of a first artificial neural network according to an embodiment.

Referring to FIG. 11, first, in step S1101, the device 100 may perform learning through a first artificial neural network in the device 100 to obtain a learning result through the first artificial neural network, and the first artificial neural network The performance of the first artificial neural network can be confirmed through the learning result through.

In step S1102, the server 200 may perform learning through a second artificial neural network in the server 200 to obtain a learning result through the second artificial neural network. 2 You can check the performance of the artificial neural network.

In step S1103, the server 200 compares the learning result through the second artificial neural network and the performance of the learning result through the first artificial neural network, and the learning result through the second artificial neural network is a learning result through the first artificial neural network. It can be checked whether or not the performance is improved by more than the third reference value. Here, the third reference value may be set differently according to embodiments.

For example, the performance of the first artificial neural network confirmed through the learning result through the first artificial neural network is 80, the performance of the second artificial neural network confirmed through the learning result through the second artificial neural network is 95, and the third When the reference value is 10, the server 200 may confirm that the learning result through the second artificial neural network has improved performance beyond the third reference value.

In step S1103, if the learning result through the second artificial neural network is confirmed to have improved performance beyond the third reference value, in step S1104, the device 100 updates the deep learning model of the first artificial neural network through the second artificial neural network. I can. Thereafter, returning to step S1101, the device 100 may re-confirm the performance of the first artificial neural network by performing learning again through the first artificial neural network in which the deep learning model has been updated.

If it is determined in step S1103 that the learning result through the second artificial neural network has not improved more than the third reference value, in step S1105, the server 200 may perform additional learning on the second artificial neural network. Thereafter, returning to step S1102, the server 200 may perform learning again through the additionally learned second artificial neural network, so that the performance of the second artificial neural network may be confirmed again.

12 is a flowchart illustrating a process of setting a final classification result according to an embodiment.

Referring to FIG. 12, first, in step S1201, the apparatus 100 may extract first data obtained by merging 2D data and 3D data by separating a union region of 2D data and 3D data.

In step S1202, the device 100 may generate a first classification result for the concrete surface based on the first data. In this case, the device 100 may transmit the first data and the first classification result to the server 200.

In step S1203, the server 200 may analyze the first data, divide it into a normal region and a damaged region, and extract second data obtained by deleting the damaged region from the first data.

In step S1204, the server 200 may generate a second classification result for the concrete surface based on the second data.

In step S1205, the server 200 may compare the first classification result generated in step S1202 with the second classification result generated in step S1204 to determine whether they are the same.

If it is determined in step S1205 that the first classification result and the second classification result are the same, in step S1206, the server 200 may set any one of the first classification result and the second classification result as the final classification result. That is, since the first classification result and the second classification result are the same, one of the first classification result and the second classification result can be set as the final classification result.

If it is confirmed that the first classification result and the second classification result are different in step S1205, in step S1207, the server 200 acquires third data using the first classification result, and in step S1208, the server 200 The fourth data may be obtained by using the second classification result.

Specifically, in the database 250 of the server 200, representative images are stored for each classification of the concrete surface, and representative images are stored for each classification, such as CSP 1, CSP 2, and CSP 3, respectively.

When it is confirmed through the first classification result that the concrete surface is classified as the first classification, the server 200 may obtain data registered as the representative image of the first classification as third data. For example, when the concrete surface is classified as CSP 2 through the first classification result, a representative image of CSP 2 may be obtained as third data.

In addition, when it is confirmed through the second classification result that the concrete surface is classified as the second classification, the server 200 may acquire data registered as the representative image of the second classification as fourth data. For example, when the concrete surface is classified as CSP 3 through the second classification result, a representative image of CSP 3 may be obtained as fourth data.

In step S1209, the server 200 may analyze the first degree of consistency by comparing the first data extracted in step S1201 with the third data acquired in step S1207. In this case, the server 200 may calculate a first degree of correspondence with respect to how much the first data and the third data are matched through image comparison analysis.

In step S1210, the server 200 may analyze the second degree of match by comparing the second data extracted in step S1203 with the fourth data acquired in step S1208. In this case, the server 200 may calculate a second degree of correspondence with respect to how much the second data and the fourth data are matched through image comparison analysis.

In step S1211, the server 200 may check whether the first degree of agreement is greater than the second degree of agreement by comparing the first degree of agreement analyzed in operation S1209 with the second degree of agreement analyzed in operation S1210.

If it is determined in step S1211 that the first degree of agreement is greater than the second degree of agreement, in step S1212, the server 200 may set the first classification result as the final classification result. That is, since the degree of agreement between the first data and the third data is greater than the degree of agreement between the second data and the fourth data, the first classification result generated through the first data may be set as the final classification result.

If it is determined in step S1211 that the second degree of agreement is greater than the first degree of agreement, in step S1213, the server 200 may set the second classification result as the final classification result. That is, since the degree of agreement between the second data and the fourth data is greater than the degree of agreement between the first data and the third data, the second classification result generated through the second data may be set as the final classification result.

13 is a diagram for describing a process of acquiring pressure sensing data for crack recognition according to an exemplary embodiment.

According to an embodiment, the crack recognition means 140 may be installed at the lower end of the device 100 and may include a plurality of pressure sensors. Here, each pressure sensor may be implemented with various sensors.

A plurality of pressure sensors included in the crack recognition means 140 are arranged for each zone, so that the pressure applied to the concrete surface in each zone can be recognized. At this time, concrete is placed on the upper part of the crack recognition means 140, and a device for applying pressure to the upper part of the concrete may be additionally provided.

As shown in FIG. 13, a second sensor is disposed in contact with the upper left of the first sensor, a third sensor is disposed in contact with the upper right of the first sensor, and a fourth sensor is disposed on the left side of the first sensor. The sensor is placed in abutment, the fifth sensor is placed in abutment on the right side of the first sensor, the sixth sensor is placed on the lower left of the first sensor, and the seventh sensor is placed on the lower right of the first sensor. It is placed.

The first sensor detects the pressure applied to the first area where the first sensor is disposed, the second sensor senses the pressure applied to the second area where the second sensor is disposed, and the third sensor detects the pressure applied to the second area where the second sensor is disposed. The pressure applied to the disposed third area is sensed, the fourth sensor senses the pressure applied to the fourth area in which the fourth sensor is disposed, and the fifth sensor is applied to the fifth area in which the fifth sensor is disposed. The pressure is sensed, the sixth sensor may sense a pressure applied to the sixth region in which the sixth sensor is disposed, and the seventh sensor may sense the pressure applied to the seventh region in which the seventh sensor is disposed. The pressure value sensed by each sensor may be transmitted to the first processor 150 of the device 100, and the device 100 combines the pressure values received from each sensor, Can be distinguished.

Specifically, the apparatus 100 may classify a region in which the pressure value is uniformly distributed as a normal region, and a region in which the pressure value is not uniformly distributed as a damage region.

A plurality of pressure sensors included in the crack recognition means 140 detect pressure in each area, and when it is determined that the detected pressure value is greater than a preset pressure reference value, the pressure sensing data is transmitted to the first of the device 100. It can be transmitted to the processor 150. Here, the pressure reference value may be set differently according to embodiments. The pressure sensing data includes a coordinate value at which the sensor is located on the crack recognition means 140, and the device 100 can check the position of the sensor that has transmitted the pressure sensing data by using the coordinate value. The pressure detection data only includes information indicating that a pressure above the reference value is detected, and may not include an accurate pressure value.

That is, each of the plurality of pressure sensors included in the crack recognition means 140 can detect the magnitude of the pressure applied to the concrete surface for each area, and when at least one of the plurality of pressure sensors detects a pressure greater than the pressure reference value, The apparatus 100 may acquire pressure sensing data for a coordinate value of a sensor located in a region in which a pressure greater than the pressure reference value is sensed.

Each of the plurality of pressure sensors may transmit pressure sensing data to the first processor 150 of the device 100 when a pressure value higher than the pressure reference value is measured, and the device 100 may transmit the pressure sensing data to the coordinates of the sensor from which the pressure sensing data is received. The area at the location corresponding to the value can be divided into a normal area.

For example, the device 100 obtained pressure sensing data from the first sensor while acquiring the pressure values sensed by each of the first sensor and the second sensor, but did not acquire the pressure sensing data from the second sensor. In this case, the first area in which the first sensor is disposed may be divided into a normal area, and the second area in which the second sensor is disposed may be divided into a damaged area.

That is, the device 100 may identify the coordinate value of the sensor through the pressure sensing data, identify a region at a location corresponding to the determined coordinate value, and divide the region into a normal region.

According to an embodiment, when the first sensor detects a pressure greater than a predetermined pressure reference value, the first sensor may transmit pressure detection data to the first processor 150 of the device 100, wherein the first sensor is greater than the pressure reference value. Even if the pressure is sensed, if at least one of the surrounding sensors in contact with the first sensor does not detect a pressure greater than the pressure reference value, the first sensor transmits pressure sensing data on the coordinate value of the first sensor to the device 100. May not be transferred to. That is, when at least one of the surrounding sensors in contact with the first sensor does not detect a pressure greater than the pressure reference value, the device 100 may control the pressure detection data not to be obtained from the first sensor.

For example, as shown in (a) of FIG. 13, not only the first sensor, but also the third sensor, the fourth sensor, the fifth sensor, the sixth sensor, and the seventh sensor sensed a pressure greater than the pressure reference value. , When the second sensor does not detect a pressure greater than the pressure reference value, the second sensor among the surrounding sensors in contact with the first sensor did not detect a pressure greater than the pressure reference value, so that the device 100 is the first sensor. Pressure sensing data may not be obtained from

On the other hand, when the first sensor detects a pressure greater than the pressure reference value, and detects a pressure greater than the pressure reference value even in all of the surrounding sensors in contact with the first sensor, the first sensor corresponds to the coordinate value of the first sensor. The pressure sensing data for the device 100 may be transmitted to the first processor 150 of the device 100. That is, when the device 100 detects a pressure greater than the pressure reference value in all of the surrounding sensors in contact with the first sensor, the device 100 may obtain pressure detection data from the first sensor.

For example, as shown in (b) of FIG. 13, not only the first sensor, but also the second sensor, the third sensor, the fourth sensor, the fifth sensor, the sixth sensor, and the third sensor surrounding the first sensor. 7 When a pressure greater than the pressure reference value is sensed in the entire sensor, since a pressure greater than the pressure reference value is sensed in all surrounding sensors in contact with the first sensor, the device 100 may obtain pressure sensing data from the first sensor. I can.

14 is a diagram illustrating an artificial neural network according to an embodiment.

The artificial neural network 1400 according to an embodiment may be any one of a first artificial neural network and a second artificial neural network. In the case of the first artificial neural network, a first input signal generated by encoding of the first data may be input, and information on which stage of the CSP is classified into the concrete surface may be output. In the case of the second artificial neural network, a second input signal generated by encoding of the second data may be input, and information on which stage of the CSP is classified into the concrete surface may be output.

The encoding according to an embodiment may be performed by storing color information for each pixel of an image in the form of a numerical data sheet, and the color information includes RGB color, brightness information, saturation information, and depth information of one pixel. Can, but is not limited to this.

According to an embodiment, the artificial neural network 1400 is composed of a feature extraction neural network 1410 and a classification neural network 1420, and the feature extraction neural network 1410 performs a task of separating a concrete region and a background region from an image. In addition, the classification neural network 1420 may perform a task of determining which stage of the CSP the concrete surface is classified within the image.

The method of distinguishing the concrete area and the background area by the feature extraction neural network 1410 is that the change of each value of color information from the data sheet of the input signal encoding the image is 30% in 6 or more of 8 pixels including one pixel. A bundle of pixels detected as having the above change may be used as a boundary between the concrete area and the background area, but is not limited thereto.

The feature extraction neural network 1410 proceeds by sequentially stacking the convolutional layer and the pooling layer on the input signal. The convolution layer includes convolution operations, convolution filters, and activation functions. The calculation of the convolution filter is adjusted according to the size of the matrix of the target input, but generally a 9X9 matrix is used. The activation function generally uses a ReLU function, a sigmoid function, and a tanh function, but is not limited thereto. The pooling layer is a layer that reduces the size of an input matrix, and uses a method of extracting a representative value by grouping pixels in a specific area. In general, an average value or a maximum value is often used for the operation of the pooling layer, but is not limited thereto. This operation is performed using a square matrix, usually a 9X9 matrix. The convolutional layer and the pooling layer are alternately iterated until the corresponding input is small enough while maintaining the difference.

The classification neural network 1420 checks the surface of the concrete region separated from the background through the feature extraction neural network 1410, and checks whether the surface state of the concrete region is similar to the stepwise surface state of the predefined CSP. It is possible to determine whether or not it is classified into stages. Information stored in the memory 160 or the database 250 may be used to compare the surface state of the CSP in stages.

The classification neural network 1420 has a hidden layer and an output layer, is composed of 5 or less hidden layers, and includes a total of 50 or less hidden layer nodes, and the activation function of the hidden layer is a ReLU function and a sigmoid function. And tanh function, but are not limited thereto.

The classification neural network 1420 may include only one output layer node in total.

The output of the classification neural network 1420 is an output value indicating which stage of the CSP is classified as the concrete surface, and may indicate which stage of the CSP corresponds to. For example, when the output value is 1, it is indicated that the concrete surface corresponds to CSP 1, and when the output value is 2, it is possible to indicate that the concrete surface corresponds to CSP 2.

According to an embodiment, the artificial neural network 1400 may receive and learn a first learning signal generated by a correct answer input by the user when a user finds a problem with an output according to the artificial neural network 1400. The problem of output according to the artificial neural network 1400 may refer to a case in which output values classified into different stages of CSP are output to the concrete surface.

The first learning signal according to an embodiment is generated based on an error between a correct answer and an output value, and in some cases, an SGD using a delta, a batch method, or a method following a backpropagation algorithm may be used. The artificial neural network 1400 performs learning by modifying an existing weight by using the first learning signal, and may use momentum in some cases. The cost function can be used to calculate the error, and the cross entropy function can be used as the cost function. Hereinafter, the learning contents of the artificial neural network 1400 will be described later with reference to FIG. 15.

15 is a diagram illustrating a method of learning an artificial neural network according to an embodiment.

According to an embodiment, the learning device may train the artificial neural network 1400. The learning device may be a separate subject different from the device 100 and the server 200, but is not limited thereto.

According to an embodiment, the artificial neural network 1400 includes an input layer to which training samples are input and an output layer to output training outputs, and may be trained based on a difference between the training outputs and the first labels. Here, the first labels may be defined based on the representative image registered in each step of the CSP. The artificial neural network 1400 is connected by a group of a plurality of nodes, and is defined by weights between the connected nodes and an activation function that activates the nodes.

The learning device may train the artificial neural network 1400 using a gradient decent (GD) technique or a stochastic gradient descent (SGD) technique. The learning device may use a loss function designed by labels and outputs of the artificial neural network 1400.

The learning device may calculate a training error using a predefined loss function. The loss function may be predefined with a label, an output, and a parameter as input variables, where the parameter may be set by weights in the artificial neural network 1400. For example, the loss function may be designed in a Mean Square Error (MSE) form, an entropy form, or the like, and various techniques or methods may be employed in an embodiment in which the loss function is designed.

The learning device may find weights that affect the training error using a backpropagation technique. Here, the weights are relationships between nodes in the artificial neural network 1400. The learning apparatus can use the SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning apparatus may update weights of a loss function defined based on labels, outputs, and weights using the SGD technique.

According to an embodiment, the learning apparatus may obtain representative images 1501 of each labeled training CSP step from the memory 160 or the database 250. The learning apparatus may obtain pre-labeled information on each of the representative images 1501 for each CSP step, and the representative images 1501 for each CSP step may be labeled according to a pre-classified CSP step.

According to an embodiment, the learning apparatus may acquire 1000 labeled training CSP step-by-step representative images 1501, and based on the labeled training CSP step-by-step representative images 1501, first training CSP step-by-step vectors ( 1502) can be created. Various methods may be employed to extract the vectors 1502 for each step of the first training CSP.

According to an embodiment, the learning apparatus may obtain first training outputs 1503 by applying the first training CSP step vectors 1502 to the artificial neural network 1400. The learning apparatus may train the artificial neural network 1400 based on the first training outputs 1503 and the first labels 1504. The learning apparatus may train the artificial neural network 1400 by calculating training errors corresponding to the first training outputs 1503 and optimizing the connection relationship between nodes in the artificial neural network 1400 in order to minimize the training errors. .

The embodiments described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices, methods, and components described in the embodiments are, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. Further, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to operate as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or, to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

As described above, although the embodiments have been described by the limited drawings, a person of ordinary skill in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order from the described method, and/or components such as systems, structures, devices, circuits, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the following claims.

Claims (3)

In the method of analyzing the concrete surface through the machine vision and 3D profile performed by the system,
Obtaining 3D data on the concrete surface through a lidar and obtaining 2D data on the concrete surface through a camera;
Separating the union region of the 2D data and the 3D data, and extracting first data obtained by merging the 2D data and the 3D data;
Encoding the first data to generate a first input signal;
Inputting the first input signal to a first artificial neural network in a device, and obtaining a first output signal based on a result of the input of the first artificial neural network;
Generating a first classification result for the concrete surface based on the first output signal;
Transmitting the first data and the first classification result to a server;
Analyzing the first data transmitted to the server to detect cracks occurring on the concrete surface;
Identifying a crack generated on the concrete surface by region, and dividing a normal region in which a crack is detected below a first reference value and a damaged region in which a crack is detected above the first reference value;
Extracting second data from which the damaged area is deleted from the first data;
Encoding the second data to generate a second input signal;
Inputting the second input signal to a second artificial neural network in the server, and obtaining a second output signal based on a result of the input of the second artificial neural network;
Generating a second classification result for the concrete surface based on the second output signal; And
If the first classification result and the second classification result are the same, comprising the step of setting any one of the first classification result and the second classification result as a final classification result for the concrete surface,
Concrete surface analysis method through machine vision and 3D profile. delete The method of claim 1,
Analyzing the concrete surface by performing image processing and pattern recognition on the second data according to the final classification result;
Obtaining a result of the operator's judgment on the final classification result;
If the determination result is collected above a second reference value, transmitting the collected determination results to a cloud connected to the server;
Processing the second artificial neural network to perform additional learning based on the determination results; And
When the learning result through the second artificial neural network is compared with the performance of the learning result through the first artificial neural network, and the learning result through the second artificial neural network is improved by more than a third reference value, the second artificial neural network Further comprising the step of updating the deep learning model of the first artificial neural network through a neural network,
Concrete surface analysis method through machine vision and 3D profile.

Images