KR20220151777A - Method and device for classifying building defect using multi task channel attention - Google Patents

Abstract

The present invention relates to a method for classifying a building defect using multi-task channel attention and a device for the same. According to an embodiment of the present invention, the method for classifying a building defect using multi-task channel attention and the device for the same provide effect of classifying a defect of a building by each category with only a short text. The method for classifying a building defect using multi-task channel attention includes: a step of receiving input on text data about the building defect by an analysis device; a step of extracting multiple sharing characteristics by receiving the input on the text data by multiple first encoders having filters of different sizes; a step of extracting multiple work characteristics by receiving a value in which the characteristics output by the first encoders are combined, by multiple second encoders; and a step of outputting a classification value about different items by receiving the input of the value output by each of multiple work specialization characteristic encoders, by multiple classifiers.

Description

Building defect classification method and apparatus using multi-task channel attention {METHOD AND DEVICE FOR CLASSIFYING BUILDING DEFECT USING MULTI TASK CHANNEL ATTENTION}

The disclosed technology relates to a method and apparatus for classifying categories of building defects using multi-working channel attention.

Various defects such as cracks or leaks may occur in the exterior or interior of a building due to various causes. In the event of such a defect, repairs should be made appropriate for the characteristics or types of the defect in order to secure the safety of the building and maintain its lifespan.

Conventionally, when a defect occurs in a building, a building manager or a resident roughly grasps the phenomenon of the defect and conveys it orally or by hand to a worker, or the worker directly goes to the site and checks the state of the defect. That is, since the worker has to prepare for unnecessary repair work only with ambiguous information about the defect transmitted from another person, or he himself visits the site and prepares for the repair work, there is a problem of low work efficiency.

On the other hand, in order to accurately identify building defects, many technologies are used to identify the state of defects by inputting images of the defect locations into a deep learning model, but there is no technology to identify building defects based on text such as natural language. Therefore, a technique for identifying defects using text data on building defects is required.

Korean Patent Registration No. 10-2221317

The disclosed technology provides a method and apparatus for classifying categories of building defects using multi-working channel attention.

The first aspect of the technology disclosed to achieve the above technical problem is the step of receiving text data about building defects by an analysis device, and a plurality of first encoders having filters of different sizes receive the text data and each of a plurality of Extracting a shared feature, extracting a plurality of task features by receiving a combined value of the features output by the first encoders by a plurality of second encoders, and outputting a plurality of task-specific feature encoders by a plurality of classifiers, respectively It is to provide a building defect automatic classification method comprising the step of receiving a value to output a classification value for each different item.

The second aspect of the technology disclosed to achieve the above technical problem includes an input device for receiving text data on building defects, a plurality of first encoders having filters of different sizes, a plurality of second encoders, and a plurality of classifiers. A storage device for storing a deep learning model that stores the text data and a plurality of different shared features are extracted by inputting the text data to the plurality of first encoders, and a value obtained by combining the plurality of shared features is transmitted to the plurality of second encoders. It is to provide a building defect classification device including an arithmetic device that extracts a plurality of work features by inputting them, respectively, and outputs classification values for different items by inputting the plurality of work features to the plurality of classifiers, respectively.

Embodiments of the disclosed technology may have effects including the following advantages. However, this does not mean that the embodiments of the disclosed technology must include all of them, so the scope of rights of the disclosed technology should not be understood as being limited thereby.

According to an embodiment of the disclosed technology, a building defect classification method and apparatus using multi-working channel attention has an effect of classifying building defects by category using only short text.

In addition, there is an effect of extracting high-level features without increasing the depth of the convolutional layer.

In addition, even if there is noise in one task, there is an effect of guaranteeing accuracy for task classification by using features shared by other tasks.

1 is a diagram illustrating a process of classifying building defects using multi-working channel attention according to an embodiment of the disclosed technology.
2 is a flowchart of a building defect classification method according to an embodiment of the disclosed technology.
3 is a block diagram of an apparatus for classifying building defects according to an embodiment of the disclosed technology.
4 is a diagram showing the structure of a deep learning model using multi-working channel attention according to an embodiment of the disclosed technology.
5 is a diagram illustrating a process of processing embedded vectors in parallel according to an embodiment of the disclosed technology.
6 is a diagram showing a detailed structure of an encoder.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, A, B, etc. may be used to describe various components, but the components are not limited by the above terms, and are only used for the purpose of distinguishing one component from another. used only as For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

In the terms used herein, singular expressions should be understood to include plural expressions unless the context clearly dictates otherwise. And the term "includes" means that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features or number, step, operation component, or part. or the possibility of the presence or addition of combinations thereof.

Prior to a detailed description of the drawings, it is to be clarified that the classification of components in the present specification is merely a classification for each main function in charge of each component. That is, two or more components to be described below may be combined into one component, or one component may be divided into two or more for each more subdivided function.

In addition, each component to be described below may additionally perform some or all of the functions of other components in addition to its main function, and some of the main functions of each component may be performed by other components. Of course, it may be dedicated and performed by . Therefore, the existence or nonexistence of each component described through this specification should be interpreted functionally.

1 is a diagram illustrating a process of classifying building defects using multi-working channel attention according to an embodiment of the disclosed technology. Referring to FIG. 1 , the analysis device may classify categories of defects by receiving text data about building defects and analyzing them through a deep learning model.

The text data on the building defects received by the analysis device means text written or typed by the user. For example, it may be a short text in which the user describes a defect in a building. In FIG. 1 , a situation in which a short text “contamination due to poor waterproofing of the bathroom floor” is input as text data is assumed. The analysis device may have a device such as a keyboard or a touch pad so that a user can directly input text, and text data can be input through the device. Alternatively, text data transmitted from the user's terminal may be received.

Meanwhile, the analysis device may embed the input text data into a deep learning model in a form that can be input. The deep learning model includes two different embedding models. For example, Word2vec and FastText can be included as embedding models. Unlike the conventional 2D Convolution Neural Network (TWO dimensional Convolution Neural Network), which performs a convolution operation on a matrix according to the kernel size for image or video processing, the deep learning model mentioned in the disclosed technology is a natural language processing-based 1D convolution neural network ( One dimensional convolution neural network) can be used. That is, it is possible to perform an operation while moving the kernel only in the height direction, not in the matrix. The analysis device may tokenize sentences included in text data through embedding.

On the other hand, text data embedded in a form that can be input to a deep learning model is first input to the shared feature encoder of the deep learning model. The shared feature encoder is composed of a plurality of encoders having a parallel structure. Hereinafter, the shared feature encoder is referred to as the first encoder. The first encoder receives the embedded text data and extracts a plurality of shared features, respectively. The shared features extracted here are used to learn the following task-specific feature encoders.

Meanwhile, the first encoder includes a plurality of convolutional layers connected to filters of different sizes in parallel. That is, convolutional layers having a parallel structure exist in the first encoder, and a plurality of different shared features are extracted through filters having different sizes connected to each layer. Each layer has a convolution block and a text squeeze and excitation (TSE) block. The convolution block may be convolution layers of a certain length, and the TSE block may be an attention layer connected to the convolution layer as a block that performs channel attention to improve the performance of the encoder. Each of the plurality of first encoders may calculate an attention score of each shared feature using the TSE block.

Meanwhile, each of the plurality of first encoders includes one or more structures in which a convolution block and a TSE block are repeated. For example, a convolution block and a TSE block may be configured as one set, and this set may be repeated. Preferably the two sets can be repeated so that the length of the encoder is not longer than necessary. In addition, features can be extracted by inputting the text data embedded in the first set and re-entering the output result into the next set. As an embodiment, the plurality of first encoders may respectively extract different shared features by inputting result values extracted through the first convolution block and the first TSE block to the second convolution block and the second TSE block, respectively. In this way, high-level features of text data can be extracted by using the features output through the first set as inputs of the second set.

Meanwhile, as described above, the TSE block corresponds to the attention layer. The Attention layer includes a Global Average Pooling (GAP) layer and a Fully Connected layer. The attention layer performs the excitation operation by multiplying the result of the squeeze operation through the global pooling layer once by the parameter of the fully connected layer, and the sigmoid ( Sigmoid) function is applied to calculate the attention score. An attention score between 0 and 1 is given to the feature extracted according to this operation.

In the conventional case, in order to extract high-level features, a block structure was repeatedly designed for a certain length or more to improve the performance of feature extraction, but in the disclosed technology, through parallel structured layers including filters or kernels of different sizes, Since the feature is extracted, a high level of feature extraction is possible without designing the depth of the convolution layer deeper than necessary. Since this has an effect similar to increasing the number of samples, it is possible to sufficiently extract local features included in text data.

Meanwhile, the extracted shared features are used as inputs of a plurality of task-specific feature encoders. Hereinafter, the task-specific feature encoder is referred to as a second encoder. The plurality of second encoders receive a concatenation value of each shared feature and extract a plurality of task features, respectively. Like the shared feature encoder, the second encoder includes a plurality of convolution blocks connected in parallel and a plurality of TSE blocks connected to each convolution block. That is, the structure itself of the two encoders is the same. However, there is a difference in that filters having different sizes exist in the first encoder and filters having the same size exist in the second encoder.

Each of the plurality of second encoders may calculate an attention score of a task feature using the TSE block. Each of the second encoders may extract a task feature by inputting result values extracted through the first convolution block and the first TSE block to the second convolution block and the second TSE block. At this time, it is possible to improve the expressiveness of the feature map affecting each task based on the combined shared features. That is, since shared features of other tasks can be learned, task feature extraction can be successfully performed even if noise is included in the input value, and the accuracy of the task feature extraction result can be increased.

Meanwhile, the analysis device outputs classification values for each different item through a plurality of classifiers. Each classifier may include, as a classification task, the type of work according to the defect, the location of the defect, the type and attribute of the defect. As shown in FIG. 1 as an example, if the output value of the model is “bathroom floor waterproof construction”, the type of work is “waterproof construction”, the location of the defect is “bathroom”, the type of defect is “water leak”, and the type of defect is “waterproof construction”. Attributes can each be classified as "floor".

Meanwhile, a parallel structure of a plurality of second encoders and a plurality of classifiers may be determined according to the number of items. For example, when there are 4 tasks, the second encoder may have a structure in which 4 layers are connected in parallel, and the classifier may also have a structure in which 4 layers are connected in parallel. Of course, the number of first encoders need not necessarily be four. According to this process, different items for defects included in the text data for building defects can be performed simultaneously.

2 is a flowchart of a building defect classification method according to an embodiment of the disclosed technology. Referring to FIG. 2, the building defect classification method 200 may be sequentially performed through a deep learning model loaded in an analysis device, and includes a step of receiving text data (210) and a step of extracting a plurality of different shared features ( 220), extracting a plurality of task features (230), and outputting a classification value for each item (240).

In step 210, the analysis device receives text data about building defects. Text data on building defects consists of short texts describing defects in the building, and may be data prepared by a building manager or occupants to be delivered to repair workers. The analysis device may receive text data from a means for transmitting text data or may directly receive text data.

In step 220, the analysis device transmits text data to a plurality of shared feature encoders, which are the first encoders of the deep learning model. Each shared feature encoder extracts shared features from input text data. The shared feature encoder has a structure in which convolution layers having filters of different sizes are connected in parallel, and each layer is composed of a convolution block and a TSE block. Naturally, since the filter sizes are different, different shared features can be extracted from each layer.

In step 230, a plurality of task-specific feature encoders, which are second encoders of the deep learning model, receive a value obtained by combining a plurality of shared features and extract a plurality of task features. Since the shared feature encoder outputs the resulting value including the attention score for each shared feature in step 220, the task-specific feature encoder can extract features suitable for each task by referring to the attention score.

In step 240, a plurality of classifiers of the deep learning model classify items of a plurality of task characteristics. A plurality of classifiers can pool only the most important features per channel from task features in order to simultaneously classify each task into different items. This is determined through the attention score calculated through the TSE block included in the task-specific feature encoder. For example, one with the highest attention score may be pooled. Item classification can be multitasked by processing this process by each classifier connected in parallel.

3 is a block diagram of an apparatus for classifying building defects according to an embodiment of the disclosed technology. Referring to FIG. 3 , the building defect classification device 300 includes an input device 310 , a storage device 320 and an arithmetic device 330 .

The input device 310 receives text data about building defects. The input device 310 may include a communication module and an input interface for receiving text data. For example, text data transmitted from another device may be received through a communication module or text data input by a user may be received through an input interface. The communication module may support short-range wireless communication such as Bluetooth or mobile communication such as 4G and 5G. In addition, a device such as a keyboard or a touch pad may be provided as an input interface.

The storage device 320 stores the deep learning model. The deep learning model stored in the storage device 320 is learned to process tokenized text, and includes a plurality of shared feature encoders, a plurality of task-specific feature encoders, and a plurality of classifiers. Filters of different sizes are provided in the plurality of shared feature encoders, and filters of the same size are provided in the plurality of task-specific feature encoders. The storage device 320 may be implemented as a memory having a capacity to store such a deep learning model.

The computing device 330 may input text data to the deep learning model, extract features, and classify items for each feature. The computing device 330 may be implemented as an AP or a processor capable of performing calculations for classifying categories. The arithmetic device 330 may first embed text data about building defects into a form that can be input to the deep learning model. In addition, a plurality of different shared features may be extracted by inputting text data embedded in a plurality of shared feature encoders having filters of different sizes included in the deep learning model, respectively. In addition, a value obtained by combining a plurality of shared features is input to a plurality of task-specific feature encoders to extract a plurality of task features, respectively, and a plurality of task features are respectively input to a plurality of classifiers to output classification values for different items. calculations can be performed.

Meanwhile, the building defect classification apparatus 300 as described above may be implemented as a program (or application) including an executable algorithm that can be executed on a computer. The program may be stored and provided in a temporary or non-transitory computer readable medium.

A non-transitory readable medium is not a medium that stores data for a short moment, such as a register, cache, or memory, but a medium that stores data semi-permanently and can be read by a device. Specifically, the various applications or programs described above are CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), EPROM (Erasable PROM, EPROM) Alternatively, it may be stored and provided in a non-transitory readable medium such as EEPROM (Electrically EPROM) or flash memory.

Temporary readable media include static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), and enhanced SDRAM (Enhanced SDRAM). SDRAM, ESDRAM), Synchronous DRAM (Synclink DRAM, SLDRAM) and Direct Rambus RAM (DRRAM).

4 is a diagram showing the structure of a deep learning model using multi-working channel attention according to an embodiment of the disclosed technology. In the conventional structure of convolutional neural networks for natural language processing, a spatial measurement structure was created by increasing the depth of the convolutional layer, and then high-level features were extracted through this to improve performance. The advantage of this architecture is that low-level convolutional layers learn general patterns while extracting local features, and high-level convolutional layers learn global patterns across all inputs. However, when the length of the input text data is short, there is a problem in that sufficient performance is not shown even if the depth of the model is increased due to lack of data.

Therefore, a model capable of efficiently extracting features using only a layer with a limited depth is required. The deep learning model used in the disclosed technology is designed to solve this problem, and is called an AutoDefect model. The AutoDefect model has a structure in which a plurality of layers are connected in parallel. In addition, by extracting features using two-step parallel encoders, sufficient performance can be guaranteed even when short text data is input.

On the other hand, high performance can be exhibited by using one model connected in parallel without designing each independent model. For example, since the number of samples is increased by concurrently processing feature extraction in parallel, it is possible to prevent overfitting. In addition, by reflecting the relative importance (attention score) in the feature map for each layer using the TSE block, the model learns meaningful information and the feature extraction function can be maximized.

Meanwhile, as shown in FIG. 4, the TSE block is a redesigned SE Net (Squeeze and excitation network) suitable for 1D CNN-based text modeling. SE Net refers to a network that extracts only important information from each channel and readjusts it according to dependencies between channels. Similarly, the TSE block extracts features for each channel from the input text data and calculates an attention score, which is a weight for each channel. The attention score is defined according to Equation 1 below.

[Equation 1]

에서 채널 별 공간 정보를 요약하는 연산을 의미한다. 이는 SE Net의 GAP로부터 변경된 공식으로 아래 수학식 2로 정의된다.Global Average Polling (GAP) in the TSE block is the feature map output of the 1D CNN

It means an operation summarizing spatial information for each channel in . This is a formula modified from the GAP of SE Net and is defined as Equation 2 below.

[Equation 2]

의 출력에 완전 연결 계층(Fully Connected Layer)의

를 한번 곱하고 시그모이드 함수를 활성화하여 계산된다. 그리고 입력

의 채널 별 특징맵에 어텐션 스코어를 곱하는 과정은 SE Net과 동일하게 수행된다. 공간 정보를 조정하는 대신 채널 단위 정보를 조정하는 이 과정은 건물 하자에 대한 텍스트 데이터와 같이 길이가 짧은 텍스트의 특징을 효율적으로 추출하는데 영향을 미치게 된다.The calculation process of the TSE block corresponding to the activation of the SE Net is changed to directly calculate the relationship per channel without bottlenecks. Eliminating the bottleneck means omitting the process of projecting features into a low-dimensional space and remapping them. That is, in the case of the conventional SE Net, low-dimensional mapping is performed once more to reduce the cost of image processing, but the TSE block used in the disclosed technology is basically similar to the SE Net, but by removing the bottleneck section in the SE Net, a short It is changed to a form suitable for extracting features of text of length. According to this change, the TSE block does not lose the connection relationship with the input value, so that the attention score of the TSE block can indicate clear channel-specific relationship information. TSE's attention score

The output of the Fully Connected Layer

It is calculated by multiplying once and activating the sigmoid function. and enter

The process of multiplying the feature map of each channel by the attention score is performed in the same way as SE Net. This process of adjusting channel-level information instead of adjusting spatial information has an effect on efficiently extracting features of short texts such as text data about building defects.

이며,

은 문장의 길이를 의미한다. 그리고

는 단어 임베딩 벡터의 차원을 의미한다. 모델에 입력할 데이터를 구성하기 위해 사전에 훈련된 임베딩 행렬을 조회하고 텍스트의 정수로 인코딩된 단어 토큰을 실제 값인 임베딩 벡터로 변환할 수 있다. 즉, 고유한 단어 토큰의 고차원 행렬을 사용하는 대신 저차원의 임베딩 값인

를 사용한다. 그리고 하나의 임베딩 모델이 아닌 서로 다른 복수의 임베딩 모델을 이용할 수 있다. 이러한 모델은 임베딩을 위한 행렬 형태일 수 있으며 서로 다른 두 모델을 함께 이용하여 텍스트를 임베딩할 수 있다. 예컨대, Word2vec과 FastText를 함께 이용하여 텍스트를 임베딩할 수 있다. Word2vec과 FastText에 따라 AutoDefect 모델의 입력 차원은 300차원 임베딩 벡터를 더한 600이 되는데, 이와 같이 큰 단어 임베딩 데이터는 입력 정보가 짧은 텍스트 길이에는 충분하지 않기 때문에 유용할 수 있으며 하나의 행렬에 임베딩 벡터가 없는 특정 단어에 대한 임베딩 정보를 다른 행렬을 통해 획득하는 것이 가능하다.On the other hand, the size of the input dimension of the AutoDefect model is

is,

represents the length of the sentence. and

denotes the dimension of the word embedding vector. To construct the data to be input to the model, we can look up a pre-trained embedding matrix and convert word tokens encoded as integers in the text into real-valued embedding vectors. That is, instead of using a high-dimensional matrix of unique word tokens, a low-dimensional embedding value

Use In addition, a plurality of different embedding models may be used instead of one embedding model. This model may be in the form of a matrix for embedding, and text may be embedded using two different models together. For example, text can be embedded using both Word2vec and FastText. According to Word2vec and FastText, the input dimension of the AutoDefect model is 600 plus the 300-dimensional embedding vector, which can be useful because large word embedding data like this is not sufficient for short text lengths, and embedding vectors in one matrix It is possible to obtain embedding information for a specific word that does not exist through another matrix.

5 is a diagram illustrating a process of processing embedded vectors in parallel according to an embodiment of the disclosed technology. Referring to FIG. 5 , the shared feature encoder of the AutoDefect model may be composed of three convolutional layers connected in parallel. Each convolutional layer contains filters of different sizes. Various shared features can be extracted by learning local correlations of different spatial sizes through filters of different sizes. The features extracted as shown in FIG. 5 are data of different lengths of 3-grams, 4-grams, and 5-grams, and mean relational text information compressed between word tokens. The features primarily extracted in this way can be input to the next convolution layer and extracted as high-level features.

6 is a diagram showing a detailed structure of an encoder. As shown in FIG. 6, the encoder has a structure in which convolution blocks are repeated, and each convolution block is composed of a convolution layer, a batch normalization layer, a PReLU layer, and a TSE block.

As shown in FIG. 6, a convolution block using a filter having a size of 5 among several filters is used, but batch normalization and a learnable activation function can be used to increase learning accuracy. In addition, in order to prevent loss of the gradient value, the attention score may be multiplied by the feature map for each channel output from the convolution layer according to the TSE block. The TSE block multiplies the result of the squeeze operation through the global pooling layer once by the parameters of the fully connected layer to perform the excitation operation, and the excitation operation result is sigmoid ( Sigmoid) function can be applied to calculate the attention score.

Meanwhile, according to the calculation of the TSE block, a feature having a high attention score may be reflected in a feature map in which important common patterns affecting the four tasks are learned. As a result, since the expressiveness of the feature map for each channel is strengthened, it is possible to extract features suitable for the work.

에서 추출된 특징의 채널 별 연결이다. 이는 공유 특징 인코더의 서로 다른 복수의 필터로부터 추출된 특징을 채널 별로 합산한 것을 의미한다. 이 입력은 작업 특화 특징 인코더의 4개의 분할된 병렬 컨볼루션 블록에 입력되어 4개의 작업에 적합한 특징을 추출하는데 이용된다.On the other hand, the input of the task-specific feature encoder is

It is a channel-specific connection of features extracted from . This means that features extracted from a plurality of different filters of the shared feature encoder are summed for each channel. This input is input to the four divided parallel convolution blocks of the task-specific feature encoder and used to extract features suitable for the four tasks.

The four task characteristics extracted in this way are input to a classifier, and a multitasking process of classifying four task-specific categories may be performed. In order to output categories classified for each task, only the most important features per channel among the features extracted for each task can be selected through Global Max Polling (GMP). And it is connected to the dense layer and can calculate the classification probability for each task through soft max activation. In this way, the AutoDefect model can classify tasks through multitasking, outputting the results of four tasks simultaneously.

The method and apparatus for classifying building defects using multi-working channel attention according to an embodiment of the disclosed technology have been described with reference to the embodiments shown in the drawings to aid understanding, but this is only exemplary, and conventional in the art. Those skilled in the art will understand that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true technical scope of protection of the disclosed technology should be defined by the appended claims.

Claims (14)

receiving text data about building defects by an analysis device;
receiving the text data and extracting a plurality of shared features by a plurality of first encoders having filters of different sizes;
extracting, by a plurality of second encoders, a plurality of working features by receiving values obtained by combining the features output by the first encoders; and
A method of automatically classifying building defects comprising a plurality of classifiers receiving values output from each of the plurality of task-specific feature encoders and outputting classification values for different items, respectively. According to claim 1,
The building defect classification method comprising the analysis device to embed the text data using two different embedding models. According to claim 1,
The plurality of first encoders each include one or more structures in which a convolution layer and an attention layer are repeated, and the convolution layers have different filter sizes. According to claim 1,
The plurality of second encoders each include one or more structures in which a convolution layer and an attention layer are repeated, and the convolution layers have the same filter size. According to claim 3,
The attention layer includes a Global Average Pooling (GAP) layer and a Fully Connected layer,
Performing an excitation operation by multiplying a result value obtained by performing a squeeze operation through the global pooling layer by a parameter of the fully connected layer once;
A building defect classification method for calculating an attention score by applying a sigmoid function to a result of performing the actuation operation. According to claim 1,
Each of the plurality of first encoders calculates an attention score of the shared feature using a Text Squeeze and Excitation (TSE) block,
The building defect classification method of claim 1 , wherein each of the plurality of second encoders calculates an attention score of the task feature using a TSE block. According to claim 1,
The plurality of classifiers respectively output the type of work according to the defect, the location of the defect, the type of defect, and the property of the defect as the classification value. an input device for receiving text data about building defects;
a storage device for storing deep learning models including a plurality of first encoders having filters of different sizes, a plurality of second encoders, and a plurality of classifiers; and
Inputting the text data to the plurality of first encoders to extract a plurality of different shared features, respectively, and inputting a value obtained by combining the plurality of shared features to the plurality of second encoders to extract a plurality of working features, respectively; and an arithmetic unit inputting the plurality of work characteristics to the plurality of classifiers and outputting classification values for different items, respectively. According to claim 8,
The deep learning model further includes two different embedding models, and the building defect classification apparatus including embedding the text data using the two embedding models. According to claim 8,
The plurality of first encoders each include one or more structures in which a convolution layer and an attention layer are repeated, and the convolution layers have different filter sizes. According to claim 8,
The plurality of second encoders each include one or more structures in which a convolution layer and an attention layer are repeated, and the convolution layer has the same filter size. According to claim 10,
The attention layer includes a Global Average Pooling (GAP) layer and a Fully Connected layer,
Performing an excitation operation by multiplying a result value obtained by performing a squeeze operation through the global pooling layer by a parameter of the fully connected layer once;
A building defect classification device that calculates an attention score by applying a sigmoid function to a result of the execution of the actuation operation. According to claim 8,
Each of the plurality of first encoders calculates an attention score of the shared feature using a Text Squeeze and Excitation (TSE) block,
The plurality of second encoders each calculate an attention score of the work feature using a TSE block. According to claim 8,
The plurality of classifiers respectively output the type of work according to the defect, the location of the defect, the type of defect, and the property of the defect as the classification value.

Images