KR102544700B1 - Mehtod and apparatus for detecting object contained within paragraph - Google Patents

Abstract

A method and apparatus for detecting an object in a paragraph are disclosed. A method for detecting an object in a paragraph according to an embodiment uses a Recurrent Neural Network (RNN) encoder to generate a hidden state vector for each of a plurality of tokens included in a token sequence generated based on an input paragraph. creating a vector); generating a self-attention vector for each of the plurality of tokens based on the hidden state vector for each of the plurality of tokens; and detecting one or more target objects included in the input paragraph from the self attention vector for each of the plurality of tokens by using a forward pointer network decoder and a backward pointer network decoder. includes

Description

Method and device for detecting objects within a paragraph {MEHTOD AND APPARATUS FOR DETECTING OBJECT CONTAINED WITHIN PARAGRAPH}

The disclosed embodiments relate to techniques for detecting objects within paragraphs.

Recently, research on question-response systems has been actively conducted. A machine reading comprehension system is one of the representative tasks of a question-response system, and is a system that finds the correct answer to a question in a given document. Since these machine reading comprehension systems collect data through tasks such as understanding documents and writing questions, high costs are required to build data. Recently, in order to alleviate this cost problem, a model for automatically generating questions using deep learning is being studied. The automatic question generation model generates questions suitable for correct answers based on paragraphs and correct answer information. However, since it requires high cost and time to manually extract correct answer information from paragraphs, a method for automatically extracting correct answer candidates from paragraphs is required.

The disclosed embodiments are intended to provide a method and apparatus for detecting an object in a paragraph.

A method for detecting an object in a paragraph according to an embodiment uses a Recurrent Neural Network (RNN) encoder to generate a hidden state vector for each of a plurality of tokens included in a token sequence generated based on an input paragraph. creating a vector); generating a self-attention vector for each of the plurality of tokens based on the hidden state vector for each of the plurality of tokens; and detecting one or more target objects included in the input paragraph from the self attention vector for each of the plurality of tokens by using a forward pointer network decoder and a backward pointer network decoder. includes

The forward pointer network decoder, when a self attention vector for the t token (where t is a natural number of 0≤t≤n, and n is the total number of the plurality of tokens) among the plurality of tokens is input, the Among one or more target objects, a first target object having the t-th token as a starting position is detected, and the reverse pointer network decoder, when a self attention vector for the t-th token is input, among the one or more target objects A second target object having the t-th token as an end position may be detected.

The forward pointer network decoder calculates a first score for each of the plurality of tokens based on a self attention vector for the t-th token and a position vector corresponding to a position of the t-th token in the token sequence; A token having the highest calculated first score is determined as an end position of the first target object, and the reverse pointer network decoder determines the plurality of values based on a self-attention vector for the t-th token and the position vector. A second score for each token may be calculated, and a token having the highest calculated second score among the plurality of tokens may be determined as a start position of the second target object.

The token sequence includes a token generated by tokenizing the input paragraph and a predetermined token not included in the input paragraph, and in the detecting step, the first score is the highest among the plurality of tokens. When the high token is the preset token, it is determined that the first target object does not exist, and when the token having the highest second score among the plurality of tokens is the preset token, the second target object It can be determined that it does not exist.

The generating of the hidden state vector may include generating the token sequence by tokenizing the input paragraph in units of predetermined tokens; generating a token vector for each of the plurality of tokens; and generating a hidden state vector for each of the plurality of tokens by using the token vector for each of the plurality of tokens as an input of the RNN encoder.

The operation of generating the token sequence may tokenize the input paragraph in units of morphemes.

The operation of generating a token vector for each of the plurality of tokens may include generating a token vector for each of the plurality of tokens based on an embedding vector for each of the plurality of tokens and a feature vector for each of the plurality of tokens. can create

The quality of each of the plurality of tokens may include parts of speech of each of the plurality of tokens.

The one or more target objects may be correct answer candidates for a query to be generated based on the input paragraph.

The RNN encoder may be a bidirectional RNN encoder.

An apparatus for detecting an object within a paragraph according to an embodiment includes a memory storing one or more commands; And one or more processors that execute the one or more instructions, wherein the one or more processors use a Recurrent Neural Network (RNN) encoder to generate a plurality of tokens included in a token sequence generated based on an input paragraph. generating a hidden state vector for each of the plurality of tokens, generating a self-attention vector for each of the plurality of tokens based on the hidden state vector for each of the plurality of tokens, and forward One or more target objects included in the input paragraph are detected from the self-attention vector for each of the plurality of tokens using a forward pointer network decoder and a backward pointer network decoder.

The forward pointer network decoder, when a self attention vector for the t token (where t is a natural number of 0≤t≤n, and n is the total number of the plurality of tokens) among the plurality of tokens is input, the Among one or more target objects, a first target object having the t-th token as a starting position is detected, and the reverse pointer network decoder, when a self attention vector for the t-th token is input, among the one or more target objects A second target object having the t-th token as an end position may be detected.

The forward pointer network decoder calculates a first score for each of the plurality of tokens based on a self attention vector for the t-th token and a position vector corresponding to a position of the t-th token in the token sequence; A token having the highest calculated first score is determined as an end position of the first target object, and the reverse pointer network decoder determines the plurality of values based on a self-attention vector for the t-th token and the position vector. A second score for each token may be calculated, and a token having the highest calculated second score among the plurality of tokens may be determined as a start position of the second target object.

The token sequence includes a token generated by tokenizing the input paragraph and a predetermined token not included in the input paragraph, and the one or more processors determine that the first score is the highest among the plurality of tokens. When the high token is the preset token, it is determined that the first target object does not exist, and when the token having the highest second score among the plurality of tokens is the preset token, the second target object It can be determined that it does not exist.

The one or more processors tokenize the input paragraph in predetermined token units to generate the token sequence, generate a token vector for each of the plurality of tokens, and convert the token vector for each of the plurality of tokens to the RNN. A hidden state vector for each of the plurality of tokens may be generated by using it as an input of an encoder.

The one or more processors may tokenize the input paragraph in units of morphemes.

The one or more processors may generate a token vector for each of the plurality of tokens based on an embedding vector for each of the plurality of tokens and a feature vector for each of the plurality of tokens.

The quality of each of the plurality of tokens may include parts of speech of each of the plurality of tokens.

The one or more target objects may be correct answer candidates for a query to be generated based on the input paragraph.

The RNN encoder may be a bidirectional RNN encoder.

According to the disclosed embodiments, by using a forward pointer network decoder and a backward pointer network decoder together to extract a target object from an input paragraph, when a plurality of target objects having the same start position or end position are included in the input paragraph It also enables effective target object detection.

1 is a block diagram of an object detection device according to an embodiment.
2 is a diagram showing the configuration of a target object detection model according to an embodiment.
3 is a diagram showing the configuration of an RNN encoder according to an embodiment.
4 is a diagram showing configurations of a forward pointer network decoder and a backward pointer network decoder according to an embodiment.
5 is an exemplary view illustrating an example of target object detection according to an exemplary embodiment.
6 is a flowchart of an object detection method according to an embodiment.
7 is a flowchart illustrating a method for generating a hidden state vector according to an embodiment.
8 is a block diagram illustrating a computing environment including a computing device according to an exemplary embodiment.

Hereinafter, specific embodiments will be described with reference to the drawings. The detailed descriptions that follow are provided to provide a comprehensive understanding of the methods, devices and/or systems described herein. However, this is only an example and the present invention is not limited thereto.

In describing the embodiments, if it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the contents throughout this specification. Terminology used in the detailed description is only for describing the embodiments and should in no way be limiting. Unless expressly used otherwise, singular forms of expression include plural forms. In this description, expressions such as "comprising" or "comprising" are intended to indicate any characteristic, number, step, operation, element, portion or combination thereof, one or more other than those described. It should not be construed to exclude the existence or possibility of any other feature, number, step, operation, element, part or combination thereof.

1 is a block diagram of an object detection device according to an embodiment.

Referring to FIG. 1 , an object detection apparatus 100 according to an embodiment includes a tokenization unit 110 , an embedding unit 120 and a detection unit 130 .

In one embodiment, the tokenization unit 110, the embedding unit 120, and the detection unit 130 are implemented using one or more physically separated devices, or implemented by one or more processors or a combination of one or more processors and software. It may be, and unlike the illustrated example, it may not be clearly distinguished in specific operations.

The tokenization unit 110 tokenizes the input paragraph in units of predetermined tokens, and generates a token sequence including a plurality of tokens generated through tokenization.

In this case, the input paragraph may be composed of one or more natural language sentences.

According to an embodiment, the tokenization unit 110 may tokenize the input paragraph by dividing the input paragraph into morpheme units through morpheme analysis of the input paragraph. However, a token unit for tokenizing an input paragraph may be a word unit or a word unit in addition to a morpheme unit, and may be changed according to embodiments.

Meanwhile, the token sequence means that a plurality of tokens generated through tokenization of an input paragraph are arranged in the order of positions within the input paragraph. According to an embodiment, the token sequence may include one or more preset tokens in addition to tokens generated by tokenizing an input paragraph.

Specifically, according to an embodiment, the token sequence may include a start token indicating the start of an input paragraph and an end token indicating the end of an input paragraph.

In addition, according to one embodiment, the token sequence is a token (hereinafter referred to as "no_answer token referred to as ").

The embedding unit 120 generates a token vector for each token included in the token sequence generated by the tokenization unit 110 .

According to an embodiment, the embedding unit 120 generates a token vector for each token by concatenating an embedding vector for each token included in the token sequence and a feature vector for each token. can

In this case, the embedding vector and feature vector for each token may be generated using a known word embedding technology such as Word2Vec, GloVe, FastText, and the like.

On the other hand, according to an embodiment, the quality of each token may be, for example, a part of speech tagged to each token through morphological analysis, and according to the embodiment, location information corresponding to a position of each token in a token sequence may be further included. can do.

The detection unit 130 detects one or more target objects included in the input paragraph using a target object detection model.

According to an embodiment, each target object included in the input paragraph may be a candidate for a correct answer to a query to be generated based on the input paragraph. For example, an input paragraph in which a target object is detected may be used as an input of a query generation model together with the detected target object, and in this case, the query generation model may generate a question suitable for the input paragraph and the target object. .

On the other hand, the target object detection model is an artificial neural network-based model that receives a token vector for each of a plurality of tokens included in a token sequence, and the start position and end position of each of one or more target objects included in the input paragraph. can be pretrained to detect

2 is a diagram showing the configuration of a target object detection model according to an embodiment.

Referring to FIG. 2 , an object detection model 100 according to an embodiment includes an encoding layer 210, an attention layer 220, and a decoding layer 230. do.

The encoding layer 210 may include a Recurrent Neural Network (RNN) encoder that encodes a token vector for each of a plurality of tokens included in the token sequence.

According to an embodiment, the RNN encoder may be, for example, a bidirectional RNN (RNN) encoder implemented using a gated recurrent unit (GRU) or long short-term memory (LSTM).

Specifically, FIG. 3 is a diagram showing the configuration of an RNN encoder according to an embodiment.

는 아래의 수학식 1과 같이 t-1번째 토큰의 토큰 벡터

에 대한 순방향 RNN(211)의 은닉 상태 벡터

및 t번째 토큰의 토큰 벡터

를 이용하여 생성될 수 있다.Referring to FIG. 3, the forward RNN 211 included in the encoding layer 210 receives a token vector for each of a plurality of tokens included in the token sequence in order within the token sequence, and generates a hidden state vector for each of the plurality of tokens. (hidden state vector) is created. Specifically, the hidden state vector of the forward RNN 211 for the tth token in the token sequence (where t is a natural number of 0≤t≤n, and n is the total number of tokens included in the token sequence)

Is the token vector of the t-1th token as shown in Equation 1 below

Hidden state vector of forward RNN (211) for

and the token vector of the tth token

can be created using

[Equation 1]

In Equation 1, f represents a non-linear active function, and may be, for example, Long-Short Term Memory (LSTM) or Gated Recurrent Unit (GRU). Hereinafter, f is used with the same meaning.

는 아래의 수학식 2와 같이 t+1번째 토큰의 토큰 벡터

에 대한 역방향 RNN(212)의 은닉 상태 벡터

및 t번째 토큰의 토큰 벡터

를 이용하여 생성될 수 있다.Meanwhile, the reverse RNN 212 included in the encoding layer 210 receives a token vector for each of a plurality of tokens included in the token sequence in the reverse order of the order in the token sequence, and generates a hidden state vector for each of the plurality of tokens do. Specifically, the hidden state vector of the reverse RNN 212 for the tth token in the token sequence

is the token vector of the t+1th token as shown in Equation 2 below

Hidden state vector of reverse RNN (212) for

and the token vector of the tth token

can be created using

[Equation 2]

와 역방향 RNN(212)의 은닉 상태 벡터

를 연결(concatenate)한 벡터

를 토큰 시퀀스에 포함된 t번째 토큰에 대한 은닉 상태 벡터로 출력할 수 있다.On the other hand, the encoding layer 210 is the hidden state vector of the forward RNN 211 as shown in Equation 3 below

and the hidden state vector of the reverse RNN (212)

vector concatenated

can be output as a hidden state vector for the tth token included in the token sequence.

[Equation 3]

에 대한 자가 주의 집중(self-attention) 벡터를 생성한다.Referring back to FIG. 2, the attentional focus layer 220 generates each hidden state vector generated by the encoding layer 210

Create a self-attention vector for .

에 대한 자가 주의 집중 벡터

는 아래의 수학식 4에 기초하여 산출될 수 있다.According to one embodiment, the hidden state vector

Self attention vector for

Can be calculated based on Equation 4 below.

[Equation 4]

이고,

는

의 차원(dimension)을 나타낸다. 이하, Q, K, V 및

는 동일한 의미로 이용된다.At this time,

ego,

Is

represents the dimension of Hereinafter, Q, K, V and

is used with the same meaning.

에 대한 자가 주의 집중 벡터

는 멀티-헤드 주의 집중(multi-head attention) 기법을 통해 결정될 수 있다. 구체적으로, 멀티-헤드 주의 집중 기법을 이용한 자가 주의 집중 벡터는 아래의 수학식 5에 기초하여 산출될 수 있다.On the other hand, according to an embodiment, the hidden state vector

Self attention vector for

may be determined through a multi-head attention technique. Specifically, the self-attention vector using the multi-head attention-focusing technique can be calculated based on Equation 5 below.

[Equation 5]

At this time, N represents a preset value as the number of heads, and W ^c represents a learned weight matrix.

는 아래의 수학식 6을 이용하여 산출될 수 있다.On the other hand, in Equation 5

Can be calculated using Equation 6 below.

[Equation 6]

,

및

를 곱함으로써 산출될 수 있다.In addition, in Equation 6, Q _i , K _{i ,} and V _i are learned weight matrices after dividing Q i , K i , and V i into N heads, respectively, as shown in Equations 7 to 9 below.

,

and

It can be calculated by multiplying

[Equation 7]

[Equation 8]

[Equation 9]

로부터 입력 문단에 포함된 하나 이상의 대상 객체를 탐지한다.The decoding layer 230 uses a forward pointer network decoder and a backward pointer network decoder to generate the self attention vector generated by the attention layer 220.

Detect one or more target objects included in the input paragraph from

In this case, according to an embodiment, one or more target objects included in the input paragraph may be correct answer candidates for a query to be generated based on the input paragraph, as described above.

4 is a diagram showing configurations of a forward pointer network decoder and a backward pointer network decoder according to an embodiment.

에 대한 주의 집중 벡터

에 기초하여, 입력 문단에 포함된 하나 이상의 대상 객체 중 t번째 토큰을 시작 위치로 하는 대상 객체를 탐지할 수 있다.Referring to FIG. 4, the forward pointer network decoder 231 calculates the hidden state vector of the t-th token included in the token sequence.

Attention vector for

Based on , a target object having the tth token as a starting position among one or more target objects included in the input paragraph may be detected.

및 t번째 토큰의 토큰 시퀀스 내 위치인 t에 대응되는 위치 벡터

에 기초하여 토큰 시퀀스에 포함된 복수의 토큰 각각에 대한 스코어를 산출할 수 있다. 이때, 위치 벡터

는 토큰의 위치에 따라 특정한 값을 가지도록 사전 설정된 벡터일 수 있다.Specifically, the forward pointer network decoder 231 generates an attention vector corresponding to the t-th token included in the token sequence.

and a position vector corresponding to t, the position in the token sequence of the tth token.

It is possible to calculate a score for each of a plurality of tokens included in the token sequence based on. In this case, the position vector

may be a vector preset to have a specific value according to the position of the token.

In addition, the forward pointer network decoder 231 may determine a token having the highest calculated score among a plurality of tokens included in the token sequence as an end position of the target object having the tth token as a start position.

는 아래의 수학식 10를 이용하여 산출될 수 있다.Specifically, the hidden state vector of the forward pointer network decoder 231

Can be calculated using Equation 10 below.

[Equation 10]

In addition, scores for each of a plurality of tokens included in the token sequence may be calculated using Equations 11 and 12 below.

[Equation 11]

[Equation 12]

는 스코어 함수를 나타내며, 이하 동일한 의미로 이용된다. 스코어 함수

는 예를 들어, tanh 함수를 활성화 함수로 이용하는 앞 먹임 신경망(Feed-Forward Neural Network, FNN)로 구현될 수 있으며, 이 경우, 수학식 11은 아래의 수학식 13와 같이 표현될 수 있다. On the other hand, in Equation 11

denotes a score function, and is used in the same meaning below. score function

Can be implemented as, for example, a Feed-Forward Neural Network (FNN) using the tanh function as an activation function, and in this case, Equation 11 can be expressed as Equation 13 below.

[Equation 13]

및

는 각각 학습된 가중치 행렬을 나타낸다.At this time,

and

denotes a learned weight matrix, respectively.

Meanwhile, the forward pointer network decoder 231 selects a token having the maximum score calculated according to Equations 11 and 12 among a plurality of tokens included in the token sequence as the end position of the target object having the t-th token as the start position. can decide

On the other hand, according to an embodiment, as described above, when the no_answer token is included in the token sequence and the score of the no_answer token is the highest among the scores calculated by the forward pointer network decoder 231, the detection unit 120 It may be determined that the input paragraph does not include a target object starting at the tth token.

에 대한 주의 집중 벡터

에 기초하여, 입력 문단에 포함된 하나 이상의 대상 객체 중 토큰 시퀀스에 포함된 t번째 토큰을 종료 위치로 하는 대상 객체를 탐지할 수 있다.The reverse pointer network decoder 232 uses the hidden state vector of the tth token

Attention vector for

Based on , a target object having the tth token included in the token sequence as an end position among one or more target objects included in the input paragraph may be detected.

및 t번째 토큰의 토큰 시퀀스 내 위치인 t에 대응되는 위치 벡터

에 기초하여 토큰 시퀀스에 포함된 복수의 토큰 각각에 대한 스코어를 산출할 수 있다. Specifically, the reverse pointer network decoder 232 generates an attention vector corresponding to the tth token included in the token sequence.

and a position vector corresponding to t, the position in the token sequence of the tth token.

It is possible to calculate a score for each of a plurality of tokens included in the token sequence based on.

In addition, the reverse pointer network decoder 232 may determine a token having the highest calculated score among a plurality of tokens included in the token sequence as the start position of the target object having the t-th token as the end position.

는 아래의 수학식 14를 이용하여 산출될 수 있다.Specifically, the hidden state vector of the reverse pointer network decoder 232

Can be calculated using Equation 14 below.

[Equation 14]

In addition, scores for each of a plurality of tokens included in the token sequence may be calculated using Equations 15 and 16 below.

[Equation 15]

[Equation 16]

가 예를 들어, tanh 함수를 활성화 함수로 이용하는 FNN으로 구현된 경우, 수학식 15는 아래의 수학식 17와 같이 표현될 수 있다. On the other hand, the score function

When is implemented as an FNN using, for example, the tanh function as an activation function, Equation 15 may be expressed as Equation 17 below.

[Equation 17]

The reverse pointer network decoder 232 determines the token having the maximum score calculated according to Equations 15 and 16 among the plurality of tokens included in the token sequence as the start position of the target object having the t-th token as the end position. there is.

On the other hand, according to an embodiment, as described above, when the no_answer token is included in the token sequence and the score of the no_answer token is the highest among the scores calculated by the reverse pointer network decoder 232, the detection unit 120 It can be determined that the input paragraph does not include the target object whose end position is the t-th token.

5 is an exemplary view illustrating an example of target object detection according to an exemplary embodiment.

In FIG. 5, it is assumed that the input paragraph is composed of one sentence “Sejong is the fourth monarch of Joseon and a linguist.” It is assumed that he is an overlord and a linguist".

In this case, "four monarchs" and "linguist" are target objects included in "four monarchs and linguists", respectively.

In this case, the detection unit 130 uses the forward pointer network decoder 231 as shown in FIG. It is a monarch and can detect the target object "linguist" with "linguist" and "language" as the starting position and "scholar" as the ending position.

That is, in the case of using only the forward pointer network decoder 231, only one of the target objects "four monarchs" and "four monarchs and linguists" having the same start position can be detected.

On the other hand, the detection unit 130 uses the reverse pointer network decoder 232 as shown in FIG. It is possible to detect the target object "four rulers and linguists" with " and "scholar" as the end position and "four generations" as the start position.

That is, in the case of using only the reverse pointer network decoder 232, only one of the target objects "linguist" and "four monarchs and linguists" having the same end position can be detected.

As can be seen from the example shown in FIG. 5, when only one of the forward pointer network decoder 231 and the reverse pointer network decoder 232 is used, all of a plurality of target objects having the same start position or end position cannot be detected. In the case of using both the forward pointer network decoder 231 and the reverse pointer network decoder 232, it is possible to detect both "four monarchs", linguists and "four monarchs and linguists".

6 is a flowchart of an object detection method according to an embodiment.

The method illustrated in FIG. 6 may be performed by, for example, the object detection apparatus 100 illustrated in FIG. 1 .

Referring to FIG. 6 , the object detection apparatus 100 first generates a hidden state vector for each of a plurality of tokens included in a token sequence generated based on an input paragraph using an RNN encoder (610).

In this case, according to an embodiment, the RNN encoder may be a bidirectional RNN encoder.

Thereafter, the object detection apparatus 100 generates a self attention vector for each of the plurality of tokens based on the concealment state vector for each of the plurality of tokens (620).

Thereafter, the object detection apparatus 100 detects one or more target objects included in the input paragraph from the self-attention vector for each of a plurality of tokens included in the token sequence by using the forward pointer network decoder and the reverse pointer network decoder. (630).

In this case, according to an embodiment, the forward pointer network decoder selects the first target object having the t-th token as a starting position when the self attention vector for the t-th token among the plurality of tokens included in the token sequence is input. can detect Specifically, the forward pointer network decoder calculates a score for each of a plurality of tokens based on a self-attention vector for the t-th token and a position vector corresponding to a position of the t-th token in the token sequence, and the calculated score is The highest token may be determined as an end position of the first target object.

In addition, according to an embodiment, the reverse pointer network decoder, when the self attention vector for the t-th token among the plurality of tokens included in the token sequence is input, selects the second target object having the t-th token as an end position can detect Specifically, the reverse pointer network decoder calculates a score for each of a plurality of tokens based on a self-attention vector for the t-th token and a position vector corresponding to a position of the t-th token in the token sequence, and the calculated score is The highest token may be determined as the start position of the second target object.

Meanwhile, according to an embodiment, the token sequence may include a no_answer token. In this case, the object detection apparatus 100 may determine that the first target object does not exist when a token having the highest score calculated by the forward pointer network decoder among a plurality of tokens included in the token sequence is the no_answer token. there is. In addition, the object detection apparatus 100 may determine that the second target object does not exist when a token having the highest score calculated by the reverse pointer network decoder among a plurality of tokens included in the token sequence is a no_answer token. .

Meanwhile, in the flowchart shown in FIG. 5, at least some of the steps may be combined with other steps and performed together, omitted, divided into detailed steps, or performed by adding one or more steps not shown.

7 is a flowchart illustrating a method for generating a hidden state vector according to an embodiment.

The method illustrated in FIG. 7 may be performed by, for example, the object detection apparatus 100 illustrated in FIG. 1 .

Referring to FIG. 7 , first, the object detection apparatus 100 tokenizes an input paragraph in units of preset tokens to generate a token sequence (710).

In this case, according to an embodiment, the object detection apparatus 100 may tokenize the input paragraph in units of morphemes.

Thereafter, the object detection apparatus 100 generates a token vector for each of a plurality of tokens included in the token sequence (720).

According to an embodiment, the object detection apparatus 100 may generate a token vector for each of a plurality of tokens based on an embedding vector for each of the plurality of tokens and a feature vector for each of the plurality of tokens. In this case, the quality of each of the plurality of tokens may include parts of speech of each of the plurality of tokens.

Thereafter, the object detection apparatus 100 generates a hidden state vector for each of the plurality of tokens by using the token vector for each of the plurality of tokens as an input of the RNN encoder (730).

8 is a block diagram illustrating a computing environment including a computing device according to an exemplary embodiment.

In the embodiment shown in FIG. 8 , each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.

The illustrated computing environment 10 includes a computing device 12 . In one embodiment, computing device 12 may be one or more components included in object detection device 100 shown in FIG. 1 .

Computing device 12 includes at least one processor 14 , a computer readable storage medium 16 and a communication bus 18 . Processor 14 may cause computing device 12 to operate according to the above-mentioned example embodiments. For example, processor 14 may execute one or more programs stored on computer readable storage medium 16 . The one or more programs may include one or more computer-executable instructions, which when executed by processor 14 are configured to cause computing device 12 to perform operations in accordance with an illustrative embodiment. It can be.

Computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. Program 20 stored on computer readable storage medium 16 includes a set of instructions executable by processor 14 . In one embodiment, computer readable storage medium 16 includes memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, other forms of storage media that can be accessed by computing device 12 and store desired information, or any suitable combination thereof.

Communications bus 18 interconnects various other components of computing device 12, including processor 14 and computer-readable storage medium 16.

Computing device 12 may also include one or more input/output interfaces 22 and one or more network communication interfaces 26 that provide interfaces for one or more input/output devices 24 . An input/output interface 22 and a network communication interface 26 are connected to the communication bus 18 . Input/output device 24 may be coupled to other components of computing device 12 via input/output interface 22 . Exemplary input/output devices 24 include a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices, and/or a photographing device. input devices, and/or output devices such as display devices, printers, speakers, and/or network cards. The exemplary input/output device 24 may be included inside the computing device 12 as a component constituting the computing device 12, or may be connected to the computing device 12 as a separate device distinct from the computing device 12. may be

Although the present invention has been described in detail through representative examples above, those skilled in the art can make various modifications to the above-described embodiments without departing from the scope of the present invention. will understand Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, and should be defined by not only the claims to be described later, but also those equivalent to these claims.

10: Computing environment
12: computing device
14: Processor
16: computer readable storage medium
18: communication bus
20: program
22: I/O interface
24: I/O device
26: network communication interface
100: object detection device
110: tokenization unit
120: embedding unit
130: detection unit
200: target object detection model
210: encoding layer
220: Attentional Hierarchy
230: decoding layer
211: forward RNN
212: reverse RNN
231: forward pointer network decoder
232: reverse pointer network decoder

Claims (20)

generating a hidden state vector for each of a plurality of tokens included in a token sequence generated based on an input paragraph by using a Recurrent Neural Network (RNN) encoder;
generating a self-attention vector for each of the plurality of tokens based on the hidden state vector for each of the plurality of tokens; and
An operation of detecting one or more target objects included in the input paragraph from the self attention vector for each of the plurality of tokens using a forward pointer network decoder and a backward pointer network decoder. include,
The forward pointer network decoder, when a self attention vector for the t token (where t is a natural number of 0≤t≤n, and n is the total number of the plurality of tokens) among the plurality of tokens is input, the Detecting a first target object having the t-th token as a starting position among one or more target objects;
The reverse pointer network decoder detects a second target object having the t-th token as an end position among the one or more target objects when the self attention vector for the t-th token is input. delete The method of claim 1,
The forward pointer network decoder calculates a first score for each of the plurality of tokens based on a self attention vector for the t-th token and a position vector corresponding to a position of the t-th token in the token sequence; determining a token having the highest calculated first score as an end position of the first target object;
The reverse pointer network decoder calculates a second score for each of the plurality of tokens based on the self attention vector and the position vector for the tth token, and the calculated second score among the plurality of tokens is A method of detecting an object within a paragraph for determining a highest token as a start position of the second target object. The method of claim 3,
The token sequence includes a token generated by tokenizing the input paragraph and a preset token not included in the input paragraph;
The detecting may include determining that the first target object does not exist when a token having the highest first score among the plurality of tokens is the predetermined token, and determining that the second target object among the plurality of tokens has the highest score. and determining that the second target object does not exist when the highest token is the preset token. The method of claim 1,
The generating of the hidden state vector may include generating the token sequence by tokenizing the input paragraph in units of preset tokens;
generating a token vector for each of the plurality of tokens; and
and generating a hidden state vector for each of the plurality of tokens by using the token vector for each of the plurality of tokens as an input of the RNN encoder. The method of claim 5,
The operation of generating the token sequence tokenizes the input paragraph in units of morphemes. The method of claim 5,
The operation of generating a token vector for each of the plurality of tokens may include generating a token vector for each of the plurality of tokens based on an embedding vector for each of the plurality of tokens and a feature vector for each of the plurality of tokens. How to detect objects in the paragraph you create. The method of claim 7,
The feature of each of the plurality of tokens includes a part of speech of each of the plurality of tokens. The method of claim 1,
The method of claim 1 , wherein the at least one target object is an answer candidate for a query to be generated based on the input paragraph. The method of claim 1,
The RNN encoder is a bidirectional RNN encoder. a memory that stores one or more instructions; and
comprising one or more processors to execute the one or more instructions;
The one or more processors,
Using a Recurrent Neural Network (RNN) encoder, a hidden state vector for each of a plurality of tokens included in a token sequence generated based on an input paragraph is generated,
generating a self-attention vector for each of the plurality of tokens based on the hidden state vector for each of the plurality of tokens;
Detecting one or more target objects included in the input paragraph from a self attention vector for each of the plurality of tokens using a forward pointer network decoder and a backward pointer network decoder,
The forward pointer network decoder, when a self attention vector for the t token (where t is a natural number of 0≤t≤n, and n is the total number of the plurality of tokens) among the plurality of tokens is input, the Detecting a first target object having the t-th token as a starting position among one or more target objects;
The reverse pointer network decoder detects a second target object having the t-th token as an end position among the one or more target objects when the self-attention vector for the t-th token is input. delete The method of claim 11,
The forward pointer network decoder calculates a first score for each of the plurality of tokens based on a self attention vector for the t-th token and a position vector corresponding to a position of the t-th token in the token sequence; determining a token having the highest calculated first score as an end position of the first target object;
The reverse pointer network decoder calculates a second score for each of the plurality of tokens based on the self attention vector and the position vector for the tth token, and the calculated second score among the plurality of tokens is An apparatus for detecting an object within a paragraph for determining a highest token as a start position of the second target object. The method of claim 13,
The token sequence includes a token generated by tokenizing the input paragraph and a preset token not included in the input paragraph;
The one or more processors, if the token having the highest first score among the plurality of tokens is the predetermined token, it is determined that the first target object does not exist, and the second score among the plurality of tokens is and determining that the second target object does not exist when the highest token is the preset token. The method of claim 11,
The one or more processors generate the token sequence by tokenizing the input paragraph in units of preset tokens;
generating a token vector for each of the plurality of tokens;
A device for detecting an object in a paragraph for generating a hidden state vector for each of the plurality of tokens by using the token vector for each of the plurality of tokens as an input of the RNN encoder. The method of claim 15
The one or more processors tokenize the input paragraph in units of morphemes. The method of claim 15
wherein the one or more processors generate a token vector for each of the plurality of tokens based on an embedding vector for each of the plurality of tokens and a feature vector for each of the plurality of tokens. The method of claim 17
The apparatus for detecting an object in a paragraph, wherein the qualities of each of the plurality of tokens include parts of speech of each of the plurality of tokens. The method of claim 11,
The one or more target objects are candidates for an answer to a query to be generated based on the input paragraph. The method of claim 11,
The RNN encoder is a bidirectional RNN encoder.

Images