KR102566928B1 - Electronic apparatus which generates a training set for performing reinforcement learning of the deep learning model for distinguishing user intention, and the operating method thereof - Google Patents

Abstract

Disclosed are an electronic device for generating a training set for performing reinforcement learning of a deep learning model that determines user intention and an operating method thereof. In the present invention, when a command for generating a training set for reinforcement learning of a deep learning model is applied by an administrator, a plurality of collected sentences are obtained from a plurality of pre-designated web pages, and the plurality of collected sentences are classified according to user intention. After doing so, an electronic device and method of operating the same generate a training set composed of the plurality of collected sentences and information on user intention to which each collected sentence belongs.

Description

Electronic device generating a training set for performing reinforcement learning of a deep learning model that determines user intention and its operating method OPERATING METHOD THEREOF}

The present invention relates to an electronic device for generating a training set for performing reinforcement learning of a deep learning model that determines user intention and an operating method thereof.

The ARS (Automatic Response System) system is a state in which various information is stored in voice, and when a user accesses the system through a phone call, it tells how to use it to search for necessary information by voice, and when the user finds the necessary information, It means a system that provides users with voice.

Since this ARS system can be built at a relatively low cost, it is widely used in various fields for the purpose of reducing manpower and costs when there is a need to repeatedly provide the same information to users, there is.

On the other hand, as various technologies related to voice recognition have recently developed, in the ARS system, when a user inputs a voice, a machine learning-based deep learning model is used to determine the user's intention, and from the voice input by the user , Research on technology for determining what information the user needs by determining the user's intention is being actively conducted.

In this regard, a deep learning model for determining user intention refers to a model built by performing machine learning based on pre-collected sentences and information on user intention to which each sentence belongs, and an administrator adds a training set. , it is possible to increase the accuracy of the deep learning model by performing reinforcement learning for strengthening the deep learning model based on the secured training set.

However, since it takes a lot of time and money for a manager to individually create a training set for performing reinforcement learning on a deep learning model, there is a problem in that it is difficult for a manager to efficiently secure a necessary training set.

Therefore, by acquiring a plurality of collected sentences from predetermined web pages and generating a training set for performing reinforcement learning on a deep learning model based on the acquired plurality of collected sentences, the administrator can more efficiently, deep Research on a technology that can help secure a training set for performing reinforcement learning on a learning model is needed.

In the present invention, when a command for generating a training set for reinforcement learning of a deep learning model is applied by an administrator, a plurality of collected sentences are obtained from a plurality of pre-designated web pages, and the plurality of collected sentences are classified according to user intention. After that, by presenting an electronic device and its operation method for generating a training set composed of the plurality of collected sentences and information on user intention to which each collected sentence belongs, a deep learning manager more efficiently determines user intention. We want to help you secure a training set to perform reinforcement learning on your learning model.

According to an embodiment of the present invention, an electronic device for generating a training set for performing reinforcement learning of a deep learning model for determining user intention includes a user intention storage unit storing information on a plurality of preset user intentions; An example sentence storage unit in which a plurality of pre-collected example sentences are stored, and a pre-built deep learning model - When a sentence is input, the deep learning model determines the user intention matching the input sentence among the plurality of user intentions. A determination result storage unit in which a determination result for a user's intention matched to each of the plurality of example sentences, which has been completed through execution, is stored, and a training set for reinforcement learning of the deep learning model by a manager. When the generation command is applied, a collection sentence acquisition unit that obtains a plurality of collection sentences by accessing a plurality of pre-specified web pages and performing web crawling on the plurality of web pages, and the plurality of collection sentences are obtained Then, an order assigning unit that randomly assigns an order number for classification to each of the plurality of collection sentences, and a target of classification work for one collection sentence in descending order among the plurality of collection sentences. A classification task of classifying the selected collection sentence as belonging to the user intention that matches the maximum among the plurality of user intentions based on the degree of association between the selected collection sentence and the plurality of example sentences. When the classification task performer classifies the plurality of collected sentences according to user intention by performing repetition, and the plurality of collected sentences are classified according to user intention, information about the plurality of collected sentences and the user's intention to which each collected sentence belongs. and a training set generation unit for generating a training set consisting of

In addition, a method of operating an electronic device for generating a training set for performing reinforcement learning of a deep learning model for determining user intention according to an embodiment of the present invention includes storing information about a plurality of preset user intentions. Maintaining a user intention storage unit, maintaining an example sentence storage unit in which a plurality of pre-collected example sentences are stored, a pre-built deep learning model - when a sentence is input to the deep learning model, the plurality of user intentions are stored. A model for determining the user's intention matching the input sentence of - maintaining a determination result storage unit in which the determination result for the user's intention matched with each of the plurality of example sentences, which has been completed through the process, is stored, to the manager , when a command for generating a training set for reinforcement learning of the deep learning model is applied, a plurality of collected sentences are collected by accessing a plurality of pre-designated web pages and performing web crawling on the plurality of web pages. Obtaining, if the plurality of collected sentences are acquired, randomly allocating an order number for a classification task to each of the plurality of collected sentences, in order of decreasing order among the plurality of collected sentences. A user who selects collected sentences one by one as a target sentence for classification, and matches the selected collected sentence to the maximum among the plurality of user intentions based on the degree of association between the selected collected sentence and the plurality of example sentences. Classifying the plurality of collected sentences according to user intention by repeatedly performing a classification operation of classifying them as belonging to the intention, and when the plurality of collected sentences are classified according to user intention, the plurality of collected sentences and each collected sentence and generating a training set composed of information about user intent to which it belongs.

In the present invention, when a command for generating a training set for reinforcement learning of a deep learning model is applied by an administrator, a plurality of collected sentences are obtained from a plurality of pre-designated web pages, and the plurality of collected sentences are classified according to user intention. After that, by presenting an electronic device and its operation method for generating a training set composed of the plurality of collected sentences and information on user intention to which each collected sentence belongs, a deep learning manager more efficiently determines user intention. It can assist in securing a training set for performing reinforcement learning on a learning model.

1 is a diagram showing the structure of an electronic device for generating a training set for performing reinforcement learning of a deep learning model for determining a user's intention according to an embodiment of the present invention.
2 is a flowchart illustrating an operating method of an electronic device for generating a training set for performing reinforcement learning of a deep learning model for determining a user's intention according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. This description is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. While describing each drawing, similar reference numerals have been used for similar components, and unless otherwise defined, all terms used in this specification, including technical or scientific terms, are common knowledge in the art to which the present invention belongs. has the same meaning as commonly understood by the person who has it.

In this document, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated. In addition, in various embodiments of the present invention, each component, functional block, or means may be composed of one or more sub-components, and the electrical, electronic, and mechanical functions performed by each component are electronic It may be implemented with various known elements or mechanical elements such as circuits, integrated circuits, ASICs (Application Specific Integrated Circuits), and may be implemented separately or two or more may be integrated into one.

On the other hand, the blocks of the accompanying block diagram or the steps of the flowchart are computer program instructions that perform designated functions by being loaded into a processor or memory of a device capable of data processing, such as a general-purpose computer, a special purpose computer, a portable notebook computer, and a network computer. can be interpreted as meaning Since these computer program instructions may be stored in a memory included in a computer device or in a computer readable memory, the functions described in blocks of a block diagram or steps of a flowchart are produced as a product containing instruction means for performing them. It could be. Further, each block or each step may represent a module, segment or portion of code that includes one or more executable instructions for executing specified logical function(s). Also, it should be noted that in some alternative embodiments, functions mentioned in blocks or steps may be executed out of a predetermined order. For example, two blocks or steps shown in succession may be performed substantially simultaneously or in reverse order, and in some cases, some blocks or steps may be omitted.

1 is a diagram showing the structure of an electronic device for generating a training set for performing reinforcement learning of a deep learning model for determining a user's intention according to an embodiment of the present invention.

Referring to FIG. 1 , the electronic device 110 according to the present invention includes a user intention storage unit 111, an example sentence storage unit 112, a determination result storage unit 113, a collection sentence acquisition unit 114, and order allocation. It includes a unit 115, a classification task performing unit 116, and a training set generating unit 117.

Information on a plurality of preset user intentions is stored in the user intention storage unit 111 .

In this regard, when a plurality of preset user intentions are referred to as 'User Intent 1, User Intent 2, and User Intent 3', the user intention storage unit 111 includes information about 'User Intent 1, User Intent 2, and User Intent 3'. Information may be stored.

Here, user intention means a thought or plan to do something, such as whether the user wants to talk to a counselor or whether the user wants to inquire about his or her personal information.

The example sentence storage unit 112 stores a plurality of pre-collected example sentences.

Here, the plurality of example sentences may be pre-collected sentences based on a user's voice input through an Automatic Response System (ARS) system.

In this regard, when the plurality of pre-collected example sentences are referred to as 'Example Sentence 1, Example Sentence 2, Example Sentence 3, Example Sentence 4, and Example Sentence 5', the example sentence storage unit 112 includes 'Example Sentence 1, Example Sentence Sentence 2, Example Sentence 3, Example Sentence 4, Example Sentence 5' may be stored.

The determination result storage unit 113 stores the determination result of the user's intention matched with each of the plurality of example sentences, which has been performed through a pre-built deep learning model.

Here, the deep learning model refers to a model for determining a user intention matching an input sentence among the plurality of user intentions when a sentence is input.

For example, as in the above example, when the plurality of example sentences are 'Example Sentence 1, Example Sentence 2, Example Sentence 3, Example Sentence 4, and Example Sentence 5', the determination result storage unit 113 has the following table As shown in 1, the determination result of the user's intention matching each of 'Example Sentence 1, Example Sentence 2, Example Sentence 3, Example Sentence 4, and Example Sentence 5' may be stored.

example sentence Discrimination result Example Sentence 1 user intent 2 example sentence 2 User Intent 3 example sentence 3 user intent 1 example sentence 4 user intent 1 example sentence 5 User Intent 3

When a command to generate a training set for reinforcement learning of the deep learning model is applied by the manager 130, the collection sentence acquisition unit 114 accesses a plurality of pre-designated web pages, and the plurality of web pages By performing web crawling on , a plurality of collection sentences are obtained.

When the plurality of collected sentences are acquired by the collected sentence acquisition unit 114, the order assigning unit 115 randomly assigns a sequential number for classification to each of the plurality of collected sentences.

Hereinafter, operations of the collection sentence acquisition unit 114 and the order allocation unit 115 will be described in detail, for example.

First, when it is assumed that a command to generate a training set for reinforcement learning of the deep learning model is applied to the electronic device 110 by the administrator 130, the collection sentence acquisition unit 114 provides a plurality of pre-specified web pages. A plurality of collected sentences may be obtained by accessing the web pages and performing web crawling on the plurality of web pages.

At this time, when it is assumed that 'collection sentence 1, collection sentence 2, collection sentence 3, and collection sentence 4' are acquired as the plurality of collection sentences, the order assignment unit 115 performs 'collection sentence' as shown in Table 2 below. 1, collection sentence 2, collection sentence 3, and collection sentence 4' may be randomly assigned to each of the order numbers for the classification task.

collection crest turn collection sentence 1 3 collection sentence 2 One collection sentence 3 4 collection sentence 4 2

In this way, when a sequence number for classification is randomly assigned to each of the plurality of collected sentences by the order assigning unit 115, the classification task performing unit 116 determines that the order number among the plurality of collected sentences is smaller. In order, one collection sentence is selected as a sentence to be classified one by one, and the selected collection sentence is selected among the plurality of user intentions based on the degree of association between the selected collection sentence and the plurality of example sentences. The plurality of collected sentences are classified according to user intention by repeatedly performing a classification operation of classifying the collected sentences as those belonging to the maximally matched user intention.

At this time, according to one embodiment of the present invention, the classification task performer 116 is a specific configuration for classifying the plurality of collected sentences by user intention, and includes a keyword dictionary database 118 and a first keyword selection unit 119 , a second keyword selection unit 120, and a classification processing unit 121 may be included.

In the keyword dictionary database 118, preset n (n is a natural number of 5 or more) keywords are stored.

For example, if n is '5' and '5' preset keywords are 'keyword 1, keyword 2, keyword 3, keyword 4, and keyword 5', the keyword dictionary database 118 includes 'keyword 1, keyword 2, keyword 3, keyword 4, and keyword 5' may be stored.

The first keyword selection unit 119 selects the first collection sentence to which the t (t is a natural number) th order number is assigned among the plurality of collection sentences as a sentence to be classified, among the n keywords. At least one first keyword appearing one or more times in the first collection sentence is selected.

The second keyword selection unit 120 refers to the determination result storage unit 113, checks example sentences determined to match each of the plurality of user intentions, and then, for each of the plurality of user intentions, In the example sentences determined to match the user intention, k (k is a natural number equal to or greater than 2) second keywords are selected in the order of the number of occurrences of each of the n keywords, and thus the plurality of user intentions are selected. Second keywords corresponding to each of the .

Hereinafter, operations of the first keyword selection unit 119 and the second keyword selection unit 120 will be described in detail, for example.

First, as in the above example, let n be '5' and k be '3', and the plurality of collection sentences 'collection sentence 1, collection sentence 2, collection sentence 3' by the order assigning unit 115 , Collected sentences 4' are randomly assigned the order numbers shown in Table 2 above, and then 'collected sentences' to which the order number of '1' is assigned as sentences subject to classification by the classification task performer 116. Assume that 2' is selected.

Then, the first keyword selector 119 selects at least one keyword that appears one or more times in 'collected sentence 2' among the '5' keywords 'keyword 1, keyword 2, keyword 3, keyword 4, and keyword 5'. 1 keyword can be selected.

In this regard, when the keywords appearing one or more times in the collection sentence 2 of 'keyword 1, keyword 2, keyword 3, keyword 4, and keyword 5' are 'keyword 1 and keyword 3', the first keyword selection unit (119 ) may select 'keyword 1, keyword 3' as the at least one first keyword.

Then, the second keyword selection unit 120 refers to the determination result storage unit 113 as shown in Table 1, and matches each of the plurality of user intentions, 'user intention 1, user intention 2, and user intention 3'. After confirming the example sentences determined to be true, for each of 'User Intent 1, User Intent 2, and User Intent 3', in the example sentences determined to match each user intention, 'Keyword 1, Keyword 2, Keyword 3' By selecting '3' second keywords in the order of the number of occurrences of keyword 4 and keyword 5, respectively, the second keywords corresponding to 'user intention 1, user intention 2, and user intention 3' are designated. can

In this regard, in the determination result storage unit 113 as shown in Table 1, the example sentences determined to match 'user intention 1' are 'example sentence 3 and example sentence 4', so the second keyword selection unit 120 are example sentences determined to match 'user intention 1', after checking 'example sentence 3 and example sentence 4', in 'example sentence 3 and example sentence 4', 'keyword 1, keyword 2, keyword 3 , Keyword 4, Keyword 5' can check the number of times each appears.

As a result, the number of appearances of each of 'keyword 1, keyword 2, keyword 3, keyword 4, and keyword 5' in 'example sentence 3 and example sentence 4' was '1 time, 2 times, 0 times, 4 times, If it is confirmed as '3 times', '3' keywords in the order of the highest number of occurrences among 'keyword 1, keyword 2, keyword 3, keyword 4, and keyword 5' are 'keyword 4, keyword 5, keyword 2' Therefore, the second keyword selection unit 120 selects 'keyword 2, keyword 4, and keyword 5' from 'example sentence 3 and example sentence 4', thereby providing second keywords corresponding to 'user intention 1'. 'Keyword 2, Keyword 4, Keyword 5' can be specified.

In this way, the second keyword selector 120 may designate second keywords corresponding to 'user intention 1, user intention 2, and user intention 3' as shown in Table 3 below.

user intent second keywords user intent 1 keyword 2, keyword 4, keyword 5 user intent 2 keyword 1, keyword 4, keyword 5 User Intent 3 keyword 1, keyword 2, keyword 3

In this way, when at least one first keyword is selected by the first keyword selector 119 and second keywords corresponding to each of the plurality of user intentions are designated by the second keyword selector 120, classification is performed. The processing unit 121 compares the at least one first keyword with second keywords corresponding to each of the plurality of user intentions, and converts the first collected sentence to the first collected sentence among the plurality of user intentions. It is classified as belonging to the first user intention having the maximum matching degree with the sentence.

At this time, according to an embodiment of the present invention, the classification processing unit 121 has a specific configuration for classifying the first collection sentence as belonging to the first user intention among the plurality of user intentions, and stores a one-hot vector. It may include a unit 122, a keyword vector generator 123, a feature vector generator 124, and a user intention selector 125.

The one-hot vector storage unit 122 stores different n-dimensional one-hot vectors preset to correspond to each of the n keywords.

For example, as in the above example, when n is '5' and '5' keywords are 'Keyword 1, Keyword 2, Keyword 3, Keyword 4, Keyword 5', the one-hot vector storage unit 122 ) may store different '5'-dimensional one-hot vectors preset to correspond to 'keyword 1, keyword 2, keyword 3, keyword 4, and keyword 5', respectively, as shown in Table 4 below.

keyword one-hot vector keyword 1 [0 0 0 0 1] keyword 2 [0 0 0 1 0] keyword 3 [0 0 1 0 0] keyword 4 [0 1 0 0 0] keyword 5 [1 0 0 0 0]

When the at least one first keyword is selected by the first keyword selection unit 119, the keyword vector generator 123 refers to the one-hot vector storage unit 122, and each of the at least one first keyword A keyword vector for the first collected sentence is generated by summing the one-hot vectors corresponding to .

When the second keywords corresponding to each of the plurality of user intentions are selected by the second keyword selection unit 120, the feature vector generator 124 refers to the one-hot vector storage unit 122 to perform the plurality of keywords. For each of the user intentions of , a feature vector corresponding to each of the plurality of user intentions is generated by summing all one-hot vectors of second keywords corresponding to each user intention.

The user intention selection unit 125 calculates a cosine similarity between the keyword vector and a feature vector corresponding to each of the plurality of user intentions, and selects a user intention having the largest cosine similarity among the plurality of user intentions. By selecting the first user intention having the maximum matching degree with one collection sentence, the first collection sentence is classified as belonging to the first user intention.

Here, the cosine similarity may be calculated according to Equation 1 below.

In Equation 1, S is the cosine similarity between vector A and vector B, and has a value between -1 and 1, A _i denotes the i-th component of vector A, and B _i denotes the i-th component of vector B. do. In this regard, the greater the cosine similarity between two vectors, the more similar the two vectors are to each other.

Hereinafter, operations of the keyword vector generator 123, the feature vector generator 124, and the user intention selector 125 will be described in detail, for example.

First, as in the above example, it is assumed that 'keyword 1 and keyword 3' are selected as the at least one first keyword by the first keyword selection unit 119, and by the second keyword selection unit 120, Assume that the second keywords corresponding to each of the plurality of user intentions, 'user intention 1, user intention 2, and user intention 3', are selected as shown in Table 3 above.

Then, the keyword vector generator 123 refers to the one-hot vector storage unit 122 as shown in Table 4 above, and sums the one-hot vectors corresponding to 'keyword 1 and keyword 3', respectively, to obtain 'collected sentences'. You can create a keyword vector for 2'.

In relation to this, in the one-hot vector storage unit 122 as shown in Table 4, the one-hot vectors corresponding to 'keyword 1 and keyword 3' are '[0 0 0 0 1], [0 0 1 0 0 ]', the keyword vector generator 123 adds up '[0 0 0 0 1] and [0 0 1 0 0]' to create a keyword vector for 'collection sentence 2', '[0 0 1 0 1]'.

Then, the feature vector generator 124 refers to the one-hot vector storage unit 122 as shown in Table 4 above, and for each of 'user intention 1, user intention 2, and user intention 3', each user intention By summing up all the one-hot vectors of the corresponding second keywords, feature vectors corresponding to 'user intention 1, user intention 2, and user intention 3' may be generated.

Relatedly, in Table 3, the second keywords corresponding to 'user intention 1' are 'keyword 2, keyword 4, and keyword 5', and in the one-hot vector storage unit as shown in Table 4, 'keyword 2, keyword Since the one-hot vectors corresponding to each of '4, keyword 5' are '[0 0 0 1 0], [0 1 0 0 0], [1 0 0 0 0]', the feature vector generator 124 ' By summing [0 0 0 1 0], [0 1 0 0 0], and [1 0 0 0 0], the feature vector corresponding to 'User Intention 1' is '[1 1 0 1 0]' and can be created together.

In this way, the feature vector generator 124 generates feature vectors corresponding to 'user intention 1, user intention 2, and user intention 3', respectively, '[1 1 0 1 0], [1 1 0 0 1], [0 0 1 1 1]'.

Then, the user intention selection unit 125 performs characteristics corresponding to the keyword vector '[0 0 1 0 1]' and 'user intention 1, user intention 2, and user intention 3', respectively, according to Equation 1 above. The cosine similarity between the vectors '[1 1 0 1 0], [1 1 0 0 1], and [0 0 1 1 1]' can be calculated as '0, 0.41, 0.82'.

Then, the user intention selector 125 selects 'user intention 3' having the largest cosine similarity among 'user intention 1, user intention 2, and user intention 3', and the second highest matching degree with 'collected sentence 2'. By selecting with user intention 1, 'collected sentence 2' can be classified as belonging to 'user intention 3'.

In this way, when the classification task for 'collected sentence 2' is completed, the classification task performing unit 116 repeats the classification task for 'collected sentence 1, collected sentence 3, collected sentence 4, and collected sentence 5' in the same way. By performing this, 'collection sentence 1, collection sentence 2, collection sentence 3, collection sentence 4, and collection sentence 5' can be classified according to user intention as shown in Table 5 below.

collection crest user intent collection sentence 1 user intent 2 collection sentence 2 User Intent 3 collection sentence 3 user intent 2 collection sentence 4 user intent 1 collection sentence 5 User Intent 3

In this way, when the plurality of collected sentences are classified according to user intention by the classification task performer 116, the training set generator 117 provides information on the plurality of collected sentences and the user's intention to which each collected sentence belongs. Create a training set consisting of

For example, as in the foregoing example, as shown in Table 5, 'Collected Sentence 1, Collected Sentence 2, Collected Sentence 3, Collected Sentence 4, and Collected Sentence 5' are classified according to user intention by the classification task performing unit 116. In this case, the training set generation unit 117 creates a training set composed of 'collected sentence 1, collected sentence 2, collected sentence 3, collected sentence 4, and collected sentence 5' and information about user intention to which each collected sentence belongs. can create

Meanwhile, according to an embodiment of the present invention, in order to generate a higher quality training set, the electronic device 110 even if the first collected sentence is classified as belonging to the first user intention, the first collected sentence processing so that the first collected sentence is not included in the training set by excluding the first collected sentence from the plurality of collected sentences when a matching degree between the first collected sentence and the first user intention does not exceed a preset criterion; can do.

In this regard, the classification task performing unit 116 may further include a sentence exclusion processing unit 126 .

If the sentence exclusion processing unit 126 classifies the first collected sentences as belonging to the first user intention by the classification processing unit 121, the first cosine similarity between the feature vector corresponding to the first user intention and the keyword vector A checks whether or not it exceeds a preset criterion similarity, and if it is confirmed that the first cosine similarity does not exceed the criterion similarity, by deleting the first collected sentence from the plurality of collected sentences, 1 Collect sentences are processed so that they are not included in the training set.

For example, suppose that the preset standard similarity is '0.85' and, as in the above-described example, the 'collected sentence 2' is classified as belonging to 'user intention 3' by the classification processing unit 121 .

In this case, according to the above example, the first cosine similarity between '[0 0 1 1 1]', which is the feature vector corresponding to 'user intention 3', and '[0 0 1 0 1]', which is the keyword vector, is ' 0.82', the classification task performer 116 may determine whether the first cosine similarity exceeds the reference similarity and confirm that the first cosine similarity does not exceed the reference similarity.

Then, the sentence exclusion processing unit 126 deletes 'collection sentence 2' from 'collection sentence 1, collection sentence 2, collection sentence 3, and collection sentence 4', which are the plurality of collected sentences, so that 'collection sentence 2' is It can be processed so that it is not included in the training set.

For this reason, the training set generation unit 117 generates a training set composed of only collected sentences whose matching degree with the user's intention to which each sentence belongs among the plurality of collected sentences exceeds the reference similarity, thereby providing higher quality training. A set can be created.

In this way, when a training set is generated through the electronic device 110 of the present invention, the manager 130 performs reinforcement learning on the deep learning model using the training set, thereby determining the user's intention of the deep learning model. The discrimination accuracy can be further increased.

2 is a flowchart illustrating an operating method of an electronic device for generating a training set for performing reinforcement learning of a deep learning model for determining a user's intention according to an embodiment of the present invention.

In step S210, a user intention storage unit in which information on a plurality of preset user intentions is stored is maintained.

In step S220, an example sentence storage unit in which a plurality of pre-collected example sentences are stored is maintained.

In step S230, the execution is completed through a pre-built deep learning model (the deep learning model is a model for determining a user intention matching an input sentence among the plurality of user intentions when a sentence is input), the plurality of A determination result storage unit in which a determination result of a user's intention matched with each of the example sentences of is stored is maintained.

In step S240, when a command to generate a training set for reinforcement learning of the deep learning model is applied by the administrator, a plurality of pre-designated web pages are accessed, and web crawling is performed on the plurality of web pages. By doing so, a plurality of collection sentences are obtained.

In step S250, when the plurality of collected sentences are obtained, an order for classification is randomly assigned to each of the plurality of collected sentences.

In step S260, any one collected sentence from among the plurality of collected sentences in descending order is selected as a sentence to be classified, and the relationship between the selected collected sentence and the plurality of example sentences is also selected. The plurality of collected sentences are classified according to user intention by repeatedly performing a classification operation of classifying the selected collected sentence as belonging to the user's intention that most matches among the plurality of user's intentions.

In step S270, when the plurality of collected sentences are classified according to user intention, a training set composed of the plurality of collected sentences and information on the user's intention to which each collected sentence belongs is generated.

At this time, according to one embodiment of the present invention, in step S260, maintaining a keyword dictionary database in which preset n (n is a natural number of 5 or more) keywords are stored, t (of the plurality of collected sentences) t is a natural number) when the first collection sentences to which the th order number is assigned are selected as sentences to be classified, at least one first keyword that appears one or more times in the first collection sentences among the n keywords Selecting, referring to the determination result storage unit, checking example sentences determined to match each of the plurality of user intentions, and then determining that each of the plurality of user intentions matches each user intention. In the example sentences, the second keyword corresponding to each of the plurality of user intentions is selected by selecting k (k is a natural number equal to or greater than 2) second keywords in the order of the number of occurrences of each of the n keywords. designating and comparing the at least one first keyword and second keywords corresponding to each of the plurality of user intentions to obtain the first collected sentence, and classifying the object as belonging to a first user intention having a maximum matching degree with the sentence.

In this case, according to an embodiment of the present invention, the step of classifying the first collected sentence as belonging to the first user intention having the maximum matching degree with the first collected sentence among the plurality of user intentions includes the n maintaining a one-hot vector storage unit storing different n-dimensional one-hot vectors corresponding to each of the keywords; selecting the at least one first keyword; If a first keyword is selected, generating a keyword vector for the first collection of sentences by adding one-hot vectors corresponding to each of the at least one first keyword with reference to the one-hot vector storage unit; When second keywords corresponding to each of the plurality of user intentions are selected by the step of designating second keywords corresponding to each of the plurality of user intentions, the one-hot vector storage unit refers to the plurality of user intentions. Generating a feature vector corresponding to each of the plurality of user intentions by summing all of the one-hot vectors of second keywords corresponding to each user intention for each of the intentions, and the keyword vector and the plurality of user intentions. A cosine similarity between feature vectors corresponding to each of the intentions is calculated, and the user intention having the maximum cosine similarity among the plurality of user intentions is converted into the first user intention having the maximum matching degree with the first collection sentence. By making a selection, classifying the first collection sentence as belonging to the first user intention.

At this time, according to one embodiment of the present invention, in step S260, if the first collection sentence is classified as belonging to the first user intention, the first between the feature vector corresponding to the first user intention and the keyword vector By checking whether the cosine similarity exceeds a preset criterion similarity, and when it is confirmed that the first cosine similarity does not exceed the criterion similarity, the first collection sentence is deleted from the plurality of collection sentences, The method may further include processing so that the first collected sentences are not included in the training set.

In the above, a method of operating an electronic device for generating a training set for performing reinforcement learning of a deep learning model for determining a user's intention has been described with reference to FIG. 2 . Here, since the operating method of the electronic device according to an embodiment of the present invention may correspond to the configuration of the operation of the electronic device 110 described with reference to FIG. 1 , a detailed description thereof will be omitted.

An operating method of an electronic device generating a training set for performing reinforcement learning of a deep learning model for determining user intention according to an embodiment of the present invention is a computer program stored in a storage medium for execution through combination with a computer. can be implemented

In addition, a method of operating an electronic device for generating a training set for performing reinforcement learning of a deep learning model for determining user intention according to an embodiment of the present invention is implemented in the form of program instructions that can be executed through various computer means. and can be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be those specially designed and configured for the present invention or those known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

As described above, the present invention has been described by specific details such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , Those skilled in the art in the field to which the present invention belongs can make various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the described embodiments, and it will be said that not only the claims to be described later, but also all modifications equivalent or equivalent to these claims belong to the scope of the present invention. .

110: Electronic device for generating a training set for performing reinforcement learning of a deep learning model that determines user intention
111: user intention storage unit 112: example sentence storage unit
113: determination result storage unit 114: collection sentence acquisition unit
115: sequence number allocation unit 116: classification operation unit
117: training set generator 118: keyword dictionary database
119: first keyword selection unit 120: second keyword selection unit
121: classification processing unit 122: one-hot vector storage unit
123: keyword vector generator 124: feature vector generator
125: user intention selection unit 126: sentence exclusion processing unit
130: manager

Claims (10)

An electronic device for generating a training set for performing reinforcement learning of a deep learning model that determines user intention,
a user intention storage unit storing information on a plurality of preset user intentions;
an example sentence storage unit in which a plurality of pre-collected example sentences are stored;
Each of the plurality of example sentences that have been executed through a pre-built deep learning model - the deep learning model is a model that, when a sentence is input, determines a user intention matching the input sentence among the plurality of user intentions a determination result storage unit for storing a determination result for a user's intention matched with;
When a command for generating a training set for reinforcement learning of the deep learning model is applied by an administrator, a plurality of collections are performed by accessing a plurality of pre-designated web pages and performing web crawling on the plurality of web pages. a collection sentence acquisition unit that acquires sentences;
If the plurality of collected sentences are obtained, a sequence allocator randomly assigns a sequence number for classification to each of the plurality of collected sentences;
Among the plurality of collected sentences, one of the collected sentences in the order of the smallest order is selected as a sentence to be classified, and based on the degree of association between the selected collected sentence and the plurality of example sentences, a classification task performing unit that classifies the plurality of collected sentences according to user intention by repeatedly performing a classification task of classifying the selected collection sentence as belonging to the user intention that most matches among the plurality of user intentions; and
When the plurality of collected sentences are classified according to user intentions, a training set generator for generating a training set composed of the plurality of collected sentences and information about user intentions to which each collected sentence belongs.
including,
The classification task performing unit
a keyword dictionary database in which n (n is a natural number of 5 or more) preset keywords are stored;
When the first collection sentence to which the t (t is a natural number) th order number is assigned among the plurality of collection sentences is selected as a sentence to be classified, one or more times in the first collection sentence among the n keywords a first keyword selector selecting at least one first keyword to appear;
After checking the example sentences determined to match each of the plurality of user intentions with reference to the determination result storage unit, in the example sentences determined to match each user intention with respect to each of the plurality of user intentions. , Designating second keywords corresponding to each of the plurality of user intentions by selecting k (k is a natural number equal to or greater than 2) second keywords in the order of the number of occurrences of each of the n keywords. 2 keyword selection unit; and
A degree of matching of the first collection sentence with the first collection sentence among the plurality of user intentions by comparing the at least one first keyword with second keywords corresponding to each of the plurality of user intentions. Classification processing unit for classifying as belonging to the first user intention with a maximum
Including,
The classification processing unit
a one-hot vector storage unit storing different n-dimensional one-hot vectors preset to correspond to each of the n keywords;
When the at least one first keyword is selected by the first keyword selector, the one-hot vector storage unit sums up one-hot vectors corresponding to each of the at least one first keyword, with reference to the one-hot vector storage unit. 1 a keyword vector generator for generating keyword vectors for collected sentences;
When second keywords corresponding to each of the plurality of user intentions are selected by the second keyword selector, with reference to the one-hot vector storage unit, for each of the plurality of user intentions, corresponding to each user intention a feature vector generating unit generating a feature vector corresponding to each of the plurality of user intentions by summing all one-hot vectors of second keywords; and
By calculating the cosine similarity between the keyword vector and the feature vector corresponding to each of the plurality of user intentions, the user intention having the largest cosine similarity among the plurality of user intentions has a matching degree with the first collection sentence. A user intention selector configured to classify the first collection sentence as belonging to the first user intention by selecting the maximum first user intention.
An electronic device comprising a. delete delete According to claim 1,
The classification task performing unit
If the first collection sentence is classified as belonging to the first user intention, it is checked whether a first cosine similarity between a feature vector corresponding to the first user intention and the keyword vector exceeds a preset reference similarity, When it is determined that the first cosine similarity does not exceed the criterion similarity, the first collection sentence is deleted from the plurality of collection sentences so that the first collection sentence is not included in the training set. exclusion processing unit
An electronic device further comprising a. A method of operating an electronic device that generates a training set for performing reinforcement learning of a deep learning model that determines user intention,
maintaining a user intention storage unit in which information on a plurality of preset user intentions is stored;
maintaining an example sentence storage unit in which a plurality of pre-collected example sentences are stored;
Each of the plurality of example sentences that have been executed through a pre-built deep learning model - the deep learning model is a model that, when a sentence is input, determines a user intention matching the input sentence among the plurality of user intentions maintaining a determination result storage unit in which a determination result for a user's intention matched with is stored;
When a command for generating a training set for reinforcement learning of the deep learning model is applied by an administrator, a plurality of collections are performed by accessing a plurality of pre-designated web pages and performing web crawling on the plurality of web pages. obtaining sentences;
randomly allocating a sequence number for a classification task to each of the plurality of collected sentences when the plurality of collected sentences are obtained;
Among the plurality of collected sentences, one of the collected sentences in the order of the smallest order is selected as a sentence to be classified, and based on the degree of association between the selected collected sentence and the plurality of example sentences, classifying the plurality of collected sentences according to user intentions by repeatedly performing a classification task of classifying the selected collected sentences as those belonging to the user's intention that best matches among the plurality of user's intentions; and
If the plurality of collected sentences are classified according to user intention, generating a training set composed of the plurality of collected sentences and information on user intention to which each collected sentence belongs.
including,
The step of classifying the plurality of collection sentences by user intention
maintaining a keyword dictionary database in which preset n (n is a natural number equal to or greater than 5) keywords are stored;
When the first collection sentence to which the t (t is a natural number) th order number is assigned among the plurality of collection sentences is selected as a sentence to be classified, one or more times in the first collection sentence among the n keywords selecting at least one first keyword to appear;
After checking the example sentences determined to match each of the plurality of user intentions with reference to the determination result storage unit, in the example sentences determined to match each user intention with respect to each of the plurality of user intentions. designating second keywords corresponding to each of the plurality of user intentions by selecting k (k is a natural number equal to or greater than 2) second keywords in the order of the number of occurrences of each of the n keywords. ; and
A degree of matching of the first collection sentence with the first collection sentence among the plurality of user intentions by comparing the at least one first keyword with second keywords corresponding to each of the plurality of user intentions. classifying as belonging to the first user intention in which
Including,
Classifying the first collection sentence as belonging to a first user intention having the maximum matching degree with the first collection sentence among the plurality of user intentions;
maintaining a one-hot vector storage unit in which different n-dimensional one-hot vectors preset to correspond to each of the n keywords are stored;
If the at least one first keyword is selected by the step of selecting the at least one first keyword, one-hot vectors corresponding to each of the at least one first keyword are obtained with reference to the one-hot vector storage unit. generating a keyword vector for the first collection sentence by summing;
When second keywords corresponding to each of the plurality of user intentions are selected by the step of designating second keywords corresponding to each of the plurality of user intentions, the one-hot vector storage unit refers to the plurality of user intentions. generating a feature vector corresponding to each of the plurality of user intentions by summing all one-hot vectors of second keywords corresponding to each user intention for each of the intentions; and
By calculating the cosine similarity between the keyword vector and the feature vector corresponding to each of the plurality of user intentions, the user intention having the largest cosine similarity among the plurality of user intentions has a matching degree with the first collection sentence. classifying the first collection sentence as belonging to the first user intent by selecting the first user intent that is maximal;
A method of operating an electronic device comprising a. delete delete According to claim 5,
The step of classifying the plurality of collection sentences by user intention
If the first collection sentence is classified as belonging to the first user intention, it is checked whether a first cosine similarity between a feature vector corresponding to the first user intention and the keyword vector exceeds a preset reference similarity, When it is determined that the first cosine similarity does not exceed the reference similarity, processing the first collection sentence not to be included in the training set by deleting the first collection sentence from the plurality of collection sentences.
Method of operating an electronic device further comprising a. A computer readable recording medium recording a computer program for executing the method of any one of claims 5 or 8 through a combination with a computer. A computer program stored in a storage medium for executing the method of any one of claims 5 or 8 through a combination with a computer.

Images