KR20210070768A - Learning data generating apparatus, and control method thereof - Google Patents

Abstract

The present invention relates to a device for generating learning data for reflecting text composition information included in a word when embedding an English word as a vector for learning; and an operation method thereof. The device for generating learning data comprises: a confirmation part; and a conversion part.

Description

Learning data generating device and its operation method {LEARNING DATA GENERATING APPARATUS, AND CONTROL METHOD THEREOF}

The present invention relates to the generation of learning data, and relates to a method for reflecting character composition information included in the word when embedding an English word as a vector for learning.

In order to apply deep learning technology in the fields of NLP (Natural Language Processing) and text mining, an embedding process that converts text (words or characters) into numerical vectors is inevitably required.

For example, Word2Vec, Glove, and Fasttext are used as a pre-trained word embedding method that learns a vector of words from large text data in advance.

However, these existing technologies are a method of learning the vector of the word from the distribution of words around each word, that is, inter-word info., and the character composition of each word (uppercase and lowercase alphabets, numbers, symbols, etc.) ) has a limitation in that it cannot contain information about

As a way to supplement this, embedding in character units or adding information to classify into 4 or 5 typical character composition types (eg allCaps, initCap, mixedCaps, lowecase, etc.) is used, but similarly, various character composition of words is not properly covered.

The present invention was created in view of the above circumstances, and an object of the present invention is to reflect the character composition information included in the word when embedding an English word as a vector for learning.

According to an embodiment of the present invention, there is provided an apparatus for generating learning data for achieving the above object, comprising: a confirmation unit for confirming a word attribute of an English word from character composition information of an English word to be learned; and a conversion unit that abbreviates and converts the English word into an abstract word according to an abstraction rule defined for each word attribute.

Specifically, the learning data generating apparatus may further include a generator for generating a vector for learning the English word through embedding of the abstract word.

Specifically, the word attribute may be determined by a word type of at least one of alphabets, numbers, and symbols that may be included as character composition information of the English word.

Specifically, when it is confirmed that the word attribute is an alphabet, the conversion unit replaces the uppercase and lowercase letters included in the English word with a predefined representative uppercase letter and lowercase letter, respectively, and displays two or more words included in the English word. With respect to the continuous uppercase and lowercase character strings, each of the uppercase and lowercase letters of the representative alphabet may be abbreviated in a form in which a predefined number is arranged.

Specifically, when it is confirmed that the word attribute is a number, the conversion unit replaces each number included in the English word with a predefined representative number, and for the number of digits of the number included in the English word, a number It can be abbreviated in the form of arranging the designated alphabetic character indicating that , and the number of the numerical value corresponding to the number of digits.

Specifically, when it is confirmed that the word attribute is a symbol, the conversion unit may abbreviate in a form in which a predetermined number of the specific symbols are listed for two or more consecutive specific symbols included in the English word.

In order to achieve the above object, an operating method of an apparatus for generating learning data according to an embodiment of the present invention includes: a confirmation step of confirming a word attribute of an English word from character composition information of an English word to be learned; and a conversion step of converting the English word into an abstract word by abbreviated according to an abstraction rule previously defined for each word attribute.

Specifically, the method may further include a generating step of generating a vector for learning the English word through embedding of the abstract word.

Specifically, the word attribute may be determined by a word type of at least one of alphabets, numbers, and symbols that may be included as character composition information of the English word.

Specifically, when it is confirmed that the word attribute is an alphabet, uppercase and lowercase letters included in the English word are replaced with predefined representative uppercase and lowercase letters, respectively, and two or more consecutive uppercase character strings included in the English word and for the lowercase character string, each of the uppercase and lowercase letters of the representative alphabet may be abbreviated in a form in which a predefined number is listed.

Specifically, in the conversion step, when it is confirmed that the word attribute is a number, each number included in the English word is replaced with a predefined representative number, and with respect to the number of digits of the number included in the English word, It can be abbreviated in a form in which a designated alphabetic character indicating a number and a number of a numerical value corresponding to the number of digits are arranged adjacent to each other.

Specifically, in the conversion step, when it is confirmed that the word attribute is a symbol, for two or more consecutive specific symbols included in the English word, the specific symbols may be abbreviated in a form in which a predefined number is listed.

Accordingly, in the learning data generating apparatus and the operating method of the present invention, when embedding an English word as a vector for learning, by reflecting the character composition information included in the word, in the field of NLP (Natural Language Processing) and text mining It can increase the learning effect.

1 is an exemplary diagram illustrating a learning data generation environment according to an embodiment of the present invention.
2 is a schematic configuration diagram of an apparatus for generating learning data according to an embodiment of the present invention;
3 is a flowchart illustrating an operating method of an apparatus for generating learning data according to an embodiment of the present invention;

1 schematically shows a learning data generation environment according to an embodiment of the present invention.

An embodiment of the present invention deals with an embedding process, which is a procedure for converting English words into numerical vectors for application of deep learning techniques in the fields of natural language processing (NLP) and text mining.

The English word may have character composition information including uppercase letters and lowercase letters of the alphabet, numbers, and various symbols.

In the case of the English alphabet, the distinction between uppercase and lowercase letters can be an important hint in determining whether or not it is a proper noun (name). It also has meaning whether only the first letter is capitalized or the whole word is capitalized.

In the case of numbers, information about how many digits is important is important.

For example, a number is used to represent a date (year, month, day). The year is usually represented by two or four digits, and the month and day are represented by one or two digits.

Therefore, the number of digits in a number can provide a hint as to whether the number can be interpreted as a year, month, or day.

Various symbols can also have different meanings when used in a sentence.

As such, the character composition information plays an important role in interpreting the meaning of English words,

Therefore, in order to improve the learning effect of English words in the fields of NLP and text mining, it can be seen that learning of such character composition information is absolutely necessary.

Accordingly, in the learning data generating environment according to an embodiment of the present invention, when embedding an English word as a vector for learning, the learning data generating device ( 100) will be included.

The learning data generating device 100 refers to a device for generating learning data in the fields of NLP and text mining, for example, a computing device (eg, PC) loaded with software (eg, application), or through a wired/wireless communication network. It can be implemented in the form of a connectable server.

For reference, when the learning data generating apparatus 100 is implemented in the form of a server, it may be implemented in the form of, for example, a web server, a database server, a proxy server, etc., and a network load balancing mechanism, or a service device is provided on the Internet Alternatively, one or more of various software to be operated on another network may be installed, and may be implemented as a computerized system through this.

As described above, in the learning data generation environment according to an embodiment of the present invention, when an English word is embedded as a vector for learning through the above-described configuration, character composition information included in the word may be reflected. The configuration of the learning data generating apparatus 100 for realizing this will be described in more detail.

2 shows a schematic configuration of an apparatus 100 for generating learning data according to an embodiment of the present invention.

As shown in FIG. 2 , the learning data generating apparatus 100 according to an embodiment of the present invention includes a confirmation unit 10 that checks the word properties of English words, and a conversion unit that converts English words into abstract words ( 20) may be included.

In addition, the learning data generating apparatus 100 according to an embodiment of the present invention may further include a generating unit 30 that generates a vector for learning English words in addition to the above-described configuration.

The entire configuration or at least a part of the configuration of the learning data generating apparatus 100 including the confirmation unit 10, the conversion unit 20, and the generation unit 30 is implemented in the form of a software module or a hardware module, or software A module and a hardware module may be implemented in a combined form.

Here, the software module, for example, may be understood as an instruction executed by a processor that performs an operation in the learning data generating apparatus 100, and these instructions are mounted in a memory in the learning data generating apparatus 100. will be able to have

Meanwhile, the learning data generating apparatus 100 according to an embodiment of the present invention may further include a configuration of the communication unit 40 for data transmission and reception in addition to the aforementioned configuration.

The communication unit 40 includes, for example, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec (CODEC) chipset, and a memory, and the like, All known circuits to perform may be included.

After all, the learning data generating apparatus 100 according to an embodiment of the present invention can convert an English word into an abstract word reflecting character composition information through the above configuration and generate a vector for learning. For each configuration in the learning data generating apparatus 100 will be described in more detail.

The confirmation unit 10 performs a function of confirming the word attribute of an English word for learning.

More specifically, the check unit 100 checks the word attribute of the English word from the character composition information of the English word as a learning target.

Here, the character configuration information includes uppercase letters and lowercase letters of the alphabet, numbers, and various symbols, as briefly mentioned above.

Accordingly, the confirmation unit 100 may confirm, as a word attribute of an English word, a word type of at least one of alphabets, numbers, and symbols that may be included as character configuration information of an English word.

The conversion unit 20 performs a function of converting an English word into an abstract word.

More specifically, when the word attribute of the English word is confirmed, the English word is abbreviated and converted into an abstract word according to an abstraction rule predefined for each word attribute.

At this time, when it is confirmed that the word attribute of the English word is an alphabet, the conversion unit 20 converts uppercase and lowercase letters included in the English word into predefined representative uppercase letters (eg, A) and lowercase letters (eg, a), respectively. can be represented by substitution.

For example, [Table 1] below exemplarily shows an alphabet conversion method according to an embodiment of the present invention.

english word my alphabet representative alphabet abstract word A to Z A A a-z a a

On the other hand, the conversion unit 20, for two or more consecutive uppercase character string and lowercase character string included in the English word, each of the representative alphabet uppercase letters (eg, A) and lowercase letters (eg, a) a predefined number (eg, 2) ) can be abbreviated. In this regard, [Table 2] below exemplarily shows an alphabetic character string conversion method according to an embodiment of the present invention.

english word my alphabet representative alphabet abstract word AA+ A AA a+ a aa

Here, + means that the corresponding alphabetic character exists more than described. In addition, when it is confirmed that the word attribute is a number, the conversion unit 20 converts each number included in the English word to a predefined representative number (eg, : 9) can be expressed in a way that replaces

In this regard, [Table 3] below exemplarily shows a number conversion method according to an embodiment of the present invention.

numbers in english words representative number abstract word 0-9 9 9

Meanwhile, the conversion unit 20 may abbreviate the number of digits included in the English word in a form in which a designated alphabetic character (eg, D) indicating that it is a number and a number of a numerical value corresponding to the number of digits are arranged adjacent to each other. However, the number of numerical values corresponding to the number of digits may be defined not to exceed a threshold value (eg, 5).

In this regard, [Table 4] below exemplarily shows a digit expression method according to an embodiment of the present invention.

numbers in english words designated alphabet abstract word 99999+ D D5 9999 D4 999 D3 99 D2 9 D1

Here, + means that the number of digits of the corresponding number is greater than described. In addition, when it is confirmed that the word attribute of the English word is a symbol, the conversion unit 20 includes two or more consecutive specific symbols included in the English word. can be abbreviated in a form in which the specific symbols are listed by a predefined number (eg, two).

In this regard, [Table 5] below exemplarily shows a symbol conversion method according to an embodiment of the present invention.

numbers in english words abstract word --+ --

Here, + means that the corresponding symbol exists beyond the description. As such, in an embodiment of the present invention, as the conversion into an abstract word is performed according to the abstract rule for each word attribute of an English word, one English word Even if my alphabets, numbers, and symbols are complex, it is possible to apply the above-described property conversion method identically.

In this regard, [Table 6] below exemplarily shows conversion results of compound word attributes according to an embodiment of the present invention.

english words abstract word Bill aaa USA AA iPhone aaaa 2019-09-27 D4-D2-D2 O157 AD3

The generator 30 performs a function of generating a vector for learning. More specifically, when the conversion of the English word into the abstract word is completed, the generator 30 performs embedding of the abstract word. A vector for learning the English word is generated.

As described above, according to the configuration of the learning data generating apparatus 100 according to an embodiment of the present invention, in various tasks of NLP and text mining, it is possible to add (concatenate) character abstraction word embeddings to existing word embeddings. and, through this, the final learning effect (performance) of the task can be increased.

Hereinafter, an operating method of the learning data generating apparatus 100 according to an embodiment of the present invention will be described with reference to FIG. 3 .

First, the check unit 100 checks the word attribute of the English word from the character configuration information of the English word as a learning target ( S10 ).

Here, the character configuration information includes uppercase letters and lowercase letters of the alphabet, numbers, and various symbols, as briefly mentioned above.

Accordingly, the confirmation unit 100 may confirm, as a word attribute of an English word, at least one word type among alphabets, numbers, and symbols that may be included as character configuration information of an English word.

Then, when the word attribute of the English word is checked, the English word is abbreviated and converted into an abstract word according to an abstraction rule predefined for each word attribute.

At this time, when it is confirmed that the word attribute of the English word is the alphabet, the converting unit 20 converts the uppercase and lowercase letters included in the English word into a predefined representative uppercase letter (eg, A) and a lowercase letter (eg, a), respectively. It can be represented by substitution (S20-S30).

Here, for two or more consecutive uppercase character strings and lowercase character strings included in the English word, the conversion unit 20 converts each of the representative alphabet uppercase letters (eg, A) and lowercase letters (eg, a) to a predefined number (eg, 2). It can be abbreviated in the form listed as

In addition, when it is confirmed that the word attribute is a number, the conversion unit 20 may represent each number included in the English word in a manner of replacing each number included in the English word with a predefined representative number (eg, 9) (S40-S50).

Here, the conversion unit 20 may abbreviate the number of digits included in the English word in a form in which a designated alphabetic character (eg, D) indicating that it is a number and a number of a numerical value corresponding to the number of digits are arranged adjacent to each other.

However, the number of numerical values corresponding to the number of digits may be defined not to exceed a threshold value (eg, 5).

In addition, when it is confirmed that the word attribute of the English word is a symbol, the conversion unit 20 lists the specific symbols by a predefined number (eg, two) for two or more consecutive specific symbols included in the English word. It can be abbreviated to the form (S60-S70).

As such, in one embodiment of the present invention, as the conversion into an abstract word is performed according to the abstract rule for each word property of an English word, even if the alphabet, number, and symbol in one English word are compounded, the above-described property conversion method is the same. It is possible to apply

Thereafter, when the conversion of the English word into the abstract word is completed, the generator 30 generates a vector for learning the English word through embedding of the abstract word ( S80 ).

As described above, according to the method of operation of the apparatus 100 for generating learning data according to an embodiment of the present invention, in various tasks of NLP and text mining, character abstraction word embedding is added (concatenated) to the existing word embedding to be used. This can increase the final learning effect (performance) of the task.

On the other hand, the functional operations and implementations of the subject matter described in this specification are implemented as digital electronic circuits, computer software, firmware or hardware including the structures disclosed in this specification and structural equivalents thereof, or at least one of these It can be implemented by combining. Implementations of the subject matter described herein are one or more computer program products, ie, one or more modules of computer program instructions encoded on a tangible program storage medium for processing or execution by a processing system. can be implemented.

The computer readable medium may be a machine readable storage device, a machine readable storage substrate, a memory device, a composition of matter that affects a machine readable radio wave signal, or a combination of one or more thereof.

As used herein, the term “system” or “device” encompasses all devices, devices and machines for processing data, including, for example, programmable processors, computers, or multiple processors or computers. The processing system may include, in addition to hardware, code that, upon request, forms an execution environment for a computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. .

A computer program (also known as a program, software, software application, script or code) may be written in any form of programming language, including compiled or interpreted language or declarative or procedural language, as a stand-alone program or module; It can be deployed in any form, including components, subroutines or other units suitable for use in a computer environment. A computer program does not necessarily correspond to a file in a file system. A program may be placed in a single file provided to the requested program, or in multiple interacting files (eg, files that store one or more modules, subprograms, or portions of code), or portions of files that hold other programs or data. (eg, one or more scripts stored within a markup language document). The computer program may be deployed to be executed on a single computer or multiple computers located at one site or distributed over a plurality of sites and interconnected by a communication network.

On the other hand, computer-readable media suitable for storing computer program instructions and data include, for example, semiconductor memory devices such as EPROMs, EEPROMs and flash memory devices, such as magnetic disks such as internal hard disks or external disks, magneto-optical disks and CDs. -Can include all types of non-volatile memory, media and memory devices, including ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated into, special purpose logic circuitry.

An implementation of the subject matter described herein may include a backend component, such as a data server, or a middleware component, such as an application server, or a web browser or graphical user, such as a user capable of interacting with an implementation of the subject matter described herein. It may be implemented in a front-end component, such as a client computer having an interface, or in a computing system including any combination of one or more of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as, for example, a communication network.

While this specification contains numerous specific implementation details, they should not be construed as limitations on the scope of any invention or claim, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. should be understood Likewise, certain features that are described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Furthermore, although features operate in a particular combination and may be initially depicted as claimed as such, one or more features from a claimed combination may in some cases be excluded from the combination, the claimed combination being a sub-combination. or a variant of a sub-combination.

Also, although operations are depicted in the drawings in a specific order herein, it is not to be understood that such operations must be performed in the specific order or sequential order shown or that all illustrated operations must be performed in order to achieve desirable results. Can not be done. In certain cases, multitasking and parallel processing may be advantageous. Further, the separation of the various system components of the above-described embodiments should not be construed as requiring such separation in all embodiments, and the program components and systems described may generally be integrated together into a single software product or packaged into multiple software products. It should be understood that

As such, this specification is not intended to limit the invention to the specific terminology presented. Accordingly, although the present invention has been described in detail with reference to the above-described examples, those skilled in the art can make modifications, changes, and modifications to the examples without departing from the scope of the present invention. The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

According to the learning data generating apparatus and the method of operation thereof according to the present invention, when embedding an English word as a vector for learning, it is possible to reflect the character composition information included in the word, the limitation of the existing technology It is an invention with industrial applicability because the possibility of marketing or business of the applied device, not only the use of the related technology, as well as the degree to which it can be clearly implemented in reality is sufficient.

100: learning data generating device
10: confirmation unit 20: conversion unit
30: generator

Claims (12)

a confirmation unit for confirming a word attribute of an English word from character composition information of an English word as a learning target; and
and a conversion unit that abbreviates and converts the English word into an abstract word according to an abstraction rule defined for each word attribute. The method of claim 1,
The learning data generating device,
The apparatus for generating learning data, characterized in that it further comprises a generator for generating a vector for learning the English word through embedding of the abstract word. The method of claim 1,
The word attribute is
The learning data generating apparatus, characterized in that it is determined by the type of at least one of alphabets, numbers, and symbols that can be included as the character composition information of the English word. 4. The method of claim 3,
The conversion unit,
When it is confirmed that the word attribute is an alphabet, uppercase and lowercase letters included in the English word are replaced with predefined representative uppercase and lowercase letters, respectively,
For two or more consecutive uppercase and lowercase character strings included in the English word, each of the uppercase and lowercase letters of the representative alphabet is abbreviated in a form in which a predefined number is listed. 4. The method of claim 3,
The conversion unit,
When it is confirmed that the word attribute is a number, each number included in the English word is replaced with a predefined representative number,
With respect to the number of digits of the number included in the English word, the learning data generating apparatus, characterized in that the abbreviation of the designated alphabetic character indicating that it is a number and the number of the numerical value corresponding to the number of digits is abbreviated in a form adjacent to each other. 4. The method of claim 3,
The conversion unit,
When it is confirmed that the word attribute is a symbol, the learning data generating apparatus, characterized in that for two or more consecutive specific symbols included in the English word, the specific symbols are abbreviated in a form in which a predefined number is listed. a confirmation step of confirming the word attribute of the English word from the character composition information of the English word to be learned; and
and converting the English word into an abstract word by abbreviated according to an abstraction rule defined for each word attribute. 8. The method of claim 7,
The method is
The method of operating the learning data generating apparatus, characterized in that it further comprises a generating step of generating a vector for learning the English word through embedding of the abstract word. 8. The method of claim 7,
The word attribute is
The operating method of the learning data generating apparatus, characterized in that it is determined by the type of at least one of alphabets, numbers, and symbols that can be included as the character composition information of the English word. 10. The method of claim 9,
The conversion step is
When it is confirmed that the word attribute is an alphabet, uppercase and lowercase letters included in the English word are replaced with predefined representative uppercase and lowercase letters, respectively,
For two or more consecutive uppercase and lowercase character strings included in the English word, each of the uppercase and lowercase letters of the representative alphabet is abbreviated in a form in which a predefined number is arranged. 10. The method of claim 9,
The conversion step is
When it is confirmed that the word attribute is a number, each number included in the English word is replaced with a predefined representative number,
With respect to the number of digits of the number included in the English word, the operation method of the learning data generating apparatus, characterized in that the abbreviation of the designated alphabetic character indicating that it is a number and the number of the numerical value corresponding to the number of digits adjacent to each other. 10. The method of claim 9,
The conversion step is
When it is confirmed that the word attribute is a symbol, for two or more consecutive specific symbols included in the English word, the specific symbols are abbreviated in a form in which a predetermined number of specific symbols are listed. .

Images