JP2023064283A - Machine learning device, natural language processing device, and program - Google Patents

Abstract

To provide a machine learning device, a natural language processing device and a program that more accurately extracts the position of a subsequence to be analyzed that is included in a word string.SOLUTION: In an opinion analysis device (natural language processing device) 1, a word embedding unit 11 accepts input of a word string and outputs a word embedded expression string that corresponds to the word string. A series labeling unit 12 accepts the word embedded expression string outputted from the word embedding unit 11 as input and outputs a label series that corresponds to the word embedded expression string. A pause position prediction unit 22 accepts the word embedded expression string outputted from the word embedding unit 11 as input and outputs pause position information that corresponds to the word embedded expression string, and performs back propagation based on an error between the pause position information and correct-answer pause position information so as to adjust the parameters of an internal model. The word embedding unit 11 adjusts the parameters of the internal model by back propagation of an error from the series labeling unit 12 and back propagation of an error from the pause position prediction unit 22.SELECTED DRAWING: Figure 1

Description

The present invention relates to a machine learning device, a natural language processing device, and a program.

For example, it is effective to automatically analyze the tendency of opinions of people or society as a whole by automatically analyzing a large number of sentences posted on SNS (Social Networking Service) or the like. For example, it is unrealistic to manually analyze tens of millions to hundreds of millions of sentences or more, and there is a strong demand for automatic analysis with high accuracy.

Conventional techniques automatically label each phrase or word contained in a sentence. For example, using a deep learning model, attempts have been made to extract sections between an opinion target and an opinion part from a sentence.

Non-Patent Document 1 describes BERT (Bidirectional Encoder Representations from Transformers), which is a mechanism for analyzing sentences written in natural language using a deep learning model.

Non-Patent Document 2 describes a technique of “sequence labeling” that uses machine learning to label a sequence of constituent elements of a sentence.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding, Proceedings of NAACL-HLT 2019, pages 4171-4186, Association for Computational Linguistics, 2019. Takatomo Ishikawa, Machine Learning for Language Processing “5. Sequence Labeling”, [online], Internet <URL: https://www.slideshare.net/Takatymo/ss-64274683, July 22, 2016>.

However, in the prior art, when labeling constituent elements of a sentence using a deep learning analysis model, there is a problem that division into phrases to be labeled cannot always be correctly performed. In other words, there is a problem that an error may occur in the division into phrases, which is the process preceding the labeling process. This is a problem that, in the task of extracting an opinion target segment or an opinion segment in a sentence, an error may occur in the punctuation of the opinion target segment or the opinion segment. Such segment extraction errors can also result in incorrect labeling. In other words, in the task of extracting an opinion section or an opinion section, some expressions that should be included in those sections are missing, or incorrect expressions that should be outside the section are extracted. I get laid off. In addition, if an error such as the one described above occurs in labeling, a problem may arise in that the accuracy of analysis in subsequent processing also deteriorates. For example, when the trends of opinions on SNS are statistically aggregated, it becomes a factor of degraded accuracy.

The present invention is based on the recognition of the problem as described above. It is intended to provide a machine learning device, a natural language processing device, and a program for making it possible to extract more accurately.

[1] In order to solve the above problems, a machine learning device according to an aspect of the present invention includes a word embedding unit that inputs a word string and outputs a word embedding expression string corresponding to the word string, and from the word embedding unit: A sequence labeling unit that inputs a string of word embedding expressions to be output and outputs a label sequence corresponding to the string of word embedding expressions, and a sequence labeling unit that inputs a string of word embedding expressions output from the word embedding unit and corresponds to the string of word embedding expressions. and a word string to be input to the word embedding unit, and a correct label sequence corresponding to the label sequence output by the sequence labeling unit. and a learning data supply unit for supplying correct phrase break position information corresponding to the phrase break position information output by the phrase break position prediction unit, wherein the sequence labeling unit includes the label sequence and the correct label The parameter of the internal model is adjusted by performing backpropagation based on the error with the series, and the phrase break position prediction unit performs back propagation based on the error between the phrase break position information and the correct phrase break position information. and the word embedding unit adjusts the parameters of the internal model by backpropagating errors from the sequence labeling unit and backpropagating errors from the phrase break position prediction unit. ,

[2] Further, according to one aspect of the present invention, in the above machine learning device, the phrase break position prediction unit is configured to perform a full-joint regression based on all word-embedding expressions included in the input word-embedding expression string. By using the model, the information on the position of punctuation corresponding to the word embedding expression string is output.

[3] In one aspect of the present invention, in the machine learning device described above, the sequence labeling unit labels a predetermined substring included in the input original word string as an opinion target. , and a label indicating that the predetermined subsequence is an opinion.

[4] Further, according to one aspect of the present invention, in the machine learning device described above, the phrase break position prediction unit outputs numerical values representing the start position and end position of the substring as the phrase break position information. , and the correct phrase cutoff position information supplied by the learning data supply unit is numerical information representing the correct start position and end position of the subsequence.

[5] In one aspect of the present invention, a word embedding unit for inputting a word string and outputting a word embedding expression string corresponding to the word string, and a word embedding expression string output from the word embedding unit are input. 5. The machine learning according to any one of claims 1 to 4, further comprising a sequence labeling unit that outputs a label sequence corresponding to the word embedding expression sequence, wherein at least the model that the word embedding unit has therein is the machine learning. It is a natural language processing device that has been learned by the device.

[6] In one aspect of the present invention, a word embedding section for inputting a word string and outputting a word embedding expression string corresponding to the word string, and a word embedding expression string output from the word embedding section are input. a sequence labeling unit for outputting a label sequence corresponding to the word embedding expression string; and a phrase break for inputting the word embedding expression string output from the word embedding unit and outputting phrase break position information corresponding to the word embedding expression string. a position prediction unit, supplying a word string to be input to the word embedding unit, supplying a correct label sequence corresponding to the label sequence output by the sequence labeling unit, and further output by the phrase break position prediction unit; and a learning data supply unit that supplies correct phrase break position information corresponding to the correct phrase break position information, wherein the sequence labeling unit performs backpropagation based on an error between the label sequence and the correct label sequence. The parameters of the internal model are adjusted by this, and the phrase break position prediction unit adjusts the parameters of the internal model by performing backpropagation based on the error between the correct phrase break position information and the correct phrase break position information. , the word embedding unit adjusts parameters of an internal model by backpropagation of errors from the sequence labeling unit and backpropagation of errors from the phrase break position prediction unit, causing the computer to function as a machine learning device. It is a program for

According to the present invention, it is possible to improve the accuracy when assigning a label sequence (tag sequence) to a word sequence.

1 is a block diagram showing a schematic functional configuration of an opinion analysis device according to an embodiment of the present invention; FIG. FIG. 4 is a configuration diagram showing in more detail the configuration of a model for predicting a punctuation position in the same embodiment; FIG. 4 is a schematic diagram showing an example of text input to the opinion analysis device according to the same embodiment, and an example of labeling performed on the input text; FIG. 4 is a schematic diagram showing an example of a correct answer of a word sequence corresponding to the input text example (FIG. 3) and a tag sequence corresponding to the sequence in the same embodiment; FIG. 4 is a schematic diagram showing another example of a correct answer of a word sequence corresponding to the input text example (FIG. 3) and a tag sequence corresponding to the sequence in the same embodiment; FIG. 4 is a schematic diagram showing an example of correct data for learning a phrase break position prediction model in the same embodiment; 6 is a flow chart showing the procedure of processing when the opinion analysis device according to the same embodiment performs model learning. 4 is a flow chart showing a sequence labeling process performed by the opinion analysis device according to the same embodiment using a trained model. It is a block diagram which shows an example of an internal configuration of the opinion analysis apparatus by the same embodiment.

An embodiment of the present invention will now be described with reference to the drawings. The apparatus according to this embodiment is an opinion analysis apparatus for analyzing opinions included in input text. Specifically, the opinion analysis apparatus of the present embodiment determines whether an element (phrase) constituting an input text is an opinion target section, an opinion section, or a section neither of these. tags (labels) for distinguishing between

A feature of the opinion analysis apparatus of this embodiment is that it includes a regression model for predicting the division point (punctuation) of the section to be extracted. The regression model unique to this embodiment may be referred to as a "phrase break position prediction model" below. The phrase break position prediction model shares the distributed representation in the lower layer with the model for performing sequence labeling (the sequence labeling model). The opinion analysis device of this embodiment performs machine learning on a model provided therein, and is also called a “machine learning device”. Also, the opinion analysis device of this embodiment uses a trained model to give a label sequence corresponding to a given unknown word string, and is also called a "natural language processing device".

In this embodiment, in order to carry out machine learning of the phrase break position prediction model, a loss function is used that gives the error between the correct phrase break position predicted by the phrase break position prediction model and the correct phrase break position in terms of cross entropy. .

FIG. 1 is a block diagram showing a schematic functional configuration of an opinion analysis device according to this embodiment. As shown, the opinion analysis device 1 includes a word embedding unit 11, a sequence labeling unit 12, a sequence labeling loss function calculation unit 17, a phrase break position prediction unit 22, and a phrase break position loss function calculation unit 27. , and a learning data supply unit 30 . Each of these functional units can be realized by, for example, a computer and a program. In addition, each functional unit has storage means as required. The storage means are, for example, program variables and memory allocated by execution of the program. Also, if necessary, non-volatile storage means such as a magnetic hard disk drive or a solid state drive (SSD) may be used. Also, at least part of the function of each functional unit may be realized as a dedicated electronic circuit instead of a program. The function of each part is as described below.

The word embedding unit 11 receives a word string and outputs a word embedding expression string corresponding to the word string. The word embedding unit 11 internally has a machine-learnable model. A model included in the word embedding unit 11 is also called a "word embedding layer". A model provided in the word embedding unit 11 is realized using, for example, a neural network. In this embodiment, BERT is used as a deep learning model of the opinion object extraction device. BERT itself is an existing technology and is described in Non-Patent Document 1 and the like. The BERT used by the word embedding unit 11 may use a general model pre-learned in Japanese. In this embodiment, the BERT possessed by the word embedding unit 11 can be further learned. The input sentence is segmented into words by a standard morphological analysis process. That is, an input sentence is equivalent to a word string. A word string corresponding to the input sentence is input to the word embedding section 11 and converted into a string of word embedding expressions.

The sequence labeling unit 12 receives the word embedding expression string output from the word embedding unit 11 and outputs a label sequence corresponding to the word embedding expression string. That is, the series labeling unit 12 assigns a label to each word. A specific example of the label will be described later. The sequence labeling unit 12 internally has a machine-learnable model. A model provided by the sequence labeling unit 12 is called a “sequence labeling model”.

In this embodiment, the sequence labeling unit 12 creates a label indicating that a predetermined subsequence included in an input original word string is an opinion target, and a label indicating that the predetermined subsequence is an opinion. , at least. More specific examples of labels given by the series labeling unit 12 will be described later.

The sequence labeling loss function calculation unit 17 calculates the error between the label sequence output by the sequence labeling unit 12 and the correct label sequence. The error calculated by the sequence labeling loss function calculator 17 is the error that is the basis of error backpropagation performed by the sequence labeler 12 to adjust the parameters.

The phrase break position prediction unit 22 receives the word embedding expression string output from the word embedding unit 11, and outputs phrase break position information corresponding to the word embedding expression string. The punctuation position prediction unit 22 has a machine-learnable model inside. A model included in the phrase break position prediction unit 22 is called a "phrase break position prediction model".

The phrase break position prediction unit 22 outputs the phrase break position information corresponding to the input word embedding expression string by using a fully connected regression model based on all the word embedding expressions contained in the input word embedding expression string. You can do it.

The phrase break position loss function calculator 27 calculates the error between the phrase break position information output by the phrase break position prediction unit 22 and the correct phrase break position information. The error calculated by the phrase break position loss function calculator 27 is the basis for error backpropagation performed by the phrase break position predictor 22 to adjust parameters.

Note that the word embedding unit 11 performs the machine learning of the model based on both the error backpropagation from the sequence labeling unit 12 and the error backpropagation from the phrase break position prediction unit 22. Tune the parameters of the inner model.

The learning data supply unit 30 supplies learning data for learning a model provided inside the opinion analysis device 1 . Specifically, the learning data supply unit 30 supplies a word string to be input to the word embedding unit 11 . In addition, the learning data supply unit 30 supplies a correct label sequence corresponding to the label sequence output by the sequence labeling unit 12, corresponding to the word sequence. Furthermore, the learning data supply unit 30 supplies correct phrase break position information, which is the correct answer corresponding to the phrase break position information output by the phrase break position prediction unit 22, corresponding to the word string.

The phrase break position prediction unit 22 outputs numerical values representing the start position and end position of the subsequence of the word string corresponding to the input sentence as the phrase break position information. Note that the number of subsequences may be one or more. The correct phrase break position information supplied by the learning data supply unit 30 is numerical information representing the correct start position and end position of the above subsequence. The start position and end position are numerical information indicating the order of the word. The correct starting and ending positions of a subsequence are each essentially integers. Each of the start position and the end position predicted and output by the punctuation position prediction unit 22 is not necessarily an integer. Normally, each of the start position and the end position predicted and output by the phrase break position prediction unit 22 is a non-integer. Those non-integers are taken to be expected approximations of the start and end positions, respectively.

FIG. 2 is a configuration diagram showing in more detail the configuration of the model for predicting the position of punctuation. As shown, the model for predicting phrase break locations comprises a fully connected regression layer 220 . The fully connected regression layer 220 is a model included in the punctuation position prediction unit 22 shown in FIG. Fully connected regression layer 220 includes two nodes (y ₁ and y ₂ in the figure) that are connected to all outputs from word embedding layer 110 (BERT embedding layer). The word embedding layer 110 is a model included in the word embedding section 11 shown in FIG. Nodes included _in the word embedding layer 110 (x ₁ , x ₂ , . y ₁ and y ₂ are numerical values representing the start position (start_position) and end position (end_position) of the segment break, respectively. Each of these numerical values represents the order of the word in the text. In other words, a model for predicting punctuation positions predicts the start and end positions of each of one or more intervals. In other words, the fully connected regression layer 220 is a two-dimensional regression model for predicting these two numbers.

That is, the word embedding layer 110 outputs D-dimensional continuous values x ₁ , x ₂ , . . . , x _D . The fully connected regression layer 220 outputs y ₁ and y ₂ calculated based on the above x ₁ , x ₂ , . . . , x _D . _y1 is the start position, _y2 is the end position, and _y1 and _y2 are as in equation (1) below.

f in equation (1) is an activation function. Also, a _ij and b _j are the intrinsic parameters of the fully connected regression layer 220 model. The values of these internal parameters are adjusted by learning. Note that j is either 1 or 2.

That is, the start position y ₁ and _the end position y ₂ are obtained by the word embedding layer 110 by adding the weighted sum of the output values x ₁ , x ₂ , . It is calculated by applying an activation function to the parameter value b _j added).

The outputs y ₁ and y ₂ from the fully connected regression layer 220 are checked against the intervals (correct answers) included in the training data. Specifically, the opinion analysis device 1 learns the internal parameters of the model using a criterion that minimizes the cross-entropy between the output from the fully-connected regression layer 220 and the label section appearing in the learning data. When calculating the norms of label intervals, all label intervals are used as norms, except when pruning is performed for computational efficiency without narrowing down the label intervals appearing in the training data. include.

A loss function L _span (S) represented by the following equation (2) is used during model learning.

In Expression (2), SpanSet(S) is a set of annotation intervals appearing in sentence S, which is the input text to opinion analysis device 1 . The start position of one interval l belonging to the set SpanSet(S) is Begin(l) in Equation (2), and the end position is End(l). In other words, one section (span) is defined by the value of the start position and the value of the end position. CrossEntropy(y, x) in equation (2) is y (that is, y ₁ or y ₂ ), which is one of the outputs from the fully connected regression layer 220, and the correct starting or ending position given by the training data. is the cross entropy with

In other words, L _span (S) is the sum of the cross entropy between y ₁ and the correct start position for each annotation interval and the cross entropy between y ₂ and the correct end position for all annotation intervals. It is based on

The cross entropy CrossEntropy(y,x) is calculated by equation (3) below.

In equation (3), k is an index for representing the position within sentence S. Also, max_seq_length is a value corresponding to the sequence length of the sentence S. Also, exp is an exponential function.

The opinion analysis device 1 uses the loss function L _seq (S) for the model held by the series labeling unit 12 (FIG. 1), in addition to the loss function L _span (S) for the above-described phrase break position prediction model. . The loss function L _seq (S) is the loss function in the sequence labeling problem according to the prior art. A total loss function L _total (S) of the model when one sentence (S) is input to the opinion analysis device 1 is calculated as shown in Equation (4) below.

That is, the overall loss function L _total (S) is the loss function L _span (S) for the phrase break position prediction model and the conventional loss function L _seq (S ).

Of the models held by the sequence labeling section 12 and the models held by the phrase break position prediction section 22, the model held by the phrase break position prediction section 22 is used only during learning and not used during inference. In other words, in the process of learning progressing so that the value of the overall loss function (equation (4) above) decreases, the learning accuracy of the distributed representation of the lower layer (the word embedding layer 110 of the word embedding unit 11) improves. . This improves the accuracy of labeling (a task of identifying an opinion target section or an opinion section) by the model of the series labeling unit 12 .

FIG. 3 is a schematic diagram showing an example of text input to the opinion analysis device 1 and an example of labeling performed on the input text. In other words, FIG. 3 is a schematic diagram showing an example of processing for extracting an opinion target section or an opinion section from an input text.

The illustrated input text example is a simple sentence posted on a certain SNS. This text is "No violence, no #anime". It should be noted that "#anime" in this text is an expression according to the notation of tags (called hashtags because "#" is used) for facilitating searches for specific topics. This input text is divided into eight words "violence/ha/donai/,/donai/yo/#/anime" (the slashes denote word delimiters). Here, punctuation marks and symbols such as "#" are treated as one of words for the sake of convenience. Opinion target sections extracted from such an input text are "violence" (start position 1, end position 1) and "#animation" (start position 7, end position 8). Also, the extracted opinion sections are "not good" (start position is 3, end position is 3) and "not good" (start position is 5, end position is 5).

FIG. 4 is a schematic diagram showing an example of a correct word sequence corresponding to the input text example shown in FIG. 3 and a tag sequence (label sequence) corresponding to the sequence. An example word sequence is "violence/ha/don't/,/don't/yo/#/anime." In this figure, numbers (position numbers) are given according to the order of the words. A correct tag sequence corresponding to such a word sequence is "TARGET/O/OPINION/O/OPINION/O/TARGET/TARGET". The tag "TARGET" is a tag indicating that the corresponding word belongs to the opinion target section. The tag "OPINION" is a tag indicating that the corresponding word belongs to the opinion section. The tag "O" is a tag that indicates that the corresponding word belongs to neither the opinion target section nor the opinion section.

In other words, the correct tag series is the expression "violence" (the section start position is 1 and the end position is 1) and the expression "#anime" (the section start position is 7 and the end position is 1). Each of 8) is an opinion target section. In addition, each of "Ikenai" (section start position is 3 and end position is 3) and "Ikenai" (section start position is 5 and end position is 5) in the word sequence is an opinion section. .

FIG. 5 is a schematic diagram showing another example of a correct word sequence corresponding to the example input text shown in FIG. 3 and a tag sequence corresponding to the sequence. As a difference from the tags in FIG. 4, in the example in FIG. 5, each tag of "TARGET" and "OPINION" is prefixed with "B-" or "I-". Such tags are called BIO format tags. "B-" indicates the beginning of the interval. Also, "I-" indicates that it is in the middle of the section. The tag given to the word at the end of the section also starts with "I-". In the example shown in FIG. 5, the correct tag sequence corresponding to the word sequence "violence/wa/don't/,/don't/yo/#/anime" is "B-TARGET/O/B-OPINION/O/B -OPINION/O/B-TARGET/I-TARGET".

In other words, the correct tag series is the expression "violence" (the section start position is 1 and the end position is 1) and the expression "#anime" (the section start position is 7 and the end position is 1). Each of 8) is an opinion target section. In addition, each of "Ikenai" (section start position is 3 and end position is 3) and "Ikenai" (section start position is 5 and end position is 5) in the word sequence is an opinion section. . In the BIO format tag, by adding "B-" or "I-", for example, the seventh word and the eighth word in FIG. 5 explicitly indicate that they belong to the same section. represent.

FIG. 6 is a schematic diagram showing an example of correct data for learning a phrase break position prediction model. As illustrated, this correct answer data is given as a set of pairs of section start and end positions. In the illustrated example, the correct answer data includes data regarding four sections. There are four intervals: start position 1, end position 1, start position 3, end position 3, start position 5, end position 5, start position 7, end position 8. . This correct answer data corresponds to the examples shown in FIGS. The data shown in FIG. 6 represent the correct answers for the start and end positions also shown in FIG.

FIG. 7 is a flowchart showing the procedure of processing when the opinion analysis device 1 performs model learning. The procedure for model learning will be described below along this flow chart.

In step S11, the learning data supply unit 30 supplies learning data. The learning data is a pair of input text (word sequence) and correct data corresponding to the input data. The correct answer data includes the correct answer of the tag sequence and the position of the section (start position and end position). The word embedding unit 11 reads input text (word sequence). Also, the series labeling loss function calculation unit 17 reads the correct tag series. Then, the loss function calculation unit 27 for the punctuation position reads the correct data (for example, FIG. 6) of the start position and end position of the section.

In step S12, the opinion analysis device 1 adjusts the internal parameters of each model based on the calculated loss function value. Specifically, it is as follows.

The word embedding unit 11 passes the series of distributed expressions corresponding to the input word series to the series labeling unit 12 and the phrase break position prediction unit 22 . The sequence labeling unit 12 obtains a sequence label using a sequence labeling model having internal parameter values at that time.
Also, the phrase break position prediction unit 22 obtains the phrase break position using a phrase break position prediction model based on the internal parameter value at that time. The series labeling loss function calculation unit 17 calculates a loss function value based on the series label output by the series labeling unit 12 and the correct series label. The phrase break position loss function calculator 27 calculates a loss function value based on the phrase break positions output by the phrase break position prediction unit 22 and the correct phrase break positions (start position and end position). Based on these calculated loss function values, the internal parameters of each model are adjusted by the error backpropagation method. Repeated adjustment of the internal parameters of such a model acts in the direction of decreasing the loss function value. In other words, by repeating parameter adjustment, each model can calculate an output value that is closer to the correct answer. The method of adjusting parameters of the model by the error backpropagation method itself belongs to the existing technology.

In step S13, the opinion analysis device 1 saves the state of each model after adjusting the parameters. Specifically, the opinion analysis device 1 writes the adjusted parameter values into a non-volatile memory or the like.

FIG. 8 is a flow chart showing a sequence labeling process performed by the opinion analysis device 1 using a trained model. Since the state of the model is saved in step S13 of the processing procedure shown in FIG. 7, the opinion analysis device 1 can read the state of each model. The procedure for estimating the sequence label will be described below along this flowchart.

In step S21, the word embedding section 11 reads the input text (word sequence).

At step S22, the opinion analysis device 1 uses the model to infer a tag sequence corresponding to the input text. Specifically, the word embedding unit 11 having the word embedding model outputs a series of vectors calculated by the word embedding model. A sequence labeling unit 12 having a sequence labeling model obtains a tag sequence based on the vector sequence passed from the word embedding unit 11 .

In step S23, the series labeling unit 12 outputs the tag series obtained in step S22. Examples of tag sequences corresponding to input texts (word strings) are as shown in FIGS. 4 and 5. FIG.

As described above, by using the opinion analysis apparatus of this embodiment, it is possible to improve the accuracy of tags to be added in the process of automatically adding tag sequences corresponding to input word sequences. As a specific example, for each word in an input word sequence, a tag indicating that it is a word that expresses an opinion object or a tag that indicates that it is a word that expresses an opinion can be added with high accuracy. . In other words, it becomes possible to accurately extract a part having a specific property (for example, an object of opinion, an opinion, etc.) in an input word string.

Such a device of the present embodiment can be used to extract a portion of a specific nature (for example, an opinion target portion, an opinion portion, etc.) from text posted on an SNS. For example, when automatically analyzing the reaction to viewing of specific content (broadcast program, etc.) based on the content posted on SNS, which part is the object of opinion and which part is the opinion in the text of the posted content itself. These parts can be automatically extracted with high accuracy even if they are not specified. Alternatively, even if the text of the posted content does not include a hash tag or the like indicating which content is being resonated with, it is possible to automatically extract the opinion target portion, the opinion portion, or the like. In other words, according to the apparatus of the present embodiment, it is possible to perform more powerful and accurate analysis than methods such as keyword searches. That is, according to this embodiment, it is possible to easily and accurately conduct large-scale market research.

If it is possible to automatically extract opinion targets and opinions from a large amount of texts (for example, thousands to hundreds of thousands) posted on SNS, etc., it will be possible to It will be possible to automatically tally how many opinions were expressed. At the time of this tabulation, it is also possible to perform tabulation on the same object of opinion, or to perform a clustering process to collect similar object of opinion. Techniques according to conventional techniques can be used as aggregation techniques and clustering techniques. For example, it is possible to automatically aggregate viewers' opinions on broadcast programs.

FIG. 9 is a block diagram showing an example of the internal configuration of the opinion analysis device 1 of the above embodiment. The opinion analysis device 1 can be implemented using a computer. As shown, the computer includes a central processing unit 901 , RAM 902 , input/output ports 903 , input/output devices 904 and 905 and the like, and bus 906 . The computer itself can be implemented using existing technology. The central processing unit 901 executes instructions included in programs read from the RAM 902 or the like. The central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic operations and logical operations according to each instruction. A RAM 902 stores data and programs. Each element contained in RAM 902 has an address and can be accessed using the address. Note that RAM is an abbreviation for "random access memory". The input/output port 903 is a port for the central processing unit 901 to exchange data with an external input/output device or the like. Input/output devices 904 and 905 are input/output devices. The input/output devices 904 and 905 exchange data with the central processing unit 901 via the input/output port 903 . Bus 906 is a common communication path used inside the computer. For example, central processing unit 901 reads and writes data in RAM 902 via bus 906 . Also, for example, central processing unit 901 accesses input/output ports via bus 906 .

At least part of the functions of the opinion analysis device 1 described above can be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read into a computer system and executed. It should be noted that the "computer system" referred to here includes hardware such as an OS and peripheral devices. In addition, “computer-readable recording media” refers to portable media such as flexible discs, magneto-optical discs, ROMs, CD-ROMs, DVD-ROMs, USB memories, and storage devices such as hard disks built into computer systems. Say things. In other words, the "computer-readable recording medium" may be a non-transitory computer-readable recording medium. In addition, "computer-readable recording medium" means a medium that temporarily and dynamically retains a program, such as a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line. , it may also include something that holds the program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or client in that case. Further, the program may be for realizing part of the functions described above, or may be a program capable of realizing the functions described above in combination with a program already recorded in the computer system.

Although a plurality of embodiments have been described above, the present invention can also be implemented in the following modifications.

[First modification]
In the above description, the text to be processed was written in Japanese, but the present embodiment may be applied to data other than Japanese. In the demonstration experiment described later, in addition to Japanese text data, English text data was also processed. Furthermore, texts in other languages may also be processed.

[Second modification]
In the above-described embodiment, one opinion analysis device 1 also performs model learning, and uses the learned model to assign (estimate) a label sequence. As a modification, the opinion analysis device 1 may perform either model learning or label sequence estimation using a trained model. When the opinion analysis device 1 functions as a machine learning device, it is possible to transplant the parameters of the learned model to another device and cause the transplanted device to assign (estimate) a label sequence. can. Alternatively, the opinion analysis device 1 may not perform machine learning by itself, but may acquire parameters of a model that has been learned and assign (estimate) a label sequence.

In the latter case, the opinion analysis device 1 receives a word embedding unit that inputs a word string and outputs a word embedding expression string corresponding to the word string, and a word embedding expression string that is output from the word embedding unit. and a sequence labeling unit that outputs a label sequence corresponding to the word embedding expression sequence. At this time, at least as a model that the word embedding unit 11 has inside, one that has been learned using the mechanism of machine learning (machine learning device) described in the above embodiment is used.

Although the embodiment of the present invention and its modifications have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and designs and the like are possible without departing from the gist of the present invention. included.

[Implementation and Evaluation]
The opinion analysis device 1 was implemented and evaluated by experiments using actual posted sentences on SNS (Twitter) and review sentences about movies. Evaluation results of the experiment will be described.

The data used for evaluation are Japanese tweets about TV programs and English reviews about movies. Specifically, as Japanese text, tweets in Japanese about the TV program (NHK morning drama) "Natsuzora" on Twitter for 9 days (the number of tweets is 24,610, the number of words is 984,861) was used. Also, as the English text, a review of the review site IMDb (986 reviews, 10,360 sentences, 321,807 words) was used. Correct answers were given manually to the learning data.

As evaluation scales, precision, recall, and f1 were used. These are defined by equations (5), (6) and (7), respectively. Here, TP is true positive, FP is false positive, and FN is false negative.

Table 1 shows the experimental results when the label was restricted to the first layer, the highest level, for the task of analyzing texts (tweets) in Japanese about broadcast programs.

In this experiment, the labels are systematized into three layers, and the first layer is roughly divided into two types. The two types are a REFERENCE label assigned to a part referring to the contents of a broadcast program and an OPINION label assigned to a part describing the sender's subjectivity. REFERENCE-type labels have respective labels of TITLE, MUSIC, PERSON, PROGRAM, SCENE, STORY, and QUOTE in the second hierarchy. On the other hand, OPINION-based labels have labels of EVALUATION, ACTION, and INDEX in the second hierarchy. Among them, ECALUATION in the second hierarchy has respective labels of POSITIVE, NEGATIVE, NEUTRAL, and REQUEST in the third hierarchy.

In the experiment, the REFERENCE label and the OPINION label of the first layer were individually evaluated as evaluation targets, and an overall evaluation was also performed. The conventional technology to be compared is a case in which the punctuation position predictor 22 (see FIG. 1) is not used even during learning.

For all other cases, the results of the present embodiment exceed the results of the prior art, except for the REFERENCE label of the recall scale, where the values of the results of the present embodiment were slightly below those of the prior art. was confirmed.

Note that the effect of using this embodiment varies depending on the number of epochs when learning the model. At the initial stage of learning (stage of about 10 epochs or less), there is no significant difference in the f1 value evaluation performance between the conventional technique and the present embodiment. However, the effect of the present embodiment begins to be seen after about 10 epochs, and the superiority of the present embodiment (the performance difference in the f1 value with respect to the conventional technology) becomes noticeable at about 20 epochs.

Table 2 shows the experimental results for the task of analyzing movie review texts written in English when the labels are restricted to the first layer, which is the highest level. As in the case of Table 1, also in Table 2, the REFERENCE label and the OPINION label of the first layer were evaluated individually and evaluated as a whole.

Analysis of this English text also confirms that the present embodiment outperforms the results of the prior art in most cases. The evaluation result of the present embodiment is lower than the evaluation result of the prior art only with respect to the REFERENCE label of the evaluation scale recall. In other cases, the evaluation result of this embodiment exceeds the evaluation of the prior art.

Table 3 below shows significance levels (p-values) that indicate whether or not the improvement in this embodiment is significant compared to the prior art. Shown here are the significance levels when the McNemar test is performed.

Table 4 below shows the experimental results for the task of analyzing Japanese texts (tweets) about broadcast programs when the label hierarchy is not restricted. The numerical values shown in Table 4 are the overall evaluation results (micro-average) of all labels.

As shown in Tables 1, 2, and 4 above, the accuracy of extracting an expression of a specific position in a text (extracting an opinion target) according to the present embodiment is better than that of the prior art. It was confirmed by experiments. Moreover, as shown in Table 3, it was confirmed by the test that the improvement in accuracy was significant.

INDUSTRIAL APPLICABILITY The present invention can be used, for example, to extract an element with a specific position from a sentence written in natural language. As an example, it can be used to extract a specific positioning element from a text posted on SNS. However, the scope of application of the present invention is not limited to those exemplified here.

1 Opinion analysis device (machine learning device, natural language processing device)
11 Word embedding unit 12 Sequence labeling unit 17 Sequence labeling loss function calculation unit 22 Phrase break position prediction unit 27 Phrase break position loss function calculation unit 30 Learning data supply unit 220 Fully connected regression layer 110 Word embedding layer (BERT embedding layer )
901 central processing unit 902 RAM
903 input/output ports 904, 905 input/output device 906 bus

Claims (6)

a word embedding unit that inputs a word string and outputs a word embedding expression string corresponding to the word string;
a sequence labeling unit that inputs the word embedding expression sequence output from the word embedding unit and outputs a label sequence corresponding to the word embedding expression sequence;
a phrase break position prediction unit that inputs the word embedding expression string output from the word embedding unit and outputs phrase break position information corresponding to the word embedding expression string;
A word string to be input to the word embedding unit is supplied, a correct label sequence corresponding to the label sequence output by the sequence labeling unit is supplied, and the phrase break position output by the phrase break position prediction unit is supplied. a learning data supply unit that supplies information on correct phrase break positions corresponding to information;
with
The sequence labeling unit adjusts the parameters of the internal model by performing backpropagation based on the error between the label sequence and the correct label sequence,
The phrase break position prediction unit adjusts the parameters of the internal model by performing backpropagation based on the error between the phrase break position information and the correct phrase break position information,
The word embedding unit adjusts the parameters of the internal model by backpropagating errors from the sequence labeling unit and backpropagating errors from the phrase break position prediction unit.
Machine learning device. The phrase break position prediction unit predicts the phrase break position information corresponding to the word embedding expression string by using a fully connected regression model based on all word embedding expressions contained in the input word embedding expression string. Output,
The machine learning device according to claim 1. The sequence labeling unit outputs at least a label indicating that a predetermined subsequence included in the input original word string is an opinion object and a label indicating that the predetermined subsequence is an opinion. do,
The machine learning device according to claim 1 or 2. The phrase break position prediction unit outputs numerical values representing the start position and end position of the subsequence as the phrase break position information,
The correct phrase break position information supplied by the learning data supply unit is numerical information representing the correct start position and end position of the subsequence.
The machine learning device according to claim 3. a word embedding unit that inputs a word string and outputs a word embedding expression string corresponding to the word string;
a sequence labeling unit that inputs the word embedding expression sequence output from the word embedding unit and outputs a label sequence corresponding to the word embedding expression sequence;
with
At least the model that the word embedding unit has inside has been learned by the machine learning device according to any one of claims 1 to 4,
Natural language processor. a word embedding unit that inputs a word string and outputs a word embedding expression string corresponding to the word string;
a sequence labeling unit that inputs the word embedding expression sequence output from the word embedding unit and outputs a label sequence corresponding to the word embedding expression sequence;
a phrase break position prediction unit that inputs the word embedding expression string output from the word embedding unit and outputs phrase break position information corresponding to the word embedding expression string;
A word string to be input to the word embedding unit is supplied, a correct label sequence corresponding to the label sequence output by the sequence labeling unit is supplied, and the phrase break position output by the phrase break position prediction unit is supplied. a learning data supply unit that supplies information on correct phrase break positions corresponding to information;
with
The sequence labeling unit adjusts the parameters of the internal model by performing backpropagation based on the error between the label sequence and the correct label sequence,
The phrase break position prediction unit adjusts the parameters of the internal model by performing backpropagation based on the error between the phrase break position information and the correct phrase break position information,
The word embedding unit adjusts the parameters of the internal model by backpropagating errors from the sequence labeling unit and backpropagating errors from the phrase break position prediction unit.
A program that makes a computer act as a machine learning device.

Images