JP6453942B2 - Text classification device, text classification method and program - Google Patents

Description

  The present invention relates to a text classification device, a text classification method, and a program.

  In industries such as call centers where customer support is required, it is necessary to refer to various information when responding to telephone calls and to answer customer questions while utilizing know-how based on experience.

  The contents of questions and answers that were conducted in the past in customer support are prepared for subsequent questions of the same type, printed on paper media, etc., and manuals are distributed to each operator to improve the efficiency of customer support operations. (For example, refer to Patent Document 1).

JP 2002-157445 A

  However, when an inexperienced operator is available to answer a phone call or when it is necessary to answer a phone call in an unfamiliar field, the number of manual calls tends to increase, and the number of calls held tends to increase, resulting in a longer response time. It is in. As a result, problems have been pointed out such as stressing customers and the cost of call centers rising.

  The present invention has been made in view of the circumstances described above, and provides a text classification technique suitable for realizing a prompt and appropriate response to a question from a customer in a telephone customer support operation or the like. Is one of the purposes.

  A text classification apparatus according to an aspect of the present invention is a text classification apparatus that performs natural language class classification using a classifier, and inputs a plurality of learning data including a plurality of classifiers, learning text data, and a learning category. Assigning a part of a plurality of input learning data as learning data for accuracy check and assigning the remaining learning data as learning data for learning for each input unit and classifier A learning unit for learning the corresponding classifier using the learning data for learning, and using the learning data for checking the assigned accuracy to check the accuracy of learning of the corresponding classifier, In any one classifier, the assigning unit assigns a part of the learning data to be assigned as learning data for accuracy check to any one of the remaining classifications. In allocates as learning data for learning.

  The above configuration may further include a classification unit that integrates the classifiers and classifies the text data input after the learning of the classifiers is completed.

  In the above configuration, the plurality of classifiers are text classifiers using a predetermined model, and at least one of the plurality of classifiers is a text classifier using a model different from the other classifiers. There may be.

  In the above configuration, at least one classifier may be a text classifier using an LRC model, and the other classifier may be a text classifier using a DNN model.

  A text classification apparatus according to another aspect of the present invention is a text classification apparatus that performs natural language class classification using a classifier, and records a plurality of classifiers and master word information shared by each classifier. A master word corpus, a plurality of domain word corpora that record domain word information generated for each classifier, and a classifier that classifies input text data using each classifier, The domain word corpus corresponding to the classifier is preferentially referred to, and if the word of the text data is included in the domain word corpus, the word is read from the domain word corpus and the corresponding classification is performed together with the read word. Classify text using a container.

  In the above configuration, when the domain word corpus does not include a word of the text data, the classification unit refers to the master word corpus and complements the expression recorded on the words recorded in the master word corpus. The text classification may be performed using the corresponding classifier together with the word after completion of the expression.

  ADVANTAGE OF THE INVENTION According to this invention, the text classification technique suitable for implement | achieving a prompt and appropriate response | compatibility with respect to a customer's question etc. in the customer support business etc. by telephone can be provided.

It is explanatory drawing which shows the basic principle of the text classification system which concerns on this embodiment. It is the figure which illustrated the operation | use in a call center. It is the figure which illustrated learning data. It is the figure which illustrated the operation | use in a call center. It is a block diagram which shows the function structure of a text classification device. It is the figure which illustrated multiple types of classifiers. It is a figure for demonstrating the utilization method of each corpus. It is a figure for demonstrating the utilization method of learning data. It is a figure for demonstrating the utilization method of learning data. It is the figure which displayed the distributed expression of the word two-dimensionally. It is a figure for demonstrating learning of word dispersion | distribution expression. It is a figure which shows a learning result when learning of word dispersion | distribution expression is implemented in multiple times. It is a figure which shows the state which applied the word between different corpora. It is a schematic diagram for demonstrating the complementation of expression. It is a flowchart which shows the processing flow of expression complementation.

  Embodiments of the present invention will be described below. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, the drawings are schematic. Therefore, specific dimensions and the like should be determined in light of the following description. Moreover, it is a matter of course that portions having different dimensional relationships and ratios are included between the drawings. Furthermore, the technical scope of the present invention should not be understood as being limited to the embodiment.

A. Embodiment A-1. Basic Principle FIG. 1 is an explanatory diagram showing the basic principle of a text classification system 100 according to this embodiment. The text classification system 100 is a system for improving the efficiency of “response to inquiry” that is suitable for use in a call center or the like, and performs natural language processing (class classification) using a natural sentence classifier. As shown in FIG. 1, the text classification system 100 includes a learning processing function F1 and a classification processing function F2.

<Learning processing function F1>
The natural sentence classification processing unit 120 that performs natural language class classification using deep learning is provided with a large amount of natural sentences (for example, inquiries) and classifications corresponding to the natural sentences (for example, responses to inquiries) from the user server 110. A set of “classification” (hereinafter abbreviated as “category”) is received as learning data Dl. The natural sentence classification processing unit 120 creates a text classifier 130 that performs class classification by machine learning based on the received learning data.

  The operation at the call center shown in FIG. 2 will be described as an example. First, the administrator appropriately operates the user server 110 while referring to the inquiry contents and logs during work, and the text data and categories representing the inquiry contents. Learning data Dl that is a set of FIG. 3 is a diagram illustrating the learning data Dl. As shown in FIG. 3, the learning data Dl includes text data (hereinafter referred to as learning text data) Dlt that is actually written from the inquiry content and a category (hereinafter referred to as “category”) in which the content actually answered in response to the inquiry is classified. Learning category) Cly. As an example, for example, a query “... I want to know about slowdown” (ie, learning text data Dlt) is associated with a category (ie, learning category Cly) “Speedlimit” (FIG. 3). It should be noted that how many sets of learning data D1 are prepared and how many categories are registered can be appropriately set and changed according to the system design or the like. When the administrator creates a plurality of sets of learning data Dl as shown in FIG. 3, the administrator outputs the created learning data Dl together with the learning request Rl to the natural sentence classification processing unit 120 (see FIG. 2).

<Classification processing function F2>
On the other hand, when performing the classification process, the natural sentence classification processing unit 120, when receiving the natural sentence Sn from the outside (such as the user server 110), appropriately analyzes the received natural sentence Sn using the classifier 130. The category Cy is determined, and the determined category Cy is output.

  The operation at the call center shown in FIG. 4 will be described as an example. First, the operator appropriately operates a given PC terminal or the like and inputs the customer's inquiry content (for example, “I want to change to XX”). ). When the user server 110 receives the text data Dtw indicating the inquiry content, the user server 110 outputs the received text data Dtw to the natural sentence classification processing unit 120 together with the classification request Rc. The natural sentence classification processing unit 120 analyzes the received text data Dtw using the text classifier 130 and determines the category Cy. The natural sentence classification processing unit 120 refers to the learning data of the text classifier 130, and for example, for text data Dtw “I want to change to XX”, the category Cy “change rate plan” is optimal. When the determination is made, the category Cy “charge plan change” is returned to the user server 110 as the determination result. Based on the response result from the natural sentence classification processing unit 120, the user server 110 displays a work screen optimal for changing the charge plan on the operator's PC terminal or the like. The operator asks the customer for information necessary for changing the contract plan as an answer to the inquiry while referring to the business screen displayed on the PC terminal or the like. Hereinafter, the text classification device 200 including the natural sentence classification processing unit 120 according to the present embodiment will be described.

A-2. Configuration FIG. 5 is a block diagram showing a functional configuration of the text classification device 200. The text classification device 200 is configured by a personal computer, for example, and includes a communication interface unit 140, an operation unit 150, and a storage unit 160 in addition to a natural sentence classification processing unit 120 and a classifier 130.

  The natural sentence classification processing unit 120 includes, for example, a processor such as a CPU, an ASIC, and an FPGA, and a memory such as a ROM and a RAM. The processor executes various control programs stored in the memory, so that the learning unit 121 and the classification unit 122 functions are provided. Details of the functions of each unit included in the natural sentence classification processing unit 120 will be described later.

  The classifier 130 analyzes the input text data Dtw under the control of the natural sentence classification processing unit 120 and determines the category Cy. As shown in FIG. 5, in this embodiment, two types of classifiers 130 using a DNN (Deep Neural Network) model are used. However, the number of classifiers is not limited to two and may be three or more. In this case, the classifier to be used is not limited to the one using the DNN model. For example, in addition to the two text classifiers using the DNN model, LRC (Logistic Regression Classifier) is used. One type of text classifier using the model may be used (see, for example, FIG. 6). The reason for using a plurality of classifiers in this way is to improve the performance (classification accuracy, etc.) of the machine learning classifier. For example, SVM (Support Vector Machine) may be used as the type of classifier to be used.

  The communication interface unit 140 includes a device connected to an external network and the like, and exchanges various data with the user server 110 and the like via a communication network such as the Internet or an intranet.

  The operation unit 150 includes an input device such as a keyboard and a mouse, a display panel that displays various information, and the like.

  The storage unit 160 includes, for example, a rewritable nonvolatile memory such as an HDD (Hard Disk Drive) and an SDD (Solid State Drive). The storage unit 160 includes one type of master word corpus Cm and a plurality of domain word corpora Cd.

  The master word corpus Cm is based on a large amount of text information obtained from various sites (externally) on the Internet, for example, in advance, and word and word semantic information (hereinafter collectively referred to as “master word information”). It is an accumulation. The master word corpus Cm is a corpus shared by a plurality of classifiers 130. Prior to the operation of this system, the master word corpus Cm may be created in advance by another device and maintained and managed by the natural sentence classification processing unit 120. In addition, the natural sentence classification processing unit 120 may centrally create, maintain, and manage the master word corpus Cm.

  The domain word corpus Cd is provided for each classifier 130, and in this embodiment, two domain word corpora Cd are provided. The domain word corpus Cd is a unique word or word semantic information (eg, a word or word semantic information used only in a specific industry or a specific company; Collectively referred to as “word information”). When generating each classifier 130, the natural sentence classification processing unit 120 generates each corresponding domain word corpus Cd, and maintains and manages each classifier 130.

  As is well known, when text classification is performed using machine learning, it is possible to improve classification accuracy by learning each word and word semantic information in advance. Therefore, in the present embodiment, the above-described master word corpus Cm and domain word corpus Cd are created, and text classification is performed using these master word corpus Cm and domain word corpus Cd, thereby improving classification accuracy. . Although details will be described later, the master word corpus Cm has an enormous amount of data compared to each domain word corpus Cd, and it is difficult to hold and manage multiple versions. Therefore, in this embodiment, when analyzing the input text data Dtw and determining the category Cy, the natural sentence classification processing unit 120 preferentially refers to the domain word corpus Cd of the corresponding classifier 130, and It is checked whether or not the word of the text data Dtw is included in the domain word corpus Cd (see FIG. 7). Only when it is determined that the domain word corpus Cd does not include the word of the text data Dtw, the natural sentence classification processing unit 120 refers to the master word corpus Cm and extracts the word of the text data Dtw from the master word corpus Cm. Read the data. Thereby, each corpus can be used efficiently.

A-3. Learning Data Utilization Method FIG. 8 is a diagram for explaining a learning data utilization method.
As described above, the natural sentence classification processing unit 120 receives, from the user server 110, a plurality of pieces of learning data D1 including a set of learning text data Dlt (inquiry contents) and a learning category Cly (classification of answered contents). Upon receipt, the classifier 130 is created and learned based on the received learning data Dl. When learning the classifier 130, in order to monitor the progress of learning, it is necessary to check the accuracy when the classifier 130 actually classifies the input data. As one of the methods for confirming accuracy, as shown in FIG. 6, a part of the plurality of learning data Dl input to the natural sentence classification processing unit 120 is assigned to the learning data Dlv for accuracy confirmation, and the remaining A method is conceivable in which the learning data Dl is assigned to the original learning data Dll for learning. However, simply allocating a part of the plurality of input learning data Dl to the learning data Dlv for accuracy check causes a problem that a part of the learning data is not reflected in the learning.

  Therefore, as shown in FIG. 9, the learning unit (assignment unit) 121 of the natural sentence classification processing unit 120 receives the learning data Dl input from the user server 110 via the communication interface unit (input unit) 140. Provided to two classifiers 130 (classifier A and classifier B in FIG. 9), a part of learning data Dl assigned to learning data Dlv for accuracy check in one classifier 130 is assigned to the other classifier The assignment of the learning data Dl is controlled so as to be assigned to the original learning data Dll for learning at 130.

  Referring to FIG. 9 as an example, a part of the learning data that is not used for learning in the classifier A (that is, the learning data Dlv for accuracy verification in the classifier A) is used for the original learning in the classifier B. Assigned to learning data Dll. On the other hand, part of the learning data that is not used for learning in the classifier B (that is, learning data Dlv for accuracy verification in the classifier B) is assigned to the original learning data Dll for learning in the classifier A. The learning unit 121 learns the corresponding classifier 130 using the assigned original learning data Dll for learning, and also uses the assigned accuracy verification learning data Dlv for the corresponding classifier 130. Check the accuracy of learning.

  As a result, it is possible to realize accurate accuracy confirmation (accuracy verification) while using the entire input learning data for learning. When the text data input from the outside is classified after learning is completed, the classification unit 122 of the natural sentence classification processing unit 120 includes two classifiers 130 (classifier A and classifier B in FIG. 9). By integrating and using, text classification with high accuracy is realized.

A-4. Completion of Word Corpus As already described, in the present embodiment, the domain word corpus Cd is preferentially used while assuming that both the master word corpus Cm and the domain word corpus Cd are used, and the domain word corpus Cd is used. Only when it is determined that the word of the text data Dtw is not included, the master word corpus Cm is referred to, and the word of the text data Dtw is read from the master word corpus Cm (see FIG. 7). At this time, simple word fitting cannot be performed between different corpora (that is, between the master word corpus Cm and the domain word corpus Cd) as described later. Therefore, in this embodiment, “complementation of expression” is performed between the master word information stored in the master word corpus Cm and the domain word information stored in the domain word corpus Cd. The details of the expression complement will be described below.

  As a premise for expression complementation, in natural language processing using NLC, expressions that express the meaning of words as low-dimensional real-valued vectors (so-called distributed expression of words) are widely used. Here, FIG. 10 is a diagram in which two-dimensional representations of word dispersion expressions (word dispersion expressions) are displayed.

  Each word can be expressed as a real-valued vector of any predetermined dimension, and the closeness of the meaning of each word (ie, similarity between words) is obtained by calculating the Euclidean distance between each vector. Can do. In FIG. 10, each of the words “apple”, “orange”, “smart phone”, and “high-function mobile terminal” is a two-dimensional real value vector (0.1, 0.2), (0.2, 0. 1), (0.4, 0.9), and (0.5, 0.8).

FIG. 11 is a diagram for explaining learning of word dispersion expressions.
In the example shown in FIG. 11, the word dispersion expression learning is performed so that the distance between the word vectors is closer as the meaning of each word is closer. As a result, from the state where each word is randomly arranged (see “before learning” shown in FIG. 11), the closer the meaning of each word is, the closer the distance between word vectors is (see “learning” shown in FIG. See later). In FIG. 11, it can be seen that the distance between “apple” and “orange” and the distance between “smartphone” and “high function mobile terminal” are shortened (approaching each). In learning, sentences (documents, summaries, etc.) including each word are used.

FIG. 12 is a diagram illustrating a learning result when the word dispersion expression learning is performed a plurality of times.
As is clear from comparison between post-learning A and post-learning B shown in FIG. 12, both post-learning A and post-learning B have similar words (“apple” and “orange”, “smartphone” and “ Although the high-performance portable terminal ") is relatively close, it can be seen that the value of the vector of each word itself changes with each learning. In the example of “apple” and “orange” shown in FIG. 12, the distance between each word is relatively short in both after learning A and after learning B, but the vector value of “apple” is , After learning (0.1, 0.2), after learning B (0.3, 0.8), the vector value of “orange” is A after learning (0.2, 0.1) Then, after learning, the vector value of each word changes every time learning is performed, such as (0.4, 0.9).

  Due to the above-mentioned nature of the word distribution expression, the relative distance between words does not make sense if only a word (word vector) of one corpus is applied to another corpus (that is, the closer the meaning of each word is, the more the word vector) The correlation that the distance between them becomes closer is broken.) FIG. 13 is a diagram illustrating a state in which words are applied between different corpora. As shown in FIG. 13, it is assumed that the word “multifunctional mobile phone” registered in the corpus A is copied to the corpus B. In this case, the word vector of the word “multifunctional mobile phone” is (0.3, 0.9) in both the corpus A and the corpus B. However, the relative distance between the extracted word “multi-function mobile phone” and other words (such as “smartphone”) varies greatly between corpora. That is, on the corpus A, the extracted word “multi-function mobile phone” is located near the word “smart phone” and the word “high-function mobile terminal”, whereas on the corpus B, the copied word “multi-function mobile phone” "Is closer to the word" orange "and the word" apple "than the word" smartphone "and the word" high-function mobile terminal ".

  In order to deal with such a problem, in this embodiment, “complementation of expression” is performed. Here, FIG. 14 is a schematic diagram for explaining complementation of expressions. The implementation of expression completion is an arbitrary conversion function. Specifically, as shown in FIG. 14, when an arbitrary word x of corpus A is copied to corpus B, learning is performed so that appropriate conversion can be performed by the linear conversion determinant Wx + b (W; linear conversion Matrix, b; bias). As a method for obtaining an appropriate conversion, there is a method in which a word group common between both corpuses (corpus A and corpus B in FIG. 14) can be converted most accurately by using machine learning in advance. . By using the optimum conversion function obtained in this way, the distance between each word is kept substantially constant between the transfer source corpus A and the transfer destination corpus B (see FIG. 14). Note that the concept of expression complementation is based on, for example, previous papers (R. Kiros, Y. Zhu, R. Salakhutdinov, RS Zemel, A. Torralba, R. Urtasun, and S. Fidler. Skip-Thought Vectors. In NIPS, 2015).

FIG. 15 is a flowchart showing a processing flow for complementing expressions.
When the classification unit 122 of the natural sentence classification processing unit 120 receives the text data Dtw from the user server 110, first, the classification unit 122 preferentially refers to the domain word corpus Cd of a predetermined classifier 130 (classifier A or the like) (step S1). S1), it is checked whether or not the word of the text data Dtw is included in the domain word corpus Cd (see FIG. 7; step S2). If the classification unit 122 determines that the word of the text data Dtw is included in the domain word corpus Cd (step S2; YES), the classification unit 122 reads out the corresponding word from the domain word corpus Cd (step S3), and the corresponding classifier 130. Is used to classify the text (that is, determine and output an appropriate category Cy), return the classification result to the user server 110 (step S4), and terminate the process.

  On the other hand, when the classification unit 122 determines that the word of the text data Dtw is not included in the domain word corpus Cd (step S2; NO), the classification unit 122 refers to the master word corpus Cm (step S5). Then, when the classification unit 122 finds that there is a word corresponding to the master word corpus Cm, the classification unit 122 complements the expression as shown in FIG. 14 (step S6). By performing the expression complementation, the classification unit 122 can appropriately operate the classifier 130 even in a situation where the master word information of the master word corpus Cm and the domain word information of the domain word corpus Cd are mixed. Become. The classification unit 122 performs text classification using the master word information after complementing the expression and the corresponding classifier 130, returns the classification result to the user server 110 (step S4), and ends the processing.

  As described above, according to the present embodiment, the learning data to be input is learned using the classifier A and the classifier B, and a part of the learning data that is not used for learning in the classifier A (that is, the classifier The learning data Dlv) for accuracy verification in A is assigned to the learning data Dll for original learning in the classifier B. On the other hand, a part of the learning data that is not used for learning in the classifier B (that is, learning data Dlv for accuracy verification in the classifier B) is assigned to the learning data Dll for original learning in the classifier A. As a result, it is possible to realize accurate accuracy confirmation (accuracy verification) while using the entire input learning data for learning. By increasing the learning accuracy of the classifier 130, it becomes possible to classify text quickly and accurately. By utilizing this, quick and appropriate responses to questions from customers are realized in customer support operations by telephone. can do.

  In this embodiment, the classification accuracy is improved by using a master word corpus Cm common to each classifier 130 and a domain word corpus Cd provided for each classifier 130. Here, the master word corpus Cm has an enormous amount of data compared to each domain word corpus Cd, and it is difficult to maintain and manage multiple versions. Therefore, the domain word corpus Cd of the corresponding classifier 130 is given priority. The master word corpus Cm is referred to and the word of the text data Dtw is read from the master word corpus Cm only when the domain word corpus Cd does not include the word of the text data Dtw. Thereby, each corpus can be used efficiently. By using each corpus efficiently, it is possible to classify text quickly and accurately. By using this, quick and appropriate responses to customer questions, etc. are realized in customer support operations by telephone. can do.

B. Modifications The present invention is not limited to the above-described embodiment, and can be implemented with various other modifications within the scope not departing from the gist of the present invention. In other words, the above-described embodiment is merely an example in all respects, and is not interpreted in a limited manner, and various modifications can be employed.

  In the present embodiment, when complementing expressions, the domain word corpus Cd of a predetermined classifier 130 (such as classifier A) is preferentially referenced (see FIG. 15). It is possible to arbitrarily set / change whether to refer to the domain word corpus Cd. When there are a plurality of classifiers 130 (for example, classifier A and classifier B), the classifying unit 122 first refers to, for example, the domain word corpus Cd of the classifier A, and the word of the text data Dtw is included in the domain word corpus Cd. Check if it is included or not. If the classification unit 122 determines that the word of the text data Dtw is not included in the domain word corpus Cd, the classification unit 122 refers to the domain word corpus Cd of the classifier B, and stores the text data Dtw in the domain word corpus Cd. Check if the word is included or not. The classification unit 122 may refer to the master word corpus Cm only when it is determined that none of the domain word corpus Cd includes the word of the text data Dtw.

  In the present specification, the “unit” does not simply mean a physical configuration, but includes a case where the function of the “unit” is realized by software. Further, even if the functions of one “unit” or device are realized by two or more physical configurations or devices, the functions of two or more “units” or devices are realized by one physical means or device. May be.

  In addition, the steps in each process described above in this specification can be arbitrarily changed in order or executed in parallel as long as the process contents do not contradict each other.

  A program for performing each process described in this specification may be stored in a recording medium. If this recording medium is used, the program can be installed in the text classification device 200. Here, the recording medium storing the program may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be a recording medium such as a CD-ROM.

DESCRIPTION OF SYMBOLS 100 ... Text classification system, 110 ... User server, 120 ... Natural sentence classification process part, 130 ... Classifier, 140 ... Communication interface part, 150 ... Operation part, 160 ... Memory | storage part, 200 ... Text classification apparatus, Cd ... Master Word corpus, Cd ... domain word corpus

Claims (6)

A text classification device for classifying natural language using a classifier,
Multiple classifiers;
An input unit for inputting a plurality of learning data including learning text data and a learning category;
Assigning a part of the plurality of learning data to be input for each classifier as learning data for accuracy confirmation, and assigning the remaining learning data as learning data for learning;
Learning that learns the corresponding classifier using the assigned learning data for learning, and checks the accuracy of learning of the corresponding classifier using the assigned learning data for accuracy check With
The assigning section is
A text classification device, wherein a part of the learning data assigned as learning data for accuracy check is assigned as learning data for learning in any one of the other classifiers.   The text classification device according to claim 1, further comprising a classification unit that performs text classification by integrating the classifiers with respect to text data input after learning of the classifiers is completed.   The plurality of classifiers are text classifiers using a predetermined model, and at least one of the plurality of classifiers is a text classifier using a model different from other classifiers. The text classification device according to claim 2.   The text classification device according to claim 3, wherein the at least one classifier is a text classifier using an LRC model, and the other classifier is a text classifier using a DNN model. A computer having a storage unit is a text classification method for performing natural language classification using a classifier,
The computer executes the program stored in the storage unit,
An input step for inputting a plurality of learning data including learning text data and a learning category;
Assigning a part of the plurality of learning data to be input as learning data for accuracy check and assigning the remaining learning data as learning data for learning for each of a plurality of classifiers;
Learning that learns the corresponding classifier using the assigned learning data for learning, and checks the accuracy of learning of the corresponding classifier using the assigned learning data for accuracy check Perform steps and
The assigning step includes
A text classification method in which any one of the classifiers assigns a part of the learning data to be assigned as learning data for accuracy check to the learning data for learning in any one of the remaining classifiers. A computer with a storage unit that performs natural language classification using a classifier,
Multiple classifiers;
An input unit for inputting a plurality of learning data including learning text data and a learning category;
Assigning a part of the plurality of learning data to be input for each classifier as learning data for accuracy confirmation, and assigning the remaining learning data as learning data for learning;
Learning that learns the corresponding classifier using the assigned learning data for learning, and checks the accuracy of learning of the corresponding classifier using the assigned learning data for accuracy check A program for functioning as a part,
The assigning section is
A text classification program for assigning a part of the learning data to be assigned as learning data for accuracy check in any one classifier as learning data for learning in any one of the remaining classifiers.