JP6605105B1 - Sentence symbol insertion apparatus and method - Google Patents

Abstract

There are cases where sentence symbols such as punctuation marks and punctuation marks are not properly inserted in character string text data. A punctuation insertion apparatus 300 includes morpheme and grammatical information of a target morpheme set in input text and morpheme of a target morpheme set in character string text data to be learned in which sentence symbols are appropriately inserted. Learning unit 340 that learns output data indicating whether or not is inserted as teacher data to form a learning model, and the morpheme and part information of the morpheme set of interest in the text string data to be estimated are input data An estimation unit 350 that determines whether or not a sentence symbol should be inserted between the morphemes of the target morpheme set based on the output data obtained by inputting to the learning model, and the character string text to be estimated based on the estimation result of the estimation unit 350 It comprises a punctuation point insertion unit 360 for inserting sentence symbols into data. [Selection] Figure 1

Description

  The present invention relates to a text symbol insertion device that inserts text symbols such as punctuation marks and punctuation marks.

  Currently, various voice recognition engines that output voice recognition text data by inputting voice data of one user or voice data of a plurality of users are provided. However, some voice recognition engines do not have a function to insert a punctuation mark, and even if there is a function to insert a punctuation mark, if the user does not speak for a predetermined time or longer, a simple punctuation is inserted. There are many functions.

  On the other hand, in Patent Document 1, in order to appropriately convert a free input unit into a sentence language processing unit, the likelihood of a node boundary corresponding to a phrase for a plurality of words and part-of-speech sequences based on text data subjected to morphological analysis. Is calculated in advance and stored as a statistical model, empirical knowledge of the prosodic information of the insertion point of punctuation is extracted in advance based on text data subjected to morphological analysis, and stored as an empirical rule. In addition to referring to empirical rules, in addition to speech recognition results by detecting silent sections, the speech recognition result of one input unit is divided into multiple language processing units, and the speech recognition result of multiple input units Discloses a speech language processing unit conversion device that performs a joining process for joining a single language processing unit.

Japanese Patent No. 3009642

  In the above spoken language processing unit conversion apparatus, it is necessary to create a calculation formula for a statistical model in advance, and a plurality of calculation formulas are also presented in Patent Document 1, and a plurality of calculation formulas are tested with test data. It was necessary to adopt the formula with the best performance or to improve it by trial and error.

  The present invention has been made in view of these problems, and an object of the present invention is to provide a function for more appropriately inserting sentence symbols such as punctuation marks into text.

  The sentence symbol insertion device according to the present invention uses, as input data, morpheme and grammatical information of a morpheme of interest consisting of a morpheme to be processed and a morpheme next to the process morpheme in character string text data to be learned. And a learning unit that learns, as teacher data, output data indicating whether or not a sentence symbol is inserted between morphemes of the morpheme set of interest, and forms a learning model, and character string text data is subjected to morphological analysis and divided A morpheme processing unit that outputs a morpheme and grammatical information of the morpheme, and a morpheme to be processed among the morpheme obtained by performing morpheme analysis on the character string text data to be estimated by the morpheme processing unit and the grammatical information of the morpheme And the morpheme and grammatical information of the target morpheme set consisting of the next morpheme of the processing target morpheme as input data to the learning model and output An estimator that determines whether or not a sentence symbol should be inserted between morphemes of the morpheme set of interest based on the output data, and a sentence that inserts a sentence symbol into character string text data to be estimated based on the estimation result of the estimator It consists of a symbol insertion part.

  According to the present invention, without the user creating a calculation formula for inserting a sentence symbol, the sentence symbol is appropriately inserted into the character string text data to be estimated based on the sentence symbol inserted in the teacher data. be able to.

It is a lineblock diagram of a dialog management system concerning a 1st embodiment concerning the present invention. It is an example of interactive speech recognition data according to the first embodiment of the present invention. It is an example of a morpheme analysis result concerning a 1st embodiment concerning the present invention. It is an example of teacher data concerning a 1st embodiment concerning the present invention. It is an example of the speech recognition data of the estimation process object which concerns on 1st Embodiment which concerns on this invention. It is an example of the morphological analysis result of the speech recognition data example of the estimation process target according to the first embodiment of the present invention. It is estimation processing explanatory drawing which concerns on 1st Embodiment which concerns on this invention. It is the example which inserted the punctuation into the example of the speech recognition data of the estimation process object which concerns on 1st Embodiment which concerns on this invention. It is a learning process sequence diagram concerning a 1st embodiment concerning the present invention. It is an estimation process sequence diagram concerning a 1st embodiment concerning the present invention. It is a wild card setting explanatory view concerning a 2nd embodiment concerning the present invention. It is a learning process sequence diagram concerning a 3rd embodiment concerning the present invention. It is an example of the speech recognition data classified by utterance subject type concerning other embodiments concerning the present invention. It is a contrast diagram of the example of the speech recognition data in which the punctuation has been inserted according to another embodiment of the present invention.

(First embodiment)

  Hereinafter, the same or equivalent components, members, and processes shown in the drawings are denoted by the same reference numerals, and repeated description is appropriately omitted. In addition, in the drawings, some of the members that are not important for explanation are omitted.

  FIG. 1 is a configuration diagram of a dialogue management system according to the present embodiment. The dialogue management system 1 is constructed in a call center or the like, manages the dialogue between the client and the operator, and provides a function of recording the dialogue, converting the recorded data into text, and inserting and recording a phrase in the text data. The dialogue management system 1 includes a dialogue recording device 100, a voice recognition device 200, and a phrase insertion device 300, which are connected to a network by wire or wirelessly and can communicate with each other. In the present embodiment, the dialog recording device 100, the speech recognition device 200, and the phrase insertion device 300 are configured from different hardware, but these devices can be configured on a single computer. Can be configured by a plurality of computers in functional units.

  The dialogue recording apparatus 100 records the dialogue between one speaker and another speaker as dialogue recording data that is voice data, and transmits the dialogue recording data to the voice recognition device 200. Typically, a client uses a mobile phone or a fixed phone to make a call with a call center operator via a telephone public network, and records the call via the telephone network as a target. Here, the dialogue recording data may be not only continuous voice data but also voice data for each speaker. The client's voice data sent to the call center via the telephone public network and the operator's voice data typically spoken to a telephone connected to the call center's internal network can be recorded as separate data. it can. There is also a well-known and commonly used speech separation technique for separating speech for each speaker even if the speech data is one. By using this, speech data for each speaker can be acquired.

  The voice recognition device 200 receives the dialogue recording data recorded from the dialogue recording device 100, converts the dialogue recording data into text data that is a character string using a known and common voice recognition technology, and serves as voice recognition data. It is recorded and transmitted to the phrase insertion device 300. As a speech recognition technique, there are systematically using a statistical method, a hidden Markov model, or the like. Some voice recognition devices do not have a function to insert a punctuation mark or a punctuation mark by simply converting dialogue speech into a character string, but a function to detect a speaker change or between utterances and insert a punctuation mark or a punctuation mark. Some have

  The punctuation insertion device 300 appropriately inserts punctuation into the speech recognition data that is character string text data, records the punctuation inserted speech recognition data, and records it in response to a request from the device used by the system user. It is a device that outputs speech recognition data with punctuation inserted, and is a type of sentence symbol insertion device that inserts a punctuation sentence symbol. The phrase insertion device 300 includes a pre-processing unit 310, a morpheme processing unit 320, a teacher data generation unit 330, a learning unit 340, an estimation unit 350, and a phrase insertion unit 360. Note that the wild card processing unit 370 in FIG. 1 is not used in the first embodiment, but is used in the second embodiment, and is drawn with a dotted line.

  The pre-processing unit 310 performs processing for removing predetermined symbols on the speech recognition data to be processed. In this embodiment, the predetermined symbol includes a punctuation point, but may not include a punctuation point. In this embodiment, even if the speech recognition apparatus 200 has a function of inserting a punctuation point, the punctuation point is removed from the speech recognition data to be processed by the pre-processing unit 310. Insert a new punctuation mark. In addition, the preprocessing unit 310 does not remove the punctuation points from the learning data to be learned in this embodiment.

  The morpheme processing unit 320 performs morpheme analysis on the preprocessed speech recognition data using an existing morpheme analysis technique. Here, a morpheme is the smallest linguistic unit that has meaning, and morpheme analysis is based on natural language text data based on information such as grammar and part-of-speech of words, and divides the parts of speech of the divided morphemes. It is to discriminate. In the present embodiment, the original form, the part of speech, and the utilization form are determined and assigned to the morphemes in the text data.

  The teacher data generation unit 330 converts the morphological analysis data into teacher data to be input to the learning unit 340, and outputs the teacher data to the learning unit 340. FIG. 2 shows an example of voice recognition data of the dialogue between the customer and the operator, and for the purpose of learning, the punctuation is appropriately inserted manually. FIG. 3 shows a morphological analysis of the learning speech recognition data example. The training data converted from this learning morpheme analysis data example is shown in FIG. 4, the numbers given in ascending order in the morpheme order, the original form (basic form) of the target morpheme, the part of speech and the utilization form, and the next morpheme of the target morpheme and the target morpheme It consists of the presence or absence of a punctuation mark indicating whether or not a punctuation mark is inserted between them. That is, in the conversion process, a punctuation point is specified in the morpheme of FIG. Removed.

  The learning unit 340 learns the input teacher data and forms a learning model. The input data of the teacher data includes the target morpheme, the next morpheme (here, the pair of the target morpheme and the next morpheme is referred to as the target morpheme set), and the original form of each of the two morphemes adjacent to the target morpheme set , Part-of-speech, inflection form, and the output data of the teacher data is the presence / absence of punctuation of the target morpheme. FIG. 4 shows one input data example and corresponding output data example of teacher data. Here, two morphemes adjacent to the target morpheme set are referred to as first adjacent morpheme sets, and two morphemes adjacent to the first adjacent morpheme set on the opposite side of the target morpheme set are referred to as second adjacent morpheme sets. Hereinafter, the two morphemes adjacent to the nth as viewed from the target morpheme set are referred to as the nth adjacent morpheme set. Here, the morpheme itself, not the morpheme itself, is used as input data because the morpheme is used differently depending on the morpheme, and there are many variations to learn, so the original form is used in this embodiment. The morpheme itself may be used instead of the original form, or the morpheme itself may be used in addition to the original form. Various methods for learning artificial intelligence / machine learning / deep learning have been proposed, and any method may be used.

  The estimation unit 350 acquires input data to be estimated via the preprocessing unit 310 and the morpheme processing unit 320, inputs the input data to the learning model, acquires the output data, and associates the input data with the output data. Record. The pre-processing unit 310 performs a process of removing a predetermined symbol from the speech recognition data to be estimated as shown in FIG. 5 as an example. The morpheme processing unit 320 performs morpheme analysis on the preprocessed speech recognition data using an existing morpheme analysis technique to obtain morpheme analysis data shown in FIG. Based on this morpheme analysis data, the estimation unit 350 inputs the original form, part of speech, and utilization form of the two morphemes of the morpheme set of interest and the two morphemes of the first adjacent morpheme set as input data to the learning model, and acquires output data Then, it is determined whether or not a punctuation point should be inserted between the morphemes of the morpheme set of interest, and the determination result is recorded in the presence or absence of the punctuation point of the target morpheme (see FIG. 7). As specific examples of the output data of the learning model, a value indicating the degree to which a punctuation point should be inserted (for example, a value of 0 to 1 indicating that the closer to 1 should be inserted) and a punctuation point should not be inserted. It is a value indicating the degree to which it is not present (for example, a value of 0 to 1 is closer to 1 indicating that it should not be inserted). Here, in the case of this specific example, the presence / absence of a punctuation point is determined based on the respective values. For example, a value indicating the degree to which a punctuation point should be inserted and a value indicating the degree to which a punctuation point should not be inserted If the value indicating the degree to which a punctuation should be inserted is large, it is determined that there is a punctuation, and if the value indicating the degree to which no punctuation should be inserted is large, it is determined that there is no punctuation.

  The morpheme processing unit 320 also records the position of the morpheme on the speech recognition data to be estimated before the morpheme analysis at the time of the morpheme analysis, and the punctuation insertion unit 360 performs the above estimation for the morpheme whose presence / absence of the punctuation is “present”. Phrases are inserted at corresponding positions on the target speech recognition data (see FIG. 8).

  Next, regarding the operation of the phrase insertion device according to the present embodiment, the learning operation will be described with reference to FIG. 9, and the estimation operation will be described with reference to FIG.

  As shown in FIG. 9, the preprocessing unit 310 performs preprocessing on speech recognition data that is a large number of conversation data to be learned (step 105). The morpheme processing unit 320 performs morpheme analysis on all pre-processed speech recognition data (step 110). The teacher data generation unit 330 converts all morpheme analysis data into teacher data (step 115). The learning unit 340 extracts one teacher data from all the teacher data, extracts one input data and output data from the target teacher data, and executes learning (step 120). The learning unit 340 determines whether or not it is the last input data and output data (step 125), and if it is not the last input data and output data, takes out the next input data and output data of the target teacher data (step 125). 130), step 120 is executed. In the case of the last input data and output data, the learning unit 340 determines whether the target dialogue data is the last dialogue data (step 135), and if it is not the last dialogue data, the learning unit 340 determines the next dialogue data. Data is extracted (step 140) and step 120 is executed. In the case of the last dialogue data, the learning operation is terminated.

  As shown in FIG. 10, the pre-processing unit 310 performs pre-processing on the speech recognition data that is the dialogue data to be estimated (step 205). The morpheme processing unit 320 performs morpheme analysis on the preprocessed speech recognition data (step 210). The estimation unit 350 inputs the first input data from the morphological analysis data to the learning model, and obtains output data (step 215). The estimation unit 350 determines whether or not a punctuation point should be inserted next to the target morpheme based on the obtained output data, and records the determination result in association with the target morpheme (step 220). The estimation unit 350 determines whether the target input data is the last input data (step 225). If the target input data is not the last input data, the next input data is extracted (step 230), and the process proceeds to step 215. If it is the last input data, the phrase insertion unit 360 inserts a phrase into the speech recognition data to be estimated based on the phrase presence / absence determination result for all input data (step 235).

  As described above, according to the dialogue management system of the present embodiment, by preparing and inputting speech recognition data for learning, learning without learning the statistical formulas in advance and learning corresponding to the learning data A model is formed, and at the time of estimation, it is determined whether or not a punctuation point should be inserted between each morpheme using this learning model, and the punctuation point to be inserted is appropriately inserted.

  In this embodiment, the original morpheme, part-of-speech and utilization form of the two morphemes of the target morpheme set and the two morphemes of the first adjacent morpheme set are learned as input data, and the formed morpheme set is The original form, part of speech, and utilization form for each of the two morphemes and the first adjacent morpheme set are input as input data, and the output data is used to determine whether a punctuation point should be inserted between the morphemes of the target morpheme set. However, in the same way as the first adjacent morpheme set, in addition to these, it may be estimated by learning using the original form, part of speech and utilization form of the two morpheme sets of the second adjacent morpheme set, It may be estimated by learning using two morphemes of the morpheme and the first adjacent morpheme set or two morphemes of the nth adjacent morpheme. Alternatively, learning may be performed using the original form of the target morpheme set in the speech recognition data to be learned, and estimation may be performed using the original form of the target morpheme set in the estimation target speech recognition data.

(Second Embodiment)

  The dialogue management system 1 according to the second embodiment is the same as the dialogue management system 1 according to the first embodiment except that the phrase insertion device 300 additionally includes a wildcard processing unit 370. It is a configuration.

  The wild card processing unit 370 refers to the wild card setting information, and performs replacement with a wild card symbol such as “*” (asterisk) if there is data corresponding to the learning morpheme analysis data and the estimation morpheme analysis data. In the wild card setting information, the system setter sets in advance a combination that the wild card setting is desired by combining the morpheme, the part of speech, and the utilization form. Even if a lot of speech recognition data, which is past dialogue data, is prepared, it is difficult to cover all combinations of morphemes, parts of speech and usage forms that may be used in dialogue. Of the combinations that cannot be covered, it is desirable to set wildcards for morphemes, parts of speech and inflected forms that are likely to appear in the future. Any of the morpheme, the part of speech, and the utilization form can be designated as a wild card, and two of the morpheme, the part of speech and the utilization form can be designated as a wild card. As a specific processing example of the wild card processing unit 370, as shown in FIG. 11, there is one wild card setting in which the morpheme, the part of speech, and the utilization form are “difficult”, “adjective”, and “*”, respectively. As shown in FIG. 3, the learning data includes “difficult”, “adjective”, and “continuous form-sounding stool”, and corresponds to the one wildcard setting. Therefore, as shown in FIG. Change from “Sounding flight” to “*”. Similarly, in the learning data, as shown in FIG. 6, the morpheme, the part of speech, and the utilization form have “sama”, “suffix”, and “(null)”, respectively. Since the form has “*”, “suffix”, and “(null)” and matches, the morpheme “like” of the learning data is converted to “*”. After the processing by the wild card processing unit 370, as in the first embodiment, teacher data is generated and learned from the morphological analysis data wild-carded by the learning unit 340 during learning, and the estimation unit during estimation The estimation is performed based on the learning model that has been formed using the morphological analysis data that has been subjected to the wild card processing in 350.

  As described above, according to the dialogue management system of the present embodiment, by setting wildcards for morphemes, parts of speech and utilization forms that are not in the learning data, morphemes, parts of speech and utilization that did not appear in the learning data in the estimation data. Even if a combination of shapes appears, the estimation by the learning model is appropriately executed by being covered by the wild card setting.

  In this embodiment, the system setter sets wildcard setting information in advance and performs processing with reference to the wildcard setting information. However, the voice recognition data to be learned is not used without using wildcard setting information. Can be designated as a wild card. In this case, the estimation target speech recognition data is designated as a wild card with reference to the wild card designation of the speech recognition data to be learned, and the estimation process is performed.

  In addition, as for the wild card setting information, the system setter sets the combination of the morpheme, the part of speech, and the utilization form that the user wants to set the wildcard in advance, but the system setter arbitrarily sets the wildcard at the timing noticed, For example, a list in which the input data of the teacher data in FIG. 4 is tabulated by appearance frequency (or a list in ascending or descending order according to the number of appearance frequencies as necessary) is displayed to a user such as a system setter to set a wild card. It may be configured to accept, and depending on the user, a vocabulary with a low appearance frequency can be set as a wild card by cutting off the number of appearance frequency cases.

(Third embodiment)

  The dialogue management system 1 according to the third embodiment is the same as the dialogue management system 1 according to the first embodiment, and includes two morphemes or first morphemes of the first morpheme set and a first morpheme set of interest. Learning by using two morphemes of n adjacent morphemes and performing the estimation, n is 1 as an initial value, n is performed up to a predetermined threshold value, and learning of the value of n that gives the best estimation result of the test result This is a configuration in which subsequent estimation processing is performed using a model. Here, the n learning models to be formed are tested using test data having the same configuration as the teacher data. Of the teacher data, about 70% of the teacher data is used for learning, and the remaining 30%. A test may be executed using the teacher data as test data. Comparing the output data obtained by inputting the test data input data into the learning model and the output data of the test data, the correctness of the test can be determined, and the number of correct answers in the number of test data becomes the correct answer rate. The learning model with the highest correct answer rate among the learning models is the learning model used in actual operation.

  Next, the operation of the dialogue management system 1 according to the present embodiment will be described with reference to FIG. 12, but steps with the same step numbers are omitted because of the operation of the first embodiment. Also, the predetermined threshold value of n in this embodiment is 3. After the teacher data is created in step 115, n is set to an initial value of 1, and the subsequent operations are executed (step 305). That is, a learning model with n = 1 is formed by executing the learning process, which is each operation of step 120 to step 140, for the target morpheme set and the first adjacent morpheme set. In step 315, whether or not n is a predetermined threshold of 3 is compared. Since n is 1, the increment is made 1 and the process returns to step 120. Similarly, the target morpheme set, the first adjacent morpheme set, and the second adjacent morpheme A learning process is executed on the pair to form a learning model with n = 2. Similarly, the process proceeds to step 315, where n is 3 and learning processing is performed, and a learning model in which n is 3 is formed. Since n is 3, the process proceeds to the end in the decision block of the next step 315. As described above, a learning model with n = 1, a learning model with n = 2, and a learning model with n = 3 are formed. By executing a test using test data, the correct answer rate of each learning model can be increased. The learning model that is obtained and has the highest correct answer rate is used as a learning model to be used in future estimation processing.

  As described above, according to the dialogue management system of the present embodiment, learning models corresponding to the threshold number n are formed, and the learning model with the highest correct answer rate is adopted to perform future estimation processing. An estimation process optimum for the input data can be performed.

(Other embodiments)

  In each of the embodiments, even when the utterance subject or the utterance subject type is different, the pre-processing and the entire utterance content by the plurality of utterance subjects or utterance subject types regardless of the utterance subject or the utterance subject type Although learning is performed by executing morpheme analysis processing and estimation is performed using the formed learning model, estimation may be performed by learning for each utterance subject or utterance subject type. The utterance subject / speaking subject type can be identified when, for example, the utterance content is recorded. If the customer is talking to the call center operator, the call channel differs between the customer and the operator. The voice data output from the call channel is assigned operator identification information (operator identification information assigned to each operator or utterance subject type identification information indicating the role of the operator) and output from the customer's call channel. By adding customer identification information (customer identification information given to each customer or utterance subject type identification information indicating the role of an operator) to the voice data, and using the identification information of those utterance subjects during voice recognition The operator's voice recognition text data and the customer's voice recognition text data. And it is possible save. There are other methods for identifying the utterance subject. In any case, when speech recognition data is input from the speech recognition device 200 to the punctuation insertion device 300, the punctuation insertion device 300 performs learning and estimation processing for each utterance subject. As long as it is possible, speech recognition text data may be input to the phrase insertion device 300 for each utterance subject (speech recognition data separable according to the utterance subject type as shown in FIG. 13). In the case of the operator and the customer, the operator's speech recognition text data is learned to form the operator's learning model, and the operator's speech recognition text data to be estimated is input at the time of estimation. Get data to reflect the insertion of punctuation.

  Further, in each of the above embodiments, the original form, part of speech and utilization form of the two morphemes of the target morpheme set and the two morphemes of the first adjacent morpheme set are used as input data, and the presence / absence of a punctuation between the two morphemes of the target morpheme set is determined. The output data shown is learned as learning data, and for the formed learning model, the original form, part of speech, and utilization form for each of the two morphemes of the target morpheme set and the two morphemes of the first adjacent morpheme set are input as input data, It was determined whether or not a punctuation point should be inserted between the morphemes of the target morpheme set using the output data. Using the shape as input data, learning output data indicating the presence or absence of reading points between the two morphemes of the target morpheme set as learning data, The original form, part of speech, and utilization form for each of the two morphemes of the morpheme and the first adjacent morpheme set are input as input data, and the output data is used to determine whether a punctuation mark should be inserted between the morphemes of the target morpheme set It may be a configuration. The same applies to sentence symbols such as line breaks instead of punctuation marks.

  In the present embodiment, the speech recognition data for dialogue is the target of processing. However, the speech recognition data is not limited to dialogue, but may be speech of one speaker or speech, and the dialogue is not limited to two, but more than three people. It may be a discussion. Further, it may not be voice recognition data, but may be a sentence created by word processing software. In speech recognition data, there is a problem that a sentence symbol is not properly inserted by a speech recognition engine, but there is a problem that a sentence symbol is not properly inserted in a sentence input by a writer. For example, even if there is no grammatical mistake in how to write a punctuation mark, there is a selection criterion that is artificially determined. You can insert readings that meet the criteria.

  In each of the above embodiments, as a specific example of the output data of the learning model, a value indicating the degree to which a punctuation point should be inserted and a value indicating the degree to which a punctuation point should not be inserted have been exemplified. For example, if the value indicating the degree to which a punctuation should be inserted is equal to or greater than a predetermined threshold, it may be determined that a punctuation should be inserted. In this case, the predetermined threshold is set by the user. It is possible to record the speech recognition data to be estimated with the punctuation inserted at each predetermined threshold, compare the speech recognition data with any punctuation inserted with the speech recognition data with the other punctuation inserted, and The structure which reveals the presence or absence may be sufficient. Here, for example, by reducing the predetermined threshold value indicating the degree to which a punctuation point should be inserted, it becomes easier to insert a punctuation point as shown in FIG. In comparison, added punctuation points may be underlined, and deleted punctuation points may be double strikethrough.

  INDUSTRIAL APPLICABILITY The present invention can be suitably used for a text symbol insertion device that appropriately inserts text symbols into character string text data.

Dialogue recording device 100
Speech recognition device 200
Punctuation insertion device 300
Pre-processing unit 310
Morphological processing unit 320
Teacher data generation unit 330
Learning part 340
Estimator 350
Punctuation insertion part 360

Claims (6)

In the character string text data to be learned, the morpheme and grammar information of the target morpheme set consisting of the morpheme to be processed and the next morpheme of the process morpheme are set as input data,
A learning unit that learns output data indicating whether or not a sentence symbol is inserted between morphemes of the morpheme set of interest as teacher data and forms a learning model;
A morpheme processing unit that performs morphological analysis on character string text data and outputs the divided morpheme and grammatical information of the morpheme;
The morpheme of the target morpheme set consisting of the morpheme to be processed and the next morpheme of the morpheme to be processed among the morpheme obtained by performing the morpheme analysis on the character string text data to be estimated and the morpheme of the morpheme And an estimator that inputs grammatical information as input data into the learning model and determines whether or not to insert a sentence symbol between morphemes of the morpheme group of interest based on the output data output;
A sentence symbol insertion unit that inserts a sentence symbol into the character string text data to be estimated based on the estimation result of the estimation unit,
The character string text data is given identification information for each utterance subject,
The learning unit uses the identification information for each utterance subject to learn for each utterance subject to form a learning model,
The estimation unit determines whether or not to insert a sentence symbol using a learning model for each utterance subject according to the identification information for each utterance subject ,
In addition to the target morpheme set in the text text data to be learned, the learning unit uses the morpheme of the first adjacent morpheme that is adjacent to the target morpheme set and the grammatical information of the morpheme. To the nth adjacent morpheme that is the nth adjacent morpheme and the grammatical information of the morpheme as input data,
The estimation unit is the second morpheme adjacent to the target morpheme set from the morpheme of the first adjacent morpheme and the grammatical information of the morpheme in addition to the target morpheme set in the character string text data to be estimated. n morphemes of adjacent morphemes and grammatical information of the morphemes are input as input data to the learning model and estimated
The learning unit has n as an initial value, learns each of n up to a predetermined threshold, and forms each learning model,
The said estimation part performs the test execution using test data with respect to each formed learning model, The sentence symbol insertion apparatus which uses a learning model with the highest correct answer rate as a result of the said test execution by subsequent estimation. The sentence symbol insertion device according to claim 1, further comprising a preprocessing unit that removes a predetermined sentence symbol from the character string text data to be estimated. Grammatical information the morphological processing section outputs the original part-of-speech, text symbol insertion apparatus according to claim 1 or 2 comprising at least one conjugated form. The grammatical information output by the morpheme processing unit includes at least an original form,
The sentence symbol insertion device according to claim 1 or 2 , wherein the morpheme original is used instead of each morpheme for learning of the learning unit and estimation of the estimation unit. Refer to the wildcard setting information or wildcard specification of the input data of learning data,
Including a wildcard processing unit that replaces the input data to be estimated with a wildcard symbol;
The sentence symbol insertion device according to any one of claims 1 to 4 , wherein the estimation unit uses the estimation target input data replaced with the wild card by the wild card processing unit. The morpheme and grammatical information of the target morpheme set consisting of the morpheme to be processed and the next morpheme of the process target morpheme are used as input data in the text text data to be learned. A learning step of learning output data indicating whether or not a symbol is inserted as teacher data to form a learning model,
Morphological analysis of character string text data and outputting the divided morpheme and grammatical information of the morpheme,
The morpheme of the target morpheme set consisting of the morpheme to be processed and the next morpheme of the morpheme to be processed among the morpheme obtained by performing the morpheme analysis on the character string text data to be estimated And grammatical information as input data input to the learning model, based on the output data that is output, the estimation step to determine whether to insert a sentence symbol between the morphemes of the morpheme group of interest,
A sentence symbol insertion step for inserting a sentence symbol into the character string text data to be estimated based on the estimation result of the estimation step,
The character string text data is given identification information for each utterance subject,
In the learning step, a learning model is formed by learning for each utterance subject using identification information for each utterance subject,
In the estimation step, it is determined whether or not to insert a sentence symbol by using a learning model for each utterance subject by identification information for each utterance subject ,
The learning step includes, in addition to the target morpheme set in the text text data to be learned, the morpheme of the first adjacent morpheme that is adjacent to the target morpheme set and the grammatical information of the morpheme. To the nth adjacent morpheme that is the nth adjacent morpheme and the grammatical information of the morpheme as input data,
In the estimation target character string text data, in addition to the target morpheme set, in addition to the morpheme of the first adjacent morpheme and the grammatical information of the morpheme, the estimation step includes two morphemes adjacent to the target morpheme set. n morphemes of adjacent morphemes and grammatical information of the morphemes are input as input data to the learning model and estimated,
In the learning step, n is 1 as an initial value, n is learned for each of up to a predetermined threshold value, and each learning model is formed.
The estimation step is a sentence symbol insertion method in which a test execution using test data is performed on each formed learning model, and the learning model having the highest correct answer rate is used in the subsequent estimation as a result of the test execution .

Images