JP7055529B1 - Meaning judgment program and meaning judgment system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a meaning determination program and a meaning determination system capable of accurately determining semantic data. SOLUTION: An acquisition step for acquiring sound data, an extraction step for extracting sound element data indicating a sound element and segmentation data regarding a start time point and an end time point of the sound element from the sound data acquired by the acquisition step, and an extraction step in advance. Machine learning using a plurality of judgment learning data in which the input data including the acquired reference phonetic data and the reference segmentation data and the output data including the semantic data indicating the meaning of the word are set as a set of data. It is characterized in that a computer is made to execute a determination step for determining semantic data for the phonetic data and the segmentation data with reference to the determination model generated by. [Selection diagram] Fig. 2

Description

The present invention relates to a meaning determination program and a meaning determination system.

In recent years, segmentation data regarding the start time and end time of phonemes are useful from the viewpoint of automatic increase of learning data and data reuse, and it is expected that the amount of data can be used in the smallest unit. For this reason, a speech recognition technique using phoneme segmentation data has attracted attention, and for example, a phoneme segmentation method based on waveform shape analysis of Patent Document 1 and a phoneme segmentation device using the same are known.

In Patent Document 1, the zero cross period of the voice waveform is calculated as the first parameter for the voice data, and the sharpness of the first maximum value between the zero cross and the zero cross of the voice waveform is calculated as the second parameter. , The maximum amplitude for each predetermined period updated between zero cross and zero cross of the voice waveform is calculated as the third parameter, and the maximum number of values between zero cross and zero cross of the voice waveform is calculated as the fourth parameter, and the voice waveform is calculated. The angle at which the amplitude value at zero crossing to the zero point is calculated as the fifth parameter, weights are given to each of the first to fifth parameters, and the first to fifth weighted parameters are multiplied and multiplied. The result is compared with the threshold to determine the phonetic boundary. As a result, Patent Document 1 discloses a technique capable of performing phoneme segmentation with high accuracy for spoken voice.

Japanese Unexamined Patent Publication No. 2006-284907

However, the disclosed technique of Patent Document 1 does not assume that when the determined segmentation data is used for voice recognition, the semantic data indicating the meaning of the sound data with respect to the phonetic data and the segmentation data is determined. .. Therefore, in Patent Document 1, for example, even if there are individual differences in the segmentation data for each user, the semantic data for the phoneme data and the segmentation data is uniquely determined, so that the semantic data can be determined accurately. Can not. Therefore, it is required to accurately determine the semantic data.

Therefore, the present invention has been devised in view of the above-mentioned problems, and an object of the present invention is to provide a meaning determination program and a meaning determination system capable of accurately determining semantic data.

The meaning determination program according to the first invention extracts sound element data indicating a sound element and segmentation data relating to a start time point and an end time point of the sound element from the acquisition step of acquiring sound data and the sound data acquired by the acquisition step. Judgment training data that includes the extraction step to be performed, the input data including the reference phonetic element data and the reference segmentation data acquired in advance, and the output data including the semantic data indicating the meaning of the word, as a set of data sets. It is characterized in that a computer is made to execute a determination step for determining semantic data for the phonetic data and the segmentation data by referring to a determination model generated by machine learning.

In the first invention, the meaning determination program according to the second invention is the extraction step, in which the extraction step includes extraction input data including reference sound data acquired in advance, reference phonetic element data, and reference segmentation data. Using a plurality of extraction training data using the output data as a set of data, the phonetic data and the segmentation data can be extracted from the sound data by referring to the extraction model generated by machine learning. It is a feature.

In the second invention, the meaning determination program according to the third invention is the extraction model, which includes extraction input data including the reference sound data pseudo-generated, reference phonetic data, and reference segmentation data. It is characterized in that it is generated by machine learning using the extraction training data having the extraction output data including and as a set of data sets.

The meaning determination system according to the fourth aspect of the present invention includes acquisition means for acquiring sound data, phonetic data indicating sound elements from the sound data acquired by the acquisition means, and segmentation data indicating a time difference from the start time to the end time of the sound elements. For determination that the extraction means for extracting and the input data including the reference phonetic data and the reference segmentation data acquired in advance and the output data including the semantic data indicating the meaning of the word are set as a set of data sets. It is provided with a database in which a determination model generated by machine learning is stored using a plurality of training data, and a determination means for determining semantic data for the phonetic data and the segmentation data with reference to the determination model. It is characterized by.

According to the first to third inventions, it is generated by machine learning using a plurality of judgment learning data in which a predetermined data set of semantic data for reference phonetic data and reference segmentation data is used. With reference to the determined determination model, the semantic data for the phonetic data and the segmentation data is determined. That is, by preparing the learning data for determination based on the individual difference of the user who uses the semantic determination program, it is possible to determine appropriate semantic data for the phoneme data and the segmentation data. Therefore, it is possible to accurately determine the semantic data in consideration of individual differences for each user.

In particular, according to the second invention, the extraction step uses a plurality of extraction training data in which a reference sound element data and a reference segmentation data for the reference sound data acquired in advance are used as a set of data, and the machine is used. The phonetic data and the segmentation data are extracted from the sound data with reference to the extraction model generated by the training. As a result, phoneme data and segmentation data can be extracted with high accuracy. Therefore, the semantic data can be accurately determined.

In particular, according to the third invention, the extraction model uses the pseudo-generated reference sound data and the extraction training data in which the reference sound element data and the reference segmentation data are a set of data sets. , Generated by machine learning. As a result, it becomes possible to use the pseudo-generated sound data as learning data, and it is possible to easily acquire learning data suitable for the assumed environment. Therefore, the semantic data can be accurately determined.

According to the fourth invention, the determination means determines the semantic data for the phoneme data and the segmentation data with reference to the determination model. That is, by preparing the learning data for judgment based on the individual difference of the user who uses the meaning judgment system, it is possible to judge appropriate semantic data for the phoneme data and the segmentation data. Therefore, it is possible to accurately determine the semantic data in consideration of individual differences for each user.

FIG. 1 is a schematic diagram showing an example of a meaning determination system in the embodiment. FIG. 2 is a schematic diagram showing an example of the operation of the meaning determination system in the embodiment. FIG. 3A is a schematic diagram showing an example of sound data, and FIG. 3B is a schematic diagram showing an example of a spectrogram. FIG. 4A is a schematic diagram showing an example of a learning method of an extraction model, and FIG. 4B is a schematic diagram showing an example of a learning method of a determination model. FIG. 5 is a schematic diagram showing an example of an extraction model. FIG. 6 is a schematic diagram showing an example of the determination model. FIG. 7A is a schematic diagram showing an example of the configuration of the meaning determination device in the embodiment, and FIG. 7B is a schematic diagram showing an example of the function of the meaning determination device in the embodiment. c) is a schematic diagram showing an example of a DB generation unit. FIG. 8 is a schematic diagram showing an example of the processing unit. FIG. 9 is a flowchart showing an example of the operation of the meaning determination system in the embodiment.

Hereinafter, an example of the meaning determination program and the meaning determination system in the embodiment to which the present invention is applied will be described with reference to the drawings.

An example of the meaning determination system 100 and the meaning determination device 1 in the present embodiment will be described with reference to the drawings. FIG. 1 is a schematic diagram showing an example of the meaning determination system 100 in the present embodiment. FIG. 2 is a schematic diagram showing an example of the operation of the meaning determination system 100 in the present embodiment. FIG. 4 is a schematic diagram showing an example of the learning method in the present embodiment.

<Meaning judgment system 100>
The meaning determination system 100 acquires sound data and is used to determine the semantic data of the acquired sound data.

The semantic determination system 100 extracts the phonetic data and the segmentation data from the sound data, refers to a database generated by machine learning using, for example, the training data, and determines the semantic data for the phonetic data and the segmentation data.

The meaning determination system 100 includes, for example, a meaning determination device 1 as shown in FIG. The meaning determination system 100 may include, for example, at least one of the terminal 2 and the server 3. The meaning determination device 1 is connected to the terminal 2 and the server 3 via, for example, the communication network 4.

In the meaning determination system 100, for example, as shown in FIG. 2, the meaning determination device 1 acquires sound data. After that, the meaning determination device 1 extracts phoneme data and segmentation data from the sound data, for example, with reference to the extraction model. After that, the semantic determination device 1 determines the semantic data for the phoneme data and the segmentation data with reference to the determination model. Thereby, the meaning determination system 100 can determine the phoneme data and the segmentation data extracted from the sound data.

The extraction model is a model that outputs phoneme data and segmentation data from the input sound data. The extraction model may be generated, for example, by machine learning. The extraction model is generated by using, for example, a set of pre-acquired reference phoneme data, reference phoneme data, and reference segmentation data as training data (learning data for extraction). As the extraction model, a trained model constructed by machine learning using a plurality of extraction training data may be used. As the extraction model, a known technique such as automatic phoneme labeling may be used as long as it is a means capable of extracting phoneme data and segmentation data from sound data.

The judgment model is generated by machine learning. As a judgment model, for example, reference phonetic data and reference segmentation data are used as input data, reference semantic data indicating judgment on the input data is used as output data, and input data and output data are used as a set of training data (for judgment). It is generated by using it as training data). As the judgment model, a trained model for judging semantic data from phonetic data and segmentation data constructed by machine learning using a plurality of judgment learning data is used.

The sound data is used, for example, to extract phoneme data and segmentation data output by the meaning determination system 100. Sound data is generated based on the sound collected by a sound collecting device such as a microphone (not shown). The sound data is, for example, a sound based on a human conversation. As the reference sound data, for example, the same data format as the sound data described later is used. Further, the sound data may include image data of a spectrogram generated from the sound.

Sound data is a coded version of sound. For example, sound coding is based on the pulse code modulation (PCM) method, which is represented as a bit string with a length determined by the number of quantization bits, sampling frequency, and time, and the density of sound waves is expressed in 1 bit. Some are based on the pulse density modulation (PDM) method, which samples at regular intervals. Further, the sound data may be indicated by an amplitude with respect to the time axis, for example, as shown in FIG. 3A.

The spectrogram is shown in intensity (amplitude) with respect to the time axis and the frequency axis, for example, as shown in FIG. 3 (b), and is converted from the sound data by using, for example, a Fourier transform (for example, a short-time Fourier transform). The spectrogram is used as image data, for example, 1 pixel × 1 pixel corresponds to a range of 0.064 sec × 15.624 Hz.

Phoneme data is data indicating phonemes of sound. The phoneme data is, for example, data indicating a phoneme of a voice included in the sound data. The phoneme data is, for example, data showing phonemes based on voice vowels and consonants included in the sound data. The phoneme data may be data indicating a phoneme in which a vowel and a consonant are paired, and includes text data such as "O: HA: YO: U".

The segmentation data is data relating to the start time point and the end time point of the phoneme. The segmentation data may be, for example, text data indicating the start time point and the end time point of the phoneme. The segmentation data may be, for example, data indicating the length of time from the start time to the end time of the phoneme. The segmentation data may be, for example, data indicating segmentation associated with each phoneme included in the phoneme data. The segmentation data may be, for example, data indicating each segmentation of each phoneme included in the phoneme data and arranged in association with each other. The phoneme data and the segmentation data may be tables linked to each other, for example, as shown in Table 1.

The phoneme data and the segmentation data are used by the semantic determination device 1 to refer to, for example, a determination model and determine the semantic data. The phoneme data and the segmentation data are extracted from the sound data by the meaning determination device 1.

The reference phoneme data and the reference segmentation data are used as the extraction learning data when the extraction model is generated by machine learning. As the reference phoneme data and the reference segmentation data, for example, the phoneme data and the segmentation data acquired in advance are used. As the reference phoneme data and the reference segmentation data, for example, the same data formats as the phoneme data and the segmentation data are used. The reference phoneme data and the reference segmentation data may be, for example, at least one of sound data based on actually recorded sound and pseudo-generated sound data.

The reference sound data is used as the judgment learning data when the judgment model is generated by machine learning. As the reference sound data, for example, sound data based on a learning sound acquired in advance is used. The reference swimming data may be in the same data format as the sound data.

The reference sound data, the reference phonetic data, and the reference segmentation data are, for example, a generation database generated by using machine learning modeled on a GAN (Generative Adversarial Network) or an autoencoder, in particular, a kind of Conditional GAN. It may be generated in a pseudo manner using the generated generation database by using machine learning using a certain pix2pix as a model.

Semantic data is data that indicates the meaning of a word. The semantic data may be, for example, data indicating the meaning of a word in sound data. The semantic data may be, for example, text data referring to a dictionary indicating the meaning of a word. Further, the semantic data may be semantic data indicated by the acoustic characteristics of the sound data.

The reference semantic data is semantic data used as the determination learning data when the determination model is generated by machine learning. As the reference semantic data, data in the same format as the semantic data may be used.

<Learning method>
The learning method in the present embodiment generates an extraction model used for extracting phoneme data and segmentation data from input sound data, and a judgment model used for determining semantic data for phoneme data and segmentation data. Used when.

The learning method generates an extraction model, for example, as shown in FIG. 4 (a). The learning method is for extraction in which the reference sound data is used as the first input data, the reference sound element data and the reference segmentation data are used as the first output data, and the first input data and the first output data are used as a pair of data sets. Get training data. The learning method generates an extraction model for extracting phonetic data and segmentation data from sound data by machine learning using a plurality of learning data for extraction.

Further, the learning method generates a determination model, for example, as shown in FIG. 4 (b). The learning method is for determination that the reference phonetic data and the reference segmentation data are used as the second input data, the reference semantic data is used as the second output data, and the second input data and the second output data are used as a pair of data sets. Get training data. The learning method generates a judgment model for judging semantic data from phonetic data and segmentation data by machine learning using a plurality of judgment learning data.

In the learning method, for example, machine learning using a neural network as a model is used to generate the above-mentioned database. The above-mentioned database is generated by using machine learning using a neural network such as CNN (Convolution Neural Network) as a model, or any model may be used.

In the extraction model, for example, the first association having a degree of association between the reference sound data (first input data), the reference phoneme data, and the reference segmentation data (first output data) is stored. The degree of association indicates the degree of connection between the first input data and the first output data. For example, it can be determined that the higher the degree of association, the stronger the connection of each data. The degree of association may be indicated by three values or more (three stages or more) such as percentage, or may be indicated by two values (two stages).

For example, the first association is constructed by the degree of connection between many-to-many information (a plurality of first input data, a pair, a plurality of first output data). The first association indicates a function (classifier) that is appropriately updated in the process of machine learning and is optimized based on, for example, a plurality of first input data and a plurality of first output data. The first association may have a plurality of association degrees indicating the degree of connection between the data, for example. The degree of association can correspond to a weight variable, for example, when the database is constructed with a neural network.

Therefore, in the meaning determination system 100, for example, the phoneme data and the segmentation data suitable for the sound data are selected by using the first association based on all the determination results of the classifier. Thereby, the phoneme data and the segmentation data suitable for the sound data can be quantitatively selected not only when the sound data is the same as or similar to the reference sound data but also when the sound data is dissimilar.

The first association may indicate the degree of connection between the plurality of first output data and the plurality of first input data, for example, as shown in FIG. In this case, by using the first association, a plurality of first output data (in FIG. 5, "reference phonetic data A" + "reference segmentation data A" to "reference phonetic data C" + "reference segmentation" Each of the data C) can be stored in association with the degree of relationship between the plurality of first input data (“reference sound data A” to “reference sound data C” in FIG. 5). Therefore, for example, a plurality of first input data can be associated with one first output data via the first association. As a result, it is possible to realize the selection of multifaceted phoneme data and segmentation data for the sound data.

The first association has, for example, a plurality of association degrees for associating each first output data with each first input data. The degree of association is shown in three or more steps such as percentage, 10 steps, or 5 steps, and is shown by, for example, the characteristics of the line (for example, thickness). For example, the "reference sound data A" included in the first input data has a degree of association AA "73%" between the "reference sound element data A" and the "reference segmentation data A" included in the first output data. , And indicates the degree of association AB “12%” between the “reference phonetic data B” and the “reference segmentation data B” included in the first output data. That is, the "degree of association" indicates the degree of connection between each data, and for example, the higher the degree of association, the stronger the connection of each data.

Further, the extraction model may be provided with at least one hidden layer between the first input data and the first output data so as to be machine-learned. The above-mentioned degree of association is set in either one or both of the first input data and the hidden layer data, and this is the weighting of each data, and the output is selected based on this. Then, when this degree of association exceeds a certain threshold value, the output may be selected.

In the determination model, for example, a second association having a degree of association between the reference phoneme data and the reference segmentation data (second input data) and the reference semantic data (second output data) is stored. The degree of association indicates the degree of connection between the second input data and the second output data. For example, it can be determined that the higher the degree of association, the stronger the connection of each data. The degree of association may be indicated by three values or more (three stages or more) such as percentage, or may be indicated by two values (two stages).

For example, the second association is constructed by the degree of connection between many-to-many information (plurality of second input data, pair, plurality of second output data). The second association indicates a function (classifier) that is appropriately updated in the process of machine learning and is optimized based on, for example, a plurality of second input data and a plurality of second output data. The second linkage may have a plurality of linkage degrees indicating the degree of connection between the data, for example. The degree of association can correspond to a weight variable, for example, when the database is constructed with a neural network.

Therefore, in the semantic determination system 100, for example, the semantic data suitable for the phoneme data and the segmentation data is selected by using the second association based on all the determination results of the classifier. As a result, even when the phonetic data and the segmentation data are the same as or similar to the reference phonetic data and the reference segmentation data, or even when they are dissimilar, the semantic data suitable for the phonetic data and the segmentation data is quantitatively obtained. Can be selected for.

The second association may indicate the degree of connection between the plurality of second output data and the plurality of second input data, for example, as shown in FIG. In this case, by using the second association, a plurality of second input data ("reference semantic data A" to "reference semantic data C" in FIG. 6) are used for each of the plurality of second output data ("reference semantic data A" to "reference semantic data C" in FIG. 6). In FIG. 6, the degree of the relationship of "reference phonetic data A" + "segmentation data A" to "reference phonetic data C" + "reference segmentation data") can be stored in association with each other. Therefore, for example, a plurality of second input data can be associated with one second output data via the second association. As a result, it is possible to realize the selection of multifaceted semantic data for phoneme data and segmentation data.

The second association has, for example, a plurality of association degrees for associating each second output data with each second input data. The degree of association is shown in three or more steps such as percentage, 10 steps, or 5 steps, and is shown by, for example, the characteristics of the line (for example, thickness). For example, the "reference phonetic data A" + "reference segmentation data A" included in the second input data has a degree of association AA "73%" with the "reference semantic data A" included in the second output data. , And indicates the degree of association AB “12%” with the “reference semantic data B” included in the second output data. That is, the "degree of association" indicates the degree of connection between each data, and for example, the higher the degree of association, the stronger the connection of each data.

Further, in the determination model, at least one hidden layer may be provided between the second input data and the second output data, and machine learning may be performed. The above-mentioned degree of association is set in either one or both of the second input data and the hidden layer data, and this is the weighting of each data, and the output is selected based on this. Then, when this degree of association exceeds a certain threshold value, the output may be selected.

<Meaning determination device 1>
Next, an example of the meaning determination device 1 in the present embodiment will be described with reference to FIGS. 7 and 8. FIG. 7A is a schematic diagram showing an example of the configuration of the meaning determination device 1 in the present embodiment, and FIG. 7B is a schematic diagram showing an example of the function of the meaning determination device 1 in the present embodiment. be. FIG. 7C is a schematic diagram showing an example of the DB generation unit 16. FIG. 8 is a schematic diagram showing an example of the processing unit 12.

As the meaning determination device 1, for example, an electronic device such as a laptop (notebook) PC or a desktop PC is used. As shown in FIG. 7A, for example, the meaning determination device 1 stores the housing 10, the CPU (Central Processing Unit) 101, the ROM (Read Only Memory) 102, the RAM (Random Access Memory) 103, and the storage. A unit 104 and I / F 105 to 107 are provided. Each configuration 101 to 107 is connected by an internal bus 110.

The CPU 101 controls the entire meaning determination device 1. The ROM 102 stores the operation code of the CPU 101. The RAM 103 is a work area used when the CPU 101 operates. The storage unit 104 stores various information such as a database and learning target data. As the storage unit 104, for example, in addition to an HDD (Hard Disk Drive), a data storage device such as an SSD (Solid State Drive) is used. For example, the meaning determination device 1 may have a GPU (Graphics Processing Unit) (not shown).

The I / F 105 is an interface for transmitting and receiving various information to and from the terminal 2, the server 3, the website, etc., as needed, via the communication network 4. The I / F 106 is an interface for transmitting / receiving information to / from the input unit 108. For example, a keyboard is used as the input unit 108, and the user or the like of the meaning determination device 1 inputs various information, a control command of the meaning determination device 1, or the like via the input unit 108. Further, the input unit 108 may include a microphone (not shown) capable of collecting sound. The I / F 107 is an interface for transmitting and receiving various information to and from the display unit 109. The display unit 109 displays various information stored in the storage unit 104, semantic data, and the like. A display is used as the display unit 109, and for example, in the case of a touch panel type, it is provided integrally with the input unit 108. Further, a speaker may be used for the display unit 109.

FIG. 7B is a schematic diagram showing an example of the function of the meaning determination device 1. The meaning determination device 1 includes an acquisition unit 11, a processing unit 12, an output unit 14, and a storage unit 15, and may include, for example, a DB generation unit 16. The DB generation unit 16 has, for example, an extraction model generation unit 161 and a determination model generation unit 162, as shown in FIG. 7 (c). The functions shown in FIGS. 7 (b), 7 (c), and 8 are realized by the CPU 101 executing a program stored in the storage unit 104 or the like using the RAM 103 as a work area, for example. It may be controlled by artificial intelligence or the like.

<< Acquisition unit 11 >>
The acquisition unit 11 acquires sound data. The acquired data is used when extracting the above-mentioned phoneme data and segmentation data. In addition to acquiring the sound data input from the input unit 108, for example, the acquisition unit 11 may acquire the sound data from the terminal 2 or the like via, for example, the communication network 4.

The acquisition unit 11 may acquire training data used for generating the various models described above, for example. In addition to acquiring the learning data input from the input unit 108, for example, the acquisition unit 11 may acquire the learning data from the terminal 2 or the like via, for example, the communication network 4.

For example, as the extraction training data used for generating the extraction model, reference sound data, reference phoneme data, and reference segmentation data can be mentioned. Further, for example, as the judgment learning data used for generating the judgment model, reference phoneme data, reference segmentation data, and reference semantic data can be mentioned.

<< Processing unit 12 >>
The processing unit 12 refers to, for example, an extraction model, extracts phoneme data and segmentation data from sound data, refers to a determination model, and determines semantic data for phoneme data and segmentation data.

As shown in FIG. 8, the processing unit 12 has an extraction processing unit 121 connected to the acquisition unit 11. Further, the processing unit 12 has a determination processing unit 122 connected to the extraction processing unit 121.

The extraction processing unit 121 refers to, for example, an extraction model, and extracts phoneme data and segmentation data from the sound data. The extraction processing unit 121 refers to, for example, an extraction model, and extracts phoneme data and segmentation data having the highest relevance to the sound data. Further, for example, a plurality of reference phoneme data and reference segmentation data having a degree of association equal to or higher than a preset threshold value may be selected as the phoneme data and the segmentation data. Further, the number of selected reference phoneme data and reference segmentation data can be arbitrarily set.

The determination processing unit 122 refers to, for example, a determination model, takes phoneme data and segmentation data as inputs, and determines semantic data for the input. The determination processing unit 122 refers to, for example, a determination model, and extracts semantic data having the highest association with the phoneme data and the segmentation data. Further, for example, a plurality of reference semantic data having a degree of association equal to or higher than a preset threshold value may be selected as the semantic data. In addition, the number of selected semantic data for reference can be set arbitrarily.

<< Output unit 14 >>
The output unit 14 outputs semantic data. The output unit 14 outputs the semantic data to the display unit 109 via the I / F 107, and also outputs the semantic data to the terminal 2 or the like via, for example, the I / F 105.

<< Memory unit 15 >>
The storage unit 15 retrieves various data such as a database stored in the storage unit 104 as needed. The storage unit 15 stores various data acquired or generated by the acquisition unit 11, the processing unit 12, and the DB generation unit 16 in the storage unit 104 as needed.

<< DB generation unit 16 >>
The DB generation unit 16 generates a database by machine learning using a plurality of learning data. For machine learning, for example, the above-mentioned neural network or the like is used.

The DB generation unit 16 has, for example, an extraction model generation unit 161 and a determination model generation unit 162.

The extraction model generation unit 161 uses, for example, a set of reference sound data, reference phonetic data, and reference segmentation data as learning data for extraction, and creates an extraction model by machine learning using a plurality of learning data for extraction. Generate.

The judgment model generation unit 162 uses, for example, a set of reference phonetic data, reference segmentation data, and reference semantic data as learning data for judgment, and uses machine learning using a plurality of learning data for judgment to generate a judgment model. Generate.

<Terminal 2>
The terminal 2 is owned by, for example, a user of a service using the meaning determination system 100, and is connected to the meaning determination device 1 via a communication network 4. The terminal 2 may indicate, for example, an electronic device that generates a database. As the terminal 2, for example, an electronic device such as a personal computer or a tablet terminal is used. The terminal 2 may have at least a part of the functions of the meaning determination device 1, for example. Further, the terminal 2 may include a microphone (not shown) for acquiring sound data and each learning data. The terminal 2 may include a display (not shown) or a speaker capable of presenting semantic data to the user.

<Server 3>
The server 3 is connected to the meaning determination device 1 via the communication network 4. Various past data and the like are stored in the server 3, and various data are transmitted from the meaning determination device 1 as needed. The server 3 may have at least a part of the functions of the meaning determination device 1, for example, and may perform at least a part of the processing instead of the meaning determination device 1, for example. The server 3 stores, for example, at least a part of various data stored in the storage unit 104 of the meaning determination device 1, and may be used in place of the storage unit 104, for example.

<Communication network 4>
The communication network 4 is, for example, an internet network or the like to which the meaning determination device 1 is connected via a communication circuit. The communication network 4 may be configured by a so-called optical fiber communication network. Further, the communication network 4 may be realized by a known communication technology such as a wireless communication network in addition to the wired communication network.

(Operation of the meaning determination system 100)
Next, an example of the operation of the meaning determination system 100 in the present embodiment will be described. FIG. 9 is a flowchart showing an example of the operation of the meaning determination system 100 in the present embodiment.

<Acquisition means S110>
The acquisition means S110 acquires sound data generated based on the sound collected by, for example, a sound collecting device. In the acquisition means S110, for example, the acquisition unit 11 acquires sound data. Further, the acquisition unit 11 may acquire sound data in the same format as the reference sound data included in the extraction learning data of the extraction model described above, for example. In addition to acquiring sound data from, for example, a terminal 2, the acquisition unit 11 may store the acquired sound data in the storage unit 104 via, for example, the storage unit 15.

<Extraction means S120>
The extraction means S120 extracts phoneme data and segmentation data from the sound data acquired by the acquisition means S110. In the extraction means S120, for example, the extraction processing unit 121 refers to the extraction model generated in advance by the learning method described above, and extracts phoneme data and segmentation data from the sound data. Further, the extraction means S120 may extract, for example, phoneme data and segmentation data in the same format as the reference phoneme data and the reference segmentation data included in the determination learning data of the determination model. Further, the extraction means S120 may extract phoneme data and segmentation data from sound data by another known technique without using an extraction model.

Further, the extraction means S120 may store the phoneme data and the segmentation data extracted from the sound data in the storage unit 104, for example, via the storage unit 15. The acquired data may be transmitted to, for example, the server 3 or another meaning determination device 1.

<Determining means S130>
The determination means S130 refers to, for example, a determination model, and determines semantic data for the phoneme data and the segmentation data extracted by the extraction means S120. In the determination means S130, for example, the determination processing unit 122 refers to the determination model generated in advance, and determines the semantic data for the phoneme data and the segmentation data extracted by the extraction means S120. This makes it possible to determine appropriate semantic data for the phoneme data and the segmentation data.

The determination means S130 may store the determined semantic data in the storage unit 104, for example, via the storage unit 15. The acquired semantic data may be transmitted to, for example, the terminal 2, the server 3, or another semantic determination device 1. As the semantic data to be determined, a plurality of semantic data may be acquired for one phoneme data and segmentation data.

<Output means S140>
In the output means S140, for example, the output unit 14 outputs the semantic data determined by the determination means S130 to the display unit 109, the terminal 2, and the like.

By performing each of the above-mentioned means, the operation of the meaning determination system 100 in the present embodiment is completed.

Although embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

1: Meaning determination device 2: Terminal 3: Server 4: Communication network 10: Housing 11: Acquisition unit 12: Processing unit 14: Output unit 15: Storage unit 16: DB generation unit 100: Meaning determination system 101: CPU
102: ROM
103: RAM
104: Storage unit 105: I / F
106: I / F
107: I / F
108: Input unit 109: Display unit 110: Internal bus 121: Extraction processing unit 122: Judgment processing unit 161: Extraction model generation unit 162: Judgment model generation unit S110: Acquisition means S120: Extraction means S130: Judgment means S140: Output means

Claims (4)

The acquisition step to acquire the sound data and
An extraction step for extracting phoneme data indicating a phoneme and segmentation data relating to a start time point and an end time point of the phoneme from the sound data acquired by the acquisition step.
A machine using a plurality of judgment training data in which input data including reference phonetic element data and reference segmentation data acquired in advance and output data including semantic data indicating the meaning of a word are set as a set of data. A semantic determination program characterized in that a computer is made to execute a determination step for determining semantic data for the phonetic data and the segmentation data with reference to a determination model generated by learning. The extraction step is learning data for extraction in which a set of data sets is input data for extraction including reference sound data acquired in advance and output data for extraction including reference phonetic element data and reference segmentation data. The meaning determination program according to claim 1, wherein the phonetic element data and the segmentation data are extracted from the sound data with reference to an extraction model generated by machine learning. The extraction model is
Extraction input data including the reference sound data generated in a pseudo manner, and
2. The meaning judgment program described in. The acquisition method for acquiring sound data and
An extraction means for extracting phoneme data indicating a phoneme from the sound data acquired by the acquisition means and segmentation data indicating a time difference from the start time to the end time of the phoneme.
A machine using a plurality of judgment learning data in which input data including reference phonetic element data and reference segmentation data acquired in advance and output data including semantic data indicating the meaning of a word are set as a set of data. A database that stores the judgment model generated by training, and
A semantic determination system comprising a determination means for determining semantic data for the phoneme data and the segmentation data with reference to the determination model.

Images