JP2016212331A - Articulation rating server device, articulation rating method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To overcome the long-suffered difficulty to effectively articulate, resulting from the problem of inadequate processing speed.SOLUTION: High-speed articulation rating can be accomplished with an articulation rating server device equipped with a receiving unit that accepts voice uttered by a user regarding a content to be uttered and receives from a terminal device voice-related data, which is the result of first partial processing for articulation rating; a rating result acquiring unit that performs on the voice-related data second partial processing for articulation rating, to be accomplished after the first partial processing, and acquires the result of rating; and a transmitting unit that transmits the result of rating to the terminal device.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a pronunciation rating system that performs pronunciation rating processing in a distributed environment, a pronunciation rating server apparatus, and the like.

  Conventionally, there has been a stand-alone pronunciation rating device (see, for example, Patent Document 1 and Patent Document 2).

Japanese Patent No. 4859125 Japanese Patent No. 5007401

  However, in the prior art, the entire pronunciation rating program is installed in the pronunciation rating device, which is the user's terminal, and pronunciation training is performed, so it is difficult to protect the confidentiality of the algorithm that forms the core of the pronunciation rating.

  Further, in the conventional technology, processing is performed by one computer, and effective pronunciation training may be difficult due to a problem of processing speed. In addition, when using a pronunciation rating device for pronunciation training in language learning, there was a problem that if there was a delay after the user uttered until the rating result was output, the motivation for language learning would be cut off .

  The pronunciation rating server device according to the first aspect of the invention receives the voice uttered by the user for the instructed utterance content, and performs the first partial process, which is the first partial process for the pronunciation rating, on the voice A receiving unit that receives the voice-related data as a result from the terminal device, and a second partial process that is a second partial process for pronunciation evaluation to be performed after the first partial process is performed on the voice-related data, and the evaluation result Is a pronunciation rating server device comprising a rating result acquisition unit for acquiring the rating result and a transmission unit for transmitting the rating result to the terminal device.

  With this configuration, it is possible to reduce the risk of knowing the pronunciation rating algorithm to the outside. In addition, high-speed pronunciation rating processing can be performed.

  In addition, the pronunciation rating server device of the second aspect of the present invention provides the voice related data, the utterance content data indicating the instructed utterance content or the utterance content identifier for identifying the utterance content, and the evaluation result for the first invention. The pronunciation rating server apparatus according to claim 1, further comprising a storage unit that accumulates one or more associated pronunciation rating information and a processing unit that performs processing on the one or more pronunciation rating information.

  With this configuration, the result of pronunciation evaluation can be used effectively.

  In addition, the pronunciation rating server device according to the third aspect of the invention is obtained by using the utterance content identifier for identifying the utterance content or the utterance content identifier for identifying the utterance content, as compared with the second invention. The utterance content data is also received, and the storage unit is a pronunciation rating server device that stores the voice related data, the utterance content data or the utterance content identifier, and the rating result in association with each other.

  With such a configuration, the result of pronunciation evaluation can be used effectively with a small amount of communication.

  In addition, the pronunciation rating server device according to the fourth aspect of the present invention is the data received by the receiving unit with respect to any one of the first to third inventions, and the received data including the voice-related data is converted into the rating result acquiring unit. Is changed to data that can be processed by the second partial processing, and further comprises a middleware part that changes the evaluation result acquired by the evaluation result acquisition part to data that can be transmitted by the transmission part. The evaluation result acquisition part is changed by the middleware part The second partial process is performed on the voice-related data after the acquisition, the evaluation result is acquired, and the transmission unit is a pronunciation evaluation server device that transmits the evaluation result after the middleware unit changes to the terminal device.

  With such a configuration, it becomes easy to cope with changes in communication protocols and advancement of pronunciation rating algorithms.

  In the pronunciation rating server device according to the fifth aspect of the present invention, in contrast to the fourth aspect of the invention, the receiving unit is data obtained as a result of processing performed by the terminal middleware unit of the terminal device. This is a pronunciation rating server device that receives data including voice-related data, which is data after the voice-related data, which is the result of the above, has been changed to data that can be transmitted.

  With such a configuration, it becomes easy to cope with changes in communication protocols and advancement of pronunciation rating algorithms.

  In the pronunciation rating server device according to the sixth aspect of the present invention, with respect to any one of the first to third inventions, the receiving unit receives voice-related data from two or more terminal devices, and the rating result acquiring unit is This is a pronunciation rating server device that performs second partial processing on two or more pieces of audio-related data received by the receiving unit in parallel to obtain two or more rating results.

  With this configuration, a pronunciation rating service can be provided for a plurality of terminal devices, and the risk of knowing the pronunciation rating algorithm to the outside can be reduced. In addition, a high-speed pronunciation rating service can be provided to a plurality of terminal devices.

  In addition, the pronunciation rating server device according to the seventh invention is different from the sixth invention in that the receiving unit is a result of different first partial processing performed by two or more terminal devices, and different types of voice related data. Are received from two or more terminal devices, and the rating result acquisition unit is a pronunciation rating server device that performs different second partial processing on each type of audio-related data and acquires the rating result.

  With this configuration, it is possible to provide a pronunciation rating service to a plurality of terminal devices while supporting specifications of different terminal devices.

  In addition, the pronunciation rating server device according to the eighth aspect of the invention further includes a storage unit that stores two or more teacher data, which is an acoustic model for each of one or more phonemes, with respect to the sixth or seventh aspect. Then, the rating result acquisition unit uses a different teacher data in accordance with the user attribute value that is the user attribute value, and the second part is a second partial process for pronunciation evaluation to be performed after the first partial process. It is a pronunciation rating server device that performs processing on voice-related data and obtains a rating result.

  With this configuration, it is possible to perform an appropriate pronunciation rating according to the user attribute value.

  The pronunciation rating server device according to the ninth aspect of the present invention is one pronunciation rating server device out of two or more pronunciation rating server devices for any one of the sixth to eighth inventions, and the rating result The acquisition unit is a pronunciation rating server device that performs second partial processing on voice-related data transmitted from some of the two or more terminal devices and acquires a rating result.

  With this configuration, it is possible to distribute the load on the pronunciation rating server.

  In addition, the pronunciation rating server device of the tenth aspect of the invention relates to each of the two or more audio-related data received by the receiving unit in order to adjust the load of the two or more pronunciation rating server devices. On the other hand, the pronunciation rating server apparatus further includes a middleware unit including an adjusting unit that determines a pronunciation rating server apparatus that performs the second partial processing.

  With this configuration, it is possible to distribute the load on the pronunciation rating server.

  In addition, the pronunciation rating server device according to the eleventh aspect of the invention relates to the tenth invention, in which the middleware unit sends voice-related data to the pronunciation rating server device determined by the adjusting means; And a receiving unit that receives the rating result acquired by the pronunciation rating server device determined by the adjusting unit, and the transmitting unit is a pronunciation rating server device that transmits the rating result received by the receiving unit to the terminal device.

  With this configuration, it is possible to distribute the load of the pronunciation rating server, and to easily cope with changes in the communication protocol and advancement of the pronunciation rating algorithm.

  Further, in the pronunciation rating server device according to the twelfth aspect of the present invention, in relation to any one of the first to eleventh aspects, the first partial process is a process of digitizing sound and acquiring sound data, or sound Is obtained, and voice data is obtained, and feature data that is a set of feature vectors is obtained from the voice data. The second partial process obtains feature data from the voice data, and combines the feature data and the teacher data. And calculating the posterior probability that the feature data is the phoneme to be evaluated, calculating the speech rating value from the posterior probability, and obtaining the rating result from one or more rating values, or the feature data and the teacher data Is used to calculate the posterior probability that the feature data is the phoneme to be evaluated, calculate the speech rating value from the posterior probability, and obtain the rating result from one or more rating values. A.

  With this configuration, it is possible to reduce the risk of knowing the pronunciation rating algorithm to the outside. In addition, high-speed pronunciation rating processing can be performed.

  According to the pronunciation rating server device of the present invention, it is possible to reduce the risk of knowing the pronunciation rating algorithm to the outside. Alternatively, the pronunciation rating server device according to the present invention can perform high-speed pronunciation rating processing.

Conceptual diagram of pronunciation rating system A in the first embodiment Block diagram of the same pronunciation rating system A A flowchart for explaining the operation of the pronunciation rating server device 1 A flowchart for explaining the operation of the terminal device 4 Block diagram of the same pronunciation rating system A Conceptual diagram of pronunciation rating system B in the second embodiment Block diagram of the pronunciation rating system B A flowchart for explaining the operation of the pronunciation rating server device 2 Flow chart explaining details of the second partial process Figure showing the same user information management table Figure showing the teacher data management table Figure showing a list of pronunciation practice Block diagram of the pronunciation rating system B Conceptual diagram of pronunciation rating system C in Embodiment 3 Block diagram of the pronunciation rating system C Flow chart for explaining the operation of the pronunciation pronunciation server device 3A A flowchart for explaining the operation of the pronunciation rating server device 3B Figure showing the load information management table Overview of the computer system in the above embodiment Block diagram of the computer system

  Hereinafter, embodiments of the pronunciation rating server device and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment 1)
In the present embodiment, a pronunciation rating system including one terminal device and one pronunciation rating server apparatus will be described.

  In the present embodiment, a pronunciation rating system in which a terminal device performs a first partial process, which is a part of the entire pronunciation rating process, and a pronunciation rating server apparatus performs a second partial process will be described.

  In the present embodiment, a pronunciation rating system including a pronunciation rating server apparatus that stores and uses voice-related data, utterance content data, and rating results in association with each other will be described. Details of the voice related data, utterance content data, and the evaluation result will be described later.

  Further, in the present embodiment, an utterance content identifier for identifying an utterance rating target is held, utterance content data is acquired using the utterance content identifier as a key, and a pronunciation rating provided with a terminal device that outputs the utterance content data The system will be described.

  In the present embodiment, a pronunciation rating system including a pronunciation rating server apparatus having middleware that performs processing corresponding to a communication protocol and the like will be described.

  Furthermore, in this embodiment, a pronunciation rating system including a terminal device having middleware that performs processing corresponding to a communication protocol or the like will be described.

  FIG. 1 is a conceptual diagram of a pronunciation rating system A in the present embodiment. The pronunciation rating system A includes a pronunciation rating server device 1 and a terminal device 4. The pronunciation rating server device 1 is usually a server device and includes a so-called cloud server and is widely understood. The pronunciation rating server device 1 may be any device as long as it has a considerable processing capability, and the type thereof is not limited. The terminal device 4 may be of any type, such as a so-called smartphone, tablet terminal, mobile phone, notebook computer, or the like. The pronunciation rating server device 1 and the terminal device 4 can communicate with each other, and are connected via, for example, the Internet. Furthermore, the processing capability of the pronunciation rating server device 1 is usually larger than the processing capability of the terminal device 4.

  FIG. 2 is a block diagram of the pronunciation rating system A in the present embodiment.

  The pronunciation rating server device 1 includes a storage unit 11, a receiving unit 12, a middleware unit 13, a rating result acquisition unit 14, a transmission unit 15, a storage unit 16, and a processing unit 17.

  The terminal device 4 includes a terminal storage unit 41, a terminal reception unit 42, a terminal first partial processing unit 43, a terminal middleware unit 44, a terminal transmission / reception unit 45, and a terminal output unit 46.

  The storage unit 11 constituting the pronunciation rating server device 1 can store various types of information. The various information is, for example, one or more pronunciation rating information. Note that one or more pronunciation rating information may be associated with a user identifier. The pronunciation rating information is information in which voice related data, utterance content data or utterance content identifier, and a rating result are associated with each other.

  The voice related data is a result of the first partial process in the terminal device 4. The voice related data is, for example, voice data. The audio-related data is, for example, one or more frame audio data that is an audio data for each of the divided frames. The voice related data is, for example, feature data that is a set of feature vectors acquired from the voice data. The voice data is voice data issued and accepted by the user of the terminal device 4. The audio data is usually data obtained by digitizing the received audio. Further, one or more frame audio data may be considered as audio data. Also, the voice-related data is a result of receiving a voice uttered by the user for the instructed utterance content and performing a first partial process that is a first partial process for pronunciation evaluation on the voice. I can say that.

The feature data is a set of feature vectors “O = o ₁ , o ₂ ,..., O _T ” acquired for each frame from one or more frame sound data. The feature vector (o _t ) constituting the feature data is obtained by performing spectral analysis on the frame speech data. The feature vector is, for example, an MFCC obtained by performing discrete cosine transform on a filter bank output of 24 channels using a triangular filter, and the static parameter, the delta parameter, and the delta delta parameter are further normalized to 12 dimensions, respectively. Power and delta power and delta delta power (39th dimension). In spectral analysis, it is preferable to perform cepstrum average removal. However, it goes without saying that the voice analysis conditions may be other conditions. Note that the process of acquiring the feature data can be realized by a known technique.

  Furthermore, the detail about a 1st partial process and a 2nd partial process is mentioned later. However, the pronunciation evaluation process is usually completed by the first partial process and the second partial process. In general, the load when executing the first partial process is smaller than the load when executing the second partial process. Further, the first partial process may be referred to as a pronunciation rating client process or a pronunciation rating front-end process, and the second partial process may be referred to as a pronunciation rating server process or a pronunciation rating back-end process.

  The utterance content data is data indicating the utterance content of the pronunciation evaluation target. The utterance content data is, for example, data acquired using an utterance content identifier that identifies the utterance content. The utterance content data is information on correct words and sentences that the user has to pronounce, for example, information on phonemes constituting the words and sentences. The utterance content data may be a correct word or a word string that the user must pronounce.

  The utterance content identifier is information for identifying utterance content data. It can be said that the utterance content identifier is information for identifying the utterance content. The utterance content identifier is, for example, an ID.

  The rating result is information indicating a result of pronunciation rating for the utterance content uttered by the user. The rating result may be a rating value or a rating value indicating a user's pronunciation level obtained from the rating value.

  A user identifier is information which identifies a user or the terminal device 4, for example, ID, a telephone number, a name, etc., The kind is not ask | required.

  The various information is, for example, an utterance content database having one or more sets of utterance content identifiers and utterance content data.

  Various types of information are, for example, one or more pieces of user information. User information is information about the user. The user information has a user identifier and one or more user attribute values. The user attribute value is, for example, age, age group, sex or the like.

  The various information is, for example, one or more teacher data. Teacher data is an acoustic model used for pronunciation evaluation. The acoustic model is usually an acoustic model for each one or more phonemes. The acoustic model is preferably data based on a Hidden Markov Model (HMM). Further, the acoustic model is, for example, an acoustic model connected from the terminal state of one phoneme HMM to the starting state of the one phoneme or all other phonemes. However, the acoustic model may not be an acoustic model connected from the terminal state of one phoneme HMM to the starting state of the one phoneme or all other phonemes.

  The teacher data does not necessarily need to be data based on the HMM in which the HMMs for each phoneme are connected. The teacher data may be a simple set of all phoneme HMMs. The teacher data does not necessarily need to be data based on the HMM. The teacher data may be data based on other models such as a single Gaussian distribution model, a probability model (GMM: Gaussian mixture model), and a statistical model. Note that the acoustic model is, for example, a set of an ID for identifying a sound and a feature vector that characterizes the sound.

  The receiving unit 12 receives voice related data from the terminal device 4. The receiving unit 12 may receive the voice related data and the user identifier from the terminal device 4.

  It is preferable that the receiving unit 12 also receives an utterance content identifier or utterance content data.

  The receiving unit 12 is data of a result of processing performed by the terminal middleware unit 44 of the terminal device 4 and is changed to data that can be transmitted with respect to voice-related data that is a result of performing the first partial processing on voice. It is preferable to receive data including the voice-related data from the terminal device 4 after being processed. The data received by the receiving unit 12 may include a user identifier, utterance content identifier, or utterance content data.

  The receiving unit 12 may receive a processing instruction for the pronunciation rating information. This processing instruction is processing for one or more pronunciation rating information by the processing unit 17 described later. Further, the processing instruction for the pronunciation rating information may include a user identifier.

  The middleware unit 13 is data received by the receiving unit 12 and changes received data including audio-related data to data that can be subjected to the second partial processing by the evaluation result acquisition unit 14. Such processing is referred to as middleware first processing. The result of the first processing of the middleware includes voice related data.

  Note that the data change in the first middleware processing includes decryption of encrypted data, decompression of compressed data, structural conversion of data received by the receiving unit 12, and pronunciation from data received by the receiving unit 12. This is one or more processes in the removal of information unrelated to the rating, and usually includes a process corresponding to a communication protocol used for communication with the terminal device 4. The communication protocol is, for example, HTTP, TCP / IP, or the like. Further, the middleware unit 13 normally does not perform processing related to the pronunciation evaluation itself.

  Further, the middleware unit 13 changes the evaluation result acquired by the evaluation result acquisition unit 14 to data that can be transmitted by the transmission unit 15. Such processing is referred to as second middleware processing.

  The data change in the second processing of the middleware is one or more of the encryption of the evaluation result, the compression processing of the evaluation result, the structure conversion of the evaluation result, and the like, and is used for communication with the terminal device 4 This is processing corresponding to the communication protocol to be performed.

  When the middleware unit 13 and a rating result acquisition unit 14 described later operate on the same computer (pronunciation rating server device 1), a local loop free from the influence of irrelevant network packets is used for higher-speed communication. It is preferable to communicate through the back interface.

  The rating result acquisition unit 14 performs the second partial process on the voice-related data and acquires the rating result. The second partial process is a second partial process for pronunciation evaluation to be performed after the first partial process.

  Moreover, the rating result acquisition part 14 may perform a 2nd partial process with respect to the audio | voice related data after the middleware part 13 changed, and may acquire a rating result.

  Here, the second partial process is usually a process of collating with teacher data based on voice-related data and utterance content data, performing pronunciation evaluation, and calculating a rating value. The second partial processing is, for example, collating with teacher data based on voice-related data and utterance content data, performing pronunciation rating, calculating a rating value, and calculating the calculated one or more rating values. It is a process which acquires a rating result using.

  Also, the second partial processing is, for example, acquiring feature data from voice data, calculating the posterior probability that the feature data is the phoneme to be evaluated using the feature data and the teacher data, This is a process of calculating a rating value and acquiring a rating result from one or more rating values.

  Further, the second partial processing is, for example, using feature data and teacher data, calculating a posterior probability that the feature data is a phoneme to be evaluated, calculating a speech rating value from the posterior probability, and calculating one or more This is a process for obtaining a rating result from a rating value.

  The second partial process includes, for example, a feature vector partial series acquisition process and a rating value calculation process. The feature vector partial sequence acquisition process is a sequence of one or more phonemes as utterance content data to be evaluated from feature data (also referred to as a feature vector sequence) that is speech-related data, and the same phoneme is continuous. This is a process of acquiring one or more feature vector partial sequences that are a set of one or more feature vectors corresponding to an optimal phoneme partial sequence that is a set of one or more optimal phoneme sequences. The rating value calculation process reads teacher data from the storage unit 11, uses the teacher data to calculate a posterior probability that the acquired feature vector partial series is a phoneme to be evaluated, and evaluates a speech rating value from the posterior probability Is a process for calculating.

  The second partial process includes, for example, a frame audio data acquisition process, a feature data acquisition process, a feature vector partial series acquisition process, and a rating value calculation process. The frame audio data acquisition process is a process of dividing audio data that is audio-related data into frames and obtaining one or more frame audio data that are audio data for each of the divided frames. The feature data acquisition process is a process of acquiring feature data that is a set of feature vectors for each frame from one or more frame audio data.

  In addition, the algorithm which acquires a rating result does not ask | require. Examples of the algorithm for obtaining the evaluation result are described in Patent Document 1, Patent Document 2, and the like described above.

  The transmission unit 15 transmits the evaluation result acquired by the evaluation result acquisition unit 14 to the terminal device 4.

  The transmission unit 15 may transmit information after the middleware unit 13 performs the second middleware process to the terminal device 4. Such information includes rating results.

  In addition, the transmission unit 15 may transmit a processing result, which is a result of the processing unit 17 performing processing on one or more pronunciation rating information, to the terminal device 4.

  The accumulating unit 16 accumulates one or more pronunciation rating information in which voice related data, utterance content data or utterance content identifier, and a rating result are associated with each other. The pronunciation rating information includes, for example, voice related data, utterance content data or utterance content identifier, and a rating result. The storage unit 16 normally stores one or more pronunciation rating information in the storage unit 11. Note that the utterance content data is data indicating the utterance content presented to the user who is practicing pronunciation. Further, the storage unit 16 may store one or more pronunciation rating information in a recording medium of an external device (not shown).

  The storage unit 16 may store one or more pronunciation rating information in association with the user identifier. Further, the storage unit 16 may store one or more pronunciation rating information for each user attribute value. The user attribute value is, for example, information for identifying age, age group, sex, and sound quality. The user attribute value is accumulated in the storage unit 11 in association with the user identifier, for example.

  Here, the utterance content data or the utterance content identifier may be acquired in any way. The reception unit 12 may receive the utterance content data or the utterance content identifier from the terminal device 4. Moreover, the utterance content data or the utterance content identifier may be stored in the storage unit 11 in advance.

  The processing unit 17 processes one or more pronunciation rating information. The processing here is, for example, display, transmission to an external device (terminal device 4 or the like), statistical processing, and the like.

  The processing unit 17 may perform processing on one or more pronunciation rating information when the receiving unit 12 receives an instruction for processing on the pronunciation rating information. The trigger and timing at which the processing unit 17 performs processing does not matter.

  The processing unit 17 may perform processing on one or more pronunciation rating information corresponding to the user identifier included in the processing instruction for the pronunciation rating information by the receiving unit 12.

  The statistical process is, for example, a process of calculating and outputting an average, variance, deviation value, etc. of two or more evaluation results. Statistical processing means, for example, calculating the average, variance, deviation value, etc. of two or more evaluation results according to the user's age, and analyzing the difference in voice quality of voice data and the bias of evaluation results due to background noise. Etc. In the statistical process, some processed information may be acquired using two or more evaluation results.

  The statistical processing means, for example, statistical analysis of a user's learning progress, pronunciation success or failure tendency, question difficulty, learning effect, and the like. For example, a passing score of the pronunciation evaluation result corresponding to the question is held in the storage unit 11, and the processing unit 17 acquires a pass / fail result for the question for each user, and acquires a learning progress for each user. The learning progress is, for example, the ratio of questions that have passed, the number of questions that have passed, the level value corresponding to the passed questions, and the like. In addition, for example, the characteristics of each question are held in the storage unit 11, and the processing unit 17 has a characteristic of a question whose pronunciation value of one user is equal to or greater than a threshold value (successful pronunciation) (for example, pronunciation of “th”, Pronunciation of “l” or “r”, etc.), or the characteristics of a question whose rating value of one user is less than a threshold (pronunciation failure), to analyze the tendency of pronunciation success or failure To do. In addition, for example, the storage unit 11 holds information indicating the relationship between the difficulty level of the question and the rating value, and the processing unit 17 calculates an average value of the rating values of each question of two or more users. Get the difficulty of each question. Further, for example, the processing unit 17 acquires the degree of increase in the rating value for each user and outputs it as a processing result indicating a learning effect. It is preferable that the processing unit 15 transmits the processing result to the terminal device 4.

  The terminal storage part 41 which comprises the terminal device 4 can store various information. The various information includes, for example, a user identifier, one or more user attribute values, and the like. The various types of information are, for example, one or more utterance content identifiers. The various types of information are, for example, one or more utterance content data. The utterance content identifier or the utterance content data corresponds to a question for pronunciation evaluation. In addition, two or more utterance content identifiers or two or more utterance content data can be information constituting a question list of pronunciation ratings.

  The terminal reception unit 42 receives various data and instructions from the user. The terminal reception unit 42 receives voice from the user. The various data and instructions are, for example, operation instructions, processing instructions for pronunciation evaluation information, and the like. The operation instruction is an instruction to start a pronunciation rating service.

  Here, reception is reception from input means, such as a microphone, a keyboard, a mouse, and a touch panel, for example. The voice input means may be anything such as a microphone or reading from a recording medium. The terminal reception unit 42 can be realized by a device driver of an input unit such as a microphone, a keyboard, a mouse, and a touch panel.

  The terminal first partial processing unit 43 performs a first partial process on the voice received by the terminal receiving unit 42 and acquires voice related data. Here, the first partial process is, for example, a process of digitizing the received voice and acquiring voice data. The first partial process is, for example, a process of digitizing sound, acquiring sound data, and acquiring feature data that is a set of feature vectors from the sound data. The first partial processing is, for example, a process of digitizing sound, obtaining sound data, dividing the sound data into frames, and obtaining one or more frame sound data that is sound data for each divided frame. It is. That is, the voice related data is, for example, voice data, feature data, one or more frame voice data, and the like.

  The terminal middleware unit 44 changes the voice-related data acquired by the terminal first partial processing unit 43 and acquires data to be transmitted. For example, the terminal middleware unit 44 encrypts voice-related data, compresses voice-related data, and converts a data structure for transmission. Such processing usually includes processing corresponding to a communication protocol used for communication with the pronunciation rating server device 1. In addition, when the terminal middleware part 44 compresses audio | voice related data, a system with little deterioration is desirable. When the terminal middleware unit 44 compresses audio-related data, it is particularly preferable that the terminal middleware unit 44 compresses the audio frequency band information without degrading the information. Furthermore, it is preferable that the terminal middleware unit 44 adopts a method that can be executed at high speed in the compression / decompression processing of the voice-related data. The process of the terminal middleware unit 44 is referred to as a first process of the terminal middleware.

  The terminal middleware unit 44 is data received by the terminal transmitting / receiving unit 45, converts data including the evaluation result, and obtains data that can be output. The terminal middleware unit 44, for example, decodes received data, decompresses it, or converts the data structure. Such processing usually includes processing corresponding to a communication protocol used for communication with the pronunciation rating server device 1. The process of the terminal middleware unit 44 is referred to as a second process of the terminal middleware.

  Note that the terminal middleware unit 44 normally does not perform processing related to the pronunciation evaluation itself.

  The terminal transmitting / receiving unit 45 transmits the voice related data acquired by the terminal first partial processing unit 43 to the pronunciation rating server device 1. The terminal transmission / reception unit 45 may be data changed by the terminal middleware unit 44 and may transmit data including voice related data to the pronunciation rating server device 1.

  The terminal transmission / reception unit 45 may also transmit the user identifier in the storage unit 11 to the pronunciation rating server device 1.

  The terminal transmission / reception unit 45 may also transmit the utterance content data or the utterance content identifier to the pronunciation rating server device 1.

  In addition, the terminal transmission / reception unit 45 may transmit an instruction to process the pronunciation rating information received by the terminal reception unit 42 to the pronunciation rating server apparatus 1.

  Further, the terminal transmission / reception unit 45 receives data including a rating result from the pronunciation rating server device 1.

  The terminal output unit 46 preferably uses the utterance content identifier of the terminal storage unit 41 to acquire and output the utterance content data corresponding to the utterance content identifier. The terminal output unit 46 may output the utterance content data in the terminal storage unit 41.

  Further, the terminal output unit 46 outputs the evaluation result received by the terminal transmission / reception unit 45. The terminal output unit 46 is data obtained by conversion by the terminal middleware unit 44, and outputs a rating result from data including a rating result.

  Note that the terminal output unit 46 may acquire two or more utterance content identifiers in the terminal storage unit 41, acquire two or more utterance content data, and output the data. In such a case, it is preferable that the terminal output unit 46 outputs the utterance content data identified by the utterance content identifier next to the utterance content identifier corresponding to the evaluation result after outputting the one evaluation result. That is, the terminal output unit 46 sequentially outputs the utterance content data and the rating result corresponding to the utterance content data, and sequentially outputs two or more utterance content data and the rating result. It is preferable to carry out continuously.

  Further, the terminal output unit 46 may acquire and output two or more utterance content data in the terminal storage unit 41. In such a case, it is preferable that the terminal output unit 46 outputs the utterance content data next to the utterance content data corresponding to the evaluation result after outputting the one evaluation result. That is, the terminal output unit 46 sequentially outputs the utterance content data and the rating result corresponding to the utterance content data, and sequentially outputs two or more utterance content data and the rating result. It is preferable to carry out continuously.

  Here, output refers to display on a display, projection using a projector, printing with a printer, sound output, transmission to an external device, storage in a recording medium, and output to other processing devices or other programs. It is a concept that includes delivery of processing results.

  The storage unit 11 and the terminal storage unit 41 are preferably non-volatile recording media, but can also be realized by volatile recording media. The process in which information is stored in the storage unit 11 or the like is not limited. For example, information may be stored in the storage unit 11 or the like via a recording medium, and information transmitted via a communication line or the like may be stored in the storage unit 11 or the like. Alternatively, information input via the input device may be stored in the storage unit 11.

  The receiving unit 12 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast.

  The middleware unit 13, the evaluation result acquisition unit 14, the storage unit 16, the processing unit 17, the terminal first partial processing unit 43, and the terminal middleware unit 44 can be usually realized by an MPU, a memory, or the like. The processing procedure of the middleware unit 13 and the like is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The transmission unit 15 is usually realized by a wireless or wired communication unit, but may be realized by a broadcasting unit.

  The terminal transmission / reception unit 45 is usually realized by a wireless or wired communication means.

  The terminal output unit 46 may be considered as including or not including an output device such as a display or a speaker. The terminal output unit 46 can be implemented by output device driver software, or output device driver software and an output device.

  Next, the operation of the pronunciation rating system A will be described. First, the operation of the pronunciation rating server device 1 will be described using the flowchart of FIG.

  (Step S <b> 301) The receiving unit 12 determines whether voice related data or the like has been received from the terminal device 4. When the voice related data or the like is received, the process goes to step S302, and when not received, the process goes to step S308. Note that the voice-related data and the like are, for example, voice-related data and at least one of a user identifier, an utterance content identifier, and an utterance content data.

  (Step S302) The middleware unit 13 performs a first middleware process on the voice-related data received in step S301, and acquires data for which the evaluation result acquisition unit 14 can perform the second partial process.

  (Step S303) The rating result acquisition unit 14 performs a second partial process on the data acquired in step S302, and acquires the rating result. The rating result acquisition unit 14 usually performs pronunciation rating processing on the data acquired in step S302 using the utterance content data and the teacher data, and acquires the rating result.

  Note that the utterance content data is, for example, utterance content data corresponding to the utterance content identifier received by the receiving unit 12. The teacher data is teacher data in the storage unit 11. That is, the rating result acquisition unit 14 acquires utterance content data corresponding to the utterance content identifier received by the reception unit 12 from a database (not shown), an external device, or the like. Further, the evaluation result acquisition unit 14 acquires teacher data in the storage unit 11. Then, the rating result acquisition unit 14 collates the teacher data using the voice-related data and the utterance content data, performs a pronunciation rating, and calculates a rating value. In addition, since the process of performing pronunciation evaluation and calculating the evaluation value is a known technique, detailed description thereof is omitted.

  (Step S304) The middleware unit 13 performs second processing of middleware on the evaluation result acquired in step S303, and configures data that can be transmitted by the transmission unit 15.

  (Step S305) The transmission unit 15 transmits data including the evaluation result configured in step S304 to the terminal device 4.

  (Step S306) The storage unit 16 configures pronunciation rating information in which voice related data, utterance content data or utterance content identifier, and a rating result are associated with each other.

  (Step S307) The accumulating unit 16 accumulates the pronunciation rating information configured in step S306 in the storage unit 11. The process returns to step S301.

  (Step S308) The receiving unit 12 determines whether or not an instruction for processing the pronunciation rating information has been received. If a processing instruction is received, the process proceeds to step S309. If a processing instruction is not received, the process returns to step S301.

  (Step S309) The processing unit 17 acquires one or more pronunciation rating information from the storage unit 11. The processing unit 17 may obtain one or more pronunciation rating information corresponding to the user identifier included in the processing instruction from the storage unit 11.

  (Step S310) The processing unit 17 performs processing on one or more pronunciation rating information acquired in step S309, and acquires a processing result.

  (Step S311) The transmission unit 15 transmits the processing result acquired in Step S310 to the terminal device 4. The process returns to step S301.

  In the flowchart of FIG. 3, the steps performed by the middleware unit 13 may not exist.

  In the flowchart of FIG. 3, it goes without saying that the algorithm of the pronunciation rating process in step S303 is not questioned.

  In the flowchart of FIG. 3, the middleware unit 13 may perform processing on the pronunciation instruction information in step S308 and the processing result in step S310.

  Further, in the flowchart of FIG. 3, the process is ended by power-off or a process end interrupt.

  Next, operation | movement of the terminal device 4 is demonstrated using the flowchart of FIG.

  (Step S401) The terminal receiving unit 42 determines whether an operation instruction has been received from the user. When the operation instruction is accepted, the process goes to step S402, and when the operation instruction is not accepted, the process goes to step S414.

  (Step S402) The terminal output unit 46 substitutes 1 for the counter i.

  (Step S403) The terminal output unit 46 determines whether or not the i-th question exists. If the i-th question exists, the process goes to step S404, and if the i-th question does not exist, the process returns to step S401. The terminal output unit 46 determines whether, for example, the i-th utterance content identifier corresponding to the i-th question is stored in the storage unit 11.

  (Step S404) The terminal output unit 46 acquires the utterance content data corresponding to the utterance content identifier from the storage unit 11 or an external device using the utterance content identifier corresponding to the i-th question. The terminal output unit 46 may acquire the utterance content data corresponding to the i-th question from the storage unit 11 or an external device.

  (Step S405) The terminal output unit 46 outputs the utterance content data acquired in step S404.

  (Step S406) The terminal receiving unit 42 determines whether or not a voice is received from the user. When the voice is accepted, the process goes to step S407, and when the voice is not accepted, the process returns to step S406. Note that the received voice is the voice of the user who pronounced the utterance content data.

  (Step S407) The terminal first partial processing unit 43 performs first partial processing on the voice received in Step S406, and acquires voice-related data and the like.

  (Step S408) The terminal middleware unit 44 executes the first process of the terminal middleware on the voice related data acquired in Step S407, and acquires voice related data and the like to be transmitted. The voice related data has voice related data. The voice related data or the like includes, for example, one or more data of a user identifier, utterance content data, and utterance content identifier.

  (Step S409) The terminal transmission / reception unit 45 transmits the sound-related data and the like acquired in step S408 to the pronunciation rating server device 1.

  (Step S410) The terminal transmitting / receiving unit 45 determines whether or not data including a rating result has been received from the pronunciation rating server device 1. When the data including the evaluation result is received, the process goes to step S411. When the data is not received, the process returns to step S410.

  (Step S411) The terminal middleware unit 44 performs the second process of the terminal middleware on the data received in Step S410, and acquires data including the evaluation result.

  (Step S412) The terminal output unit 46 outputs the rating result acquired in step S411.

  (Step S413) The terminal output unit 46 increments the counter i by one. The process returns to step S403.

  (Step S414) The terminal receiving unit 42 determines whether or not a processing instruction for pronunciation rating information has been received from the user. If a processing instruction is accepted, the process proceeds to step S415. If a processing instruction is not accepted, the process returns to step S401.

  (Step S415) The terminal transmission / reception unit 45 transmits a processing instruction for the pronunciation rating information received in Step S414 to the pronunciation rating server apparatus 1. Here, it is preferable that the terminal transmission / reception unit 45 transmits an instruction including the user identifier to the pronunciation rating server device 1.

  (Step S416) The terminal transmission / reception unit 45 determines whether or not data including a processing result has been received from the pronunciation rating server device 1. When data is received, the process goes to step S417, and when data is not received, the process returns to step S416.

  (Step S417) The terminal output unit 46 outputs the processing result received in Step S416. The process returns to step S401.

  In the flowchart of FIG. 4, the terminal middleware unit 44 may perform terminal middleware processing on the data including the instruction of processing on the pronunciation rating information and the received rating result.

  In the flowchart of FIG. 4, the steps performed by the terminal middleware unit 44 may not exist.

  Further, in the flowchart of FIG. 4, the process is terminated by powering off or a process termination interrupt.

  As described above, according to the present embodiment, the pronunciation rating service can be provided to the user without installing the program that implements the algorithm of the part that forms the pronunciation rating core in the terminal device, so the pronunciation rating core is formed. The risk that the algorithm is known to the outside can be reduced. Alternatively, according to the pronunciation rating server device according to the present invention, high-speed pronunciation rating processing can be performed without relying on a terminal device having a low processing capability.

  In the pronunciation rating server device 1 according to the present embodiment, it is preferable that the device in which the middleware unit 13 operates and the device in which the rating result acquisition unit 14 operate are separated. Moreover, it is preferable that the apparatus including the evaluation result acquisition unit 14 is not connected to a network that can be accessed by an external apparatus such as the Internet. This is because it is possible to prevent leakage of the algorithm of the part that forms the core of pronunciation evaluation.

  In this case, a block diagram of the pronunciation rating system A is shown in FIG. The pronunciation rating system A includes a middleware device A1, a pronunciation rating device A2, and a terminal device 4. The middleware apparatus A1 includes a reception unit 12, a middleware unit 13, and a transmission unit 15. The pronunciation rating device A2 includes a storage unit 11, a rating result acquisition unit 14, a storage unit 16, and a processing unit 17. The middleware device A1 and the pronunciation rating device A2 are usually connected via a LAN or the like, and the pronunciation rating device A2 cannot be accessed from the outside.

  Note that the pronunciation rating system A in FIG. 5 is an example of a separation mode of the middleware unit 13 and the rating result acquisition unit 14. The middleware part 13 and the evaluation result acquisition part 14 should just exist in a different apparatus. That is, the middleware device A1 includes only the reception unit 12 and the middleware unit 13, and the pronunciation rating device A2 includes the storage unit 11, the rating result acquisition unit 14, the transmission unit 15, the storage unit 16, and the processing unit 17. May be. The middleware device A1 includes a storage unit 11, a reception unit 12, a middleware unit 13, a transmission unit 15, a storage unit 16, and a processing unit 17, and the pronunciation evaluation device A2 includes only the evaluation result acquisition unit 14. Also good.

  Since the middleware unit 13 and the rating result acquisition unit 14 are separated and the rating result acquisition unit 14 cannot be accessed from an external device, leakage of the algorithm of the rating process can be avoided.

  Furthermore, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. In addition, the software which implement | achieves the pronunciation rating server apparatus 1 in this Embodiment is the following programs. In other words, this program is a result of accepting the voice uttered by the user for the utterance content instructed by the computer and performing the first partial process, which is the first partial process for pronunciation evaluation, on the voice. A receiving unit that receives voice-related data from the terminal device, and a second partial process that is a second partial process for pronunciation evaluation to be performed after the first partial process is performed on the voice-related data, and a rating result is obtained. It is a program for functioning as a rating result acquisition unit to be acquired and a transmission unit for transmitting the rating result to the terminal device.

  In the program, a storage unit that stores the voice related data, the utterance content data indicating the instructed utterance content or the utterance content identifier for identifying the utterance content, and one or more pronunciation rating information in association with the rating result It is preferable that the program further causes a computer to function as the processing unit that processes the one or more pronunciation rating information.

  In the above program, the receiving unit also receives an utterance content identifier for identifying the utterance content, or utterance content data acquired using the utterance content identifier for identifying the utterance content, and the storage unit It is preferable that the program causes a computer to function as the related data, the utterance content data or the utterance content identifier, and the rating result.

  In the above program, the received data including the voice-related data that is received by the receiving unit is changed to data that can be subjected to the second partial processing by the rating result acquisition unit, and the rating result acquisition unit acquires the data. The computer further functions as a middleware unit that changes the rating result to data that can be transmitted by the transmission unit, and the rating result acquisition unit performs second partial processing on the voice-related data after the middleware unit has changed. It is preferable that the transmission unit is a program that causes a computer to function as transmitting the evaluation result after the change by the middleware unit to the terminal device.

  In the above program, the receiving unit is data of a result of processing performed by a terminal middleware unit of the terminal device, and for voice related data that is a result of performing the first partial processing on the voice, It is preferable that the program is a program that causes a computer to function as data that has been changed to data that can be transmitted and that includes data related to voice, from the terminal device.

(Embodiment 2)
In the present embodiment, the difference from the first embodiment is that the pronunciation rating system includes a plurality of terminal devices. That is, in this embodiment, it is a multi-client.

  In the present embodiment, a pronunciation rating system in which the first partial processing performed by the pronunciation rating server apparatus may be different depending on the terminal device will be described. That is, in such a case, for example, one terminal device stores an old version of the pronunciation rating client program, and another terminal device stores a new version of the pronunciation rating client program. It is.

  Furthermore, in the present embodiment, a pronunciation rating system including a pronunciation rating server apparatus that uses different teacher data for pronunciation evaluation according to user attribute values will be described.

  FIG. 6 is a conceptual diagram of the pronunciation rating system B in the present embodiment. The pronunciation rating system B includes a pronunciation rating server device 2 and two or more terminal devices 4.

  FIG. 7 is a block diagram of the pronunciation rating system B in the present embodiment. The pronunciation rating server device 2 includes a storage unit 11, a receiving unit 22, a middleware unit 23, a rating result acquisition unit 24, a transmission unit 15, a storage unit 16, and a processing unit 17.

  The receiving unit 22 constituting the pronunciation rating server device 2 receives voice-related data from two or more terminal devices 4. The receiving unit 22 normally receives voice-related data and user identifiers from two or more terminal devices 4.

  The receiving unit 22 may receive different types of audio-related data from two or more terminal devices 4. Different types of sound-related data are results of two or more terminal devices 4 performing different first partial processes. The difference in the voice related data is caused by, for example, a difference in the model of the terminal device 4 or a difference in the version of the pronunciation rating terminal program installed in the terminal device 4.

  That is, for example, the first type terminal device 4 acquires audio data from the received audio and transmits the audio data to the pronunciation rating server device 2. The first partial process performed by the first type terminal device 4 is an audio data acquisition process. The second type terminal device 4 acquires voice data from the received voice, acquires feature data from the voice data, and transmits the feature data to the pronunciation rating server device 2. The first partial processing performed by the second type of terminal device 4 is voice data acquisition processing and feature data acquisition processing.

  Further, for example, the first type terminal device 4 performs terminal middleware processing and transmits the result to the pronunciation rating server device 2. The result is, for example, voice data or feature data. Further, the second type terminal device 4 transmits voice data or feature data to the pronunciation rating server device 2 without performing terminal middleware processing.

  It is preferable that the receiving unit 22 further receives an utterance content identifier or utterance content data from each terminal device 4.

  The receiving unit 22 is data as a result of processing performed by the terminal middleware unit 44 of the terminal device 4, and it is preferable to receive data including audio-related data from two or more terminal devices 4.

  The middleware unit 23 is data received by the receiving unit 22 from the two or more terminal devices 4, and changes the received data including the voice-related data to data that the evaluation result acquisition unit 24 can perform the second partial processing. For example, when the receiving unit 22 receives audio-related data, the middleware unit 23 performs a first middleware process in parallel by a process such as forking a process or starting a thread.

  Further, the middleware unit 23 changes the evaluation result acquired by the evaluation result acquisition unit 24 to data that can be transmitted by the transmission unit 15. This process is the second middleware process.

  The rating result acquisition unit 24 performs the second partial process on the voice-related data and acquires the rating result. The rating result acquisition unit 24 may perform the second partial process on the voice-related data after the middleware unit 23 has changed and acquire the rating result.

  The rating result acquisition unit 24 performs a second partial process on the two or more audio-related data received by the receiving unit 22 from the two or more terminal devices 4 in parallel, and acquires two or more rating results. It is preferable that two or more evaluation results are immediately transmitted to each terminal device 4. The parallel processing may be parallel processing. For example, when the reception unit 22 receives audio-related data, the evaluation result acquisition unit 24 may perform the second partial processing in parallel by processing such as forking a process or starting a thread.

  The rating result acquisition unit 24 may perform different second partial processing on different types of audio-related data and acquire the rating result. Different types of voice-related data are, for example, voice data, feature data, and the like. That is, when the received voice-related data is voice data, for example, the evaluation result acquisition unit 24 extracts feature data from the voice data and collates with teacher data based on the feature data and the utterance content data. Perform pronunciation rating and calculate the rating value. Further, when the received voice-related data is feature data, the evaluation result acquisition unit 24 compares the teacher data with the teacher data based on the received feature data and utterance content data, for example, Is calculated. That is, when the received voice related data is feature data, the process of extracting feature data from the voice data is not necessary. When the received voice-related data is one or more voice frame data, the evaluation result acquisition unit 24 extracts, for example, feature data from the one or more voice frame data, and based on the feature data and the utterance content data. Then, the pronunciation value is checked against the teacher data, and the rating value is calculated.

  It is preferable that the evaluation result acquisition unit 24 performs the second partial process on the voice-related data using different teacher data according to the user attribute value that is the user attribute value, and acquires the evaluation result. . The teacher data is stored in the storage unit 11 in association with the user attribute value, for example. Here, the user attribute value is, for example, one or more pieces of information of age group and sex.

  The receiving unit 22 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast.

  The middleware unit 23 and the evaluation result acquisition unit 24 can be usually realized by an MPU, a memory, or the like. The processing procedure of the middleware unit 23 and the like is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the pronunciation rating system B will be described. First, the operation of the pronunciation rating server device 2 will be described using the flowchart of FIG. In the flowchart of FIG. 8, the description of the same steps as those in the flowchart of FIG. 3 is omitted. When the receiving unit 22 receives the voice-related data in step S301, the middleware unit 23 performs first middleware processing (processing in S302) in parallel by processing such as forking a process and starting up a thread. I do. Further, when the first middleware process is not performed, when the receiving unit 22 receives the voice-related data in step S301, the evaluation result acquisition unit 24 performs a process such as forking a process and starting a thread. In parallel, the second partial process of step S801 is performed.

  (Step S801) The rating result acquisition unit 24 performs second partial processing on the data acquired in step S302, and acquires the rating result. This pronunciation evaluation process is different from the second partial process in the first embodiment. Details of the second partial process in step S801 will be described with reference to the flowchart of FIG.

  In the flowchart of FIG. 8, the process ends when the power is turned off or the process is terminated.

  Next, details of the second partial processing in step S801 will be described with reference to the flowchart of FIG.

  (Step S901) The evaluation result acquisition unit 24 acquires one or more user attribute values. The rating result acquisition unit 24 acquires, from the storage unit 11, one or more user attribute values that are paired with the user identifier received in step S301, for example. The rating result acquisition unit 24 may acquire, for example, one or more user attribute values received in step S301.

  (Step S902) The evaluation result acquisition unit 24 acquires teacher data corresponding to one or more user attribute values acquired in step S901 from the storage unit 11.

  (Step S903) The rating result acquisition unit 24 determines the type of the received voice-related data. The evaluation result acquisition unit 24 may determine the type of audio-related data from the content of data such as the format and structure of the audio-related data received in step S301, or the type of audio-related data received in step S301. The type of voice-related data may be determined from the type identifier for identifying. Note that determining the type is, for example, obtaining the type identifier of the voice related data.

  (Step S904) The evaluation result acquisition unit 24 performs the second partial process according to the type of the voice related data determined in Step S903. Return to upper process.

  The second partial process corresponding to the type of audio related data is, for example, as follows. When the type of speech-related data is feature data, for example, the second partial process calculates the posterior probability that the feature data is the phoneme to be evaluated using the feature data and the teacher data. This is a process of calculating a rating value and acquiring a rating result from one or more rating values. In addition, when the type of speech related data is speech data, for example, feature data is obtained from speech data, and the posterior probability that the feature data is a phoneme to be evaluated is calculated using the feature data and the teacher data. This is a process of calculating a voice rating value from the posterior probability and obtaining a rating result from one or more rating values.

  Hereinafter, a specific operation of the pronunciation rating system B in the present embodiment will be described. A conceptual diagram of the pronunciation rating system B is shown in FIG.

  Now, the storage unit 11 holds the user information management table shown in FIG. The user information management table has “ID”, “user identifier”, “name”, and “user attribute value”. The “user attribute value” has “age” and “sex”. “ID” is information for identifying a record.

  The storage unit 11 holds a teacher data management table shown in FIG. The teacher data management table has “ID”, “user attribute value condition”, “teacher data identifier”, and “teacher data”. The “user attribute value condition” is a condition for selecting teacher data, and is a condition related to a user attribute value. Here, the “user attribute value condition” is a condition corresponding to the age group and sex of the user.

  Further, it is assumed that a version 1 pronunciation rating client program is installed in the terminal device 4 of the user A “Taro Yamada”. Further, it is assumed that a version 2 pronunciation rating client program is installed in the terminal device 4 of the user B “Hanako Ota”. In the version 1 pronunciation rating client program, the first partial process is a process of digitizing the received voice and acquiring voice data. In the version 2 pronunciation rating client program, the first partial process is a process of digitizing the received voice, acquiring voice data, and acquiring feature data from the voice data.

  Further, it is assumed that the terminal device 4 of user A and the terminal device 4 of user B store the pronunciation practice list shown in FIG. The pronunciation practice list is a set of utterance content identifiers for identifying utterance content data.

  In the above situation, it is assumed that the user A inputs an operation instruction to his / her terminal device 4. Next, the terminal reception unit 42 receives an operation instruction from the user A.

  Next, the terminal output unit 46 of the terminal device 4 of the user A acquires the utterance content identifier “53” of the first question from the pronunciation practice list. Then, using the utterance content identifier “53”, the terminal output unit 46 acquires utterance content data corresponding to the utterance content identifier from an external device.

  Next, the terminal output unit 46 displays the acquired utterance content data. Then, it is assumed that the user A looks at the displayed utterance content data and pronounces it so as to match the utterance content data.

  Then, the terminal reception part 42 receives a voice from a user. Next, the terminal first partial processing unit 43 digitizes the received voice and acquires voice data that is voice-related data. Next, the terminal middleware unit 44 executes a first process of terminal middleware on the audio data. Here, the terminal middleware unit 44 adds the user identifier “1001” and the utterance content identifier “53” to the voice data, encrypts and compresses it, changes the data structure to match the HTTP communication protocol, and transmits it. Suppose you have acquired voice related data.

  Next, the terminal transmission / reception unit 45 of the terminal device 4 of the user A transmits the acquired voice related data and the like to the pronunciation rating server device 1.

  Then, it is assumed that the user B inputs an operation instruction to his / her terminal device 4 with a slight delay from the user A. Next, the terminal reception unit 42 of the terminal device 4 of the user B receives an operation instruction from the user B.

  Next, the terminal output unit 46 acquires the utterance content identifier “53” of the first question from the pronunciation practice list. Then, the terminal output unit 46 acquires the utterance content data corresponding to the utterance content identifier from the terminal storage unit 41 using the utterance content identifier “53”. It is assumed that utterance content data is stored in the terminal storage unit 41 of the terminal device 4 of the user B in association with the utterance content identifier.

  Next, the terminal output unit 46 of the terminal device 4 of the user B displays the acquired utterance content data. Then, it is assumed that the user B looks at the displayed utterance content data and pronounces it so as to match the utterance content data.

  Then, the terminal reception unit 42 of the terminal device 4 of the user B receives voice from the user. Next, the terminal first partial processing unit 43 digitizes the received voice and acquires voice data that is voice-related data. And the terminal 1st partial process part 43 acquires feature data from the acquired audio | voice data.

  Next, the terminal middleware unit 44 performs a first process of terminal middleware on the feature data. Here, the terminal middleware unit 44 of the terminal device 4 of the user B adds the user identifier “1002” and the utterance content identifier “53” to the voice data, encrypts and compresses them, and matches the HTTP communication protocol. Assume that the data structure is changed and the voice related data to be transmitted is acquired.

  Next, the terminal transmission / reception unit 45 of the terminal device 4 of the user B transmits the acquired voice related data and the like to the pronunciation rating server device 1.

  Next, the receiving unit 12 of the pronunciation rating server device 1 receives voice-related data and the like from the terminal device 4 of the user A.

  Then, the middleware unit 23 forks the process and performs the first middleware processing within the process (referred to as process 1). The evaluation result acquisition unit 24 performs the second partial process in the process 1.

  In addition, the receiving unit 12 of the pronunciation rating server device 1 receives voice related data and the like from the terminal device 4 of the user B.

  Then, the middleware unit 23 forks the process and performs the first middleware processing within the process (referred to as process 2). The evaluation result acquisition unit 24 performs the second partial process in the process 2.

Hereinafter, details of the process 1 and the process 2 will be described.
(Process 1)

  The middleware unit 23 performs a first process of middleware, and acquires data that the evaluation result acquisition unit 24 can perform the second partial process. That is, the middleware unit 23 decompresses the received voice-related data and the like, decodes it, and obtains data that can be subjected to the second partial processing. Here, the data that can be subjected to the second partial processing includes voice data, a user identifier “1001”, and an utterance content identifier “53”.

  Next, the rating result acquisition unit 24 performs the following second partial process on the acquired data, and acquires the rating result. That is, the rating result acquisition unit 24 acquires one or more user attribute values “age: 35” and “gender: male” paired with the user identifier “1001” from the user information management table of FIG. Next, the evaluation result acquisition unit 24 converts the teacher data identified by the teacher data identifier “teacher male_1.dat” corresponding to the user attribute values “age: 35” and “gender: male” into the teacher data management table of FIG. Get from.

  Next, the evaluation result acquisition unit 24 acquires utterance content data corresponding to the utterance content identifier “53”. The rating result acquisition unit 24 may acquire the utterance content data corresponding to the utterance content identifier “53” from the storage unit 11 or may be acquired from an external device.

  Next, the rating result acquisition unit 24 determines the type from the data content of the received voice-related data, and acquires the type identifier “1”. The type identifier “1” is “voice data”, and the type identifier “2” is “feature data”.

  Next, the evaluation result acquisition unit 24 uses the teacher data and the utterance content data identified by the teacher data identifier “teacher male_1.dat” to the voice-related data, using the type identifier “1” of the voice-related data. The second partial processing according to “is performed. That is, the rating result acquisition unit 24 extracts feature data from the voice data. Then, the rating result acquisition unit 24 applies the extracted feature data and utterance content data to the teacher data “teacher male_1.dat”, and calculates a rating value. Here, it is assumed that the rating value is the rating result.

  Next, the middleware unit 23 performs a second middleware process on the acquired evaluation result, and configures data that can be transmitted by the transmission unit 15. Here, it is assumed that the middleware unit 23 compresses the evaluation result and converts it into a data structure that can be transmitted using the HTTP communication protocol.

  Next, the transmission unit 15 transmits data including the configured evaluation result to the terminal device 4 of the user A.

  Next, the storage unit 16 configures pronunciation rating information in which voice related data, utterance content data or utterance content identifier, and a rating result are associated with each other.

  Next, the storage unit 16 stores the configured pronunciation rating information in the storage unit 11 in association with the user identifier “1001”.

  Next, the terminal transmitting / receiving unit 45 of the terminal device 4 of the user A receives data including the evaluation result. And the terminal middleware part 44 performs the 2nd process of terminal middleware with respect to the received data, and obtains the data which can be output. And the terminal output part 46 outputs the data containing the evaluation result obtained by the terminal middleware part 44 converting.

Thereafter, in the terminal device 4 of the user A, similarly, processing corresponding to the utterance content data identified by the utterance content identifiers “108” and “109” is performed.
(Process 2)

  The middleware unit 23 performs a first process of middleware, and acquires data that the evaluation result acquisition unit 24 can perform the second partial process. That is, the middleware unit 23 decompresses the received voice-related data and the like, decodes it, and obtains data that can be subjected to the second partial processing. Here, the data that can be subjected to the second partial processing includes voice data, a user identifier “1002”, and an utterance content identifier “53”.

  Next, the rating result acquisition unit 24 performs the following second partial process on the acquired data, and acquires the rating result. That is, the rating result acquisition unit 24 acquires one or more user attribute values “age: 23” and “sex: female” that are paired with the user identifier “1002” from the user information management table. Next, the rating result acquisition unit 24 acquires the teacher data identified by the teacher data identifier “teacher woman_n.dat” corresponding to the user attribute values “age: 23” and “gender: woman” from the teacher data management table. .

  Next, the evaluation result acquisition unit 24 acquires utterance content data corresponding to the utterance content identifier “53”. The rating result acquisition unit 24 may acquire the utterance content data corresponding to the utterance content identifier “53” from the storage unit 11 or may be acquired from an external device.

  Next, the rating result acquisition unit 24 determines the type from the data content of the received voice-related data, and acquires the type identifier “2”.

  Next, the evaluation result acquisition unit 24 uses the teacher data and the utterance content data identified by the teacher data identifier “teacher_n.dat”, and the voice related data type identifier “2” for the voice related data. The second partial processing according to “is performed. That is, the rating result acquisition unit 24 applies the feature data and the utterance content data to the teacher data “teacher_n.dat”, and calculates a rating value. Here, it is assumed that the rating value is the rating result.

  Next, the middleware unit 23 performs a second middleware process on the acquired evaluation result, and configures data that can be transmitted by the transmission unit 15. Here, it is assumed that the middleware unit 23 compresses the evaluation result and converts it into a data structure that can be transmitted using the HTTP communication protocol.

  Next, the transmission unit 15 transmits data including the configured evaluation result to the terminal device 4 of the user B.

  Next, the storage unit 16 configures pronunciation rating information in which voice related data, utterance content data or utterance content identifier, and a rating result are associated with each other.

  Next, the storage unit 16 stores the configured pronunciation rating information in the storage unit 11 in association with the user identifier “1002”.

  Next, the terminal transmitting / receiving unit 45 of the terminal device 4 of the user B receives data including the evaluation result. And the terminal middleware part 44 performs the 2nd process of terminal middleware with respect to the received data, and obtains the data which can be output. And the terminal output part 46 outputs the data containing the evaluation result obtained by the terminal middleware part 44 converting.

  Thereafter, in the terminal device 4 of the user B, similarly, processing corresponding to the utterance content data identified by the utterance content identifiers “108” and “109” is performed.

  As described above, according to the present embodiment, in the multi-client pronunciation rating system, the pronunciation rating service can be provided to the user without installing the program that implements the algorithm for forming the pronunciation rating core in the terminal device 4. The risk of knowing the algorithm that forms the core of pronunciation rating to the outside can be reduced. Alternatively, according to the pronunciation rating server device 2 according to the present invention, high-speed pronunciation rating processing can be performed without relying on the terminal device 4 having a low processing capability.

  In addition, according to the present embodiment, it is possible to provide a pronunciation rating service to a plurality of terminal devices 4 while corresponding to specifications of different terminal devices 4.

  Further, according to the present embodiment, it is possible to perform pronunciation evaluation with high accuracy according to the user attribute value.

  In the pronunciation rating server device 1 according to the present embodiment, it is preferable that the device in which the middleware unit 23 operates and the device in which the rating result acquisition unit 24 operate are separated. FIG. 13 is a block diagram of the pronunciation rating system B in such a case. The pronunciation rating system B includes a middleware device B1, a pronunciation rating device B2, and a terminal device 4. The middleware device B1 includes, for example, a reception unit 12, a middleware unit 23, and a transmission unit 15. The pronunciation rating device B2 includes, for example, a storage unit 11, a rating result acquisition unit 24, a storage unit 16, and a processing unit 17. The middleware device B1 and the pronunciation rating device B2 are normally connected via a LAN or the like, and the pronunciation rating device B2 cannot be accessed from the outside.

  Note that the pronunciation rating system B in FIG. 13 is an example of a separation mode of the middleware unit 23 and the rating result acquisition unit 24. The middleware part 23 and the evaluation result acquisition part 24 should just exist in a different apparatus. That is, the middleware device B1 includes only the reception unit 12 and the middleware unit 23, and the pronunciation rating device B2 includes the storage unit 11, the rating result acquisition unit 24, the transmission unit 15, the storage unit 16, and the processing unit 17. May be. The middleware device B1 includes a storage unit 11, a receiving unit 12, a middleware unit 23, a transmission unit 15, a storage unit 16, and a processing unit 17. The pronunciation rating device B2 includes only a rating result acquisition unit 24. Also good.

  Furthermore, the software that implements the pronunciation rating server device 2 in the present embodiment is the following program. In other words, this program is a result of accepting the voice uttered by the user for the utterance content instructed by the computer and performing the first partial process, which is the first partial process for pronunciation evaluation, on the voice. A receiving unit that receives voice-related data from the terminal device, and a second partial process that is a second partial process for pronunciation evaluation to be performed after the first partial process is performed on the voice-related data, and a rating result is obtained. It is a program for functioning as a rating result acquisition unit to be acquired and a transmission unit for transmitting the rating result to the terminal device.

  In the program, the receiving unit receives voice-related data from two or more terminal devices, and the rating result acquisition unit performs parallel processing on each of the two or more voice-related data received by the receiving unit. It is preferable that the program further functions as a computer that performs two-part processing and obtains two or more evaluation results.

  In the above program, the receiving unit receives different types of audio-related data from two or more terminal devices, which is a result of different first partial processes performed by two or more terminal devices, and the rating result acquisition unit includes: It is preferable that the program further causes the computer to function as performing different second partial processing on each of the different types of audio-related data and obtaining the evaluation result.

  The computer-accessible storage medium includes a storage unit that stores two or more teacher data that are acoustic models of one or more phonemes. In the program, the evaluation result acquisition unit includes the attribute of the user A second partial process, which is a second partial process for pronunciation evaluation that should be performed after the first partial process, is performed on the speech-related data using different teacher data according to the user attribute value that is a value. It is preferable that the program is a program that further functions a computer as a means for obtaining the evaluation result.

(Embodiment 3)
In the present embodiment, the difference from the second embodiment is that the pronunciation rating system includes a plurality of pronunciation rating server devices. That is, in this embodiment, it is a multi-client and a multi-server.

  In this embodiment, a pronunciation rating system having an adjustment function that distributes the load of a plurality of pronunciation rating server devices will be described.

  FIG. 14 is a conceptual diagram of the pronunciation rating system C in the present embodiment. The pronunciation rating system C includes two or more pronunciation rating server devices 3 and two or more terminal devices 4.

  Of the two or more pronunciation rating server devices 3, it is preferable that only one pronunciation rating server device 3 can perform middleware processing. In this case, it is assumed that the pronunciation rating server apparatus that can perform middleware processing is 3A, and the pronunciation rating server apparatus that cannot perform middleware processing is 3B. In such a case, it is preferable that the pronunciation rating server device 3B is not connected to a network such as the Internet and cannot be accessed from an external device.

  FIG. 15 is a block diagram of the pronunciation rating system C in the present embodiment. Of the two or more pronunciation rating server devices 3, it is preferable that only one pronunciation rating server device 3 includes the middleware unit 33. The pronunciation rating server device 3A including the middleware unit 33 can communicate with two or more terminal devices 4 via a communication network such as the Internet, and the other pronunciation rating server device 3B includes two or more terminal devices 4; It is preferable that direct communication is not possible. In addition, it is preferable that one pronunciation rating server apparatus 3A including the middleware unit 33 and the other one pronunciation rating server apparatus 3B can communicate with each other via a LAN or the like.

  The pronunciation rating server device 3A includes a storage unit 31, a reception unit 22, a middleware unit 33, a rating result acquisition unit 24, a transmission unit 15, a storage unit 16, and a processing unit 17. The middleware unit 33 includes a data changing unit 331, an adjusting unit 332, a sending unit 333, and a receiving unit 334.

  The pronunciation rating server device 3B includes a storage unit 11, a receiving unit 32, a rating result acquisition unit 24, a transmission unit 35, a storage unit 16, and a processing unit 17. Note that the pronunciation rating server device 3B may include only the storage unit 11, the receiving unit 32, the rating result acquiring unit 24, and the transmitting unit 35.

  The storage unit 31 constituting the pronunciation rating server device 3A can store various types of information. The various information is, for example, one or more pieces of load information. The load information is information regarding the load for each device, and includes, for example, the device identifier of the pronunciation rating server device and the number of pronunciation rating processes being processed. The load information includes, for example, the device identifier of the pronunciation rating server device, the number of processes of pronunciation evaluation being processed, and the number of processes that can be processed in parallel.

  The various information includes, for example, one or more pronunciation rating information, one or more user information, one or more utterance content data, one or more utterance content identifiers, and the like.

  The middleware unit 33 performs processing performed by the data changing unit 331, the adjusting unit 332, the sending unit 333, and the receiving unit 334.

  The data changing unit 331 is data received by the receiving unit 22 and changes received data including voice-related data to data that can be subjected to the second partial processing by the evaluation result acquisition unit 24. Such processing is referred to as middleware first processing. Further, the first processing of the middleware is not related to, for example, decryption of encrypted data, decompression of compressed data, structural conversion of data received by the receiving unit 22, and pronunciation evaluation from data received by the receiving unit 12. And the like, and usually includes processing corresponding to a communication protocol used for communication with the terminal device 4. The communication protocol is, for example, HTTP, TCP / IP, or the like. Further, the middleware unit 33 normally does not perform processing related to the pronunciation evaluation itself.

  The data changing unit 331 changes the evaluation result acquired by the evaluation result acquisition unit 24 to data that can be transmitted by the transmission unit 15. Such processing is referred to as second middleware processing. Further, the second process of the middleware is, for example, data encryption, data compression, data structure conversion, and the like, and is a process corresponding to a communication protocol used for communication with the terminal device 4.

  The adjustment unit 332 determines the pronunciation rating server device 3 that performs the second partial processing on each of the two or more audio-related data received by the receiving unit 22. Such a process is a process for adjusting the load of two or more pronunciation rating server devices 3.

  The adjusting unit 332 refers to one or more pieces of load information stored in the storage unit 11, and determines the pronunciation rating server device 3 that performs the second partial process. For example, the adjustment unit 332 refers to one or more pieces of load information in the storage unit 11 and acquires the device identifier of the device having the smallest number of processes. For example, the adjustment unit 332 refers to one or more pieces of load information in the storage unit 11 in order, and acquires a device identifier of a device that can be processed by one process. Further, for example, the adjustment unit 332 refers to one or more pieces of load information in the storage unit 11 in order, and the device of the device having the smallest value of “number of pronunciation evaluation processes being processed / number of processes that can be processed in parallel” Get an identifier.

  The sending means 333 sends voice related data to the pronunciation rating server device 3 determined by the adjusting means 332. The pronunciation rating server device 3 determined by the adjusting unit 332 is, for example, the pronunciation rating server device 3 corresponding to the device identifier acquired by the adjusting unit 332.

  The accepting unit 334 accepts the rating result acquired by the pronunciation rating server device 3 determined by the adjusting unit 332.

  The receiving unit 32 constituting the pronunciation rating server device 3B receives voice-related data and the like from one pronunciation rating server device 3A including the middleware unit 33.

  The rating result acquisition unit 34 performs second partial processing on the voice-related data after the middleware unit 33 of the pronunciation rating server device 3A has changed, and acquires the rating result.

  The rating result acquisition unit 34 performs second partial processing on the two or more audio-related data received by the receiving unit 32 in parallel, and acquires two or more rating results. For example, when the reception unit 32 receives voice-related data, the evaluation result acquisition unit 34 may perform the second partial processing in parallel by processing such as forking a process or starting up a thread. This process is the same as that of the evaluation result acquisition unit 24.

  The rating result acquisition unit 34 may perform different second partial processing on different types of audio-related data and acquire the rating result. This process is the same as that of the evaluation result acquisition unit 24.

  The evaluation result acquisition unit 34 may acquire the evaluation result by performing the second partial process on the voice-related data using different teacher data according to the user attribute value that is the user attribute value. This process is the same as that of the evaluation result acquisition unit 24.

  The transmission unit 35 transmits the rating result to the pronunciation rating server device 3A. The transmission unit 15 transmits the evaluation result received by the receiving unit 334 to the terminal device 4.

  The data changing unit 331, the adjusting unit 332, and the evaluation result acquisition unit 34 can be usually realized by an MPU, a memory, or the like. The processing procedure of the middleware unit 33 and the like is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The transmission unit 35 and the sending unit 333 are usually realized by a wireless or wired communication unit, but may be realized by a broadcasting unit.

  The accepting unit 334 is usually realized by a wireless or wired communication unit, but may be realized at a stage of receiving a broadcast.

  Next, the operation of the pronunciation rating system C will be described. First, the operation of the pronunciation rating server device 3A will be described using the flowchart of FIG. In the flowchart of FIG. 16, the description of the same steps as those in the flowchart of FIG. 3 is omitted.

  (Step S1601) The adjustment unit 332 refers to the load information in the storage unit 11, and determines the pronunciation rating server device 3 that performs the second partial process. There may be various algorithms for the adjustment means 332 to determine the pronunciation rating server device 3.

  (Step S1602) The adjusting unit 332 updates the information on the load information corresponding to the device determined in Step S1601. For example, the adjustment unit 332 increments the number of processes paired with the device identifier acquired in step S1601 by one.

  (Step S1603) The sending means 333 determines whether or not the device determined in step S1601 is its own device (the pronunciation rating server device 3A). If it is its own device, it goes to step S303, and if it is not its own device, it goes to step S1605.

  (Step S1604) The adjustment unit 332 updates the information of the load information corresponding to the device that has performed the second partial process. For example, the adjustment unit 332 decrements the number of processes that are paired with the device identifier corresponding to the device that has performed the second partial process by one.

  (Step S1605) The sending means 333 transmits voice-related data and the like processed by the middleware to the apparatus determined in Step S1601.

  (Step S1606) The accepting unit 334 determines whether or not the rating result acquired by the pronunciation rating server device 3B determined by the adjusting unit 332 has been received from the pronunciation rating server device 3B. If received, go to step S1604; otherwise, return to step S1606.

  In the flowchart of FIG. 16, the process is terminated by power-off or a process termination interrupt.

  Next, the operation of the pronunciation rating server device 3B will be described using the flowchart of FIG. In the flowchart of FIG. 17, description of the same steps as those in the flowchart of FIG. 3 is omitted.

  (Step S1701) The receiving unit 32 determines whether voice related data or the like has been received from the pronunciation rating server device 3A. If received, the process goes to step S1702, and if not received, the process returns to step S1701. Note that the voice-related data and the like are, for example, voice-related data and a user identifier.

  (Step S1702) The transmission unit 35 transmits a rating result or the like to the pronunciation rating server device 3A. Note that the rating result and the like are, for example, a rating result, a user identifier, a device identifier, and the like.

  In the flowchart of FIG. 17, the process ends when the power is turned off or the process ends.

  Further, the sharing method of the pronunciation rating server device 3A and the pronunciation rating server device 3B may be another sharing method. For example, in the flowcharts of FIGS. 16 and 17, one or more pronunciation rating information is centrally managed by the pronunciation rating server device 3A. However, the phonetic rating server device 3A and one or more phonetic rating server devices 3B may be distributed and managed. In addition, the pronunciation rating server device 3A may not perform the second partial process.

  Hereinafter, a specific operation of the pronunciation rating system C in the present embodiment will be described. A conceptual diagram of the pronunciation rating system C is shown in FIG.

  Now, it is assumed that the pronunciation rating server device 3A includes a storage unit 11, a receiving unit 22, a middleware unit 33, a rating result acquisition unit 24, a transmission unit 15, a storage unit 16, and a processing unit 17. Further, the pronunciation rating server device 3B includes only the receiving unit 32, the rating result acquiring unit 24, and the transmitting unit 35. Further, the pronunciation rating system C includes one pronunciation rating server device 3A (device identified by the device identifier “d01”) and two pronunciation rating server devices 3B (devices identified by the device identifiers “d02” and “d03”). And a large number of terminal devices 4 receive the pronunciation rating service.

  It is assumed that the storage unit 11 of the pronunciation rating server device 3A holds the load information management table shown in FIG. The load information management table manages one or more pieces of load information. The load information includes “ID”, “device identifier”, “current process number”, and “maximum process number”. “ID” is information for identifying a record. The “device identifier” is information for identifying the pronunciation rating server device 3. “Number of current processes” is the number of pronunciation rating processes currently being processed in each pronunciation rating server device 3. The “maximum number of processes” is the maximum number of pronunciation rating processes that can be processed in parallel in each pronunciation rating server device 3. Note that the load information may not include the “maximum number of processes”.

  Further, it is assumed that the storage unit 11 holds the user information management table shown in FIG.

  Furthermore, it is assumed that the terminal device 4 of user A and the terminal device 4 of user B store the pronunciation practice list (list for advanced users) shown in FIG. The pronunciation practice list is a set of utterance content identifiers for identifying utterance content data.

  In this situation, as shown in the load information management table of FIG. 18, five pronunciation rating processes are currently executed in the pronunciation rating server apparatus 3A identified by the apparatus identifier “d01”. In addition, two pronunciation rating processes are currently being executed in each of the two pronunciation rating server devices 3B identified by the device identifiers “d02” and “d03”.

  In this situation, it is assumed that the user A has input an operation instruction to his / her terminal device 4. Next, the terminal reception unit 42 receives an operation instruction from the user A.

  Next, the terminal output unit 46 of the terminal device 4 of the user A acquires the utterance content identifier “53” of the first question from the pronunciation practice list. Then, using the utterance content identifier “53”, the terminal output unit 46 acquires utterance content data corresponding to the utterance content identifier from an external device.

  Next, the terminal output unit 46 displays the acquired utterance content data. Then, it is assumed that the user A looks at the displayed utterance content data and pronounces it so as to match the utterance content data.

  Then, the terminal reception part 42 receives a voice from a user. Next, the terminal first partial processing unit 43 digitizes the received voice and acquires voice data that is voice-related data. Next, the terminal middleware unit 44 executes a first process of terminal middleware on the audio data. Here, it is assumed that the terminal middleware unit 44 adds the user identifier “1001” and the utterance content identifier “53” to the voice data, performs encryption and compression, and acquires voice-related data to be transmitted.

  Next, the terminal transmission / reception unit 45 of the terminal device 4 of the user A transmits the acquired voice related data and the like to the pronunciation rating server device 3A.

  Next, the receiving unit 22 of the pronunciation rating server device 3A receives voice-related data and the like from the terminal device 4 of the user A.

  Then, the data changing unit 331 of the middleware unit 33 changes the data so that the received voice related data and the like can be pronounced.

  Next, the adjusting unit 332 determines the pronunciation rating server device 3 that performs the second partial process on the voice-related data and the like whose data has been changed. That is, the adjustment unit 332 refers to the load information management table shown in FIG. 18 and acquires the device identifier “d02” of the device having the smallest value of “current process number / maximum process number”. Here, it is assumed that the device identifier of the device with the smallest value of “current process number / maximum process number” is “d02” and “d03”, but the first “d02” is acquired.

  Next, the sending unit 333 sends the voice-related data whose data has been changed to the pronunciation rating server device 3B corresponding to the device identifier “d02” acquired by the adjusting unit 332.

  Next, the adjustment unit 332 updates the load information. That is, the current process number corresponding to the device identifier “d02” is incremented by 1 to “3”.

  Next, the receiving unit 32 of the pronunciation rating server device 3B corresponding to the device identifier “d02” receives voice-related data and the like.

  Then, the rating result acquisition unit 34 of the pronunciation rating server device 3B applies the received voice related data or the like to the teacher data or the like of the storage unit 11 and acquires the rating result. In addition, the acquisition method of this evaluation result is as having mentioned above.

  Next, the transmission unit 35 of the pronunciation rating server device 3B transmits a rating result or the like to the pronunciation rating server device 3A.

  Next, the receiving means 334 of the pronunciation rating server device 3A receives the rating result and the like.

  Then, the adjusting unit 332 updates the load information. That is, the adjustment unit 332 decrements the current process number corresponding to the device identifier “d02” by 1, and sets it to “2”.

  Next, the data changing unit 331 performs a second middleware process on the received evaluation result and the like.

  Next, the transmission unit 15 transmits data including the evaluation result and the like to the terminal device 4 as data obtained as a result of the second processing of the middleware performed by the data changing unit 331.

  Next, the accumulation unit 16 configures pronunciation rating information and accumulates it in the storage unit 11. The details of this processing have been described above.

  Next, the terminal transmission / reception unit 45 of the terminal device 4 receives data including the evaluation result. And the terminal middleware part 44 performs the 2nd process of terminal middleware with respect to the received data, and obtains the data which can be output. And the terminal output part 46 outputs the data containing the evaluation result obtained by the terminal middleware part 44 converting.

  The specific operation example of the pronunciation rating system C when one user enjoys the pronunciation rating service has been described above.

  As described above, according to the present embodiment, the load distribution of the pronunciation rating server device 3 can be achieved.

  In addition, according to the present embodiment, it is possible to reduce the risk of knowing the algorithm for forming the pronunciation evaluation core. Alternatively, the pronunciation rating server device according to the present invention can perform high-speed pronunciation rating processing.

  In addition, according to the present embodiment, it is possible to provide a pronunciation rating service to a plurality of terminal devices 4 while corresponding to specifications of different terminal devices 4.

  Further, according to the present embodiment, it is possible to perform pronunciation evaluation with high accuracy according to the user attribute value.

  It is preferable that the pronunciation rating server device 3A of the present embodiment does not have the rating result acquisition unit 24. That is, it is preferable that the pronunciation rating server device 3A does not perform the second partial process but performs the middleware process, the management of the pronunciation rating information, and the like. In such a case, the pronunciation rating server device 3 </ b> A includes a storage unit 31, a reception unit 22, a middleware unit 33, a transmission unit 15, a storage unit 16, and a processing unit 17.

  Furthermore, the software that implements the pronunciation rating server device 3 in the present embodiment is the following program. In other words, this program is a result of accepting the voice uttered by the user for the utterance content instructed by the computer and performing the first partial process, which is the first partial process for pronunciation evaluation, on the voice. A receiving unit that receives voice-related data from the terminal device, and a second partial process that is a second partial process for pronunciation evaluation to be performed after the first partial process is performed on the voice-related data, and a rating result is obtained. It is a program for functioning as a rating result acquisition unit to be acquired and a transmission unit for transmitting the rating result to the terminal device.

  In the above program, the rating result acquisition unit performs a second partial process on voice-related data transmitted from a part of two or more terminal devices, and acquires a rating result. It is preferable that the program is a function that functions.

  In the above program, in order to adjust the load of two or more pronunciation rating server devices, an adjustment for determining a pronunciation rating server device that performs second partial processing on each of the two or more audio-related data received by the receiving unit It is preferable that the middleware unit having the means is a program that further functions a computer.

  In the above-mentioned program, the middleware unit sends a rating result obtained by the pronunciation rating server apparatus determined by the adjusting means and a sending means for sending the voice related data to the pronunciation rating server apparatus determined by the adjusting means. It is preferable that the program further includes a receiving unit that receives the rating result received by the receiving unit and that transmits the rating result to the terminal device.

  FIG. 19 shows the external appearance of a computer that executes the program described in this specification and realizes the pronunciation rating server apparatus and the like according to the above-described embodiment. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 19 is an overview diagram of the computer system 300, and FIG. 20 is a block diagram of the computer system 300.

  In FIG. 19, the computer system 300 includes a computer 301 including a CD-ROM drive 3012, a keyboard 302, a mouse 303, a monitor 304, and a microphone 305.

  20, in addition to the CD-ROM drive 3012, a computer 301 is connected to an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012, and a ROM 3015 for storing a program such as a bootup program. A RAM 3016 for temporarily storing application program instructions and providing a temporary storage space, and a hard disk 3017 for storing application programs, system programs, and data are included. Although not shown here, the computer 301 may further include a network card that provides connection to a LAN.

  A program that causes the computer system 300 to execute the functions of the pronunciation rating server device and the like of the above-described embodiment may be stored in the CD-ROM 3101, inserted into the CD-ROM drive 3012, and further transferred to the hard disk 3017. . Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored in the hard disk 3017. The program is loaded into the RAM 3016 at the time of execution. The program may be loaded directly from the CD-ROM 3101 or the network.

  The program does not necessarily include an operating system (OS), a third-party program, or the like that causes the computer 301 to execute functions such as the pronunciation rating server device of the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 300 operates is well known and will not be described in detail.

  In the above program, in the step of transmitting information, the step of receiving information, etc., processing performed by hardware, for example, processing performed by a modem or an interface card in the transmission step (only performed by hardware) Processing) is not included.

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  Further, in each of the above embodiments, it goes without saying that two or more communication units existing in one apparatus may be physically realized by one medium.

  In each of the above embodiments, each process may be realized by centralized processing by a single device, or may be realized by distributed processing by a plurality of devices.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the pronunciation rating server apparatus according to the present invention has an effect of reducing the risk of knowing the pronunciation rating algorithm to the outside, and is useful as a pronunciation rating server apparatus.

1, 2, 3 Pronunciation rating server device A1, B1 Middleware device A2, B2 Pronunciation rating device 4 Terminal device 11, 31 Storage unit 12, 22, 32 Receiving unit 13, 23, 33 Middleware unit 14, 24, 34 Obtaining evaluation results Unit 15, 35 Transmitter 16 Storage unit 17 Processing unit 41 Terminal storage unit 42 Terminal reception unit 43 Terminal first partial processing unit 44 Terminal middleware unit 45 Terminal transmission / reception unit 46 Terminal output unit 331 Data change unit 332 Adjustment unit 333 Transmission unit 334 Reception means

Claims (14)

Receives speech uttered by the user for the instructed utterance content, and receives speech-related data as a result of performing the first partial processing, which is the first partial processing for pronunciation evaluation, on the speech from the terminal device A receiving unit to
A second partial process that is a second partial process for pronunciation evaluation to be performed after the first partial process is performed on the voice-related data, and a rating result acquisition unit that acquires a rating result;
A pronunciation rating server device comprising: a transmission unit that transmits the rating result to the terminal device. An accumulator that accumulates the voice related data, the utterance content data indicating the instructed utterance content or the utterance content identifier for identifying the utterance content, and one or more pronunciation rating information in association with the rating result;
The pronunciation rating server apparatus according to claim 1, further comprising a processing unit that performs processing on the one or more pronunciation rating information. The receiver is
The speech content identifier for identifying the speech content, or the speech content data acquired using the speech content identifier for identifying the speech content is also received,
The storage unit
The pronunciation rating server apparatus according to claim 2, wherein the voice-related data, the utterance content data or the utterance content identifier, and the rating result are stored in association with each other. The data received by the receiving unit, the received data including voice related data is changed to data that the rating result acquisition unit can perform the second partial processing, and the rating result acquired by the rating result acquisition unit, A middleware unit for changing to data that can be transmitted by the transmission unit;
The rating result acquisition unit
The second partial processing is performed on the voice-related data after the middleware part is changed, and the evaluation result is obtained.
The transmitter is
The pronunciation rating server device according to any one of claims 1 to 3, wherein the middleware unit transmits a rating result after the change to the terminal device. The receiver is
Data obtained as a result of processing performed by the terminal middleware unit of the terminal device, and after voice-related data that is a result of the first partial processing performed on the voice, is changed to data that can be transmitted. 5. The pronunciation rating server device according to claim 4, wherein the pronunciation rating server device receives data including voice-related data from the terminal device. The receiver is
Receive voice-related data from two or more terminal devices,
The rating result acquisition unit
The pronunciation according to any one of claims 1 to 3, wherein a second partial process is performed in parallel with respect to each of the two or more audio-related data received by the receiving unit, and two or more evaluation results are acquired. Rating server device. The receiver is
Two or more terminal devices receive different types of audio-related data, which are the results of different first partial processes, from two or more terminal devices,
The rating result acquisition unit
The pronunciation rating server apparatus according to claim 6, wherein a different second partial process is performed on each of the different types of voice-related data to obtain a rating result. A storage unit storing two or more teacher data, each of which is an acoustic model of one or more phonemes;
The rating result acquisition unit
Depending on the user attribute value that is the user attribute value, the second partial process, which is a second partial process for pronunciation evaluation to be performed after the first partial process, is performed using the different teacher data. The pronunciation rating server apparatus according to claim 6, wherein the rating result is obtained by performing the processing for the pronunciation rating. A pronunciation rating server device of two or more pronunciation rating server devices,
The rating result acquisition unit
The pronunciation according to any one of claims 6 to 8, wherein a second partial process is performed on voice-related data transmitted from a part of the two or more terminal devices, and a rating result is obtained. Rating server device. In order to adjust the load of two or more pronunciation rating server devices, an adjusting means is provided for determining a pronunciation rating server device that performs second partial processing on each of the two or more audio-related data received by the receiving unit. The pronunciation rating server device according to claim 9, further comprising a middleware unit. The middleware part is
Sending means for sending the voice related data to the pronunciation rating server device determined by the adjusting means;
Receiving means for receiving a rating result acquired by the pronunciation rating server device determined by the adjusting means;
The transmitter is
The pronunciation rating server device according to claim 10, wherein the rating result received by the receiving unit is transmitted to a terminal device. The first partial process is a process of digitizing the voice and obtaining voice data, or
A process of digitizing the voice, obtaining voice data, and obtaining feature data that is a set of feature vectors from the voice data;
The second partial process acquires feature data from the speech data, calculates a posterior probability that the feature data is a phoneme to be evaluated using the feature data and teacher data, A process for calculating a rating value and obtaining a rating result from one or more rating values, or
Using the feature data and the teacher data, calculate a posterior probability that the feature data is a phoneme to be evaluated, calculate a speech rating value from the posterior probability, and obtain a rating result from one or more rating values The pronunciation rating server device according to any one of claims 1 to 11, which is a process. A pronunciation rating method realized by a receiving unit, a rating result acquiring unit, and a transmitting unit,
Voice-related data as a result of the reception unit receiving voice uttered by the user with respect to the instructed utterance content and performing a first partial process, which is a first partial process for pronunciation evaluation, on the voice Receiving from the terminal device;
A rating result acquisition step in which the rating result acquisition unit performs a second partial process, which is a second partial process for pronunciation rating to be performed after the first partial process, on the voice-related data, and acquires a rating result When,
A pronunciation rating method comprising: a transmitting step in which the transmitting unit transmits the rating result to the terminal device. Computer
Receives speech uttered by the user for the instructed utterance content, and receives speech-related data as a result of performing the first partial processing, which is the first partial processing for pronunciation evaluation, on the speech from the terminal device A receiving unit to
A second partial process that is a second partial process for pronunciation evaluation to be performed after the first partial process is performed on the voice-related data, and a rating result acquisition unit that acquires a rating result;
The program for functioning as a transmission part which transmits the said evaluation result to the said terminal device.