KR20160030168A - Voice recognition method, apparatus, and system - Google Patents

Abstract

The present invention relates to a speech recognition apparatus, system, and method for enhancing speech recognition performance using personal information of a user. A voice recognition system according to an embodiment of the present invention includes a terminal for receiving a voice signal and collecting personal information of a user, a terminal for receiving voice signals and personal information from the terminal, sorting and storing personal information into predetermined categories, And a voice recognition server for performing voice recognition on the basis of voice signals and personal information transmitted from the private server and generating voice recognition results, .

Description

TECHNICAL FIELD [0001] The present invention relates to a voice recognition method, a voice recognition method,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a speech recognition apparatus, a system and a method, and more particularly, to a speech recognition apparatus, system and method for enhancing speech recognition performance using personal information of a user.

Speech recognition technology is one of the key technologies that can facilitate the interaction between the user and the terminal. Through the speech recognition technology, the terminal can listen to the user's voice, understand it, and provide the appropriate service to the user based on the understanding.

In general, speech recognition technology uses a speech recognition model that is constructed by extracting statistical features from a large amount of speech data and language data. The speech recognition apparatus analyzes the user's voice and measures the similarity with the speech recognition model, which is already established, to infer the information included in the user's voice.

However, in spite of recent remarkable developments, speech recognition technology is currently used only in a very limited field compared with its wide possibility. This is due to some limitations of speech recognition technology. The limitation is due to the use of a generalized speech recognition model, not tailored to the individual characteristics of the user in the speech recognition process. In addition, the limitation of the computing ability of the current speech recognition terminal is also a major obstacle.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and it is an object of the present invention to provide a method and apparatus for collecting information of a user and personalizing an acoustic model and a language model used in a speech recognition process And to provide a speech recognition system capable of improving performance.

In addition, the present invention also has an object to provide a voice recognition system that is robust against security and does not leak personal information of the collected user without the consent of the user.

According to an embodiment of the present invention, there is provided a terminal for receiving a voice signal from a user and collecting personal information of the user; A private server for receiving the voice signal and the personal information from the terminal, sorting the personal information into a predetermined category and storing the voice signal, and transmitting the voice signal and at least some of the stored personal information to the voice recognition server; A voice recognition server for performing voice recognition based on the voice signal transmitted from the private server and the personal information, and for generating a voice recognition result; Wherein the personal information transmitted from the private server to the voice recognition server is personal information set by the user to be public and the voice recognition server selects at least one of phonemes, And an environment controller for selecting an acoustic model and a language model to be used by the acoustic model unit and the language model unit in the speech recognition process, wherein the acoustic model unit comprises: A voice recognition system can be provided.

At this time, the environmental controller selects at least one acoustic model and at least one language model by referring to the personal information transmitted to the voice recognition server.

The personal information may include user behavior information collected from a result of measurement of user behavior and user behavior, user-specific personal information, and user status information indicating a user's situation, A user's online record of the user's online activity and internet usage record, user location information indicating the user's actual location, user connection information, which is communication identification information of the user, and user equipment collected during the interaction process between the user and the terminal The user status information includes user identification information, user attribute information indicating a personality, a body, an emotion state, and environment attribute information indicating a characteristic of a surrounding environment in which the user is located.

Also, the terminal directly receives the user state information from the user, or inferences from at least one of the voice signal and the user behavior information.

Alternatively, the private server directly receives the user status information from the user, or inferences the user status information from at least one of the voice signal and the user behavior information.

The speech recognition server derives a plurality of speech recognition results and transmits the same to the private server, and also transmits the acoustic model used in the speech recognition process and the type information of the language model.

At this time, the private server selects at least one of a plurality of voice recognition results transmitted from the voice recognition server, using the disclosed personal information and private personal information.

The private server selects at least one of a plurality of voice recognition results transmitted from the voice recognition server and stores a weight based on the public personal information and the private personal information in each probability value of the plurality of voice recognition results And selects the speech recognition result having the highest probability as a result.

According to another embodiment of the present invention, there is provided a mobile communication terminal comprising: a first terminal for receiving a voice signal from a first user and collecting personal information of the first user; A private server for receiving the voice signal and the personal information from the first terminal, sorting the personal information into a predetermined category and storing the voice signal, and transmitting the voice signal and at least some of the stored personal information to the voice recognition server; A voice recognition server for performing voice recognition based on the voice signal transmitted from the private server and the personal information to generate a voice recognition result and transmitting the voice recognition result to a second terminal; A second terminal for receiving the voice recognition result and outputting the result to a second user; Wherein the personal information transmitted from the private server to the voice recognition server is personal information set by the first user to be set to be public and the voice recognition server transmits at least one of phonemes, A language model unit for forming a character string by referring to a sentence structure of a language, and an environment controller for selecting an acoustic model and a language model to be used in the speech recognition process by the acoustic model unit and the language model unit The speech recognition system can be provided.

At this time, the environmental controller selects at least one acoustic model and at least one language model by referring to the personal information of the first user transmitted to the voice recognition server.

The first terminal directly receives the user state information from the user, or inferences from at least one of the voice signal and the user behavior information.

Alternatively, the private server directly receives the user status information from the user, or inferences the user status information from at least one of the voice signal and the user behavior information.

The speech recognition server derives a plurality of speech recognition results and transmits the same to the private server, and also transmits the acoustic model used in the speech recognition process and the type information of the language model.

At this time, the private server selects at least one of a plurality of voice recognition results transmitted from the voice recognition server, using the disclosed personal information and private personal information.

Alternatively, the private server may select at least one of a plurality of voice recognition results transmitted from the voice recognition server, and add a weight based on the disclosed personal information and the private personal information to each probability value of the plurality of voice recognition results And selects the speech recognition result having the highest probability as a result.

The second terminal outputs the voice recognition result by voice.

At this time, the second terminal receives the personal information of the first user from the private server, and when the second terminal converts the voice recognition result into voice, the second terminal refers to the personal information of the first user to form a voice do.

Alternatively, the second terminal refers to at least one of the characteristics of the speech and environment characteristics separately stored when converting the speech recognition result into speech, thereby forming the speech.

The second terminal may further include a translator for translating a language, and the translator translates the speech recognition result into a language selected by the second user.

According to another embodiment of the present invention, there is provided a terminal for receiving a voice signal from a user, collecting personal information of the user, and transmitting the voice signal and at least some personal information to a voice recognition server. And a voice recognition server for performing voice recognition based on the voice signal transmitted from the terminal and the personal information to generate a voice recognition result; Wherein the personal information transmitted from the terminal to the voice recognition server is personal information set by a user to be open, and the voice recognition server selects at least one of phonemes, syllables and words corresponding to the voice signal An acoustic model unit, a language model unit for forming a character string by referring to a sentence structure of a language, and an environment controller for selecting an acoustic model and a language model to be used in the speech model unit and the language model unit, May be provided.

At this time, the environmental controller selects at least one of an acoustic model and a language model by referring to the personal information transmitted to the voice recognition server.

The terminal receives the user state information directly from the user or infer from at least one of the voice signal and the user behavior information.

Alternatively, the speech recognition server directly receives the user status information from the user, or inferences the user status information from at least one of the voice signal and the user behavior information.

In addition, the speech recognition server derives a plurality of speech recognition results and transmits the same to the terminal, but also transmits the acoustic model used in the speech recognition process and the type information of the language model.

The terminal selects at least one of a plurality of voice recognition results transmitted from the voice recognition server using the disclosed personal information and private personal information.

Alternatively, the terminal may select at least one of a plurality of voice recognition results transmitted from the voice recognition server, and add a weight based on the disclosed personal information and the private personal information to each probability value of the plurality of voice recognition results And selects the speech recognition result having the highest probability as a result.

The speech recognition server derives a plurality of speech recognition results and selects at least one of the plurality of speech recognition results using the disclosed personal information.

Alternatively, the speech recognition server may derive a plurality of speech recognition results, add a weight based on the disclosed personal information to a probability value of the plurality of speech recognition results, Select the result of voice recognition.

According to another embodiment of the present invention, Collecting personal information of the user; Generating a voice recognition result from the voice signal based on the voice signal and the personal information; Selecting a final speech recognition result from the speech recognition result; Wherein the step of generating a voice recognition result from the voice signal based on the voice signal and the personal information comprises the steps of: selecting an acoustic model and a language model with reference to the personal information set by the user; The speech recognition method according to the present invention may further comprise the steps of:

The collecting of the user's personal information may include acquiring the personal information directly input by the user, Deriving user state information from at least one of a voice signal and user behavior information; .

The step of generating a voice recognition result from the voice signal based on the voice signal and the personal information may include generating a plurality of voice recognition results and performing voice recognition on each of the plurality of voice recognition results And generates the acoustic model and the language model type information together.

In addition, in the step of selecting the final speech recognition result, the final speech recognition result is selected using the public personal information and the private personal information.

The step of selecting the final speech recognition result may include adding a weight based on the disclosed personal information and the private personal information to each probability of the plurality of speech recognition results and outputting a result of the speech recognition result having the highest probability value .

According to the present invention, the user's personal information can be collected, and the personalized acoustic model and the language model can be selected by the user using the collected personal information. And, by performing the speech recognition by selecting the personalized acoustic model and the language model, the success rate of speech recognition can be increased.

Also, according to the embodiment of the present invention, the personal information of the user is stored only in private space such as the user's terminal or the private server, and only the public personal information is transmitted to the voice recognition server, Can be strongly protected.

In addition, according to the embodiment of the present invention, the user can transmit the voice recognition result to the other person, and thereby can transmit and receive the voice recognition result in real time to the other person.

In addition, according to the embodiment of the present invention, it is possible to provide a voice recognition system capable of freely arranging each component according to the performance of a terminal, a private server, and a voice recognition server.

1 is a block diagram of a speech recognition apparatus according to an embodiment of the present invention.
2 is a block diagram of a speech recognition system according to an embodiment of the present invention.
3 is a diagram illustrating a speech recognition system according to another embodiment of the present invention.
4 is a diagram illustrating a speech recognition system according to another embodiment of the present invention.
5 is a diagram showing an embodiment of a speech recognition system including a private server.
6 is a diagram showing another embodiment of a speech recognition system including a private server.
7 is a diagram illustrating another embodiment of a speech recognition system including a terminal and a speech recognition server.
8 is a diagram illustrating an embodiment of a speech recognition system for transmitting a speech recognition result of a first user to a second user.
9 is a diagram illustrating a speech recognition method according to an embodiment of the present invention.

Best Mode for Carrying Out the Invention

The present invention relates to a voice recognition device system and method for enhancing voice recognition performance using personal information of a user and protecting personal information of a user, and a preferred embodiment of the present invention will be described in detail with reference to the drawings.

1 is a block diagram of a speech recognition apparatus 100 according to an embodiment of the present invention.

1, the speech recognition apparatus 100 according to the present invention may include an input unit 112, a feature extraction unit 130, an acoustic model unit 152, a language model unit 140, and an output unit 114 have. In Fig. 1, the solid line indicates the flow of the voice signal and the voice recognition result, and the dotted line indicates the flow of additional information required for speech recognition. 1, the acoustic model unit 152 may include a plurality of acoustic models, and the language model unit 154 may include a plurality of language models. The details related to this will be described when the acoustic model unit 152 and the language model unit 154 are described.

Although the speech recognition apparatus 100 is referred to as a " device " for convenience of description, it may exist in the form of software, and may exist in at least one of a form of hardware and a form in which software and hardware are fused have. The voice recognition apparatus 100 may exist in the form of a PC installed in a specific place, or may exist in the form of a portable terminal such as a smart phone, a notebook, and a wearable device.

The input unit 112 is a component for collecting the voice of the user 800 and converting it into an electrical signal. Typically, a microphone or the like may be used, but the present invention is not limited thereto. The input unit 112 can collect video signals as well as voice signals, and can capture the face shape of the user 800 using a video signal input device such as a camera. A video signal input device may be used in the input unit 112 of the speech recognition apparatus 100 according to the present invention to provide a configuration for inferring the sound currently being pronounced from the face or mouth shape of the user 800. [

The feature extraction unit 130 may generate basic information necessary for speech recognition from the collected speech signal. The voice signal collected through the input unit 112 is divided into specific intervals to extract information such as energy distribution for each frequency band of voice. The frequency-band-based information may be vector-quantized, and the vector-quantized information may be used as a voice feature. (LPC) Cepstrum, PLP (Perceptual Linear Prediction) cepstrum, Mel Frequency Cepstral Coefficient (MFCC), and Filter Bank Energy Analysis can be used as a method of extracting characteristics of a speech signal. It does not.

The acoustic modeling unit 152 can determine a basic unit of the language corresponding to the voice feature extracted by the feature extracting unit 130. Here, the basic unit of the language may be phonemes, syllables, words, and the like. For example, the acoustic modeling unit 152 analyzes whether a user pronounces 'dog' in English in correspondence to phonemes / d /, / o /, / g / of the word 'dog' The user's voice signal is recognized as each phoneme.

In the case of a voice signal, even the same word can be expressed in different sounds depending on the person who pronounces and the order in which the word is located in the sentence. Therefore, a large number of utterance data are required to determine which speech features correspond to a basic unit of a language. According to a preferred embodiment of the speech recognition apparatus 100 according to the present invention, the acoustic modeling unit 152 can communicate with the speech database 372 storing the large amount of speech data. In the training phase, the acoustic modeling unit 152 may generate a statistical acoustic model that determines a basic unit of the language corresponding to each speech feature by referring to a large amount of speech data stored in the speech database 372 . The acoustic model unit 152 can select the phoneme having the highest similarity by measuring the similarity between the speech characteristic corresponding to each phoneme in the generated acoustic model and the speech characteristic transmitted from the feature extraction unit 130. The acoustic modeling unit 152 can generate words by combining selected phonemes. The acoustic modeling unit 152 can select at least one result as a result of determining a basic unit of a language such as phonemes, syllables, and words corresponding to the acoustic model. Meanwhile, HMM (Hidden Markov Model) or Neural Network may be used in the process of generating the acoustic model in the acoustic model unit 152, but the present invention is not limited thereto.

The language modeling unit 154 can form a character string by referring to the sentence structure of the language. In all languages, words in a sentence are listed according to certain rules. The language modeling unit 154 refers to the sentence structure of such a language to grasp the sequence of characters and predicts a character that can be placed after the specific character if recognized. Under the assumption that the user 800 has uttered in accordance with the grammar or rules of the language, the language model unit 154 can recognize that characters not conforming to the structure of such a string are misidentified and can be omitted from the candidate character, The success rate of string recognition can be increased.

However, since people often do not utter the correct grammar in everyday life, consideration should be given to this. In addition, sentences with similar meanings can be expressed in different sentences according to the age, sex, and place of residence of the person who uttered them. The language modeling unit 154 may undergo a separate training step in order to correctly recognize these various sentence structures, and through this training step, a statistical language model can be formed. In order to form the language model by the language model unit 154, it is necessary to communicate with the language database 374 storing a vast amount of sentence structure, as in the case of the acoustic model unit 152 described above. The language modeling unit 154 can generate at least one character string as a result of character string recognition. The language modeling unit 154 displays the words included in at least one character string as a result of the character string recognition as a node, connects the characters, and displays a branch indicating the probability of each connection You can create a set of strings with a lattice structure. A plurality of types of characters that can be positioned after one character in a sentence can be selected. The probability of having a combination of the first character and the next character can be determined differently depending on the type of the language model. Even if one language model is used in the speech recognition process, a plurality of character strings can be formed according to the order of each character, and the probability that each character string can be formed can be calculated differently.

The speech recognition apparatus 100 can refer to a plurality of acoustic models in a speech recognition process, and can refer to a plurality of language models, thereby generating a plurality of speech recognition results. The speech recognition apparatus 100 may select a character string having the highest probability as the final speech recognition result and transmit the selected character string to the output unit 114 by referring to the probability included in each character string. Even when a single acoustic model and a language model are used, a plurality of strings can be generated. Even at this time, the speech recognition server 300 can select a string having the highest probability as the final speech recognition result.

The output unit 114 outputs the speech recognition result of the speech recognition apparatus 100. Here, the speech recognition result may include at least one of a basic unit of the language recognized by the acoustic model unit 152 and a character string recognized by the language model unit 154. The result of the speech recognition may be formed in the form of a plurality of strings, or may be formed of a set of strings of the above-described lattice type.

The input unit 112 and the output unit 114 are included in the voice recognition apparatus 100 and the input unit 112 and the output unit 114 are not included in the voice recognition apparatus 100, But may be provided as a separate component. For example, when the speech recognition apparatus 100 is provided as a smartphone, a user can attach a high-performance microphone to the smartphone and collect voice for correct speech recognition. In addition, a user can output a voice recognition result in various ways by connecting a large-sized monitor, a beam projector, or a multi-channel speaker to the smartphone.

The sound database 372 and the language database 374 described above may be provided separately from the speech recognition apparatus 100 as shown in FIG. 1, but the present invention is not limited thereto. In particular, if the computing capability of the speech recognition apparatus 100 and the size of the information store are sufficient, the two databases 372 and 374 may be included in the speech recognition apparatus 100.

2 is a diagram illustrating a speech recognition system 1000A according to an embodiment of the present invention.

Referring to FIG. 2, the speech recognition system 1000A according to the embodiment of the present invention may include a terminal 200 and a speech recognition server 300. FIG. The terminal 200 may include an input unit 212, a feature extraction unit 230, and an output unit 214. The speech recognition server 300 may include an acoustic modeling unit 352, a language modeling unit 354, an acoustic database 372, and a language database 374.

The feature extraction unit 230 may be included in the speech recognition server 300 and the sound database 372 and the language database 374 of the speech recognition server 300 may be included in the speech recognition server 300 300) may be present.

The input unit 212, the feature extraction unit 230, the output unit 214, the acoustic model unit 352, the language model unit 354, the acoustic database 372, and the language database 374 shown in FIG. Is a component common to the input unit 112, the feature extraction unit 130, the output unit 114, the acoustic model unit 152, the language model unit 154, the acoustic database 372, and the language database 374, A detailed description thereof will be omitted.

The advantages of the voice recognition system 1000A including the terminal 200 and the voice recognition server 300 of FIG. 2 are as follows. First, since the terminal 200 performs only minimum input / output and basic voice signal processing for voice recognition, the terminal 200 having a relatively low computing power is less burdened. Instead, most operations are performed at a high speed in the speech recognition server 300, which has relatively high processing capability and storage capability, and the terminal 200 only needs to receive the result. Since the wired / wireless communication environment such as the Internet has achieved many improvements in comparison with the past, communication between the terminal 200 and the voice recognition server 300 can be freely performed. In addition, terminals 200 having different computing capabilities have been developed and distributed on the market. It may be inefficient to provide a different voice recognition process for each terminal 200. As shown in FIG. 2, since the voice recognition server 300 performs most operations, it is possible to implement a system independent of the terminal 200, which is independent of the type of the terminal 200. Of course, the entire voice recognition system 1000A may also be implemented by a method of freely distributing voice recognition processing steps of the terminal 200 and the voice recognition server 300 with reference to the processing capability of the specific terminal 200. [

Meanwhile, a plurality of speech recognition servers 300 may be provided, and a plurality of speech recognition servers 300 may perform Cloud based Distributed Speech Recognition (DSR). Distributed speech recognition refers to a technology in which the characteristics of a speech signal are converted into digital data and transmitted and a speech recognition server distributes the speech signal in order to improve speech recognition performance in a wireless communication environment. According to the distributed speech recognition, the processing speed of the speech recognition operation and the use efficiency of the memory can be maximized.

2, the terminal 200 receives the voice recognition result from the voice recognition server 300 and outputs the voice recognition result through the output unit. However, the terminal 200 is not limited to the terminal 200 of FIG. 2, The voice recognition result may be transmitted to the mobile terminal.

FIG. 3 is a diagram illustrating a speech recognition system 1000B according to another embodiment of the present invention.

Referring to FIG. 3, the speech recognition system 1000B according to the embodiment of the present invention may include a terminal 200 and a speech recognition server 300. FIG. The terminal 200 may include an input unit 212, a feature extraction unit 230, and an output unit 214. The speech recognition server 300 may include an acoustic model unit 352, a language model unit 354, an acoustic database 372, a language database 374, and a configuration controller 380.

The feature extraction unit 130 may be included in the speech recognition server 300 and the sound database 372 and the language database 374 may be present outside the speech recognition server 300 depending on the computing capability of the terminal 200. [ It is also possible.

The detailed description of common elements in FIG. 3 and FIG. 2 is redundant and will be omitted.

The environment setting controller 380 selects an acoustic model and a language model to be used by the acoustic model unit 352 and the language model unit 354 in the speech recognition process. The acoustic model unit 352 and the language model unit 354 can refer to a plurality of acoustic models and language models in the speech recognition process. Depending on the speaker's age, sex, and the use of dialects, the voice characteristics of the acoustic model may be different from each other, and the voice characteristics may be changed depending on background noise and reverberation at the place where the voice is uttered. Since the order relation of the words in the sentence and the words used can vary depending on the age, sex, and use of the dialect of the speaker, the language model can be formed in various ways according to the characteristics of the speaker used in the training step. The speech recognition server 300 can increase the success rate of speech recognition by using a plurality of acoustic models and language models of various types in the speech recognition process.

In the case of performing distributed speech recognition in the form of FIG. 3, a speech recognition process using different acoustic models and speech models is performed for each speech recognition server 300, and speech recognition results generated through various models are combined To the terminal 200 again. Alternatively, each speech recognition server 300 uses the same acoustic model and language model, but the processing speed of speech recognition can be increased by parallelly calculating each speech recognition process.

4 is a diagram illustrating a speech recognition system 1000C according to another embodiment of the present invention.

Referring to FIG. 4, the speech recognition system 1000C according to the embodiment of the present invention may include a terminal 200 and a speech recognition server 300. FIG. The terminal 200 may include an input unit 212, a personal information collection unit 220, a feature extraction unit 230, a personal information analysis unit 240, and an output unit 214. The speech recognition server 300 may include an acoustic model unit 352, a language model unit 354, an acoustic database 372, a language database 374, and a configuration controller 380.

At least one of the feature extraction unit 230 and the personal information analysis unit 240 may be included in the speech recognition server 300 and the sound database 372 of the speech recognition server 300 may be included in the speech recognition server 300, And the language database 374 may exist outside the voice recognition server 300. [

The detailed description of the components common to FIG. 4 and FIG. 3 will be omitted because they are redundant.

Meanwhile, in the present invention, the personal information may include user activity information collected from a result of measuring user activity and recording of user activity. In addition, the personal information may include user-specific personal information and user status information indicating the user's situation.

The user behavior information may include user online history, user location information, user connection information, and user device utilization information.

The user online record is information that collects online activity of the user 800 and internet use record. The user's online record may include a post such as text, photograph, music, video created by the user 800 on the Social Network Service (SNS), an emotion icon displayed by the user on the SNS, or a good-bad, A simple doctrine of behavior, a list of online neighbors and Internet browser search and visit history, a list of favorite sites, and the like.

The user location information is information indicating the actual location of the user 800. The user location information includes location information displayed by the user 800 using his or her location information obtained by using a positioning system such as GPS, a smartphone application providing location-based services, etc., And the like.

The user connection information is communication identification information of the user 800 and may include a telephone number of the user 800, an e-mail address, real address information, and the like.

User device utilization information refers to information collected during an interaction process between the user 800 and the terminal 200. The user device utilization information includes information such as the type of the device used by the user 800, the usage time and frequency of each device, the type of application that the user 800 executes through the PC or smart phone, A list of installed applications, and a list of applications downloaded from online.

On the other hand, the user status information may include user attribute information and environment attribute information.

The user attribute information is information indicating the user's personal information, personality, body, and emotional state, and is information indicating the user's age, gender, ethnic group, dialect used, occupation, income, education degree, health status, . ≪ / RTI >

The environmental attribute information is information indicating the characteristics of the surrounding environment in which the user is located, and may include background noise, degree of reverberation, and season, time, weather, and climate information, which are acoustic characteristics of the space in which the user is located.

The personal information of the user 800 listed above may be collected by the personal information collection unit 220. The personal information collection unit 220 may automatically collect personal information of the user 800 when the user 800 operates the terminal 200 and may store at least one of the user behavior information and the user status information, (800). The personal information collecting unit 220 includes the user 800 in the terminal 200 or the external terminal and the server in which the user 800 has agreed to collect authentication or private information for performing voice recognition, Of personal information.

The personal information analyzing unit 240 analyzes the collected personal information. In particular, the personal information analyzer 240 may infer user state information from at least one of the personal behavior information and the voice signal collected through the input unit 212. [ For example, the personal information analyzing unit 240 may determine whether the user 800 is male or female by grasping a frequency band in which energy is mainly distributed in the voice signal. In addition, the personal information analyzer 240 may analyze the waveform of the vowel portion of the voice signal to grasp the vocal state of the user 800, and may infer the age and health state of the user 800 through the vocal pattern analysis. On the other hand, if the user 800 frequently searches for information such as cosmetics discount information, a clothing site, a luxury goods photograph, a entertainment issue, an Internet childcare cafe, and the like via the terminal 200, (800) is highly likely to be female. Meanwhile, the personal information analyzer 240 can infer the background noise level and the degree of reverberation of the concert hall when the current position of the user 800, which is determined through GPS or the like, is inside the concert hall.

That is, the personal status information may be inferred from the personal information analysis unit 240. However, as described above, the personal status information may be input directly through the personal information collection unit 220. [

The personal information analyzer 240 may calculate the probability value of each item of the user attribute information and the environment attribute information. For example, the personal information analyzing unit 240 may analyze the energy distribution of the voice signal of the user 800 according to frequencies to determine that the user 800 has a male probability of 80%. As another example, if the user 800 directly inputs his or her age as 75 years old through the personal information collection unit 220, the personal information analysis unit 240 may determine that the user 800 is 100% . As another example, the personal information analyzing unit 240 may refer to the Internet search record of the user 800 so that the probability that the user 800 is 70% female, 90% student, 60%.

Meanwhile, the personal information analyzing unit 240 can continuously perform personal information analysis work. Since the personal information of the user is continuously collected by the personal information collection unit 220, the amount of personal information can be increased in proportion to the time. The personal information analyzing unit 240 can perform the personal information analyzing operation again every time the amount of personal information is changed. Alternatively, the personal information analyzing unit 240 may analyze the personal information again at predetermined intervals according to a preset method. The personal information analyzer 240 can more accurately deduce the individual status information as the amount of personal information increases and the variety thereof increases. Accordingly, the personal information analyzer 240 can increase the accuracy of the probability value of each item of the personal information.

In addition, the personal information analyzer 240 can infer the behavior pattern of the user 800 from the collected personal information. For example, it can be assumed that the user 800 is a student who returns home and school at a predetermined time. The personal information analyzing unit 240 can refer to the time information and the space information that the user 800 can be located by time zone by referring to the place information transmitted from the personal information collecting unit 220 such as GPS. In this case, the personal information analyzing unit 240 may collect or infer the environmental attribute information of 'school' during the time that the user 800 spends time in 'school' for a specific time.

The personal information analyzing unit 240 may analyze the personal information of the user 800 already collected or collected through the Big Data technique, but is not limited thereto.

Referring to FIG. 4, the personal information collection unit 220 and the personal information analysis unit 240 are included in the terminal 200. 4, the terminal 200 may directly receive the personal information of the user 800 through the personal information collecting unit 220 and the personal information analyzing unit 240, Information can be deduced. However, the speech recognition system 1000C according to the present invention is not limited to the configuration shown in FIG. 4, and the personal-agent analysis unit 240 may be included in the speech recognition server 300. FIG. The voice recognition server 300 is superior to the terminal 200 in terms of processing capacity and storage capacity so that the voice recognition server 300 can smoothly communicate with the voice recognition server 300 when the personal information analysis unit 240 is included in the voice recognition server 300. [ So that personal information can be deduced.

Meanwhile, the user 800 can sort the collected personal information according to a predetermined category. The classification of the personal information may be performed on all devices in which the personal information of the user 800 is stored. Preferably, the user 800 may classify the personal information into public or private personal information according to whether the personal information is disclosed or not.

4, personal information inferred through the personal information analysis unit 240 or input through the personal government collection unit 220 may be transmitted to the environment setting controller 380 of the voice recognition server 300. [ The personal information transmitted at this time can be constituted by only allowing the user to disclose it. The probability of each item of the personal information identified through the personal information analyzing unit 240 may also be transmitted to the voice recognition server 300. [

The configuration controller 380 can select at least one of an acoustic model and a language model by referring to the transmitted personal information. Also, the configuration controller 380 can select at least one acoustic model and at least one language model by referring to the transmitted personal information. For example, when the configuration controller 380 receives public personal information called 'child', it may select an acoustic model associated with the 'child'. Also, the environment setting controller 380 can select a language model associated with 'child'. If the acoustic model unit 354 and the language model unit 354 both have an acoustic model and a language model associated with 'child' Both of the above models may be selected.

The environment setting controller 380 may use the pattern information of the user inferred from the personal information analyzing unit 240. [ As described above, if the user 800 is a student, the personal information analyzer 240 can infer that the user 800 goes to the 'school' at a specific time and sends a routine there. The environment setting controller 380 can select an acoustic model and a language model corresponding to 'school' when recognizing the voice of the user 800 during the specific time period with reference to the pattern information and the time information.

On the other hand, if the user 800 does not input any personal information at all, or if there is little or no analyzed or inferred personal information, the configuration controller 380 may select all available acoustic and language models. Even when there is no acoustic model and language model associated with the received user's personal information, the configuration controller 380 can select all available acoustic and language models. When there is no acoustic model and language model directly associated with the received personal information of the user, the configuration controller 380 can select an acoustic model and a language model that are similar to the received personal information. For example, if the personal information received by the speech recognition server 300 includes only the 'student' but the language model unit 354 does not have a language model corresponding to 'student', the environment setting controller 380 performs the speech recognition The server 300 may select the 'youth' language model held by the server 300.

The environment setting controller 380 can select an acoustic model and a language model that match the personal information in this way, so that the acoustic model and the language model can be personalized to the voice of the user 800. [ In addition, by using the personalized acoustic model and the language model in the speech recognition process, the accuracy of speech recognition system 1000C can be improved more than when performing speech recognition.

The speech recognition server 300 may refer to a plurality of acoustic models and language models in performing speech recognition. In this case, the speech recognition server 300 refers to the probability value of each item transmitted from the personal information analysis unit 240, and calculates a weight based on the probability value Can be applied to each string. The speech recognition server 300 may select a character string having the highest probability value among the probabilities to which the weight is applied as the final speech recognition result.

Meanwhile, when the speech recognition system 1000C is configured with the structure shown in FIG. 4, the speech recognition system 1000C can be advantageously used in a training step for forming an acoustic model and a language model. When a large number of randomly recorded voice signals are input to the terminal 200 through the input unit 212, the characteristics of each voice signal are extracted and transmitted to the voice recognition server 300, and the analyzed personal information is also transmitted . The speech recognition server 300 can form various acoustic models and language models by performing the training step with reference to the transmitted voice features and personal information. In particular, only voice signals corresponding to a specific personal information item are selected and used in the training step, so that an acoustic model and a language model specific to a specific item can be formed. For example, when the voice recognition system 1000C can select the voice of the elderly male separately through the personal information analysis unit 240, only the voice of the elderly male is selected and used, And can be used for analyzing voice signals classified as 'old man' or 'male' at a later speech recognition step.

In the speech recognition system 1000C shown in FIG. 4, a plurality of speech recognition servers 300 may be provided, and a distributed speech recognition process may be performed.

5 is a diagram showing an embodiment of a speech recognition system 1000D including a private server 400. In FIG.

5, a speech recognition system 1000D according to an embodiment of the present invention may include a terminal 200, a speech recognition server 300, and a private server 400. [ The terminal 200 may include an input unit 212, a personal information collection unit 220, a personal information analysis unit 240, and an output unit 214. The speech recognition server 300 may include an acoustic model unit 352, a language model unit 354, an acoustic database 372, a language database 374, and a configuration controller 380. The private server 400 may include a feature extraction unit 430 and a personal information storage unit 460.

The feature extraction unit 430 included in the private server 400 may be the same as the feature extraction unit 230 included in the terminal 200 of FIG.

The feature extraction unit 430 may include the terminal according to the computing capability of the terminal 200 and at least one of the personal information collection unit 220 and the personal information analysis unit 240 may be included in the private server 400 have. The feature extraction unit 430 may be included in the speech recognition server 300. [ The acoustic database 372 and the language database 374 of the speech recognition server 300 may exist outside the speech recognition server 300. [

The detailed description of the components common to FIG. 5 and FIG. 4 will be omitted because they are redundant.

The private server 400 receives the voice signal and the personal information from the terminal 200, classifies the personal information into a predetermined category, and stores the classified personal information. Also, the private server 400 may transmit the voice signal and at least some of the stored personal information to the voice recognition server 300.

The feature extraction unit 430 of the private server 400 may extract the feature from the voice signal transmitted from the terminal 200 and transmit the feature to the voice recognition server 300. [ When the private server 400 transmits the voice characteristic to the voice recognition server 300, the voice characteristic can be encrypted and transmitted. When the feature extraction unit 430 is included in the voice recognition server 300, the private server 400 may transmit the encrypted voice signal to the voice recognition server 200. [ As such, the private server 400 can encrypt a voice feature or a voice signal, thereby preventing leakage of personal information that can be inferred from unencrypted voice.

The personal information storage unit 460 of the private server 400 stores the personal information transmitted from the terminal 200. [ The personal information storage unit 460 may store personal information directly input by the user 800, personal information deduced from the voice signal of the user 800, and other personal information derived from the personal information of the user 800. Preferably, the personal information may be transmitted from the personal information analyzer 240. 4, the personal information analyzer 240 may calculate a probability value for each item of personal information, and the probability value may also be stored in the personal information storage unit 460. FIG.

As described in FIG. 4, the user 800 can classify the personal information into predetermined categories, and the classification process can be performed in the private server 400. The user 800 can classify the personal information stored in the terminal 200 and the private server 400 according to an arbitrary category or predetermined category of the user 800 through the operation of the terminal 200, And may be stored in the personal information storage unit 460 of the private server 400. The private server 400 may classify and store the personal information of the user 800 into publicly available private information and non-public personal information, but the present invention is not limited thereto.

As described above, the private server 400 may store the personal information of the user 800 and may perform various encryption techniques for personal information security. The private server 400 contracts with the user 800 or stores the user's personal information under user authentication and personal information collection agreement. The private server 400 is provided separately from the voice recognition server 300 and can prevent user information from being leaked in the voice recognition server 300 that processes a large amount of voice recognition in the open area. Personal information of the user 800 is freely transmitted and received between the terminal 200 and the private server 400. However, due to the security of the private server 400, the private information does not flow out of the private server 400 to the subsequent network connection. In particular, the private server 400 can prevent private information, which the user does not want to disclose, from flowing out by transmitting only the personal information set by the user 800 to the voice recognition server 300.

As shown in FIG. 5, the following advantages are obtained when the private server 400 exists between the terminal 200 and the voice recognition server 300. For example, it may be assumed that the user 800 desires to receive the voice recognition service, but worries about leakage of personal information due to voice recognition, such as voice signal leakage. The user 800 can transmit private information and voice signals to the highly reliable private server 400 and the private server 400 can transmit the encrypted voice signals and the encrypted voice characteristics to at least one of the private server 400 and the private server 400, Only one piece of personal information can be transmitted to the server 200 providing the voice recognition service. The voice recognition service provider 200 can only extract the character string by using the voice signal and the personal information permitted to be disclosed and can not determine who is actually speaking or who has the characteristics of the voice signal, Can be protected. In addition, since the intermediate stage of the private server 400 is further developed between the terminal 200 and the voice recognition server 300, the voice recognition process is divided and arranged among the respective components, The load applied to the side of the main body 300 can be reduced.

In the speech recognition system 1000D shown in FIG. 5, a plurality of speech recognition servers 300 may be provided to perform distributed speech recognition processing.

Meanwhile, the voice recognition server 300 may select an acoustic model and a language model corresponding to the personal information by referring to the personal information transmitted from the private server 400, thereby increasing the voice recognition success rate. 4, when a plurality of sound models and language models are used in the speech recognition process and a plurality of strings are generated as a result of speech recognition, the speech recognition server 300 acquires a plurality of character strings from the personal information analysis unit 240 We can refer to the probability of each transmitted property and apply a weight based on it to each string. The speech recognition server 300 may select a character string having the highest probability value among the probabilities to which the weight is applied as the final speech recognition result.

5, the voice recognition result generated by the voice recognition server 300 is transmitted to the terminal 200 through the private server 400. However, the voice recognition result is not limited to the voice recognition server 300, Or may be transmitted directly from the terminal 300 to the terminal 200. Also, the voice recognition server 300 may transmit the voice recognition result to the devices other than the terminal 200 and the private server 400 described above.

6 is a diagram illustrating another embodiment 1000E of a speech recognition system including a private server.

Referring to FIG. 6, a speech recognition system 1000E according to an embodiment of the present invention may include a terminal 200, a speech recognition server 300, and a private server 400. FIG. The terminal 200 may include an input unit 212, a personal information collection unit 220, a personal information analysis unit 240, and an output unit 214. The speech recognition server 300 may include an acoustic model unit 352, a language model unit 354, an acoustic database 372, a language database 374, and a configuration controller 380. The private server 400 may include a feature extraction unit 430, a personal information storage unit 460, and an output re-

The feature extraction unit 430 may include the terminal according to the computing capability of the terminal 200 and at least one of the personal information collection unit 220 and the personal information analysis unit 240 may be included in the private server 400 have. The feature extraction unit 430 may be included in the speech recognition server 300. [ The result re-encoding unit 490 may distribute the computation burden to the terminal 200, the voice recognition server 300, and the private server 400 of the voice recognition system 1000E, Or may be included in at least one of the recognition servers 300. [ The acoustic database 372 and the language database 374 of the speech recognition server 300 may exist outside the speech recognition server 300. [

The detailed description of the components common to FIG. 6 and FIG. 5 will be omitted because they are redundant.

The speech recognition server 300 may generate a plurality of speech recognition results. Here, the plurality of speech recognition results may include a set of strings of the lattice structure generated by the language modeling unit 354. [ The voice recognition server 300 may transmit the plurality of voice recognition results to the private server 400. [

At this time, the speech recognition server 300 may transmit the acoustic model and the type information of the language model used in the speech recognition process, and the acoustic model and the type information of the language model may be included in each speech recognition result. The type information may be used to identify which acoustic model and language model each speech recognition result is derived from. The type information may be transmitted from the configuration controller 380 to the private server 400. The acoustic model unit 352, the language model unit 354, and the voice recognition server 300 ≪ / RTI >

The result reproduction unit 490 can select an optimum speech recognition result from the speech recognition result transmitted from the speech recognition server 300. [ When the speech recognition server 300 transmits a plurality of speech recognition results and acoustic model and language model type information used for each speech recognition result together, the result reproduction unit 490 outputs the acoustic model and the language model type information It is possible to select an optimal speech recognition result. At this time, the result reproduction unit 490 can refer to the personal information of the user 800 stored in the personal information storage unit 460. The referenced personal information may include both the personal information that the user 800 has set as public and the personal information that has not been set as public. A specific example in which the result reproduction unit 490 selects the optimum speech recognition result is as follows.

First, the user 800 uses English, the private server 400 has 'male', which is the personal information that is disclosed, and the private information 'old man' and 'Texas Texas dialect' And the like. The private server 400 may transmit the 'personal information' to the voice recognition server 300. The configuration controller 380 of the speech recognition server 300 can perform speech recognition using a generalized acoustic model and a language model. However, the environment setting controller 380 can select an acoustic model and a language model corresponding to personal information 'male' for more accurate voice recognition. The environment setting controller 380 can also select acoustic models and language models, such as 'New Jersey dialect' and 'Boston dialect', formed from local dialect data, and can also select acoustic models and language models for various age groups . If the speech recognition server 300 has a language model corresponding to the 'Texas dialect' but does not have an acoustic model associated therewith, the configuration controller 380 can select only the 'Texas dialect' language model. Although the speech recognition server 300 does not have an acoustic model corresponding to the 'Texan dialect', it is possible to use 'New Mexico', 'Oklahoma', 'Arkansas' 'And' Louisiana 'areas, the configuration controller 380 may select the acoustic model of the local dialect. The environment setting controller 380 can select all kinds of acoustic models and language models possessed by the speech recognition server 300. [ Even if the speech recognition server 300 does not have an acoustic model and a language model that match the disclosed personal information, the environment setting controller 380 can also store all types of acoustic models and languages You can choose a model. In the speech recognition process, acoustic and language models of 'male', 'elderly', 'children', 'youth', 'Texan dialect', 'New Jersey dialect', 'Boston dialect' were used, When the result is generated, the voice recognition server 300 transmits the voice recognition results and the respective type information to the private server 400. The result reproduction unit 490 of the private server 400 may store at least one of the voice recognition result corresponding to the 'personal information', which is the public personal information, and the voice recognition result corresponding to the private information 'senior citizen' and ' Can be selected as the final speech recognition result, and all of the three speech recognition results can be selected. The result reproduction unit 490 may select a character string having the highest probability among all speech recognition results as the final speech recognition result.

According to FIG. 5, the probability value of each item of the personal information identified through the personal information analyzing unit 240 can be stored in the personal information storage unit 460. FIG. 6, likelihood values of individual items of personal information may be stored in the personal information storage unit 460. [ Of course, various kinds of information generated by the personal information analyzing unit 240 may be transmitted to the result re-generating unit 490 without going through the personal information storing unit 460. [

In a case where a plurality of acoustic models and language models are used in the speech recognition process and a plurality of strings are generated as a result of speech recognition, the result reproduction unit 490 may apply a weight based on the probability values to each string, have. The result reproduction unit 490 may select a character string having the highest probability value among the probability values to which the weight is applied as the final speech recognition result.

In the immediately preceding example, the result reproduction unit 490 may form the weight 1, weight 2, and weight 3 values based on the probabilities of each of the male, the elderly, and the tex1 dialect, To the resultant product. The result reproduction unit 490 can obtain the final probability by multiplying each probability value of word strings formed through the acoustic model and the language model corresponding to 'male' by the weight value 1, which is equivalent to other results do. However, a method of obtaining the final probability can be variously arranged, and is not limited to the above multiplication method. The result reproduction unit 490 can select the word sequence having the highest final probability among all calculation result values.

When the user 800 receives a plurality of final speech recognition results, he / she checks the contents of the speech recognition result, selects an outcome corresponding to the intention of the user 800, and transmits the selected result to the speech recognition system 1000E . Alternatively, the user 800 may evaluate the accuracy of all final speech recognition results and send the accuracy evaluation information of each result to the speech recognition system 1000E. Here, the speech recognition system 1000E may generate the user feedback information including the final speech recognition result selection information and the accuracy evaluation information of the user 800. [ The speech recognition system 1000E may assign an accuracy weight to the acoustic model and the language model based on the user feedback information, and the accuracy weight information may be used in a subsequent speech recognition process to improve the accuracy of speech recognition. For example, the speech recognition system 1000E may add the accuracy weights to the probability of each word string formed by the speech recognition result, and may select the word string having the highest probability as the final speech recognition result.

7 is a diagram illustrating another embodiment 1000F of a speech recognition system including a terminal and a speech recognition server.

Referring to FIG. 7, the speech recognition system 1000F according to the embodiment of the present invention may include a terminal 200 and a speech recognition server 300. FIG. The terminal 200 includes an input unit 212, a personal information collection unit 220, a feature extraction unit 230, a personal information analysis unit 240, a personal information storage unit 260, an output re- (214). The speech recognition server 300 may include an acoustic model unit 352, a language model unit 354, an acoustic database 372, a language database 374, and a configuration controller 380. The speech recognition server 300 may include at least one speech signal analyzer 250 including an acoustic model unit 352 and a language model unit 354. [

The feature extraction unit 230, the personal information storage unit 260 and the result reproduction unit 290 included in the terminal 200 of FIG. 7 include the feature extraction unit 430 included in the private server 400 of FIG. 6, The personal information storage unit 460 and the output re-output unit 490 may be provided.

At least one of the personal information collection unit 220, the personal information analysis unit 240, the result reproduction unit 290, the personal information storage unit 260, and the feature extraction unit 230 may be May be included in the voice recognition server 300. The acoustic database 372 and the language database 374 of the speech recognition server 300 may exist outside the speech recognition server 300. [

The detailed description of the components common to FIG. 7 and FIG. 6 will be omitted because they are redundant.

The terminal 200 of FIG. 7 includes a feature extraction unit 430, a personal information storage unit 460 and an output re-emission unit 490 included in the private server 400 of FIG. And performs processing for personal information analysis and final speech recognition result selection. In particular, the terminal 200 shown in FIG. 7 is a configuration suitable for a high-performance smartphone which has recently been distributed in the market, and is a system structure that can be applied to a personal computer used in the home.

In the speech recognition system 1000F of FIG. 7, the terminal 200 exchanges voice signals, public personal information, and voice recognition results with the voice recognition server 300 only. In addition, the speech recognition system 1000F of FIG. 7 has a simple structure that does not require a separate speech recognition step except for the terminal 200 and the speech recognition server 300. This simplicity is an advantage of the structure of the system structure . In terms of security, the user only needs to manage personal information stored in each terminal 200, and does not have to take extra security. The voice recognition system 1000 of FIG. 7 is a combination of the terminal 200 and the private server 400 of FIG. 6, and has a great advantage in the process of processing personal information. 6, since the private information is separately stored in the private server 400, an additional step may be required to access the private server 400 when it is necessary to delete or modify personal information at the request of the user 800 have. However, in the case of FIG. 7, the user 800 can easily manage personal information through the terminal 200 directly. In addition, the user 800 can easily select an output according to his or her preference during the process of re-arising the result.

FIG. 8 is a diagram illustrating an embodiment of a speech recognition system 1000G for transmitting a speech recognition result of a first user 800a to a second user 800b.

Referring to FIG. 8, a speech recognition system 1000G according to an embodiment of the present invention may include a first terminal 500, a speech recognition server 300, a private server 400, and a second terminal 600 . The first terminal 500 may include an input unit 512, a personal information collecting unit 520 and a personal information analyzing unit 540. The second terminal 600 may include a signal receiving unit 610, a translator 620 An output signal selection unit 640, a voice signal conversion unit 650, an attribute storage unit 652, and an output unit 614, as shown in FIG.

The voice recognition server 300 and the private server 400 of FIG. 8 are the same as the voice recognition server 300 and the private server 400 of FIG. 6, and the output unit 614 of the second terminal 600 includes And may be provided in the same manner as the output unit 214 included in the terminal 200 of FIG. The first terminal 500 includes a separate output unit as in the terminal 200 of FIG. 6, so that the first user 800a can check the voice recognition result of the first user 800a.

The detailed description of common elements in FIGS. 8 and 6 will be omitted since they are redundant.

The signal receiving unit 610 receives the voice recognition result of the first user 800a. The signal receiving unit 610 can receive a plurality of speech recognition results. In addition, the signal receiving unit 610 can receive the personal information of the first user 800a and the voice characteristics of the first user 800a. At this time, the signal receiving unit 610 can receive only the disclosed personal information of the first user 800a, which is disclosed from the private server 400. [ Here, the voice feature of the first user 800a may be extracted from the voice signal of the first user 800a in the feature extraction unit 430 and stored in the private server 400. [ The personal information of the first user 800a may be used to output the voice in the second terminal 600. [

The translating unit 620 translates the voice recognition result of the first user 800a to the language of the second user 800b when the language of the first user 800a is different from the language of the second user 800b Translate. At this time, the translation unit 620 can generate a plurality of translation results for a plurality of speech recognition results. In addition, the translating unit 620 may perform a normalization process on the speech recognition result. Here, the normalization process is a process of transforming the result of speech recognition into a character string conforming to a standard phrase. However, the present invention is not limited to this, and the normalization process may be performed by other components of the speech recognition system 1000G.

The result selection unit 630 selects at least one of a plurality of speech recognition results and a plurality of translation results. At this time, the result selecting unit 630 may select an output according to the selection input of the second user 800b. In addition, the result selecting unit 630 can refer to the personal information of the second user 800b and select an output corresponding to the personal information. The result selecting unit 630 may be combined with the translating unit 620 and exist as one component.

The output signal selector 640 determines an output format for outputting the selected output. The output signal selection unit 640 may receive an output format specification input such as 'video output' and 'audio output' from the second user 800b. The output signal selector 640 may determine the output format according to the configuration of the second terminal 400. [ For example, if the second terminal 400 does not have a separate video output unit and only audio output means such as a speaker is provided, the output signal selector 640 selects the audio output format.

The output signal selector 640 may determine that the selected output is to be output by voice. In this case, the output signal selector 640 may transmit the selected output to the voice signal converter 650. The voice signal converting unit 650 converts the selected result into a voice signal. At this time, the voice signal converting unit 650 can generate an output voice by referring to the personal information and the voice feature information of the first user 800a received by the signal receiving unit 610. That is, if the first user 800a is a Korean woman and the second user 800b is a United States person, the second user 800b can listen to what the first user 800a has spoken in Korean in English, At this time, this means that the voice can have the Korean female voice characteristic unique to the first user 800a.

On the other hand, the attribute storage unit 652 may store voice characteristics and environment characteristics. Here, the voice feature may include the voice feature of the celebrity, and the environmental feature may include the reverberation feature and spatial information of the various spaces.

The voice signal converting unit 650 may generate voice by referring to voice characteristics and environment characteristics stored in the attribute storage unit 652. [ For example, the voice signal converter 650 may output the selected result to the voice of a famous entertainer by referring to the information stored in the attribute storage 652, and may generate a voice having a characteristic of a concert hall having a strong resonance It is possible.

The speech recognition system 1000G having the configuration as shown in FIG. 8 can be utilized as a real-time interpretation system. That is, the first user 800a and the second user 800b can freely speak each other in their own languages, and can listen to each other's contents in their own languages. Although FIG. 8 shows only the structure in which the voice recognition result is transmitted from the first user 800a to the second user 800b, the present invention is not limited thereto, and a bi-directional system in which two users can simultaneously speak and listen is also possible. That is, the first user 800a and the second user 800b each have a private server 400 to be used by them, and correspond to the first terminal 500 corresponding to the input means of the voice recognition system and the output means Directional interpretation system can be implemented if each of the second terminals 600 includes a single terminal.

Similar to FIG. 8, a real-time interpretation system without a private server 400 may be configured by including a second terminal in the configuration of FIG.

9 is a diagram illustrating a speech recognition method according to an embodiment of the present invention.

9, the speech recognition method according to the present invention includes a step S100 of receiving a voice signal from a user, a step S200 of collecting personal information of a user, a step of acquiring voice information (S400) of selecting a final speech recognition result from the speech recognition result, and outputting the final speech recognition result (S500).

The step of receiving a voice signal from a user (S100) is a step of receiving a voice signal, which is basic information for voice recognition, through a microphone or the like.

The step of collecting the user's personal information (S200) is a step of acquiring personal information for increasing the voice recognition success rate. The personal information may include user behavior information collected from a record of user actions and results of measuring user actions, user-specific personal information, and user status information indicating a user's situation. At this time, collecting the user's personal information (S200) includes analyzing at least one of the step S220 of acquiring the personal information directly inputted by the user and the voice signal and the collected personal behavior information, (Step S240). Meanwhile, the step of collecting personal information (S200) may be performed before the step of receiving the voice signal from the user (S100), and may be performed after the step of receiving the voice signal from the user (S100) is completed.

The step S300 of generating the voice recognition result from the voice signal based on the voice signal and the personal information may further include a step S320 of selecting the acoustic model and the language model with reference to the personal information set by the user have. The step of generating the voice recognition result from the voice signal based on the voice signal and the personal information (S300) can generate the voice recognition result with high accuracy by referring to the personalized acoustic model and the language model.

On the other hand, the step of generating a voice recognition result from the voice signal based on the voice signal and the personal information (S300) may generate a plurality of voice recognition results, and may have a probability value for each voice recognition result.

The step of generating the voice recognition result from the voice signal based on the voice signal and the personal information (S300) may display the types of the acoustic model and the language model used when the voice recognition result is generated.

The step of selecting the final speech recognition result from the speech recognition result (S400) is a step of selecting an optimum character string from a plurality of speech recognition results. In the step S400 of selecting the final speech recognition result, the speech recognition result may be selected using the user's public personal information and the private personal information. In addition, in the step S400 of selecting the final speech recognition result, a weight based on the personal information may be added to each probability of the plurality of speech recognition results, and as a result, a speech recognition result having the highest probability value may be selected.

The step of outputting the final speech recognition result (S500) outputs the speech recognition result using characters, images, sounds, and the like. At this time, outputting the final speech recognition result (S500) may display the speech recognition result to another user.

Although not shown in FIG. 9, the method further includes a method of transmitting the final speech recognition result to another user, translating the final speech recognition result into a language used by another user, and outputting the final speech recognition result to another user through video or sound You may.

By using the speech recognition method as shown in FIG. 9, it is possible to refer to the personalized acoustic model and the language model, thereby increasing the speech recognition success rate.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention. Accordingly, it is to be understood that within the scope of the appended claims, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

DETAILED DESCRIPTION OF THE INVENTION

The contents have been described in the best mode for carrying out the invention.

The present invention can be applied to voice recognition applications of smart phones and PCs, terminals for household appliances including telemarketing and voice recognition functions, speech recognition vehicles, voice-based interpreters operating in real time, and voice recognition systems.

Claims (13)

A terminal for receiving a voice signal from a user and collecting personal information of the user;
A private server for receiving the voice signal and the personal information from the terminal, sorting the personal information into a predetermined category and storing the voice signal, and transmitting the voice signal and at least some of the stored personal information to the voice recognition server;
A voice recognition server for performing voice recognition based on the voice signal transmitted from the private server and the personal information, and for generating a voice recognition result; , ≪ / RTI &
Wherein the personal information transmitted from the private server to the voice recognition server is personal information set by the user to be public,
The voice recognition server comprises:
An acoustic model unit for selecting at least one of phonemes, syllables and words corresponding to the speech signal;
A language model unit for referring to a sentence structure of a language to form a character string,
Wherein the acoustic model unit and the language model unit include an environment controller for selecting an acoustic model and a language model to be used in the speech recognition process. The method according to claim 1,
Wherein the environmental controller selects at least one acoustic model and at least one language model by referring to the personal information transmitted to the voice recognition server. The method according to claim 1,
The personal information includes:
User activity information collected from a result of measuring user activity and user activity, user status information indicating user's own personal information and user's status,
Wherein the user behavior information comprises:
User's online activity that collected user's online activity and Internet usage record,
User location information indicating the user's actual location,
User connection information, which is communication identification information of the user, and
User device information collected in the process of interaction between the user and the terminal,
Wherein the user state information comprises:
User attribute information indicating the user's personal information and personality, body, and emotional state, and
And environmental attribute information indicating a characteristic of a surrounding environment in which the user is located. The method of claim 3,
The terminal comprises:
Wherein the user state information is directly input from the user or inferred from at least one of the voice signal and the user behavior information. The method of claim 3,
The private server includes:
Wherein the user state information is input directly from the user, or the user state information is inferred from at least one of the voice signal and the user behavior information. The method according to claim 1,
The voice recognition server comprises:
Wherein the plurality of speech recognition results are derived and transmitted to the private server, and the acoustic model and the language model type information used in the speech recognition process are transmitted together. The method according to claim 6,
The private server includes:
Selecting at least one of a plurality of speech recognition results transmitted from the speech recognition server,
And selects the personal information using the public personal information and the private personal information. The method according to claim 6,
The private server includes:
Selecting at least one of a plurality of speech recognition results transmitted from the speech recognition server,
Adding a weight based on the public personal information and the private personal information to each probability value of the plurality of voice recognition results, and selecting a voice recognition result having the highest probability value as the result. Receiving a voice signal from a user;
Collecting personal information of the user;
Generating a voice recognition result from the voice signal based on the voice signal and the personal information;
Selecting a final speech recognition result from the speech recognition result; Lt; / RTI >
Wherein the step of generating a voice recognition result from the voice signal based on the voice signal and the personal information includes the steps of: selecting an acoustic model and a language model with reference to the personal information set by the user; The speech recognition method further comprising: 10. The method of claim 9,
The collecting of personal information of the user may include:
Acquiring personal information directly input by the user; Wow
Inferring user state information from at least one of a voice signal and user behavior information; Further comprising the steps of: 10. The method of claim 9,
Wherein the step of generating a voice recognition result from the voice signal based on the voice signal and the personal information comprises:
And generating a plurality of speech recognition results and generating acoustic models and language model type information used when speech recognition is performed for each of the plurality of speech recognition results. 12. The method of claim 11,
Wherein the step of selecting the final speech recognition result comprises:
And the final speech recognition result is selected using the public personal information and the private personal information. 12. The method of claim 11,
Wherein the step of selecting the final speech recognition result comprises:
Adds a weight based on the public personal information and the private personal information to each probability of the plurality of voice recognition results, and selects a voice recognition result having the highest probability value as a result.

Images