JPH08116385A - Individual information terminal equipment and voice response system - Google Patents

Abstract

PURPOSE: To preserve large quantity of data by storing encoded voice data, decoding voice data by using a codebook as necessary, and reproducing and outputting voice data. CONSTITUTION: An input/output part 402 is composed of the input/output interface of a microphone and an input/output control part and outputs the inputs of data and commands and processing results to a user. A voice data recording part 403 is an exclusive memory storing the voice data transferred from a server outputting the voice data of a voice response server. A CODEC part 404 has a function encoding voice by using the codebook of a code book control part 405 and reproducing a voice waveform and is composed of a CODEC part 404 performing an information compression and a decoding part reproducing voice data. The voice data which is received from the server and is encoded is stored in the voice data recording part 403, and the voice data is decoded, reproduced and outputted by using the codebook in the CODEC part 404. Therefore, the capacity of the recording part 403 can be increased.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a client / server type voice response system, and more particularly to a personal information terminal device used in the voice response system.

[0002]

2. Description of the Related Art In personal information terminals, it is desired to reduce the size and power consumption in order to save space and enhance portability, but there is a limit to the amount of memory and processing capacity that can be installed.
On the other hand, in order to perform voice processing such as voice recognition and voice synthesis, a large amount of memory and a huge processing capacity are required.
It is difficult to process with the current personal information terminal. As a means for solving the above problems, a client / server type system has been proposed in which a personal information terminal is connected to a network and a high-performance device on the network is used as a voice recognition server or a voice synthesis server to undertake voice processing. There is.

By the way, since the voice data has a large amount of data, if the voice data is directly used for communication, a transfer rate of several tens of kbps is required. On the other hand, in wireless communication, it is necessary to secure as many channels as possible in a limited frequency band. Therefore, generally in wireless communication, communication is often performed using audio data compressed by encoding.

As a voice encoding system, various systems have been proposed for a long time. For example, Kazuo Nakata "Digital Information Compression" (Kosaido Kogyo Publishing, Electronic Science Series)
100) describes various speech coding methods, and shows many methods related to waveform coding methods and information source coding methods (parameter coding methods).

Generally, in the high-efficiency coding system, attention is paid to the fact that the existence of speech information is biased, and speech coding is realized by thickening the allocation of codes to the portions where information exists. There is. This point is pushed even more positively, paying attention to the deviation of information as a combination of a plurality of parameters, and assigning a code to a portion where voice information exists for a set of parameter combinations (called a vector). A method of increasing the thickness (called vector quantization) is drawing attention. Such a method is described in, for example, S. Roucos et al., "
Segment quantization for very-low-rate speech codi
ng "Proc. ICASSP 82, pp1563 (1982) and the like.

In vector quantization, a codebook that stores a finite number of vectors in association with indexes is used. Vector quantization is a quantization process in the sense that an input vector is mapped to a finite number of code vectors, but since the information amount of the index can be made smaller than the original vector, it is also used for information compression.

As a vector quantization method, a CELP method (for example
BSAtal et al., "Stochastic coding of speech signa
ls at very low bit rates "Proc. ICC 84, pp.1610-161
3 (1984)) or its improved method VSELP (for example, IAG
erson et al., "Vector sum excited linear prodiction
(VSELP) "Proc. IEEE workshop on speech coding for te
telecommunications, pp.66-68 (1989)) and the like have been proposed. In Japan, in the field of digital mobile phones, VS
ELP and PSI-CELP have been established as standardization methods.

[0008]

By the way, the problem of the voice response system having the above-mentioned client / server configuration is the risk of eavesdropping. In the above vector quantization method, the client and the server communicate voice data encoded using a common codebook. Therefore, it is very difficult to reproduce encoded voice data without a codebook, but conversely, it is easy to reproduce encoded voice data if a codebook is available.

In other words, if the server uses only one codebook, it means that an unspecified number of people who use the service of this server have the same codebook. There was a problem that the possibility of eavesdropping by a malicious person was unavoidable.

In addition, there is an unavoidable risk of loss in the mobile terminal. A third party who acquired the lost mobile terminal can listen to the contents of the saved voice without the original owner's consent,
When communicating with the server, information leakage, etc.
The owner suffers a great deal of disadvantage.

In addition to the above problems, there are the following problems related to the storage of a large amount of voice data. If it takes a lot of time to process a large amount of database search, or if you want to send stock price data or news that is regularly transferred, send the audio sent from the server rather than playing it at that point. It is preferable that the voice data thus created be temporarily stored in a memory so that it can be played back by the user at his or her convenience. However, the voice data has a larger data amount than the character data, and a large-capacity memory is required to store the voice data. To store large amounts of data,
A DRAM (Dy) that is relatively inexpensive and has a large bit capacity per cell area compared to using an expensive SRAM (Static RAM)
It is more economical to use (namic RAM). However, in a personal information terminal considering portability, when a large amount of voice data is stored in a DRAM, which requires battery backup, there is a problem in that it cannot be stored for a long time due to exhaustion of the battery.

According to the present invention, a personal information terminal device used in a voice response system can store a large amount of voice data and can reproduce the voice data at any time by the user. To aim.

It is another object of the present invention to reduce the possibility of eavesdropping in the voice response system and prevent the information from being leaked to a third party even when the personal information terminal device is stolen or lost. And

[0014]

A personal information terminal device according to the present invention includes a voice data recording means for storing encoded voice data which is a response message received from a server, and a voice data represented as a multidimensional vector. Codebook storage means for storing a codebook composed of a code vector that is a vector representing a feature quantity and a code word that is a one-dimensional value corresponding to each code vector; and a codebook storage means that is stored in the codebook storage means. And a codec means for encoding and decoding voice using a codebook. Then, the received encoded voice data is temporarily stored in the voice data recording means, and when necessary, the voice data is recorded by the codebook means using the codebook stored in the codebook storage means. Decrypt and reproduce and output.

Further, one or more codebooks stored in the codebook storage means are assigned an identifier for identifying each, and are managed by the codebook management means. Then, when the command is transmitted from the personal information terminal device to the server, the identifier of the codebook to be used is transmitted to the server, and the server encodes the voice data using the codebook of the designated identifier. In the personal information terminal device which has transmitted the encoded voice data as a response message and has received the encoded voice data, the encoded voice data received from the server is temporarily stored in the voice data recording means, and when necessary. The audio data is decoded and reproduced and output by using the codebook of the specified identifier.

The command transmitted from the personal information terminal device to the server may be a voice command. A non-volatile memory can be used for the voice data recording means and the codebook storage means. Flash memory can also be used. When a flash memory is used, the memory area in which each codebook is recorded in the codebook storage means may be adjusted to the erase unit of the flash memory.

The codebook management means of the personal information terminal device has a function of registering a codebook (a function of receiving a codebook from a server and registering it in the codebook storage means) and a function of erasing the codebook (a codebook erasing signal). When received, the function of erasing the codebook recorded in the codebook storage means) and the function of checking the personal identification number (only when the correct personal identification number is input, the codebook stored in the codebook storage means) It is advisable to provide a function for permitting the change of the contents of.

[0018]

The present invention can be modified in various ways, and the typical means will be described below.

In the client-server type voice response system composed of the personal information terminal device and the voice response server, when the voice data is transferred from the server to the personal information terminal device, the codebook stored in the server side is used. Encode audio data before sending. By encoding the voice data, it is possible to significantly reduce the communication amount as compared with the case of directly transmitting the voice data.

On the personal information terminal side, the received encoded voice data is temporarily stored in the voice data recording means. For reproduction of voice, the stored encoded voice data is sequentially read and decoded by the codec means using the codebook stored in the codebook storage means. Of course, if it is not desired to reproduce it on the spot, it is possible to keep it stored in the audio data recording means.

If the codebook used on the server side and the codebook used on the personal information terminal device side are not the same, correct voice cannot be restored. Therefore, the codebook management means has a function of registering one of a large number of codebooks on the server side in advance in the codebook storage means. The codebook is managed with an identifier (for example, ID number) common to the server. Of course, it is also possible to register a plurality of codebooks in the codebook storage means. By specifying the codebook identifier to be used at the time of starting communication, the codebooks of the server and the personal information terminal device (client) can be matched.

Therefore, according to the present invention, the voice data encoded and transmitted from the server is stored in the voice data recording means in the encoded state, so that the memory capacity is smaller than that in the case of directly storing the voice. With, a large amount of voice data can be stored, and the voice data can be played back when the user likes.

Further, by specifying the identifier of the codebook to be used at the time of starting communication, it is possible to avoid the problem of eavesdropping, which is a problem when using a single standardized codebook. Further, if a function of erasing the codebook of the codebook storage means upon reception of the codebook erasing signal is prepared, even if the personal information terminal is lost, it is possible to prevent unauthorized use by the acquired third party. .

[0024]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of an embodiment of a client / server type voice response system in which a personal information terminal device is a client. In the present embodiment, various services connectable from the personal information terminal device 101 (for example, data search for searching a large amount of databases, reservation service for reservations for aircraft, hotels, etc., translation of input voice into another language) It is envisioned that the voice response system will operate voice translation services such as a translation service.

In FIG. 1, 101 is a personal information terminal device, 102 is a voice response server, 103 is a communication system,
Reference numeral 104 is a voice recognition system, 105 is an application system, and 106 is a voice synthesis system.

The personal information terminal device 101 and the voice response server 102 each have a communication function and can perform wireless communication. In the communication system 103, the voice recognition system 104, the application system 105, and the voice synthesis system 106, one machine may have these functions as the respective functions of the voice response server 102, or on the network. A configuration in which a plurality of computers are distributed as the respective server machines may be used.

The application system 105 is a part that provides various services provided by using the voice response system. For example, in the case of a database search service, the application system 105 has a database search function. The personal information terminal device 101, the communication system 103, the voice recognition system 104, and the voice synthesis system 106 will be described in detail later.

FIG. 2 is an external view of an embodiment of the personal information terminal 101. In FIG. 2, 201 is a display,
202 is a pen, 203 is a push button, 204 is a microphone, 205 is a speaker, and 206 is an antenna.

The display 201 is used for displaying text, graphics, moving images, etc.
It is also used as an input interface for pen input using 02. The push button 203 includes a switch that performs various functions such as turning the power on and off, and a switch that is used as a character input unit that substitutes for the keyboard. The microphone 204 is used for recording voice memos and inputting voice commands. The speaker 205 is
Outputs audio playback sounds and alarm sounds. Antenna 206
Is a part for transmitting and receiving radio waves. In this embodiment, the antenna 206 is provided outside, but it can be built in the housing.

FIG. 3 shows a hardware configuration of an embodiment of the personal information terminal 101. In FIG. 3, 301 is a microphone, 302 is a speaker, 303 is a display, 304 is a button, 305 is an antenna, 306 is an input / output control unit, 307 is an A / D conversion unit, 308 is a D / A conversion unit, and 309 is a VRAM. (Video RAM), 310 is a video controller, 311 is a touch sensor controller,
312 is a button controller, 313 is an RF modulator / demodulator, 314 is a CPU, 315 is a bus,
316 is a memory unit, 317 is a ROM, 318 is a RAM,
319 is a memory controller, 320 is an address line, 3
Reference numeral 21 is a data line and 322 is a control line.

A microphone 301, a speaker 302,
Display 303, button 304, and antenna 3
Reference numeral 05 is used as an interface for exchanging data with the outside. The functions of these input / output interfaces have already been described with reference to FIG.

The input / output control section 306 is a section for controlling the input / output interface, and includes the A / D conversion section 307 and the D / D conversion section 307.
It is composed of an A conversion unit 308, a VRAM 309, a video controller 310, a touch sensor controller 311, a button controller 312, and an RF modulator / demodulator 313.

The A / D converter 307 is provided for the microphone 3
It has a function of converting an analog signal input from 01 into a digital signal. Conversely, the D / A converter 308 converts digital data into an analog signal and outputs it to the speaker. The VRAM 309 is a dedicated memory for storing screen image data displayed on the display 303. The video controller 310 controls the update of the image data read into the VRAM 309 and the VRAM 30.
9 functions to display the screen image data stored in 9 on the display 303.

The touch sensor controller 311 is a sensor controller for the touch panel of the display 303. The button controller 312 is a controller for sensing ON / OFF of the button 304. R
The F modulator / demodulator 313 has a function of modulating (modulating) data into an RF (Radio frequency) signal in order to emit the data to be transmitted from the antenna 305 as a radio wave, and the R received by the antenna 305.
It has the function of demodulating the F signal.

The CPU 314 is a main processing unit that controls and operates the system based on a program. The bus 315 is a communication path for communicating control commands and data between the CPU 314 and other processing units.

The memory unit 316 includes a ROM 317 and a RAM.
318 and a memory controller 319. The ROM 317 is a read-only memory and stores various control programs and the like. RAM31
A read / write memory 8 stores various data input by the user. RAM31
A DRAM (Dynamic RAM) is usually used for the memory 8.
Also in this embodiment, the RAM 318 is a DRAM.

The DRAM is characterized by a large storage capacity per unit area and a low cost per bit. However, the DRAM cannot hold the recorded data unless it is refreshed at regular intervals. That is, in order to use it as a storage device of a mobile terminal, a backup battery is required, and data cannot be stored for a long time. Therefore, RAM318
As an SRAM (St
atic RAM) sometimes used.

In this embodiment, the ROM 317 and the RAM 318 are each composed of one chip, but they may be composed of a plurality of chips.

The memory controller 319 is a ROM 31.
7 and the reading of data from the RAM 318, RA
A controller for controlling the writing of data to the M318. The memory controller 319 and the various memories 317 and 318 include an address line 320 and a data line 32.
1 and a control line 322.

The address line 320 is a line for communicating an address designating a memory area to be accessed, the data line 321 is a line for communicating the accessed data, and the control line 322 is a line for communicating commands such as writing, erasing and reading. For example, taking the case of writing data to the RAM 318 as an example, the memory controller 319 sets a write destination address in the address line 320, sets the value of the data to be written in the data line 321, and sends a write signal from the control line 322.

Next, the function of the personal information terminal 101 will be described. FIG. 4 is a block diagram for explaining the function of one embodiment of the personal information terminal 101. In FIG.
Reference numeral 401 is a control unit, 402 is an input / output unit, 403 is an audio data recording unit, 404 is a codec unit, 405 is a codebook management unit, 406 is a codebook storage unit, 407 is a program storage unit, 408 is a data storage unit, 409. Is a communication unit.

The control unit 401 corresponds to the CPU 314 in FIG. 3, and is a part that controls the entire system. The input / output unit 402 is composed of an input / output interface such as the microphone 301 of FIG. 3 and an input / output control unit 306, and inputs data and commands from the user and outputs processing results to the user. The voice data recording unit 403 is a dedicated memory for storing voice data transferred from a server such as the voice response server 102 that outputs voice data. In this embodiment, part of the RAM 318 is used as the audio data recording unit 403.

The codec section 404 has a function of reproducing the original audio waveform by encoding audio using a codebook or decoding encoded audio data. The codec section 404 will be described later in detail. The codebook management unit 405 manages the codebook recorded in the codebook storage unit 406, and the codebook unit 4
It has a function of determining the codebook to be used in 04 and notifying the codebook to be used to the voice response server 102 side so that the codebooks used by each other are the same.

In the program storage unit 407, the CPU 31
A program for operating 4 is stored. The data storage unit 408 stores various data such as data input by the user and system processing results. Both the program storage unit 407 and the data storage unit 408 are
It corresponds to the memory unit 316 in FIG. The communication unit 409 includes an antenna 305 and an RF modulator / demodulator 31.
3, and has a function of communicating with the server machine.

FIG. 5 is a diagram for explaining an embodiment of the codec section 404 of FIG. Codec section 404
Is composed of a coding unit for compressing information by encoding voice and a decoding unit for reproducing encoded voice data. With respect to audio coding, various methods have been proposed for a long time, as already described in the section of the prior art. In this embodiment, an example of them will be described.

In FIG. 5, 501 is an analysis unit, 502 is a vector quantization unit, 503 is a vector dequantization unit, and 50 is a vector dequantization unit.
4 is a synthesis unit.

The analysis unit 501 cuts out the input voice at regular intervals (about 20 to 60 msec.), And extracts the feature amount of the voice from the cut out short-time voice waveform data. In this embodiment, a power spectrum envelope (PSE) analysis method will be described as an example. The PSE analysis method is described in detail in Nakajima et al., "Power Spectrum Envelope (PSE) Speech Analysis / Synthesis System", Journal of Acoustical Society of Japan, Vol. 44, No. 11 (Sho 63-11).

In the analysis unit 501 using the PSE analysis method, pitch information (pitch frequency or pitch period) 505, power information (short time power) 506, and spectrum information (line spectrum component) are extracted from the cut out short-time speech waveform data. ) Extract three pieces of information 507. Pitch information 505
For the extraction of R, a known method such as a correlation method or AMDF method may be used. Fourier transform is used to extract the spectrum information 507. When the Fourier coefficient obtained by the Fourier transform is squared, the power spectrum of the cutout waveform is obtained.

When the cut-out waveform has a periodic structure,
This spectrum has a line spectral structure due to pitch harmonics. Therefore, only the extracted harmonic component of the pitch frequency is extracted from the spectrum obtained by the Fourier transform. Using the data retrieved in this way,
The power spectrum envelope is approximated by a finite number of cosine series shown in Equation 1 below.

[0051]

[Equation 1]

Here, when f0 is the sampling frequency and fp is the pitch frequency, λ is expressed by the following equation 2.

[0053]

[Equation 2]

Coefficients A0 and A (1) to A (m) in Equation 1
Is calculated so that the squared error between the harmonic component of the pitch frequency extracted from the power spectrum and Y according to Equation 1 is minimized. Here, since A0 represents the short-time power of the input voice, it is output as power information 506, and A (1) to A (m) are output as spectrum information 507.

The vector quantizer 502 uses the codebook that stores the vector (code vector) representing the spectrum information of the voice and the index (codeword) corresponding to the code vector to use the spectrum information 507.
(Input vector) is vector-quantized. Details of the codebook will be described later with reference to FIG.

The input vector 507 is the vector quantizer 5
When input to 02, each code vector is read from the code book, and the distance dj between the input vector 507 and each code vector is calculated by the following formula 3. And
A code vector that minimizes the distance (for example, i-th code vector Bi (1), Bi (2), ..., Bi (m)) is obtained, and the codeword i is output.

[0057]

(Equation 3)

In this embodiment, the Euclidean distance weighting each element of the vector is used as the distance measure. Of course, other suitable distance measures may be used.

As described above, the analysis unit 501 and the vector quantization unit 5
By the processing described in 02, the audio data is coded (encoded), and the input audio becomes pitch information 50.
5, power information 506, and codeword 508 are converted into three parameters and transmitted.

Next, the operations of the vector dequantization unit 503 and the synthesis unit 504 regarding the decoding process of the voice data will be described.

Inversely to the vector quantizer 502, the vector dequantizer 503 converts the vector quantized data 509 (codeword) into vector data (spectral information) 510 using a codebook. . It goes without saying that the codebook used by the vector dequantization unit 503 must be the same as the codebook used for vector quantization.

The synthesizer 504 receives the input pitch information 51.
1, power information 512, and vector dequantization unit 503
A voice waveform is synthesized from the spectrum information 510 obtained in (1). The input pitch frequency 511 value is fp ', the power 512 value is A0', and each element of the spectrum information 510 is Bk.
(1), Bk (2), ..., Bk (m), the power spectrum Y ′ is obtained by the following equation 4.

[0063]

[Equation 4]

However, λ'is as shown in the following expression 5.

[0065]

(Equation 5)

The reproduced power spectrum is converted into an amplitude spectrum and then subjected to inverse Fourier transform to obtain a phoneme piece. A reproduced voice can be obtained by adding these phonemes while shifting them by a pitch interval.

The codebook will be described with reference to FIG. FIG. 6 is an example of a codebook used in the codec section of FIG. As shown in FIG. 6, the codebook is composed of a codeword 601 and a code vector 602. The code vector 602 is an m-dimensional vector quantity represented by m values indicating a voice feature quantity, and in this embodiment, m of B (1), B (2), ..., B (m). It is individual spectrum information.

For the code vector, voice data for learning is used in advance and is prepared as a vector representing the feature vector obtained from those voices. The LPG algorithm is well known as a code vector creation algorithm. The LPG algorithm is a method of clustering data (feature vector of learning speech) using a distortion measure, and is essentially the same as the k-means algorithm which is a clustering method used in the field of pattern recognition.

The codeword 601 is a one-dimensional value associated with these code vectors in a one-to-one correspondence. That is, when a codeword is specified, the codevector (B2 (1), B2 (2), ..., B2 (m) in this case) corresponding to the specified codeword (eg, 2) is unique. Specified in.

At the end of the explanation of the personal information terminal 101,
Codebook management unit 405 and codebook storage unit 406
Will be described. In this embodiment, the personal information terminal 101, which is a client, and the voice response server 102 use a common codebook, so that voice data is encoded and transmitted / received to reduce the communication data amount.

The problem here is communication security. In other words, if a third party has the same codebook, it is easy to eavesdrop on the communication content. Therefore, it is desirable to prepare an original codebook for each personal information terminal, or prepare a codebook with many varieties and switch and use it for each communication. Of course, communication cannot be established unless both the server and the client use a common codebook.

FIG. 7 is a memory map of one embodiment of the codebook storage unit 406. In FIG. 7, codebook management information is stored in 701, and codebooks are stored in 702 and 703, respectively. Each codebook has the configuration shown in FIG.

In FIG. 7, codebook management information 70
1 is managed by the codebook management unit 405. The codebooks stored in the codebook storage unit 406 are managed with ID numbers common to the server side. By thus unifying the ID numbers of the server and the client, it becomes possible to access the common codebook by only specifying the ID number of the codebook on the server side.

The codebook management information 701 contains this I
Start address (705) of the memory area in which the codebook having the D number (704) and the ID number is stored
Are recorded alternately. Codebook management unit 405
By accessing this codebook management information 701, a codebook having an arbitrary ID number can be used. In addition, when a codebook is newly added or deleted, the codebook management information 701 is also added.
Should be updated.

This is the end of the description of the personal information terminal 101.

Next, the details of the voice response server 102 will be described.

FIG. 8 is a diagram for explaining an embodiment of the communication system 103. In FIG. 8, 801 is a communication unit, 802 is a codec unit, 803 is a codebook management unit, and 804 is a codebook storage unit. Communication unit 801
Is the same as the RF modulator / demodulator 313 in FIG. 3, it functions to modulate the RF signal in order to emit the data to be transmitted from the antenna and the R received by the antenna.
It has a function of demodulating (detecting) the F signal.

The codec section 802, the codebook management section 803, and the codebook storage section 804 also have substantially the same functions as the codec section 404, the codebook management section 405, and the codebook storage section 406 of the personal information terminal 101 described with reference to FIG. To do. However, the codebook management unit 803 manages the codebook for each client. The codebook storage unit 804 stores all codebooks used by many clients.

FIG. 9 is a diagram for explaining one embodiment of the voice recognition system 104. In FIG. 9, 901 is an acoustic analysis unit, 902 is a language processing unit, and 903 is a semantic processing unit.

The acoustic analysis unit 901 divides the input voice data at regular intervals (usually several tens of ms), and then calculates the feature amount of the voice used for recognition determination for each division unit (analysis frame). . The LPC cepstrum is widely used as a feature quantity of speech used for speech recognition. The LPC cepstrum is a cepstrum calculated using the linear prediction coefficient obtained by the linear prediction analysis (LPC analysis). The feature analysis of speech, including the LPC cepstrum, is explained in detail in Furui's "Digital Speech Processing" (Tokai University Press).

The language processing unit 902 and the characteristic amount of the input voice obtained by the acoustic analysis unit 901 and the acoustic model 904, the word dictionary 905, and the grammar 906 prepared in advance.
And are used for voice recognition. HMM for voice recognition
The method using (Hidden Markov Model) is the mainstream.
In HMM, speech is applied to a probabilistic model for recognition.

Specifically, the acoustic model is represented by a transition model having several states, and the occurrence probability of the codebook of each phoneme and the transition probability between states are learned in advance. In the speech recognition, a model in which acoustic models are connected is created using the word dictionary 905 and the grammar 906, and the model having the maximum probability when the time series of the input speech feature amount is applied to these models is recognized. The result.

The semantic processing unit 903 semantically analyzes the result recognized by the language processing unit 902 and converts it into a command that the application system 105 can understand. For example,
For the recognition result such as "Search XXX", "search
It is converted to “XXX” and transmitted to the application system 105.

FIG. 10 is a diagram for explaining one embodiment of the voice synthesis system 106. In FIG. 10, 10
01 is a response sentence generation part, 1002 is a text analysis part, 10
Reference numeral 03 is a prosody control unit, and 1004 is a voice generation unit.

The response sentence generator 1001 receives the result processed by the application system 105 and generates a response sentence. For example, when the application system 105 is requested to search for data, according to the search result,
A sentence such as "No corresponding item was found." Or "There are three corresponding items." Is generated. Of course, when the application system 105 generates and outputs a response sentence, the processing of the response sentence generation unit 1001 is unnecessary.

The text analysis unit 1002 uses the word dictionary 100 for the response sentence generated by the response sentence generation unit 1001.
Morphological analysis is performed using 5 to give a correct reading to the input text, and then a phonetic symbol string is generated in consideration of accent combination. Several methods have been proposed for morphological analysis, and for example, the left longest matching method, which uses the result of the longest syntax fitting from the beginning of a sentence as the correct analysis result, is well known.

The prosody control unit 1003 has a text analysis unit 1
Prosodic information is added to the phonetic symbol string generated in 002 according to the prosodic rule. The prosody rule is composed of the control model 1006 created based on the result of the analysis of the actually spoken human voice by the developer. By adding prosody information such as duration control and pitch pattern control, it becomes possible to generate synthetic speech with natural rhythm and intonation.

The voice generation unit 1004 has a prosody control unit 100.
For the phonetic symbol string added with the prosody information processed in 3,
The voice is reproduced using the voice unit model 1007. The process of the voice generation unit 1004 is divided into unit voice selection and voice signal generation. In the unit voice selection, an appropriate unit voice is selected from the unit voices prepared as the voice unit model (for example, CV syllable, VC syllable, etc.) in consideration of the phoneme sequence to be synthesized and the phoneme environment of the extraction source. It In the generation of a voice signal, while controlling the duration and the pitch pattern, the selected unit voices are connected to generate a voice signal. At this time, interpolation processing is performed so that the connected portions do not become discontinuous.

The description of the voice response server 102 is completed.

Next, the processing of the client / server type voice response system using the personal information terminal device 101 and the voice response server 102 will be described.

FIG. 11 is a processing sequence of an embodiment of the client / server type voice response system. Many kinds of such processing sequences are conceivable depending on the service provided by the voice response server and the command used. Therefore, the processing sequence described here is only an example.

The processing sequence will be described below based on the processing flow of FIG. First, the personal information terminal device 10
1 user inputs a voice command (110)
1). A voice command is a request for a voice response system using voice, specifically, "search for XXX".
Or "Reserving △ to XX on XX month".

The personal information terminal device 101 compresses the input voice by compressing the information (1102), communicates with the voice response server 102 using the voice wireless function, and sends a voice command (1103). At this time, the ID number of the codebook used for encoding is transmitted to the voice response server 102, and the codebooks used are matched.

When the encoded voice command is input (1104), the voice response server 102 decodes the encoded voice in the communication system 103 to restore the original voice waveform. (110
5). Next, voice recognition is performed on the restored voice command using the voice recognition system 104 (110).
6). The recognition of voice commands is described in detail in the description of the voice recognition system 104. When the voice recognition is completed, the application system 105 receives the command sent from the voice recognition system 104 and executes the command (1107).

When the processing of the application system 105 is completed, the speech synthesis system 106 converts the processing result executed by the application system 105 into synthetic speech (1108). The speech of the processing result synthesized by the speech synthesis system 106 is encoded by the communication system 103 (1109) and the processing result is transmitted to the personal information terminal 101 (1110).

The personal information terminal 101 is the voice response server 1
When the encoded voice data of the processing result transmitted from 02 is received (1111), the received voice data is stored in the voice data recording unit 403 (1112). By storing the voice data in the voice data recording unit 403, the user can reproduce and listen to the voice at any time unless the stored data is deleted. The audio playback is
The coded audio data is decoded by the codec unit 404 (1113) and output from the speaker 302 through the D / A conversion unit 308 (1114).

As described above, according to the present embodiment, a large amount of voice data can be stored by storing the voice data encoded and transmitted from the server in the voice data recording unit as it is, It becomes possible for the user to reproduce the voice data at any time.

Further, by designating the ID number of the codebook to be used at the time of starting communication, it is possible to avoid the problem of eavesdropping which is a problem when a standardized single codebook is used. .

Next, registration of the code book described with reference to FIG. 7 will be described. The codebook registration is an operation of storing the codebook on the server side in the codebook storage unit 406 of the personal information terminal 101 on the client side.

FIG. 12 shows a flow chart of an embodiment of a codebook registration sequence. In FIG. 12, when the user of the personal information terminal 101 tries to start registration of the codebook by the registration command (1201), the system requests the user to input the personal identification number (120).
2). The request for the personal identification number is not necessarily required, but it is effective for preventing a third party other than the owner from arbitrarily registering or deleting the codebook. When the personal identification number is input (1203), it is checked whether the input personal identification number is correct (1204).

If the entered personal identification number is incorrect, the process is terminated and the codebook registration work is not performed. If the personal identification number is correct, the ID number of the codebook to be registered is set (1205). As the ID number needs to be assigned a number that uniquely specifies the codebook, the system automatically determines the ID number so that it does not overlap with the already registered ID number.

When the ID number is decided, a new codebook is transmitted from the server side (1206). Upon receiving the transmitted codebook, the personal information terminal 101 records the codebook in the codebook storage unit 812 (1207), and stores the ID of the newly registered codebook in the codebook management information and the address of the recording area. Add (1208).

The registration of a new codebook is completed when all the above processes are completed.

By the way, the portable personal information terminal can be carried anywhere as the owner moves. This has the advantage that it can be used anywhere, but it also entails the risk of unauthorized use by others due to loss or theft. Of course, it is possible to prevent such unauthorized use by setting a personal identification number. However, it is very troublesome for the owner to enter the personal identification number each time it is used.

Therefore, a codebook is used to protect security when lost. As already mentioned,
Since the encoded audio data is stored in the audio data recording unit 403, in order to reproduce the audio, the codec unit 404 needs to decode the audio data. That is, even if the recorded voice is not erased, it is impossible to reproduce the recorded voice if the codebook is erased. By utilizing this, the risk of unauthorized use by others due to loss or theft is avoided. Hereinafter, a specific description will be given.

FIG. 13 shows a codebook erasing sequence for protection of security when lost. First,
The owner who notices that his / her personal information terminal 101 has been lost is instructed by his / her own personal information terminal 1 in the voice response server 102.
Request to erase 01 codebook (130
1). Of course, since the personal information terminal 101 of one's own cannot be used for communication with this server, another terminal device capable of communicating with the server is used. Further, if a telephone line interface is prepared for the voice response server 102, it is possible to access using a telephone.

Upon receiving the codebook erasing request, the voice response server 102 asks the requester to input the personal identification number as in the codebook registration sequence (1302).
When the personal identification number is input from the requester (1303), it is checked whether the input personal identification number is correct (130).
4). If the entered PIN is incorrect,
The process is terminated and the codebook deletion request is rejected.
If the personal identification number is correct, the voice response server 102
An erasing signal is transmitted to the erasing destination personal information terminal 101 (1305).

Personal information terminal 101 that has received the erase signal
Deletes the codebook stored in the codebook storage unit 406 regardless of the current usage state (power off, communication in progress, etc.) (1306) and deletes the codebook in the codebook management information. Record (13
07).

By the sequence described above, the erasing of the codebook stored in the personal information terminal 101 is completed. After that, it becomes impossible to reproduce the voice recorded in the voice data recording unit 403. Of course, it is not possible to newly communicate with the voice response server. Further, if the original owner re-registers the deleted codebook (the security check with the personal identification number is required), the recorded voice can be reproduced again.

As described above, according to this embodiment, even when the personal information terminal is lost, it is possible to prevent the unauthorized use by the acquired third party.

Next, as a second embodiment of the personal information terminal 101, a case where a flash memory is used as a part of the memory unit 316 will be described. As I already explained, D
Since a battery is required to store data in RAM or SRAM, DR is added to the personal information terminal 101 that is supposed to be portable.
It is not possible to store large amounts of data such as codebooks and voice data for a long time using AM or SRAM. Therefore, if a part of the memory unit of the personal information terminal 101 is replaced with a non-volatile memory represented by a flash memory,
It is possible to retain the codebook and voice data for a long time without the need for battery backup.

FIG. 14 shows an embodiment of a memory section in which a part of the memory is replaced with a flash memory. In FIG. 14, 1401 is a memory controller, and 1402 is RO.
M, 1403 is a DRAM, 1404 is a flash memory, 1405 is an address line, 1406 is a data line, 14
Reference numeral 07 is a control signal line.

As is clear from comparison between FIGS. 3 and 14,
In the memory section of FIG. 14, a part of the RAM 318 of FIG.
I just set 03. Therefore, the function of each part is the same as in FIG. However, in the flash memory, there is a limitation that an erasing process is always performed when writing data, and the erasing unit is a block unit (for example, 512 bytes). Therefore, the control of the memory controller 1401 is slightly different.

FIG. 15 shows an embodiment of a memory map when a flash memory is used for the codebook storage section and the voice recording section.

Data (1501) stored in the codebook storage unit 406 and voice data (1502) stored in the voice data recording unit 403 are recorded in the flash memory. As described above, the flash memory is necessarily accompanied by an erasing process for writing data, and has a limitation that the erasing unit is a block unit (for example, 512 bytes). Therefore, if the data to be updated is mixed in the block to be updated, the memory management becomes complicated.

Therefore, by utilizing the fact that all codebooks have the same size ((m-dimensional) × (N) words in this embodiment), the area for storing the codebook is fixed in advance, and this fixed The created area is aligned with the erase block of the flash memory. For example, in the memory map shown in FIG. 16, the memory area for storing the codebook 1 is changed from block1 to block3 of the erase unit of the flash memory to
The memory area for storing codebook 2 is changed from block4 to bloc.
Fixed to k6.

With the setting as shown in FIG. 16, in order to change the codebook 1, it is sufficient to write a new codebook from block1 to block3 after sequentially erasing block1, block2 and block3. Memory management becomes easier compared to the case where data to be erased and data to be left are mixed in a block.

By using the second embodiment described above, the codebook and voice data can be stored for a long time without battery backup.

[0119]

As described above, according to the present invention, a large amount of voice data can be saved by storing the voice data encoded and transmitted from the server in the voice data recording unit in the encoded state. Therefore, it becomes possible for the user to reproduce the voice data at any time. Also, an identifier for identifying the codebook used between the server and the personal information terminal device is exchanged at the time of starting communication,
Since the codebook of the identifier is used, the possibility of eavesdropping can be reduced. further,
When the personal information terminal device is stolen or lost, an erasing signal can be sent to erase the entire codebook, so that unauthorized use by a third party can be prevented.

[Brief description of drawings]

FIG. 1 is a diagram for explaining an example of a client-server type voice response system.

FIG. 2 is an external view of an embodiment of a personal information terminal.

FIG. 3 is a diagram illustrating a hardware configuration of an embodiment of a personal information terminal.

FIG. 4 is a block diagram for explaining a function of one embodiment of the personal information terminal.

FIG. 5 is a diagram illustrating an example of a codec unit.

FIG. 6 is a diagram for explaining an example of a codebook.

FIG. 7 is a diagram for explaining an embodiment of a codebook storage unit.

FIG. 8 is a diagram for explaining an example of a communication system.

FIG. 9 is a diagram for explaining an example of a voice recognition system.

FIG. 10 is a diagram for explaining an example of a voice synthesis system.

FIG. 11 is a diagram illustrating a processing sequence of an embodiment of the client / server type voice response system.

FIG. 12 is a diagram for explaining an example of a codebook registration sequence.

FIG. 13 is a diagram for explaining an example of a codebook erasing sequence.

FIG. 14 is a diagram illustrating an example of a memory unit.

FIG. 15 is a diagram for explaining an example of a memory map of a flash memory.

FIG. 16 is a diagram for explaining an embodiment of a codebook storage unit using a flash memory.

[Explanation of symbols]

101 ... Personal information terminal, 102 ... Voice response server, 40
1 ... Control part, 402 ... Input / output part, 403 ... Audio data recording part, 404 ... Codec part, 405 ... Codebook management part, 406 ... Codebook storage part, 407 ... Program storage part, 408 ... Data storage part.

Claims (16)

[Claims] 1. A personal information terminal device, which transmits a command designating a process requested to a server and receives voice data encoded as a response message from the server, wherein data is exchanged with the server. Communication means for transmitting and receiving, voice data recording means for storing encoded voice data received from the server, code vectors that are vectors representing the feature amount of voice represented as a multidimensional vector, and respective codes Codebook storage means for storing a codebook composed of a codeword that is a one-dimensional value corresponding to a vector, and encoding and decoding speech using the codebook stored in the codebook storage means. And a codec unit for converting the encoded voice data received from the server into the voice data. It is stored in the recording means, when necessary, a personal digital assistant device, characterized in that to reproduce and output decoded audio data by using the codebook stored in the codebook storage unit by the codec unit. 2. A personal information terminal device for transmitting a command designating a process requested to a server and receiving encoded voice data transmitted as a response message from the server, the personal information terminal device communicating with the server. Communication means for transmitting and receiving data, voice data recording means for storing the encoded voice data received from the server, and a code vector which is a vector representing a feature amount of voice represented as a multidimensional vector, respectively. Using a codebook storing means for storing one or more codebooks each including a codeword that is a one-dimensional value corresponding to the code vector of: and a codebook stored in the codebook storing means. Codec means for encoding and decoding voice, and a codebook stored in the codebook storage means A codebook management unit that manages using an identifier that identifies each codebook, and when transmitting a command from the personal information terminal device to the server, transmits the identifier of the codebook to be used to the server, The server encodes the voice data using the codebook of the designated identifier and transmits it as a response message, and the personal information terminal device receiving the encoded voice data encodes the voice data received from the server. The personal information terminal device, wherein the voice data is temporarily stored in the voice data recording means, and when necessary, the voice data is decoded and reproduced and output using a codebook of the designated identifier. 3. The personal information terminal device encodes voice data representing the command using the voice code data by means of inputting a voice data command for designating a process requested to the server, and transmits the voice data to the server. Means to
The personal information terminal device according to claim 1 or 2, further comprising: 4. The personal information terminal device according to claim 1, wherein a nonvolatile memory is used as the voice data recording means. 5. The personal information terminal device according to claim 4, wherein a flash memory is used as the voice data recording means. 6. The personal information terminal device according to claim 1, wherein a nonvolatile memory is used as the codebook storage means. 7. The personal information terminal device according to claim 6, wherein a flash memory is used as the codebook storage means. 8. The personal information terminal device according to claim 7, wherein in the codebook storage means, a memory area for recording each codebook is preset according to an erasing unit of the flash memory. 9. The personal information terminal device according to claim 2, wherein the codebook management means has a function of receiving the codebook from the server and storing the received codebook in the codebook storage means. 10. The personal information terminal device according to claim 2, wherein said codebook managing means has a function of erasing the codebook recorded in said codebook storing means when receiving the codebook erasing signal. 11. The codebook management means has a function of permitting modification of the contents of the codebook stored in the codebook storage means only when a correct personal identification number is input. The personal information terminal device according to 2. 12. A server and a personal information terminal device, wherein a command is transmitted from the personal information terminal device to the server to request various processing, and the server performs processing according to the command and displays the result. In the voice response system, which transmits the encoded voice data that is represented as a response message to the personal information terminal device, and the personal information terminal device receives and reproduces the encoded voice data, wherein the server is the individual Receiving means for receiving the command transmitted from the information terminal device, application processing means for performing processing according to the command transmitted from the personal information terminal device, and outputting the processing result, and from the processing result of the application processing means A response sentence generating means for generating a response sentence, and voice synthesis for synthesizing a voice according to the generated response sentence And a transmitting means for encoding the synthesized voice and transmitting the encoded voice to the personal information terminal device, the personal information terminal device transmitting means for transmitting a command to the server, and a code transmitted from the server. Receiving means for receiving the encoded voice data, voice data recording means for storing the received encoded voice data, and a code vector which is a vector representing the feature quantity of the voice represented as a multidimensional vector Codebook storage means for storing a codebook composed of a codeword, which is a one-dimensional value corresponding to a code vector, and encoding and decoding speech using the codebook stored in the codebook storage means. The personal information terminal device, the encoded voice data received from the server. A voice characterized in that the voice data is once stored in the voice data recording means, and when necessary, the voice data is decoded and reproduced and output by the codec means using the codebook stored in the codebook storage means. Response system. 13. A server and a personal information terminal device, wherein the personal information terminal device sends a command to the server to request various processing, and the server performs processing according to the command and displays the result. In the voice response system, which transmits the encoded voice data represented as a response message to the personal information terminal device, and the personal information terminal device receives and reproduces the encoded voice data, wherein the personal information terminal device is , Store one or more codebooks each composed of a code vector that is a vector representing a feature amount of speech represented as a multidimensional vector and a codeword that is a one-dimensional value corresponding to each code vector Codebook storing means and coded speech using the codebook stored in the codebook storing means Codebook managing means for managing the codebook means for decoding, and the codebook stored in the codebook storing means, using an identifier for identifying each codebook, and transmitting a command to the server, and At this time, transmitting means for transmitting the identifier of the codebook to be used, receiving means for receiving the encoded audio data transmitted from the server, and audio data recording means for storing the received encoded audio data. The server includes a receiving unit that receives a command and a codebook identifier transmitted from the personal information terminal device, and performs processing according to the command transmitted from the personal information terminal device, and outputs a processing result. Generating a response sentence from the application processing unit that performs the processing and the processing result of the application processing unit Response sentence generating means, a voice synthesizing means for synthesizing a voice in accordance with the generated response sentence, and the synthesized voice is encoded by using a codebook having an identifier designated by the personal information terminal device to A transmission means for transmitting to the information terminal device, and at the time of starting communication, the personal information terminal device exchanges the identifiers of the codebooks used by the server and makes the codebooks used coincide with each other. Voice response system. 14. The voice response system according to claim 13, wherein the command transmitted from the personal information terminal device to the server is a voice command. 15. The voice response system according to claim 13, wherein a flash memory is used as the voice data recording means. 16. The voice response system according to claim 13, wherein a flash memory is used as the codebook storage means.