KR100634142B1 - Potable terminal device - Google Patents

Abstract

Translating the text by translating the text inputted in a predetermined language into another language (transmission server, etc.) and translating the text, and the translation result information including the text of the translation result and pronunciation data indicating the pronunciation method In response, the portable terminal device is configured to pronounce the text of the translation result based on the pronunciation data, and the speech synthesis is performed using the formant parameter as the pronunciation data. Accordingly, the translation result information can be communicated with a relatively small communication capacity, and the portable terminal device can also receive related voices, images, and the like.

Description

Portable terminal device {POTABLE TERMINAL DEVICE}

1 shows a schematic configuration of a system consisting of a mobile telephone and a delivery server connected via a communication network according to a preferred embodiment of the present invention.

2 shows an example of the contents of a dictionary registered in a dictionary database.

3 shows a relationship between event data and duration data in each sequence data.

4 is a diagram for explaining the data storage and processing structure of SMAF.

5 shows an example used in an embodiment of the present invention by adding an HV track chunk to a conventional SMAF file.

Fig. 6 shows three types of audio reproduction formats, (a) shows TSeq type, (b) shows PSeq type and (c) shows FSeq type.

7 shows an example of a data exchange format of audio reproduction sequence data.

8A shows a configuration of sequence data.

8B shows the relationship between duration and gate time.

9 is a diagram for explaining rhyme control information.

10 is a diagram illustrating a relationship between gate time and delay time.

11 shows the level and center frequency of each formant waveform.

12 shows an example of data of a body portion of an FSeq data chunk.

13A shows the meaning of symbols in HV-Script.

13B shows a pitch change indicated by a symbol in HV-Script.

Fig. 13C shows the pitch change indicated by the symbol in the HV-Script.

14 shows the parameters characterizing the formant.

15 is a block diagram showing a schematic configuration of a mobile telephone according to the present embodiment.

16 shows the configuration of the speech synthesis unit.

FIG. 17 shows the configuration of the formant generating unit shown in FIG. 16.

18 is a flowchart for explaining the operation of the cellular phone in the case of executing a translation process.

19 is a flowchart for explaining the operation of the delivery server in the case of executing a translation process.

20 is a flowchart for explaining the operation of the cellular phone in the case of executing the pre-search processing.

21 is a flowchart for explaining the operation of the delivery server in the case of executing the pre-search processing.

22 shows an example of a format type.

23 shows an example of a language type.

24 shows an example of a time base.

25 shows an example of the HV timbre parameters.

Fig. 26 shows an example of the structure of the rhyme control information.

27 shows an example of a frame data column of the FSeq type.

Explanation of symbols on the main parts of the drawings

1: mobile phone 2: delivery server

2a: controller 2b: database

100: SMAF file 113: LCD display

12: communication unit 12a: antenna

13: voice processing unit 15: microphone

16a: sound source 17: speaker

The present invention relates to a portable terminal apparatus which pronounces a translation result or a dictionary search result by a translation means by speech synthesis.

In recent years, various services and functions have been provided for mobile phones (Cellular Phone, etc.). For example, only by inputting sentences written in a language, the machine is automatically translated into another language, and the translation results are voiced. Also, a service using a wireless communication network that can be pronounced and spoken to a user is also being implemented. At present, such a mobile phone is used to realize a translation and interpretation function between other languages provided by the above-described service or the like as well as the communication function as a phone.

Patent documents related to the above translation services and the like also exist. For example, Japanese Patent Application Laid-open No. 2002-125050 discloses a user's input voice via an internet via a mobile phone to an interpretation server. A technique for returning a machine-translated (or automatically translated) voice to a mobile phone is disclosed.

However, since the public telephone line is used to provide a translation result by voice as the above-described service and patent document description technology, transmission and reception of voice data (or voice signal) in a mobile phone is necessarily limited to use of a predetermined line capacity. Will receive. Therefore, in the above-mentioned conventional technology, it cannot be said that communication resources which have recently become wideband and have a tendency of high speed and large capacity can be effectively used.

In addition, in the above-mentioned prior art, since the text display as the translation result and the audio output as the translation result are only possible in the mobile phone, in addition to the provision of the translation information thereof, the related image and the audio information are also designed to be provided simultaneously. It is difficult to say that it can meet the expectations and demands of the user side in the future with the development of communication technology.

SUMMARY OF THE INVENTION The present invention has been made in view of the above point, and in a portable terminal device that outputs a translation result or a dictionary search result by speech synthesis, the capacity required for transmission and reception of information on translation and presearch results can be reduced, and It also aims at enabling information to be provided to a user in real time using at least voice (or image).

The portable terminal device according to the present invention transmits a text to be translated to a translation means (e.g., an external translation delivery server) for translating an input text into another language, and translates the text to be translated according to the translation result. And returning the translation result information including the pronunciation data indicating the pronunciation method, and causing the text of the translation result to be pronounced in a desired language and natural intonation based on the received pronunciation data. Here, the formant parameter is used as the pronunciation data.

In this way, since the translation result can be pronounced as a voice, the user of the portable terminal device can easily recognize and grasp the translation result of the input text by hearing. The pronunciation data returned from the translation means indicates and specifies the formant parameters (for example, formant frequency, formant level, bandwidth, etc.) used for synthesizing each phoneme. It is smaller than the voice signal, and therefore, when the portable terminal apparatus receives the reply of the pronunciation data through the communication network, even a smaller transmission capacity is possible than the reply of the conventional voice signal.

In addition, when the portable terminal apparatus is provided with display means (e.g., a liquid crystal display), the text of the translation result can be displayed, so that the user can visually recognize and grasp the translation result.

The translation means described above may be provided outside the portable terminal device to execute an instruction for executing the translation process, receive a translation result, or the like through the communication network, or may provide the translation means inside the portable terminal device.

In addition, the portable terminal apparatus according to the present invention inputs predetermined voice data to pronounce a voice, transmits the headword information as a search key to a dictionary database, retrieves the semantic information, and simultaneously pronounces the semantic information. And returning the search result information including the pronunciation data indicating that the speech is pronounced based on the received pronunciation data. Here, the pronunciation data is constructed using the formant parameter.

In this way, the semantic information retrieved can be pronounced as a voice using the headword information as a search key, so that the user of the portable terminal device can easily recognize and grasp the semantic information by hearing. Further, the search result information returned from the dictionary database indicates the formant parameter, and therefore, only a small transmission capacity can be used for the return of the search result.

The above search result information includes text indicating semantic information, image data indicating an image related to a headword of a search key, and second pronunciation data indicating a sound related to a headword. In this case, text and an image can be displayed by the display means provided in the portable terminal device. Also, based on the second pronunciation data, a sound related to the headword may also occur. Accordingly, the user of the portable terminal device can obtain not only the meaning of the headword which is the search key but also the rich related information.

The pronunciation data includes a pronunciation string for displaying a translation result or a search result, and a rhyme symbol for specifying an accent or the like (innation or accent) of the pronunciation when the pronunciation string is spoken. Specifically, it is described by so-called HV-Script, which is defined by ordinary characters (in Japanese, hiragana and katakana, etc.) or a combination of numerals and predetermined symbols.

Embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 illustrates a mobile phone 1 according to a preferred embodiment of the present invention and a translation service and a dictionary retrieval service for the mobile phone 1 through a communication network (including a wireless communication network and a digital data network). The schematic structure of the system which consists of the delivery server 2 is shown.

The mobile phone 1 has a function of pronounced translation results or search results in a desired language by speech synthesis, and is connected to a delivery server 2 managed by a predetermined content provider or the like through the communication network. The delivery server 2 includes a control unit 2a and a dictionary database 2b for controlling each part of the delivery server.

The dictionary database 2b stores various dictionaries for searching for the translation dictionary used for the translation processing executed by the control unit 2a, the meaning of words, and the like. The translation dictionary includes pronunciation data (hereinafter referred to as "first pronunciation data") for speech synthesis using a translation result for a sentence or word (text data) to be translated and a pronunciation method (for example, a reading method) in a predetermined language. Is registered. The various dictionaries are referred to as text representing semantic information (i.e., the meaning of the heading to be searched, etc.) corresponding to the heading as the search key, and pronunciation data (hereinafter referred to as "second pronunciation data") of the reading method. Information (images, sounds, etc.) associated with this information is registered correspondingly.

The control unit 2a performs predetermined processing in accordance with a translation request or a retrieval request from the mobile phone 1, and in the case of a translation request, the translation target text sent from the mobile phone 1 using the above-described translation dictionary. The translation process is executed, and the translation result information including the text according to the translation result and the pronunciation data according to the reading method is generated and returned to the mobile phone 1. On the other hand, in the case of a search request, a search process is performed using a dictionary designated as a search key the headword of the search target included in the search request, the text displaying the search result, the pronunciation data of the reading method, and the headword The search result information including the pronunciation data representing the sound associated with and the image data representing the associated image is generated and returned to the mobile phone 1.

Next, the structure of the dictionary database 2b is demonstrated in detail.

As described above, the dictionary database 2b stores translation dictionaries, various dictionaries, and the like. For example, when the translation dictionary stored in the dictionary database 2b has the function of an English-Japanese dictionary, "と て も い い 天氣 で す ね。" is the text of the translation corresponding to the English word "It's very fine, isn't it?" It is registered, and mixed data of characters and symbols such as "と っ 'て も, S54' い / い て い ん ん / で す ＿ ね -2 *-" is registered as pronunciation data of the reading method. In addition, although the pronunciation sequence data mentioned later is also registered in the translation dictionary, it abbreviate | omits description here. As described above, the pronunciation data according to the voice (i.e., human voice) in the present embodiment is constituted by including a rhyme symbol that defines the intonation of the sound and the like when pronounced by voice synthesis.

As described above, the description rule of the pronunciation data described in text (hereinafter, referred to as "HV-Script" in this embodiment) will be described later. In addition, in the present embodiment, the mobile phone 1 converts the pronunciation data by HV-Script received from the delivery server 2 into formant parameters for each phoneme and at the same time based on the added rhyme symbols. Frame parameters are formed by changing the frame parameters, and speech synthesis is performed using the frame data streams. In this case, the data according to the phoneme description type or the data using the formant frame description type can also be used.

As described above, the text of the translation sentence corresponding to the short sentence or word and the pronunciation data of the reading method are registered in the translation dictionary. In the case of translating a relatively long sentence, a translation process is performed by parsing a syntax using a known technique and the like, and the translation dictionary includes various data for performing the translation. In addition, pronunciation data corresponding to each sentence or word constituting the sentence is registered in the translation dictionary, and the pronunciation data corresponding to the entire sentence is generated by gradually replacing the sentence or word constituting the sentence of the translation result with corresponding pronunciation data. do. In addition, in the dictionary database 2b, the selection and decision rules of the rhyme symbols selected and determined from the conditions under which the phrases or words are used (that is, the conditions according to the types of the sentence, the sentence, the question and the like) are also registered. The desired rhyme symbol may be determined and added, or the rhyme symbol may be appropriately changed.

Various dictionaries are also stored in the dictionary database 2b, and as shown in FIG. 2, the dictionary entry information including the heading information (index) and the meaning information corresponding to the heading information as one set is shown. It is composed of a large number. The semantic information includes data ("data 1") indicating the meaning of the headword information, first pronunciation data ("data 2") indicating the pronunciation method of the headword, and second pronunciation data (" Data 3 ") and image data (" data 4 ") for displaying an image related to the headword.

For example, when the translation dictionary has the function of an English-Japanese dictionary, an English word designated as a search key is registered in the column of the index as shown in FIG. As data 1 corresponding to the English word of each heading, a translation of a word (for example, "duck" when the heading is "Duck") is registered as data 1, and pronunciation data of the translation is registered as data 2 Sound data related to the English word of the headword as data 3 (when the table control is “Duck”, for example, pronunciation data according to the duck's cry), the image related to the English word of the headword as data 4 Data (when the table control is "Duck", for example, image data of a duck) is registered.

In addition, text sequence data is registered in the column of the index and data 1, which is composed of event data (a text string, a reproduction position, and the like) and duration data. The detailed description will be described later.

In the column of data 2, only pronunciation data by HV-Script is described when only one word is registered, while speech sequence data is described when registering a plurality of words in a series.

The speech sequence data is composed of a plurality of speech data (data expressed by HV-Script) and pronunciation sequence data, and a pronunciation number is assigned to each pronunciation data. As shown in Fig. 3, the pronunciation sequence data is composed of event data and duration data indicating an event interval. This event data has data indicating a pronunciation number indicating the corresponding pronunciation data and data indicating a pronunciation period by the pronunciation data. By reproducing the pronunciation sequence data, the pronunciation data corresponding to each event data is reproduced at a timing in accordance with the duration data, so that a plurality of words in a series are sequentially pronounced.

In the column of data 3, PCM (Pulse-Code Modulation) sequence data or FM (Frequency Modulation) sequence data is described. PCM sequence data is composed of waveform data corresponding to pronunciation data and sequence data for PCM. Waveform data is specified by waveform number. As shown in Fig. 3, the PCM sequence data is composed of event data (i.e., waveform number, pronunciation time, etc.) and duration data (indicative event interval). In addition, FM sequence data is composed of tone data, which is pronunciation data conforming to the Musical Instrument Digital Interface (MIDI) standard, and sequence data for FM. The tone data is data representing an FM synthesis algorithm and is designated by a predetermined tone number. As shown in Fig. 3, the FM sequence data indicates event data (i.e., tone number, pitch (e.g., pitch), sound field, etc.) and duration data (event interval). It is composed of

The waveform data of the PCM sequence data is recorded audio data, while the FM sequence data is data for synthesizing a musical sound by controlling an FM sound source. Therefore, when PCM sequence data is used, real sounds can be reproduced for both music and voice, and the amount of data is large, and thus the use capacity of the memory is larger than that of FM sequence data. On the other hand, the FM sequence data is suitable for realistically reproducing the sound of musical instruments and the like, and there is an advantage that the amount of data can be used at least, since the amount of data is small.

In the column of data 4, image sequence data is described. This image sequence data is composed of image data in a predetermined format (e.g., Joint Photograph Experts Group (JPEG) standard, etc.) and sequence data for image display. The image data is designated by a predetermined image number. As shown in Fig. 3, the image display sequence data is composed of event data (i.e., picture number, display time, display type, etc.) and duration data (shows event intervals).

In addition, each dictionary is constituted by a plurality of dictionary items as described above, but in addition, it is possible to jump the access destination to a predetermined dictionary item of the dictionary from a sentence having a different dictionary link to which each dictionary is linked. A predetermined link address may be given to each dictionary item.

Further, when the search result information is returned to the mobile phone 1, the reproduction data included in the search result information is reproduced on the mobile phone 1 side, so that the mobile phone 1 has a predetermined data exchange format. This is a format that SMAF specification Ver.3.06, Yamaha Co., Ltd., searched on October 18, 2014, and Internet <URL: http: //smaf.yamaha.co.jp> already disclosed by the present applicant. It is expanded to respond to the pronunciation of a human voice.

This Synthetic Music Mobile Application Format (SMAF) is a data format specification for representing multimedia contents in portable terminals.

Here, SMAF will be described with reference to FIG. 4.

In Fig. 4, reference numeral 100 denotes an SMAF file, and a group of data called chunks has a basic structure. The chunk consists of a fixed length (8 byte) header portion and an arbitrary length body portion, and the header portion is further divided into a 4-byte chunk ID and a 4-byte chunk size. The chunk ID is used as an identifier of the chunk, and the chunk size indicates the length of the body portion. The SMAF file 100 itself has a chunk structure as well as the various data contained therein.

As shown in FIG. 4, the SMAF file 100 includes one or more track chunks 102 including contents information chunks 101 in which information for management is described and sequence data for an output device. 108). Sequence data is data which defined and expressed the control content with respect to an output device with time. All sequence data included in one SMAF file 100 are defined as starting playback at the same time at time O, and as a result, all the sequence data are reproduced in synchronization.

Sequence data is represented by a combination of events and durations. An event is data indicating control contents of an output device corresponding to sequence data, and duration data is data indicating an elapsed time between the event and the event. Although the processing time of the event is not actually 0, it is substantially 0 in SMAF, and the passage of time is expressed in duration. The time at which an event is executed can be uniquely determined by integrating the duration from the beginning of the sequence data. In principle, the processing time of an event does not affect the processing start time of the next event. Thus, successive events with a duration of 0 are interpreted as being executed simultaneously.

SMAF includes an FM sound source device that pronounces with control data equivalent to MIDI, a PCM sound source device that plays back PCM data, and a display device such as an LCD (Liquid Crystal Display) that displays text or an image as an output device. It is defined.

Corresponding to each output device defined as the track chunk described above, score track chunks 102 to 105, PCM audio track chunks 106, graphics track chunk 107, and master track chunk 108 are provided. Here, except for the master track chunk 108, the score track chunks 102 to 105, the PCM audio track chunk 106, and the graphics track chunk 107 can be described up to 256 tracks, respectively. have.

In the example of FIG. 4, the score track chunks 102 to 105 store sequence data for executing playback processing in the FM sound source device (sound source 111), and the PCM track chunk 106 stores a PCM sound source device (PCM). The decoder 112 stores the wave data, such as Adaptive Differential Pulse-Code Modulation (ADPCM), MPEG Audio Layer 3 (MP3), TwinVQ, etc., which are pronounced by the decoder 112, in an event format. Image data, text data, and sequence data for reproducing them on the display device (LDC display 113) are stored. The master track chunk 108 also stores sequence data for controlling the SMAF sequencer itself.

SMAF has the above-described format and contains various sequence data such as MIDI equivalent data (music data), PCM audio data, text and image display data, and all these sequences can be synchronously reproduced. However, since expressing a human voice is not particularly defined, in the present embodiment, the conventional SMAF is expanded as follows.

That is, as shown in FIG. 5, the HV (Human Voice) track chunk (which expands the SMAF file 100 and stores voice reproduction sequence data for reproducing voice (human voice) using a predetermined sound source) h4) is further provided. This audio reproduction sequence data is a duration for specifying an audio reproduction event for instructing the reproduction of the audio based on the pronunciation data indicating the formant parameter and an elapsed time from the audio reproduction event preceding the timing at which the audio reproduction event is executed. Using data as a set, these are arranged in chronological order.

By the above-described extension, similarly to the reproduction using the conventional SMAF file 100, by simultaneously starting the reproduction of each sequence data, each data can be reproduced synchronously on the same time axis.

As the audio reproduction event, any of the following three kinds of reproduction instruction information may be used.

(1) Text descriptive information consisting of a character string indicating how to read synthesized speech and a rhyme symbol specifying a speech expression (such as an accent).

(2) Phoneme description type information consisting of phoneme information indicating synthesized speech and rhyme control information.

(3) Formant frame description type information composed of formant parameters for each frame time indicating the reproduced sound.

Text description type (referred to as TSeq type) is a format for describing a voice to be pronounced by text notation, and a symbol (rhyme symbol) indicating a voice expression such as a character string and an accent by a character code of each language. Include. During playback on the mobile phone 1 side, as shown in Fig. 6A, the middle ware process converts the sequence data of the TSeq type into the PSeq type (by the first converting process). Next, the PSeq type is converted to the FSeq type (by the second converting process) and output to the speech synthesis device.

The first converting process from the TSeq type to the PSeq type includes a character string (for example, Japanese hiragana and katakana texts), a rhyme symbol, and information representing a pronunciation not dependent on the language corresponding to the language. It is executed by referring to a first dictionary that stores (phoneme) and rhyme control information for controlling rhyme. On the other hand, the second converting process for converting from PSeq type to FSeq type stores each phoneme and corresponding formant parameters (parameters for forming each formant, such as formant frequency, bandwidth, and level). It is executed by referring to one second dictionary, and the formant parameter as a result of the conversion is changed based on the rhyme control information.

Phoneme description type (PSeq type) describes information about a voice to be pronounced in a format similar to the MIDI event defined by SMF (Standard MIDI File). This phoneme description is based on phoneme units that are not language dependent. do. As shown in Fig. 6B, the delivery server 2 converts the TSeq type pronunciation data retrieved from the dictionary stored in the dictionary database 2b into the PSeq type by the first converting process. When the PSeq type pronunciation data is reproduced by the cellular phone 1, the PSeq type data file is converted into the FSeq type and output to the speech synthesis device by the second converting process executed as the middleware process.

The formant frame description type (FSeq type) is a format in which a formant parameter is expressed as a frame data string. As shown in Fig. 6C, the delivery server 2 executes a series of conversion processing from TSeq type to first conversion process to PSeq type to second conversion process to FSeq type. Further, the third converted process, which is the same process as the normal speech analysis process, may be performed on the sampled waveform data, thereby creating FSeq-type data. At the time of reproduction on the cellular phone 1, a file of a given FSeq type can be output to a speech synthesis device as it is and reproduced.

Next, the contents of the HV track chunk h4 will be described in detail with reference to FIG.

As shown in Fig. 7, each HV track chunk h4 has a format type indicating the type of audio reproduction sequence data included in this chunk in the three formats described above, and the language being used. Data specifying language types and timebases each indicating a type is described.

22 shows an example of a format type.

23 shows an example of a language type. Although only Japanese (0x00; 0x represents hexadecimal) and Korean (0x02) are shown here, other languages such as Chinese and English can also be defined.

The timebase defines the reference time of the duration and gate time in the sequence data chunks included in this track chunk. 24 shows an example of this time base, where the time value (for example, 20 msec) can be changed as appropriate.

Next, the details of the data of the above three format types will be described.

(a) TSeq type (Format type: 0x00)

As described above, this format type is a format using a sequence representation (TSeq: Text Sequence) by text notation, and includes sequence data chunks h5 and n (where n is an integer of 1 or more) TSeq data chunks (TSeq #). 00 to TSeq # n) (h6, h7, h8) are included (see Fig. 7). The audio reproduction event (note-on event) included in the sequence data is instructed to reproduce the data included in the TSeq data chunk.

(a-1) Sequence data chunk

The sequence data chunk h5 includes sequence data in which the combination of duration and event is arranged in chronological order, similar to the sequence data chunk in SMAF. 8A shows the configuration of sequence data, where duration represents the time between events. The leading duration (Duration 1) represents the elapsed time from the time (0). 8B illustrates a relationship between a duration and a gate time included in the note message when the event is a note message, where the gate time indicates a pronunciation time of the note message. The structure of the sequence data chunks shown in Figs. 8A and 8B is also the same for the sequence data chunks of the PSeq type and the FSeq type.

The events supported by this sequence data chunk include the following three events. In addition, the initial value described below is a default value when there is no event designation.

(a-1-1) Note message `` 0x9n kk gt ''

Here, "n" represents a channel number (0x0 [fixed]), "kk" represents a TSeq data number (OxO0 to 0x7F), and "gt" represents a gate time (1 to 3 bytes).

The note message is a message for interpreting the TSeq data chunk designated by the TSeq data number kk in the channel designated by the channel number n to start pronunciation. Note that the note message whose gate time is &quot; O &quot; is not pronounced.

(a-1-2) Volume `` 0xBn 0x07 vv ''

Here, "n" represents a channel number (OxO [fixed]), and "vv" represents a control value (OxO0 to Ox7F). In addition, the initial value of a channel volume is "Ox64."

In addition, the volume is a message for designating the volume of the designated channel.

(a-1-3) Fan (panport) `` 0xBn 0x0A vv ''

Here, "n" represents a channel number (OxO [fixed]), and "vv" represents a control value (0xO0 to 0x7F). In addition, the initial value of a fan pot is "0x40 (center)".

In addition, the pan message is a message for designating a stereo sound field position of a designated channel with respect to a using device having a stereo sound system.

(a-2) TSeq data chunks (TSeq # 00 to TSeq # n)

TSeq data chunks (h6, h7, h8, etc.) are information for speech synthesis, which includes information about language or character code, setting of pronunciation sounds (according tones), and information to be read (by speech synthesis). Format, described by HV-Script.

(b) PSeq type (Format type: OxO1)

The PSeq type is a format type that uses phoneme sequence representation (PSeq: Phoneme Sequence) in a format similar to MIDI events. This format lets you describe phonemes, so it's not language dependent. In addition, since the phoneme can be represented by character information indicating pronunciation, it is possible to use, for example, an ASCII code that is common among many languages.

As shown in Fig. 7, the PSeq type includes a setup data chunk h9, a dictionary data chunk h10, and a sequence data chunk h11, and includes a channel designated as a voice reproduction event (note message) in the sequence data. Instructs reproduction of phoneme and rhyme control information.

(b-1) Setup data chunk (optional)

This is a chunk that stores the timbre data of the sound source portion, and stores an array of exclusive messages. In this embodiment, the HV timbre parameter registration message is stored as an exclusive message.

The HV timbre parameter registration message is, for example, in the format "0xF0 Size 0x43 0x79 0xO7 0x7F 0x01 PCdata ... 0xF7", where "PC" represents a program number (0x02 to 0x0F), and "data" represents an HV timbre parameter. Indicates. In this message, the HV timbre parameters of the corresponding program number "PC" are registered.

25 shows an example of the HV timbre parameters.

As shown in Fig. 25, the HV timbre parameters include pitch shift amounts, formant frequency shift amounts, formant level shift amounts, and operator waveforms for formants from first to nth (n is an integer of 2 or more). Selection information is included. The mobile phone 1, which is a user device, stores a preset dictionary (above "second dictionary") that describes each phoneme and its corresponding formant parameters (i.e., formant frequency, bandwidth, level, and the like). The HV timbre parameter defines the shift amount relative to the parameter stored in this preset dictionary. Thereby, the same shift is performed with respect to all the phonemes, so that the nature of the synthesized voice can be changed.

In addition, with this HV timbre parameter, a timbre of numbers corresponding to 0x02 to 0x0F (that is, the numerical value of the program number) can be registered.

(b-2) Dictionary data chunk (optional)

The chunk stores dictionary data including language data, for example, difference data compared with the preset dictionary, phoneme data not defined in the preset dictionary, and the like. This makes it possible to synthesize voices with different personalities.

(b-3) Sequence Data Chunk

Similar to the sequence data chunk described above, the sequence data includes sequence data in which a combination of durations and events are arranged in chronological order.

The events (or messages) supported by the sequence data chunk h11 of this PSeq type are listed below. The reader ignores only these messages. In addition, the desired setting value described below is a default value when there is no event designation.

(b-3-1) Note message `` 0x9n Nt Vel Gatetime Size Data ... ''

Here, "n" is channel number (0x0 [fixed]), "Nt" is note number (absolute value note designation: 0xO0 to 0x7F, relative value note designation: 0x80 to 0xFF), and "Vel" is velocity (velocity) ( 0x00 to 0x7F), "Gatetime" represents a gate time length (Variable), and "Size" represents the size (variable length) of the data portion.

This note message starts the pronunciation of the audio of the designated channel. Note that the MSB (Most Significant Bit) of the note number is a flag for switching the interpretation between "absolute value" and "relative value". Note bits are represented by seven bits other than this MSB. In addition, since the pronunciation of the sound is monaural only, when the gate time overlaps, the pronunciation process is performed as a post-deposition priority.

In addition, the data portion contains phonemes and rhyme control information (pitch band and volume) corresponding thereto, and has a data structure shown in FIG.

As shown in Fig. 26, the data portion is composed of the number n of phonemes (# 1), for example, individual phonemes (phonemes 1 to phoneme n) (# 2 to # 4) and rhyme control information described in ASCII codes. do. Rhyme control information includes pitch band and volume. Regarding the pitch band, the phoneme pitch band is divided into N sections defined by the number of phoneme pitch bands (# 5), and a set of phoneme pitch band positions and phoneme pitch bands as pitch band information specifying a pitch band in each section. (I.e., phoneme pitch band position 1 and phoneme pitch band 1 (# 6, # 7) to phoneme pitch band position N and phoneme pitch band N (# 9, # 10)) are described. . Further, the phoneme section is divided into M sections defined by the phoneme volume number (# 11) with respect to the volume, and a set of phoneme volume positions and phoneme volumes (i.e., phonemes) as volume information for designating the volume in each section. The volume position 1 and the phoneme volume 1 (# 12, # 13) to the phoneme volume position M and the phoneme volume M (# 15, # 16) are described.

9 is a diagram for explaining rhythm control information. Here, the case where the character information to pronounce is "お は よ う" ("ohayou") is shown as an example. In this example, N = M = 128 is set. In FIG. 9, the pronunciation section of the character information (ohayou) is divided into 128 (= N = M) sections, and the pitch and volume in each section are expressed by the above pitch band information and volume information to control the rhyme. .

FIG. 10 illustrates a relationship between a gate time length (Gatetime) and a delay time (# 0). As shown in Fig. 10, the actual pronunciation can be delayed by a delay time rather than the timing defined by the duration. In addition, in this embodiment, Gatetime = 0 is prohibited.

(b-3-2) Program Change `` 0xCn pp ''

Here, "n" represents a channel number (OxO [fixed]), and "pp" represents a program number (0x 00 to 0xFF). In addition, the initial value of a program number is set to 0x00.

The timbre of the channel specified by this program change message is set. Here, the set values of the channel numbers include "0xO0" (male preset timbre), "0x01" (female preset timbre), and "0x02" to "0x0F" (extended timbre).

(b-3-3) Control Change

This embodiment uses the following control change messages.

(b-3-3-1) Channel volume `` 0xBn 0x07 vv ''

Here, "n" represents a channel number (OxOO [fixed]), and "vv" represents a control value (OxO0 to Ox7F). In addition, the initial value of the channel volume is set to 0x64.

This channel volume message designates the volume of a predetermined channel, and aims to set the volume balance between the channels.

(b-3-3-2) Fan (fan port) `` 0xBn 0x0A vv ''

Here, "n" represents a channel number (OxOO [fixed]), and "vv" represents a control value (0x00 to 0x7F). The initial value of the fan port is set to 0x40 (center).

This message specifies the location of the stereo sound field of a given channel for a using device having a stereo sound system.

(b-3-3-3) Expression `` 0xBn 0x0B vv ''

Here, "n" represents a channel number (OxOO [fixed]), and "vv" represents a control value (0x00 to 0x7F). The initial value of this expression message is set to 0x7F (maximum value).

This message indicates a change in the volume specified by the channel volume in a given channel. This is used for the purpose of changing, for example, the volume in a piece of music.

(b-3-3-4) Pitch band `` 0xEn ll mm ''

Here, "n" represents a channel number (0x00 [fixed]), "ll" represents a band value LSB (0x00 to 0x7F), and "mm" represents a band value MSB (0x00 to 0x7F). The initial value of the pitch band is set to 0x40 for the MSB (or upper byte) and 0x00 for the LSB (or lower byte).

This message changes the pitch of a given channel up and down (i.e., up and down in frequency). The initial value of the change width (i.e., pitch band range) is ± 2 semitones, and when the combination of the above band values is OxO0 / 0xO0, the downward pitch band is maximum, and in the case of 0x7F / 0x7F, The pitch band is maximum.

(b-3-3-5) Pitch band sensitivity (0x8n bb)

Here, "n" represents a channel number (OxOO [fixed]), and "bb" represents a data value (Ox00 to 0x18). The initial value of this pitch band sensory is set to 0x02.

This message is to set the sensitivity of the pitch band of a predetermined channel, and the unit is semitone. For example, when bb = O1, it is ± 1 semitone (the change range is 2 semitones in total).

In this way, the PSeq type format is based on a phoneme unit expressed by character information indicating pronunciation, and describes voice information in a format similar to a MIDI event. The data size is larger than that of TSeq but smaller than that of FSeq.

As a result, similar to the MIDI standard, pitch and volume can be finely controlled on the time axis, and since it is described as a phoneme base, there is no language dependency, and the tone (quality) can be edited finely. In other words, in the present embodiment, voice control similar to the MIDI standard can be performed, which has the advantage of being easily mounted on a conventional MIDI device.

(c) Formant frame technology (FSeq) type (format type: 0x02)

This is a format in which formant parameters (that is, parameters such as formant frequency gain for generating each formant) are expressed as a frame data string. That is, since the formant of the voice to be pronounced within a certain time (frame) is constant, a sequence expression (FSeq: Formant) for updating the formant parameter (formant frequency or gain, etc.) corresponding to the spoken voice for each frame. Sequence). This instructs data reproduction of the FSeq data chunk specified by the note message included in the sequence data.

This format type includes a sequence data chunk h12 and n FSeq data chunks (FSeq # 00 to FSeq # n: h13, h14, h15, etc.).

(c-1) Sequence data chunk

The FSeq data chunks consist of FSeq frame data columns. That is, the speech information is cut out for each frame having a predetermined length of time (for example, 20 msec), and the formant parameters (formal frequency, gain, etc.) obtained by analyzing the speech data within each frame period are used for the speech data of each frame. A format expressed as a frame data string to indicate.

27 shows an example of a frame data column of the FSeq type.

In Fig. 27, # 0 to # 3 represent data for designating a number of formant waveforms (i.e., sine wave, square wave, and the like) used for speech synthesis (n in this embodiment). # 4 to # 11 represent parameters defining n formants by a combination of formant levels (amplitudes) (# 4 to # 7) and center frequencies (# 8 to # 11). That is, # 4 and # 8 represent parameters defining the first formant waveform # 0, and # 5 and # 9 represent parameters defining the second formant waveform # 1. Likewise below, # 7 and # 11 represent parameters defining the nth formant waveform # 3. In addition, # 12 is a flag indicating unvoiced / voiced transition.

11 shows the level and center frequency of each formant waveform. In this embodiment, n formant data from the first to the nth formant are used. The parameters relating to the first to n-th formants for each frame and the parameters relating to the pitch frequency are supplied to the speech synthesis device provided in the cellular phone 1, and the speech synthesis output for each claim is as described above. Will be generated as output.

12 shows data of the body portion of the FSeq data chunk. Of the FSeq type frame data columns shown in FIG. 27, # 0 to # 3 are data for specifying the types of formant waveforms, and therefore, it is not necessary to specify each frame. As shown in FIG. 12, all data shown in FIG. 27 is set for the first frame, and only data after # 4 in FIG. 27 is set for the subsequent frame. As shown in Fig. 12, the body portion of the FSeq data chunk can reduce the total number of data.

As described above, the FSeq type is a format in which formant parameters (formant frequency, gain, etc.) are expressed as a frame data string, so that the audio can be reproduced by outputting the FSeq type file as it is to the speech synthesis device. Therefore, the processing side does not need to perform the conversion processing like the TSeq type or the PSeq type, and the CPU (central processing unit) only needs to perform frame update processing every predetermined time. In addition, the tone data (quality) can be changed by giving a fixed offset to the pronunciation data already stored.

Any type of file created as described above is transmitted to the cellular phone 1. Accordingly, the apparatus includes a pronunciation sequencer for supplying control parameters to the speech synthesis device at a timing defined by the duration included in the sequence data, and a speech synthesis device for reproducing and outputting speech based on the control parameters supplied from the pronunciation sequencer. In the mobile phone 1, which is the user device, the audio is synchronously reproduced together with other information (meaning information, information such as a related sound or image).

Next, the present embodiment will be described below by using audio description sequence data of text description type by HV-Script as described above.

First, the pronunciation data (except the pronunciation sequence data) by HV-Script will be described in detail.

For example, an example of pronunciation data by HV-Script, "Ka3 sa＿k ほ 554 い '4 ね $ 2-", adds a predetermined innation to a sentence called "Kar'a が ほ し い ね-" and synthesizes speech. It is a technique by HV-Script. The symbols "'", "^", "＿", "$", and the like described in this example are rhyme symbols indicating the type of intonation to be added to the character (Kana character), and subsequent characters of the rhyme symbol (the If there is a numerical value immediately after, a predetermined accent is added to the character following the numerical value.

13A shows the meaning of each symbol (typical example) by HV-Script.

That is, the symbol "'" raises the pitch from the dark (see ① in FIG. 13B), the symbol "^" raises the pitch during pronunciation (see ③ in FIG. 13C), and the symbol "＿" lowers the pitch from the dark (FIG. (2) in 13b), and the symbol "$" means synthesizing the speech as in lowering the pitch during pronunciation (see (4) in FIG. 13C).

In addition, when a numerical value is added immediately after the symbol, the numerical value designates the change amount of the accent to add. For example, in the phrase "か ＿3 さ が", the first letter "か" is pronounced at the standard pitch, but the pitch is lowered by an amount of 3 in the next "さ", and in the next "が" It means to pronounce at the lowered pitch.

As described above, when HV-Script adds an accent to a character included in a pronounced word, the syntax is the same as that described by adding a rhyme symbol (a numerical value indicating the amount of variation in the tonation) immediately before the character. It is. In addition, although the above description has been described only for the symbols for controlling the pitch, the symbols for controlling the sound intensity, speed, sound quality, and the like can also be used.

For example, the pronunciation data "と っ 'て も, S54' い / い て $ ん ＿ き / で す", which corresponds to the pronunciation data "と て も い い 天氣 で す ね", the English translation of "It's very fine, isn't it?" "S" in the symbol to be "S54" contained in the word "2-2" is one of the control characters for changing the speed, and functions to change the speech speed after the control characters. "54" following this control character "S" is a numerical value indicating a speed, and since the initial value is 50, it instructs to increase the speed only by the amount of 4 rather than the initial value. Once the speech speed is changed by this control letter S, the same speed is maintained until the next speed change.

In addition, "/" is one of the accent shift clear symbols (that is, the symbol for returning the changed accent), and returns the pitch changed to the rhyme symbol to zero. In addition, the pitch or volume changed by the rhyme symbol is controlled so that the pitch or volume continues until a symbol indicating a sentence division such as "," or "。" (hereinafter referred to as a "phrase separator") occurs. . "*" Is a symbol for lowering the pitch and volume in the second half of the next character, and "-" is a symbol for instructing to extend and pronounce the previous syllable.

In addition, "2" in the phrase "ner-2 *-" relates to the immediately preceding long sign "-" and instructs to double this long sound. In other words, "ner-2 *-" extends the pronunciation period of "ner" by a total of three amounts, and instructs "*" to finally lower the pitch and volume (by the amount of one). The symbols "'" and "$" are as described above.

As described above, the pronunciation data by HV-Script, which is one of the data indicating the formant parameter, can be pronounced more naturally with a relatively small amount of information, and therefore, it is suitable for applications such as pronounced translation results. In addition, the HV-Script described here is suitable for speech synthesis in Japanese, and the above-described PSeq type or FSeq type may be used for speech synthesis in other languages.

Next, the above formant and formant parameters will be described in detail.

The formant has the form shown in FIG. 14 and is specified by various parameters (namely, formant parameters) such as formant frequency, formant level, and formant bandwidth. Therefore, the number of formants included in the human voice, the frequency, amplitude, and bandwidth of each formant are important factors that determine the characteristics of the voice, and are greatly influenced by the characteristics, physique, age, etc. of the human voice. Different.

However, even if someone speaks, the word "あ" is pronounced "あ" and the word "い" is pronounced "い", so the same word sounds the same. That is, as the human voice, a characteristic formant combination is determined for each type of word to be pronounced. If the formants are largely divided according to their type, they are classified into voiced formants having pitch information for synthesizing voiced sounds and unvoiced formants having no pitch information for synthesizing unvoiced sounds.

Here, the voiced sound refers to a voice in which the vocal cords vibrate when uttered, for example, a vowel or a half vowel, and a voiced voice used in Japanese "ba line", "ga line", "ma line", "ra line", and the like. Consonants are included. In addition, the unvoiced voice refers to a voice in which the vocal cords do not vibrate when uttered, and includes, for example, consonants used in Japanese "ha", "ka", "sa", and the like. One phoneme is composed of a plurality of formants, as shown in FIG.

Therefore, a formant for each phoneme of a particular person's pronunciation is registered in advance in the mobile phone 1, and the formant parameters (i.e., formant frequency, formant level, The formant bandwidth) and the basic waveform forming the formant by speech synthesis according to the rhyme symbols of the text description type HV-Script, or by changing the speech control information in the phoneme description type described above. In addition, voices with various intonations can be pronounced.

Further, when the control unit 2a constituted by the memory and the CPU (central processing unit) and the like in the delivery server 2 performs the translation process, a process for translating the text to be translated and returning the translation result information thereof The function is realized by loading and executing a program in sequence into a memory. Further, in the case of performing the dictionary search process, the function is realized by loading and executing a program consisting of a process procedure for searching the dictionary with the given headword as a search key and returning the search result information to the memory.

In addition, an input device, a display device, and the like (all of which are not shown) are connected to the delivery server 2 as peripheral devices. Here, the input device means an input device such as a keyboard or a mouse, and the display device means a cathode ray tube (CRT) or a liquid crystal display device.

In addition, the dictionary database 2b is comprised by nonvolatile storage devices, such as a hard disk and a magneto-optical disk, and this may be installed in the delivery server 2, or the external which can be accessed from the delivery server 2; Or you may install in another server.

Next, the configuration and operation of the cellular phone 1 according to the present embodiment will be described in detail.

FIG. 15 is a block diagram showing the schematic configuration of the mobile telephone 1. FIG.

In addition, the present invention can be applied to not only a cellular phone (Cellular Phone, etc.) but also a PHS (Personal Handyphone System), a wireless digital terminal (PDA) capable of wireless communication, and the like.

In Fig. 15, reference numeral 11 denotes a CPU (central processing unit), and controls the operation of each unit of the mobile phone 1 by executing various programs.

Reference numeral 12 denotes a communication unit, demodulates a signal received by the antenna 12a included in the communication unit 12, modulates a signal to be transmitted, and supplies it to the antenna 12a.

The CPU 11 decodes the signal from the delivery server 2 demodulated by the communication unit 12 according to a predetermined protocol, and the above-described first method for text reproduction type speech reproduction sequence data by HV-Script. The convert process and the second convert process are executed to generate a frame data string consisting of formant parameters. The communication unit 12 supplies the signal to the display sequencer 21a or the pronunciation sequencer 16a in accordance with whether the data in the received file is display data or pronunciation data.

Reference numeral 13 denotes a voice processing unit. That is, the voice signal via the telephone line demodulated by the communication unit 12 is decoded by the voice processing unit 13, and thus the corresponding voice is pronounced by the speaker 14. On the other hand, the speech signal collected by the microphone 15 is digitized and compressed and encoded by the speech processor 13. Thereafter, it is modulated by the communication unit 12 and transmitted from the antenna 12a to a base station (not shown) of the cellular phone network. The speech processing unit 13 compresses / decodes speech data with high efficiency by, for example, Code Excited Linear Predictive Coding (CELP) or Adaptive Differential Pulse-Code Modulation (ADPCM).

Reference numeral 16a indicates a sounding sequencer, receives sounding control sequence data which instructs the sound system to pronounce a predetermined sound or sound at a predetermined timing, and thus controls a sound source 16b having a speech synthesis function.

Reference numeral 16b denotes a sound source having a speech synthesis function, which is composed of an unshown speech synthesis unit, an FM sound source device, and / or a PCM sound source device. The sound source 16b having the speech synthesis function not only executes the speech synthesis processing described below, but also reproduces the music data selected as the ringing tone and sounds the sound from the speaker 17. In addition, the detail of the structure of this speech synthesis unit is mentioned later. The FM sound source device may be a WT (Wave Table) sound source, a high frequency synthesized sound source, or a square wave sound source, and the PCM sound source may be an MP3 decoder or the like.

Reference numeral 18 denotes an operation unit, which is an input means for detecting input from various buttons (not shown) or other input device including alphanumeric buttons provided on the housing of the cellular phone 1.

Reference numeral 19 denotes a RAM (Random-Access Memory), which includes a work area of the CPU 11, a storage area of downloaded music data and accompaniment data (these are used for reproducing an incoming melody, etc.), received e-mail, and the like. The mail data storage area in which the data of the data is stored, the area for storing the translation result information received from the delivery server 2, the search result information, and the like are set.

Reference numeral 20 denotes a read-only memory (ROM), which controls transmission and reception of various telephone function programs for controlling outgoing and incoming calls executed by the CPU 11, programs for assisting music reproduction processing, and e-mail. The mail transmission / reception function program and the program assisting the voice synthesis processing are stored, and various data such as contents and music data of the first dictionary and the second dictionary described above are stored.

Reference numeral 21a indicates a display sequencer, and receives display control sequence data instructing display of a predetermined image or text on the display portion 21b at a predetermined timing to control the display portion 21b.

The display portion 21b is constituted by a liquid crystal display (LCD), and under the control of the CPU 11 and the display sequencer 21a, display of desired text or images and display according to the operation of the operation unit 18 Is done.

Reference numeral 22 denotes a vibrator that notifies the user of the incoming call by vibrating the main body of the cellular phone 1 at the time of incoming call.

In addition, the above-described functional blocks are interconnected via the bus 30, and thus data and commands are transmitted and received.

Next, the configuration of the speech synthesis unit included in the sound source 16b having the speech synthesis function will be described in detail.

16 shows a schematic configuration of a speech synthesis unit.

The speech synthesis unit shown in FIG. 16 has a large number of formant generators 40a to 40m and one pitch generator 50. The formant generators 40a to 40m are configured based on formant parameters (formant frequency for forming each formant, formant level, etc.) and pitch information output from the pronunciation sequencer 16a. Generate a formant signal. These formant signals are synthesized in the mixing section 60, thus generating predetermined phonemes. In addition, although each formant generating unit 40a to 40m generates a basic waveform that is a basis for generating a formant signal, for example, a waveform generator of a known FM sound source can be used for generating the basic waveform. It is available. In addition, the pitch generator 50 has a function of generating pitch (pitch) by a predetermined operation, and adds the pitch calculated to the phoneme generated only when the phoneme to be pronounced is voiced sound.

Next, with reference to FIG. 17, the structure of said formant generating part 40a-40m is demonstrated.

As shown in FIG. 17, each formant generator 40a to 40m includes a waveform generator 41, a noise generator 42, an adder 43, and an amplifier 44.

The waveform generator 41 generates one formant constituting one phoneme according to the formant frequency, formant fundamental waveform (sine wave, triangle wave, etc.) and waveform phase specified for each formant of each phoneme. Occurs. The noise generator 42 operates according to whether the formant generated by the waveform generator 41 is a voiced sound or an unvoiced sound. In the case of the unvoiced sound, noise is generated and supplied to the addition common electrode 43.

The adder 43 adds noise supplied from the noise generator 42 to the formant generated by the waveform generator. The output of this adder 43 is amplified by the amplifier 44 at a predetermined formant level and output.

Each formant generating unit 40a to 40m relates to one formant constituting the phoneme. In addition, a plurality of formants are synthesized with respect to one phoneme. Therefore, in order to generate one phoneme, it is necessary to generate a large number of formants constituting the phoneme and to synthesize them. As a result, as shown in FIG. 16, a large number of formant generating units 40a to 40m are provided.

Next, the operation of the mobile phone 1 and the translation server 2 according to the present embodiment configured as described above will be described in detail. In addition, here, the description is abbreviate | omitted about well-known operation | movement, such as the operation | movement at the time of origination and reception by a normal telephone function.

First, the operation of the mobile phone 1 and the translation server 2 in the case of performing translation is demonstrated with reference to the flowchart shown to FIG. 18 and FIG.

The user of the mobile phone 1 inputs texts to be translated (for example, "It's very fine, isn't it?"), And also translates a translation language (here, English to Japanese). By designating, a translation request including these information is transmitted (step S101).

The delivery server 2 repeatedly executes the determination step of step S201 until the translation request from the cellular phone 1 is received, and is in a waiting state. However, when the delivery server 2 receives the translation request from the cellular phone 1, The text to be translated is translated using the translation dictionary of the dictionary database 2b (step S202).

In the meantime, the cellular phone 1 repeatedly executes the determination step of step S101 until the translation result data is received, and enters the standby state.

The delivery server 2 converts the translated text into pronunciation data by HV-Script using the translation dictionary of the dictionary database 2b (step S203). Here, conversion is made to pronunciation data by HV-Script corresponding to the document, sentence, or word unit after translation.

Thereafter, the translation result information having the above-mentioned data exchange format including the text of the translation result and its pronunciation data is generated, and the translation result information is returned to the cellular phone 1 (step S204).

When the cellular phone 1 receives the translation result information from the delivery server 2, the determination result in step S102 becomes "Yes", and the flow advances to step S103 to store the received data in the RAM 19.

After that, the determination of step S104 is repeated until the user of the cellular phone 1 performs the operation of the predetermined key, and the standby state is entered.

When the user of the cellular phone 1 reproduces the translation result by operating a predetermined key, the determination result of step S104 becomes "Yes", and the flow advances to step S105.

In step S105, the CPU 11 reads the translation result information received from the delivery server 2 from the RAM 19, displays the contents of the text data included in the extra translation result information on the display unit 21b, The pronunciation data is speech synthesized in a sound source 16b having a speech synthesis function. Until the reproduction of the above translation result information is completed (that is, until the determination result of step S106 is YES), the text display of the translation result and pronunciation by the pronunciation data are executed.

In this way, the translation work is performed between the mobile phone 1 and the delivery server 2. Next, operations of the cellular phone 1 and the delivery server 2 in the case of using the pre-search function in the delivery server 2 will be described with reference to flowcharts shown in FIGS. 20 and 21.

First, the user of the mobile phone 1 specifies a type of dictionary (eg, English-Japanese dictionary) to be used by inputting text (eg, "Duck" of an English word) to search, and search containing the information. The request is sent (step S111).

The delivery server 2 repeats the determination of step S211 until it receives the search request from the mobile phone 1, but is in the waiting state. When the delivery server 2 receives the search request from the mobile phone 1, With the included headword as the search key, a search is performed on the designated headword using the English-Japanese dictionary of the dictionary database 2b (step S212).

In the meantime, the cellular phone 1 repeats the determination of step S112 until it receives the search result data (that is, the search result information), and is in the standby state.

When the retrieval process is completed, the delivery server 2 is associated with text (for example, "duck" which is a translation of "Duck"), pronunciation data of the reading method, and heading ("Duck") indicating the semantic information of the search result. The search result information having the above-described data exchange format including the duck's crying data as the voice to be sounded and the duck's image data as the related information is returned to the cellular phone 1 (step S213).

When the cellular phone 1 receives the above search result information from the delivery server 2, the determination result of step S112 becomes "Yes", and the flow advances to step S113 to store the data in the RAM 19. .

After that, the determination of step S114 is repeated until the user of the cellular phone 1 performs a predetermined key operation, and is in the standby state.

If the user of the cellular phone 1 operates the predetermined key, the determination result of step S114 becomes "Yes", and the flow advances to step S115.

The CPU 11 reads the information designated by the user from the RAM 19 among the search result information received from the delivery server 2 in step S115, and executes the reproduction. The sequence data contained in the search result information and designated by the user are supplied to the corresponding display sequencer 21a and pronunciation sequencer 16a, which are appropriately controlled to produce the desired display and audio output. For example, when the user of the cellular phone 1 searches the English word “Duck” using the English-Japanese dictionary, when the user designates the text display of the search result, the “duck” of the text is displayed on the display unit 21b, If the pronunciation is designated, the speech synthesis is performed in the sound source 16b having the speech synthesis function, and the pronunciation is performed. Further, if the user designates the reproduction of the associated voice, the duck's cry is reproduced as the associated voice in the sound source 16b having the speech synthesis function, and if the reproduction of the associated image is instructed, the display unit 21b is related. An image of a duck is displayed as an image to be used. When the simultaneous reproduction of the data is instructed by the user of the cellular phone 1, the data (i.e., text, first) is controlled by the control of the pronunciation sequencer 16a and the display sequencer 21a. And second pronunciation data and image data) are synchronously reproduced.

The above processing of steps S115 and S116 is repeated until the reproduction of the designated search result information is completed.

In addition, the flow and steps used in the above description are examples, and therefore, the present invention is not limited to the above processing flow.

As mentioned above, although the preferred embodiment of this invention was described with reference to drawings, the specific structure of this invention is not limited to this embodiment, Therefore, the structure etc. which do not deviate from the summary of this invention are included. For example, the contents of the dictionary database 2b of the delivery server 2 may be stored in the mobile phone 1, and a translation function and a dictionary search function may be provided in the mobile phone 1. In this case, the cellular phone 1 does not need to communicate with the delivery server 2 when performing translation or dictionary search.

As described above, the present invention has various effects and technical features and will be briefly described below.

(1) In the present invention, since the translation result can be reproduced by voice by the portable terminal device, the user can recognize and grasp the translation result by hearing. The pronunciation data returned from the translation means (e.g., an external delivery server, etc.) indicates a formant parameter. In this case, since the present invention is not in the form of a reply of a voice signal as in the prior art, it is necessary to The reception of the pronunciation data of a does not require a large transmission capacity.

(2) In the present invention, the text displaying the translation result can be displayed by the display means provided in the portable terminal device, so that the user can recognize the translation result also by time.

(3) In the present invention, the semantic information retrieved as heading information as a retrieval key is pronounced as a voice, and the user can recognize this semantic information by hearing. The search result information returned from the dictionary database indicates a formant parameter, and when the search result information is received from an external device of the portable terminal device, the required transmission capacity can be at least possible.

(4) In the present invention, the user can view not only the semantic information of the search result, but also the associated image. Furthermore, not only the voice of the text displaying the semantic information by the pronunciation means but also the sound associated with the headword is pronounced, so that the user can obtain not only the meaning of the headword which is the search key but also the rich information related thereto.

Claims (17)

Transmitting means 12 for transmitting text data input in a predetermined language to a predetermined server 2 through a communication line; Receives translation result information including a character string representing a translation result of translating input text data into another language and pronunciation data for controlling the intonation of the reading method, from the server through a communication line, and the pronunciation data is a Forman Receiving means 12, which is data representing a network parameter; Generating means for generating a formant parameter in accordance with the pronunciation data included in the translation result information; And And a pronunciation means (16a, 16b) for accentuating and pronouncing a voice corresponding to the character string, in accordance with the formant parameter. A portable terminal apparatus according to claim 1, further comprising display means (21a, 21b), for displaying text data corresponding to the character string. The method of claim 1, wherein the server has a dictionary database in which heading information and semantic information are associated, and the character string is semantic information obtained by searching for the heading information corresponding to text data input from a dictionary database. Mobile phone. The portable device according to claim 3, further comprising display means (21a, 21b), and displaying text data representing the semantic information included in the translation result information and image data corresponding to the headword information. Terminal device. The mobile terminal device according to claim 4, wherein the translation result information further comprises data indicating a sound related to the headword information. delete Translates text data input in a predetermined language into another language, and generates translation result information including a character string representing the translation result and pronunciation data for controlling the intonation of the reading method, wherein the pronunciation data is a formant Translation means 11, 19, 20, which are data representing parameters; Generating means for generating a formant parameter in accordance with the pronunciation data included in the translation result information; And And a pronunciation means (16a, 16b) for pronouncing the accented voice corresponding to the character string according to the formant parameter. 8. A portable terminal device according to claim 7, further comprising display means (21a, 21b) to display text data for displaying a translation result included in the translation result information. 10. The method of claim 7, further comprising a dictionary database in which headword information and semantic information are associated, and searching for headword information corresponding to text data input from the dictionary database to generate semantic information obtained as the character string. A portable terminal device. The portable apparatus according to claim 9, further comprising display means (21a, 21b), and displaying text data representing the semantic information included in the translation result information and image data corresponding to the headword information. Terminal device. The portable terminal apparatus according to claim 10, wherein the translation result information further comprises pronunciation data indicating a sound related to the headword information. delete 8. The method according to claim 1 or 7, wherein the sounding means has a plurality of formant generators 40, and synthesizes waveforms of specific formant frequencies generated by each formant generator. A portable terminal device. The text data input by the portable terminal device in a predetermined language is transmitted to the predetermined server 2 via a communication line, Receives translation result information including a character string representing a translation result of translating input text data into another language and pronunciation data for controlling the intonation of the reading method, from the server through a communication line, and the pronunciation data is a Forman Data representing the network parameters, A formant parameter is generated according to the pronunciation data included in the translation result information, And the accent is pronounced according to the formant parameter, and the voice corresponding to the character string is pronounced. Translates text data input in a predetermined language into another language, and generates translation result information including a character string representing the translation result and pronunciation data for controlling the intonation of the reading method, wherein the pronunciation data is a formant Data representing a parameter, A formant parameter is generated according to the pronunciation data included in the translation result information, And the accented voice corresponding to the character string is pronounced according to the formant parameter. A storage medium for storing a program executed by a processing apparatus included in a portable terminal device, comprising: transmitting text data input in a predetermined language by the portable terminal device to a predetermined server 2 through a communication line, Receives translation result information including a character string representing a translation result of translating input text data into another language and pronunciation data for controlling the intonation of the reading method, from the server through a communication line, and the pronunciation data is a Forman Data representing the network parameters, Generate a formant parameter according to the pronunciation data included in the translation result information, And a program in which the accent is pronounced and the voice corresponding to the character string is stored in accordance with the formant parameter. A storage medium for storing a program executed by a processing apparatus included in a portable terminal device, comprising: translating text data input in a predetermined language into another language, and controlling an accent of a character string representing the translation result and a reading method thereof Generating translation result information including pronunciation data to be used, wherein the pronunciation data is data representing a formant parameter, A formant parameter is generated according to the pronunciation data included in the translation result information, And a program configured to pronounce an accented voice corresponding to the character string in accordance with the formant parameter.

Images