KR100612780B1 - Speech and music reproduction apparatus - Google Patents

Abstract

The sound and music reproducing apparatus according to the present invention is associated with the script data 11, the user tone parameter 12, and the user phrase synthesis dictionary data 13, and also includes a dipole tone parameter 18 and a default synthesis dictionary. A middleware for reproducing a synthesized voice signal or the like based on the data 19, a sound source 20 and a speaker for reproducing a desired voice or music based on the synthesized voice signal. In addition, in the case of using an HV script in which various events are described as the script data, the desired waveform data, the music phrase data including the note information, and the formant frame data are appropriately combined and reproduced by the event type.

HV script, formant frame data, middleware, strings, speech converter

Description

Voice and music playback device {SPEECH AND MUSIC REPRODUCTION APPARATUS}

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing the configuration of an audio reproducing apparatus according to a first embodiment of the present invention.

2 is a diagram showing an allocation relationship between a pronunciation unit and a phrase ID;

3 is a diagram showing an example of the contents of phrase synthesis dictionary data;

4 is a diagram illustrating a format example of a SMAF file.

5 is a functional block diagram illustrating an example of an HV authoring tool.

Fig. 6 is a block diagram showing the configuration of a portable communication terminal to which an audio reproducing apparatus is applied.

7 is a flowchart showing a process of creating user phrase synthesis dictionary data;

8 is a flowchart showing a reproduction process of user phrase synthesis dictionary data.

9 is a flowchart showing a process of creating an SMAF file.

10 is a flowchart showing a reproduction process of an SMAF file.

Fig. 11 is a block diagram showing the construction of an audio / music reproducing apparatus according to a second embodiment of the present invention.

Fig. 12 is a diagram showing an example of the allocation relationship between each event, waveform data, and music phrase data.

Fig. 13 is a flowchart showing an audio / music reproducing process according to the second embodiment.

Fig. 14 is a block diagram showing the configuration of a mobile telephone including a voice and music reproducing apparatus according to the second embodiment.

Fig. 15 is a block diagram showing the construction of an audio / music reproducing apparatus according to a third embodiment of the present invention.

FIG. 16 is a flowchart showing the operation of the audio / music reproducing apparatus shown in FIG. 15; FIG.

<Explanation of symbols for the main parts of the drawings>

1: voice playback device

11: script data

12: User Voice Parameter

13: User Phrase Synthesis Dictionary Data

14: application software

15: middleware API

16: converter

17: driver

18: Default Voice Parameter

19: Default Synthetic Dictionary Data

20: sound source

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech and music reproducing apparatus, and more particularly, to a speech and music reproducing apparatus that reproduces a specific language by speech synthesis and converts text information into speech or music.

Background Art Conventionally, a string-to-speech apparatus for converting character string information such as electronic mail into voice and outputting the same has been devised. Japanese Patent Laid-Open Publication No. 2001-7937 shows an example of a conventional character string-to-speech apparatus, where the character string information is divided into sentence units, and the contents are displayed on the display while the audio is output.

In addition, a single phrase phrase can be generated by reproducing waveform data (or sampling data) created by sampling a music phrase or voice phrase, or by musical note information such as SMF (standard MIDI file) or SMAF (synthetic music mobile application file). A method of constructing the music phrase and reproducing the music phrase is known. For example, Japanese Patent Laid-Open Publication No. 2001-51688 discloses an electronic mail reading device capable of separating string information and musical sound information from electronic mail and pronunciation of each.

However, in the conventional string-to-speech apparatus, since the string information is divided into phrases or phrases and outputted as speech, the speech output is a set of sounds in an pronunciation unit (or character unit). The reproduction of the joints (connection parts) in the pronunciation unit gives the listener a sense of discomfort compared to the pronunciation of the spoken language. In other words, the conventional string-to-speech converter has a problem in that it is possible to output the sound by changing its tone with high-quality voice for the whole sentence, that is, cannot output a natural voice close to the human spoken language.

In order to solve the above-mentioned problems, for example, a method of pre-sampling audio for each sentence (hereinafter referred to as a phrase) is stored in advance as audio data, and outputted as a corresponding audio waveform during reproduction. do. However, with this method, it is necessary to increase the sampling frequency in order to improve the quality of the voice output, which causes the necessity of preserving a large amount of voice data, and thus a relatively storage capacity such as a portable telephone or cellular phone. In this limited apparatus, there was a technical difficulty.

In addition, in the conventional method of reproducing waveform data created by sampling a piece of music or voice, or a single piece of music data using note information such as SMF or SMAF, and reproducing the piece of music data, the timing of reproducing the piece of music or voice It is not described in the form of a text file, and therefore it is difficult to perform a combination of voice reproduction based on character string information, waveform data reproduction or music data reproduction as desired by the user.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a speech reproducing apparatus capable of reproducing and outputting a desired sentence (or phrase) composed of character string information or the like as a sound having a good quality. do.

In addition, another object of the present invention is a voice / music reproducing apparatus which enables a user to easily perform a combination of voice reproducing, waveform data reproducing, and music data reproducing, thereby reproducing voice and music faithful to the user's intention. To provide.

The audio reproducing apparatus according to the present invention stores a database of formant frame data corresponding to a predetermined pronunciation unit in advance as composition dictionary data, and stores the database in a character string arranged by arranging the pronunciation units. Given information, speech synthesis is performed using the synthesis dictionary data. Here, the formant frame data is replaced with arbitrary user phrase data, and when character string information is given, speech synthesis is performed using the synthesis dictionary data substituted with the user phrase data. The tone phrase parameter for processing formant frame data is added to the user phrase data. In addition, a predetermined data exchange format including user phrase data is used in speech synthesis. This data exchange format is a SMAF file format, for example, and can contain not only user phrase data but also various chunks and music reproduction information.

Specifically, the audio reproducing apparatus includes a default synthesis dictionary data for storing formant frame data corresponding to a predetermined pronunciation unit in advance, and a middleware application program interface (API) for replacing the formant frame data with user phrase data. ), Converter, driver, and sound source. As a result, a desired phrase made of character string information can be reproduced as a high quality voice, and the tone can be appropriately changed and reproduced.

The audio / music reproducing apparatus according to the present invention stores script data (i.e., HV script; HV-Script) that describes the pronunciation of a character or an instruction for reproducing previously stored pronunciation data. Based on the script data, a voice signal corresponding to the character is generated to generate a desired voice, and a pronunciation signal corresponding to the pronunciation data is generated to generate a desired voice or musical sound. Here, the pronunciation data is constituted by, for example, waveform data generated by sampling of voice or music, and a synthesized pronunciation signal is generated based on the waveform data. When the pronunciation data is used as musical data including musical note information, a musical tone signal corresponding to musical note information is generated based on the musical musical data. In addition, when the formant control parameter (or formant frame data) characterizing the pronunciation of the character is stored, an audio signal is generated based on the formant control parameter. The script data may be arbitrarily created by a user. In this case, the script data takes a predetermined file format created by text input.

Specifically, when various types of events described in the HV script are analyzed and the type of the event indicates waveform data, the waveform data is read and reproduced, and when the type of the event indicates music phrase data, A music phrase data playback process is performed. In this case, the note information is read and reproduced based on the time information in the music phrase data. In response to other events, the input character string is converted into formant frame strings using the synthesis dictionary data to perform voice synthesis. In this manner, the user can easily perform a combination of voice reproduction, waveform data reproduction, and music data reproduction.

<Example>

Embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Fig. 1 is a block diagram showing the construction of an audio reproducing apparatus according to the first embodiment of the present invention.

That is, the audio reproducing apparatus 1 shown in FIG. 1 includes an application software 14, a middleware application program interface 15, a converter 16, a driver 17, and default tone parameters. (default tone color parameter) 18, default composition dictionary data 19, and a sound source 20, script data 11, user tone parameter 12, And a user phrase composition dictionary data 13 (variable length) is input to reproduce speech.

The speech reproducing apparatus 1 is a method for reproducing speech based on formant composition by composite sinusoidal modeling (CSM) speech synthesis using a resource of a frequency modulation (FM) sound source. Is basically used. Also, in the present embodiment, user phrase synthesis dictionary data 13 is defined, and the audio reproducing apparatus 1 refers to the user phrase synthesis dictionary data 13 to perform user phrases in phoneme units on tone parameters. Assign. When user phrase synthesis dictionary data 13 is assigned to the tone color parameter as described above, the audio reproducing apparatus 1 substitutes a user phrase for a phoneme registered in the default synthesis dictionary data at the time of reproduction, and based on the substitution data. Speech synthesis. In addition, the above-mentioned "phoneme" is a minimum unit of pronunciation, and is composed of vowels and consonants in Japanese.

Next, the detailed configuration of the audio reproducing apparatus 1 will be described.

In Fig. 1, the script data 11 defines a data format for reproducing &quot; HV (human voice): voice synthesized by the above method &quot;. That is, the script data 11 includes a pronunciation string including an intonation symbol, event data for setting a sound to be pronounced, event data for controlling the application software 14, and the like. A data format for performing speech synthesis is shown, and a text input format is provided to facilitate manual input by a user.

The definition of the data format in the script data 11 is language dependent and can be defined in various languages. However, in the present embodiment, the definition in Japanese is exemplified.

The user phrase synthesis dictionary data 13 and the default synthesis dictionary data 19 are analyzed by sampling and analyzing the actual human sound in phonetic character units (for example, Japanese "あ", "い", etc.). The formant frequency, formant level, and pitch of the pair are extracted as parameters, and the parameters are previously associated with phonetic character units as formant frame data, and correspond to a database stored in phonetic character units. The user phrase synthesis dictionary data 13 is a database built in addition to the middleware, and the user can register the formant frame data arbitrarily with the database, so that the contents of the user phrase synthesis dictionary data 13 can be registered in the middleware API ( 15) it is possible to completely replace the contents of the default synthesis dictionary data 19. That is, the contents of the default synthesis dictionary data 19 can be completely replaced by the contents of the user phrase synthesis dictionary data 13. On the other hand, the default synthesis dictionary data 19 is a database built in the middleware.

As the user phrase synthesis dictionary data 13 and the default synthesis dictionary data 19, it is suitable to have two types, respectively, for male sound quality and for female sound quality. Although the audio output of the audio reproducing apparatus 1 changes with the period of each frame, the frame period of the formant frame data registered in the user phrase synthesis dictionary data 13 and the default synthesis dictionary data is, for example, 20 ms. It is set.

The user timbre parameters 12 and the default timbre parameters 18 are groups of parameters for controlling the sound quality in the audio output of the audio reproducing apparatus 1. That is, the user timbre parameter 12 and the default timbre parameter 18 are, for example, eight sets of formant frequency and formant change (i.e., user phrase synthesis dictionary data 13 and default synthesis dictionary data 19). Designation of the formant frequency, change amount from formant level), and basic waveforms for formant synthesis can be specified, and various tones can be produced accordingly.

The default tone color parameter 18 is a tone color parameter set previously set as a default value in the middleware. The user tone parameter 12 is a parameter that can be arbitrarily created by the user, and is stored and stored outside the middleware, and extends the contents of the default tone parameter 18 through the middleware API 15.

The application software 14 is software for reproducing the script data 11.

The middleware API 15 constitutes an interface between the application software 14 made of software, the converter 16 made of the middleware, the driver 17, the default timbre parameters 18, and the default synthesis dictionary data 19. do.

The converter 16 interprets the script data 11 and converts the script data 11 into a formant frame data string in which frame data is finally formed continuously using the driver 17.

The driver 17 generates formant frame data strings based on the pronunciation strings and default synthesis dictionary data 19 included in the script data 11, and interprets the tone color parameters to process the formant frame data strings. .

The sound source 20 outputs a synthesized pronunciation signal corresponding to the output data of the converter 16, and the synthesized pronunciation signal is output to the speaker for viewing.

Next, technical features of the audio reproducing apparatus 1 according to the present embodiment will be described.

The user timbre parameter 12 includes parameters for allocating phrase IDs stored in the user phrase synthesis dictionary data 13 for any pronunciation unit. Fig. 2 shows an example of the assignment between the pronunciation unit and the phrase ID. Here, the allocation relationship between the mora and the phrase ID is shown. In addition, in the case of Japanese, Mora means a beat, and corresponds to a kana character unit, for example.

By allocating phrase IDs for each pronunciation unit, the pronunciation unit specified in the user timbre parameter 12 is defined by referring to the user phrase synthesis dictionary data 13 instead of the default synthesis dictionary data 19. In addition, in the user tone color parameter 12, it is desirable to be able to specify an arbitrary number of pronunciation units in one tone color parameter.

The phrase ID assignment for each pronunciation unit in the user tone parameter 12 described above shows an example of the present embodiment, and other methods may be employed as long as they correspond to the pronunciation unit.

Next, the details of the user phrase synthesis dictionary data 13 will be described. 3 shows an example of the contents of the user phrase synthesis dictionary data 13. The user phrase synthesis dictionary data 13 stores formant frame data including eight sets of formant frequencies, formant levels, and pitches. The phrase "phrase" in FIG. 3 is a phrase that has a meaning or syllable integration, such as Japanese word "ohashi," for example, and the definition scale of this phrase does not need to be specifically defined. Arbitrary scales, such as syllables and sentences, may be used.

As a tool for creating the user phrase synthesis dictionary data 13, a conventional sound file (an extension file such as * .wav, * .aif, etc.) is analyzed and a form including eight sets of formant frequencies, formant levels, and pitches is included. It is necessary to mount an analysis engine for generating fragment frame data.

The script data 11 includes event data for instructing sound quality change, but the user timbre parameter 12 can be specified by this event data.

For example, as the description example of the script data 11, when Japanese hiragana and alphanumeric characters are used, "TJK12 MINAX1010" can be set. In this example, "K" represents event data specifying the default timbre parameter 18, and "X" represents event data specifying the user timbre parameter 12. "K12" designates arbitrary specific default timbre parameters from a plurality of kinds of default timbre parameters. In addition, "X10" designates the user timbre parameter shown in FIG. 2 from a plurality of types of user timbre parameters.

In the above example, the synthesized voice to be reproduced is "Minasan Koninichi." Here, "mini-san" is a synthesized voice reproduced with reference to the default timbre parameter 18 and the default synthesis dictionary data 19. In addition, "Konnichiha" and "鈴木 で す" are the user timbre parameter 12 and the user phrase synthesis dictionary. Synthesized speech reproduced with reference to the data 13. In other words, the phrase "み な さ ん" is a synthesized speech obtained by reading formant frame data corresponding to four phonemes of "み", "な", "さ", and "ん" from the default synthesis dictionary data 19. In addition, the phrases "konichiha" and "鈴木 で す" are synthesized voices which read and reproduce formant frame data of each phrase unit from the user phrase synthesis dictionary data 13.

In the example shown in FIG. 2, although three pronunciation units of "A", "I", and "KA" are illustrated, this is not specifically defined as long as it is a character and a symbol which can be represented by text. In the above example, the phrase "Konichiha" appears and is pronounced by the pronunciation symbol "あ" following "X10", and the phrase "鈴木 で す" appears and is pronounced by the pronunciation symbol "ka". For this reason, when the original "A" is pronounced after the above-described pronunciation example, the symbol for returning the reference destination to the default synthesis dictionary data 19 (for example, "XO0" and "O" is predetermined). Number, etc.).

Next, the data exchange format of the music reproduction sequence data (SMAF: synthetic music mobile application format) used in the audio reproducing apparatus 1 according to the present embodiment will be described with reference to FIG. 4 shows the format of a SMAF file. This SMAF is a data exchange format for data distribution and mutual use for expressing music using a sound source, and for playing multimedia contents on a personal digital assistant (PDA), personal computer, cellular phone, etc. Data format specification.

The SMAF file 30 in the data exchange format shown in FIG. 4 has a basic structure of data units called chunks. The chunk consists of a header part of fixed length (8 bytes) and a body part of arbitrary length. The header portion is separated into a 4-byte chunk ID and a 4-byte chunk size. The chunk ID is used as an identifier of the chunk, and the chunk size indicates the length of the body portion. The SMAF file 30 also has a chunk structure as well as the various data contained therein.

As shown in FIG. 4, the SMAF file 30 includes a contents info chunk 31, an optional data chunk 32, and a score track chunk 33. , And HV chunks 36.

The content info chunk 31 stores various management information for the SMAF file 30, and stores, for example, the class, type, copyright information, genre name, song name, artist name, songwriter / composer name, and the like of the content. . The optional data chunk 32 stores information such as copyright information, genre name, song name, artist name, songwriter / composer name, and the like. It is also not necessary to include the optional data chunk 32 in the SMAF file 30.

The score track chunk 33 includes a setup data chunk 34 (optional) and a sequence data chunk 35 as a chunk for storing a sequence track of music to be transmitted to a sound source. .

The setup data chunk 34 stores an array of phrases of exclusive messages as chunks for storing the tone data of the sound source and the like. As an exclusive message, there is a timbre parameter registration message, for example.

The sequence data chunk 35 stores actual performance data. HV note-on ("HV" stands for human voice) and other for determining the reproduction timing of the script data 11 are described. The sequence events are mixed and stored. Here, the HV and music events other than that are distinguished by the channel designation of the HV.

The HV chunk 36 includes an HV setup data chunk 37 (optional), an HV user phrase dictionary chunk 38 (optional), and an HV-S chunk 39. ).

The HV setup data chunk 37 stores HV user timbre parameters, and messages for specifying channels for HV. The HV-S chunk 39 also stores HV script data.

The HV user phrase dictionary chunk 38 stores the contents of the user phrase synthesis dictionary data 13. In addition, the HV user timbre parameters stored in the HV setup data chunk 37 require parameters for assigning the Mora and phrase ID shown in FIG. 2.

By applying the SMAF file 30 shown in FIG. 4 to the timbre parameters of the present embodiment, the synthesized voice (HV) can be reproduced in synchronization with the piece of music, and the contents of the user phrase synthesis dictionary data 13 can also be reproduced. .

Next, the HV authoring tool which is a tool for creating the user phrase synthesis dictionary data 13 shown in FIG. 1 and the SMAF file 30 shown in FIG. 4 is demonstrated with reference to FIG. . 5 is a block diagram showing a function and an example of a specification of the HV authoring.

When the HV authoring tool 42 creates the SMAF file 30, a SMF file 41, which is previously created by a musical instrument digital interface (MIDI) sequencer, performs note-on for determining the pronunciation timing of the HV. And the SMAF file 43 (the SMAF file 30) based on information obtained from the HV script user interface 44 and the HV voice editor 45. Equivalent).

The HV voice editor 45 is an editor having a function of editing the HV user timbre parameters (corresponding to the above-described user timbre parameters 12) included in the HV user timbre file 48. In addition to editing various HV timbre parameters, the HV audio editor 45 can assign user phrases to arbitrary modes.

The interface of the HV voice editor 45 has a menu for selecting Mora, and has a function of assigning an arbitrary sound file 50 to the Mora. The sound file 50 assigned by the interface of the HV voice editor 45 is analyzed by the waveform analyzer 46 to generate formant frame data including eight sets of formant frequencies, formant levels, and pitches. . The formant frame data can be input and output as individual files (i.e., HV user timbre file 48, HV user synthesis dictionary file 49).

HV script UI 44 may directly edit HV script data. This HV script data can also be input and output as individual files (i.e., HV script file 47). In addition, the HV authoring tool 40 according to the present embodiment may be configured only with the above-described HV authoring tool 42, the HV script UI 44, the HV voice editor 45, and the waveform analyzer 46. .

Next, an example in which the audio reproducing apparatus 1 according to the present embodiment is applied to a portable communication terminal will be described with reference to FIG. 6 is a block diagram showing the configuration of the portable communication terminal 60 including the audio reproducing apparatus 1.

The portable communication terminal 60 corresponds to, for example, a cellular phone, and includes a CPU 61, a ROM 62, a RAM 63, a display unit 64, a vibrator 65, an input unit 66, and a communication unit ( 67, an antenna 68, a sound processor 69, a sound source 70, a speaker 71, and a bus 72. The CPU 61 controls the entire portable communication terminal 60. The ROM 62 stores various communication control programs, control programs such as programs for reproducing music, and various constant data.

The RAM 63 is used as a work area and stores a music file, various application programs, and the like. The display unit 64 is configured of, for example, a liquid crystal display (LCD). The vibrator 65 vibrates upon reception of the cellular phone. The input unit 66 is composed of operators such as a plurality of keys. These operators instruct the registration processing of the user timbre parameters, the user phrase synthesis dictionary data, and the HV script data based on the operation by the user. The communication unit 67 is composed of a demodulator-demodulator and the like, and is connected to an antenna.

The speech processing section 69 is connected to a telephone microphone and a sign language speaker (e.g., microphone and earphone), and has a function of encoding and decoding a speech signal for a call. The sound source 70 reproduces the music based on the music file stored in the RAM 63 or the like, and reproduces the audio signal and outputs it to the speaker 71. The bus 72 includes a CPU 61, a ROM 62, a RAM 63, a display unit 64, a vibrator 65, an input unit 66, a communication unit 67, a voice processing unit 69, and a sound source 70. Is a transmission path for performing data transmission between each component of the cellular phone.

The communication unit 67 can download the HV script file or the SMAF file 30 shown in FIG. 4 from a predetermined content server or the like and store them in the RAM 63. The ROM 62 also stores the application software 14 and the middleware program of the audio reproducing apparatus 1 shown in FIG. The programs of the application software 14 and the middleware are read by the CPU 61 and started. In addition, the CPU 61 analyzes the HV script data stored in the RAM 63 to generate formant frame data, and sends the formant frame data to the sound source 70.

Next, the operation of the audio reproducing apparatus 1 according to the present embodiment will be described. First, a method for creating user phrase synthesis dictionary data 13 will be described. 7 is a flowchart showing a method of creating user phrase synthesis dictionary data 13.

First, in step S1, the HV sound tone referring to the user phrase synthesis dictionary data 13 is selected using the HV authoring tool 42 shown in FIG. 5, and the HV voice editor 45 is started. Next, using the HV voice editor 45, select the Mora to be used to attach a sound file. As a result, in step S2, the HV voice editor 45 generates and outputs user phrase dictionary data (corresponding to the HV user synthesis dictionary file 49).

Next, the HV timbre parameters are edited using the HV voice editor 45. As a result, the HV audio editor 45 generates and outputs a user tone color parameter (corresponding to the HV user tone file 48) in step S3.

Next, using the HV script UI 44, the sound quality change event for designating the corresponding HV timbre in the HV script data is described to describe the Mora to be reproduced. As a result, the HV script UI 44 generates and outputs HV script data (corresponding to the HV script file 47) in step S4.

Next, the reproduction operation of the user phrase synthesis dictionary data 13 in the audio reproduction device 1 will be described with reference to FIG. 8 is a flowchart showing the reproduction operation of the user phrase synthesis dictionary data 13 in the audio reproduction device 1.

First, in step S11, the user tone parameter 12 and the user phrase synthesis dictionary data 13 are registered in the middleware of the audio reproducing apparatus 1. Thereafter, the script data 11 is registered in the middleware of the audio reproducing apparatus 1, and playback of the HV script data is started in step S12.

At the time of reproduction, in step S13, it is monitored whether or not the script data 11 includes a sound quality change event (X event) specifying the user tone parameter 12.

In step S13, when the sound quality change event is found, the phrase ID assigned to the Mora is found from the user timbre parameter 12, and the data corresponding to the phrase ID is read from the user phrase synthesis dictionary data 13, thereby In S14, the dictionary data of Mora corresponding to the default synthesis dictionary data 19 managed by the HV driver is replaced with the user phrase synthesis dictionary data 13. In addition, the replacement process of step S14 may be performed before reproduction of the HV script data.

After the end of step S14 or when no sound quality change event is found in step S13, the flow advances to step S15, in which case converter 16 performs script data 11 (step S14, if the process of step S14 is performed). The script data after the replacement process is analyzed and finally converted to formant frame column data using the HV driver.

In step S16, the data converted in step S15 is reproduced by the sound source 20.

Thereafter, the flowchart advances to step S17 to determine whether or not the reproduction of the script data 11 has ended. If not, the flow returns to step S13, and when finished, the user shown in FIG. The reproduction processing of the phrase synthesis dictionary data 13 is terminated.

Next, the creation method of the SMAF file 30 shown in FIG. 4 is demonstrated with reference to FIG. 9 is a flowchart illustrating a method of creating the SMAF file 30.

First, in accordance with the procedure shown in FIG. 7, the user phrase synthesis dictionary data 13, the user timbre parameters 12, and the script data 11 are created (see step S21).

Next, in step S22, an SMF file 41 including an event for controlling pronunciation of music data and HV script data is created.

Next, the SMF file 41 is input to the HV authoring tool 42 shown in FIG. 5, and the SMF file 41 is converted into the SMAF file 43 (the SMAF file by the HV authoring tool 42). (Equivalent to 30) (see step S23).

Next, the user tone parameter 12 created in step S21 is inputted to the HV setup data chunk 37 in the HV chunk 36 of the SMAF file 30 shown in FIG. 4, and the user phrase created in step S21. The synthesis dictionary data 13 is input to the HV user phrase dictionary chunk 38 in the HV chunk 36 of the SMAF file 30, whereby the SMAF file 30 is generated and output (see step S24).

Next, the reproduction processing of the SMAF file 30 will be described with reference to FIG. 10 is a flowchart showing a reproduction process of the SMAF file 30.

First, in step S31, the SMAF file 30 is registered in the middleware of the audio reproducing apparatus 1 shown in FIG. Here, the audio reproducing apparatus 1 registers the music data portion in the SMAF file 30 normally in the music reproducing section of the middleware to prepare for reproduction.

In step S32, the audio reproducing apparatus 1 determines whether or not the HV chunk 36 is included in the SMAF file 30.

If the determination result in step S32 is "YES", the flow advances to step S33, and the audio reproducing apparatus 1 analyzes the contents of the HV chunk 36.

In step S34, the audio reproducing apparatus 1 registers the user tone parameter, the registration of the user phrase synthesis dictionary data, and the registration of the HV script data.

If the determination result in step S32 is "NO" or the registration process in step S34 ends, the flow advances to step S35, and the audio reproducing apparatus 1 analyzes the chunk in the music reproducing unit.

Next, the audio reproducing apparatus 1 starts the analysis of the sequence data (that is, the actual performance data) in the sequence data chunk 35 in response to the "start" signal, thereby performing music reproduction (see step S36).

In the music reproduction, the audio reproducing apparatus 1 sequentially analyzes the events included in the sequence data, and determines whether or not each event corresponds to the HV note on in the process (see step S37).

If the determination result in step S37 is "YES", the flow advances to step S38, and the audio reproducing apparatus 1 starts reproduction of the HV script data of the HV chunk designated as the HV note on.

After the end of step S38, the audio reproducing apparatus 1 executes the reproduction processing of the user phrase synthesis dictionary data shown in FIG. That is, the audio reproducing apparatus 1 determines whether or not there is a sound quality change event (X event) specifying the user tone parameter 12 in the reproduction of the HV script data in step S38 (see step S39).

If the above-described sound quality change event exists, that is, if the determination result in step S39 is "YES", the flow advances to step S40 to find the phrase ID assigned to Mora from the user tone parameter 12, and to the phrase ID. The corresponding data is read from the user phrase synthesis dictionary data 13 and the corresponding dictionary data of the corresponding Mora is replaced with the user phrase synthesis dictionary data in the default synthesis dictionary data 19 managed by the HV driver. The replacement process of step S40 may be executed before reproduction of the HV script data.

After the end of step S40, or if no sound quality change event is found in step S39, the flow advances to step S41, where the converter 16 interprets the Moras of the script data and finally forms the formant frame using the HV driver. Convert to column data.

Next, the flow advances to step S42 to reproduce the data converted in step S41 by the HV reproducing section of the sound source 20.

Thereafter, the flow advances to step S43, and the audio reproducing apparatus 1 determines whether or not the reproduction of the piece of music has ended. When the music reproduction ends, the reproduction processing of the SMAF file 30 ends, while when the music reproduction does not end, the flow returns to step S37.

In step S37, when the event in the sequence data is not HV note on, the audio reproducing apparatus 1 recognizes the event as part of the music data and converts it to sound source reproduction event data (see step S44).

Next, the flow advances to step S45, and the audio reproducing apparatus 1 reproduces the data converted in step S44 in the music reproducing section of the sound source 20. In FIG.

As described above, the present embodiment employs a voice reproduction method by formant synthesis using an FM sound source, which has the following three advantages.

(1) The user can assign the desired phrase. In other words, voice reproduction can be performed with a tone that is close to the desired tone without depending on the fixed dictionary.

(2) In order to replace a part of the default synthesis dictionary data 19 with the user phrase synthesis dictionary data 13, an excessive increase in the data capacity in the audio reproducing apparatus 1 can be avoided. In addition, since a part of the default synthesis dictionary data 19 can be replaced with an arbitrary phrase, pronunciation of a phrase unit can be realized, so that in the seam between the pronunciation units generated in the synthesized voice by the conventional pronunciation unit, Can resolve hearing discomfort

(3) Since an arbitrary phrase can be specified in the HV script data, the speech synthesis in units of Mora and the pronunciation of speech in units of phrases can be used together.

Further, according to the present embodiment, the tone change at the formant level can be realized as compared with the method of reproducing the waveform data formed by sampling the phrase in advance. In addition, although the data size and quality in the present embodiment depend on the frame rate, high quality audio reproduction can be realized with much smaller data capacity as compared with the conventional method using sampling waveform data. Therefore, for example, the voice reproducing apparatus 1 of the present embodiment can be easily provided in a portable communication terminal such as a cellular phone, whereby contents such as electronic mail can be reproduced with high quality voice.

Fig. 11 is a block diagram showing the construction of an audio / music reproducing apparatus according to a second embodiment of the present invention. Here, the HV script (i.e., HV script data) corresponds to a file defining a format for reproducing a voice, and includes a pronunciation string including a rhyme symbol (i.e., a symbol specifying a pronunciation aspect such as an accent) and a sound to be pronounced. A file that defines data for executing speech synthesis, which is composed of a message set and a message to a playback application or the like, is created by text input to facilitate creation by a user.

The HV script may be read by application software such as a text editor and described in a file format that can be edited by text. An example is a text file created by a text editor. HV scripts have language dependencies and can be defined in various languages. However, in this embodiment, the HV scripts are defined in Japanese.

Reference numeral 101 denotes an HV script player, which controls the reproduction, stop, and the like of the HV script. Here, when the HV script is registered in the HV script player 101 and the playback instruction is received, the HV script player 101 starts the analysis of the HV script. Thereafter, according to the type of event described in the HV script, any one of the HV driver 102, the waveform reproduction player 104, and the phrase reproduction player 107 is executed based on the event.

The HV driver 102 reads and references synthesis dictionary data from a read-only memory (ROM) (not shown). The human voice has a predetermined formant (i.e., its own frequency spectrum) depending on the structure of the human body (e.g., the shape of the vocal cords, oral cavity, etc.), and the synthesis dictionary data provides parameters according to the formant of the voice. Corresponds with phonetic characters. Synthetic dictionary data is a phonetic character unit based on a parameter obtained by a result obtained by sampling and analyzing the actual sound in phonetic character units (for example, phoneme units such as Japanese 'あ' and 'い'). It is equivalent to the database memorized in advance.

For example, in the CSM (Composite Sinusoidal Modeling) speech synthesis method, the synthesis dictionary data stores eight sets of formant frequency, formant level, pitch, and the like as parameters. This speech synthesis method has the advantage that the amount of data is very small compared with the reproduction method of waveform data created by sampling of speech. In addition, the synthesis dictionary data may further store parameters (for example, parameters for specifying eight sets of formant frequencies and formant levels) for controlling the sound quality of the reproduced voice.

The HV driver 102 interprets a phonetic string including a rhyme symbol in the HV script, converts it into a formant frame sequence using synthesized dictionary data, and outputs it to the HV sound source 103. The HV sound source 103 generates a pronunciation signal based on the formant frame string output from the HV driver 102 and outputs the pronunciation signal to the adder 110.

The waveform reproducing player 104 reproduces or stops waveform data pre-sampled with voices, musics, and pseudo sounds. Reference numeral 105 denotes a waveform data random access memory (RAM) for waveform data, and the default waveform data is stored in advance. The user can store the user waveform data in the user data RAM 112 in the waveform data RAM 105 via the registration application program interface 113. When the waveform reproduction player 104 receives the reproduction instruction from the HV script player 101, the waveform reproduction player 104 reads out the waveform data from the waveform data RAM 105 and outputs it to the waveform reproducer 106. The waveform player 106 generates a pronunciation signal based on the waveform data output from the waveform reproduction player 104 and outputs it to the adder 110. In addition, the sampled waveform data is not limited to the pulse-code modulation (PCM) method, but may be a voice compression format using, for example, the moving picture experts group layer 3 (MP3) method.

The phrase replay player 107 reproduces or stops the music phrase data (or music data). The music phrase data is in SMF format and is composed of musical note information indicating the pitch and volume of the sound to be pronounced, and time information indicating the pronunciation time of the sound to be pronounced. Reference numeral 108 denotes a RAM for music phrase data, and the default music phrase data is stored in advance. The user can store user music phrase data in the user data RAM 112 in the music phrase data RAM 108 via the registration API.

When the phrase playback player 107 receives a playback instruction from the HV script player 101, the phrase playback player 107 reads the music phrase data from the music phrase data RAM 108, performs time management of the note information in the music phrase data, and plays the phrase phrase. The note information is output to the phrase sound source 109 based on the time information described in the data. The phrase sound source 109 generates a musical sound signal based on the note information output by the phrase reproducing player 107 and outputs it to the adder 110. FIG. Although the FM system, the PCM system, etc. can be employ | adopted as the phrase sound source 109, it is not necessary to limit the sound source system, as long as it has a function of playing music phrase data.

The adder 110 synthesizes the pronunciation signal output from the HV sound source 103, the audio signal output from the waveform regenerator 106, and the music signal output from the phrase sound source 109, and converts the synthesized signal into the speaker 111. Will output The speaker 111 pronounces the voice and / or the musical sound based on the synthesized signal of the adder 110.

In addition, the HV driver 102, the waveform reproducing player 104, and the phrase reproducing player 107 perform the processing at the same time, thereby integrating the pronunciation signal, the audio signal, and the sound signal (also, the voice signal and the sound signal). Sound signal ”and sound music based on each sound may be simultaneously pronounced. Alternatively, the processing timings of the HV driver 102, the waveform reproducing player 104, and the phrase reproducing player 107 are managed by the HV script player 101, and the audio and music based on the respective processing may not be reproduced simultaneously. do. In this embodiment, simultaneous processing by the HV driver 102, the waveform reproducing player 104, and the phrase reproducing player 107 is prohibited. In FIG. 11, for convenience of explanation, separate RAMs are provided as the waveform data RAM 105, the music phrase data RAM 108, and the user data RAM 112, but these functions are provided in a single manner. May be allocated to another storage area in the RAM.

Fig. 12 shows a definition example of an event for reproducing waveform data or music phrase data (hereinafter, collectively referred to as "tone data") described in the HV script. "D", which is the header of the event, means a default definition, and "O" means a user definition. For each type of event, a waveform or phrase is assigned. Default waveform data stored in the waveform data RAM 105 and default music phrase data stored in the music phrase data RAM 108 in advance are allocated to the default definitions D0 to D63. In the default definition, 64 default waveform data and default music phrase data can be assigned. In user definitions (O0 to O63), sampling waveform data and music phrase data created by the user are assigned. The user definition can be assigned 64 sampling waveform data and music phrase data.

An event in which the type shown in Fig. 12 is waveform data and the relationship between the waveform data indicated by the event is stored in the waveform data RAM 105 in advance. In addition, data indicating the relationship between the event whose type is a phrase and the music phrase data indicated by the event is stored in the music phrase data RAM 108. These data are updated when the waveform data or music phrase data is registered in the user data RAM 112 by the user.

As the HV script, for example, "TJK12 MinaO0 desu. D20" is described. In "TJK12" described at the beginning, "T" is a code indicating the start of the HV script, "J" designates a country character code, and shows that the HV script is described in Japanese here. "K12" is a code for setting the sound quality, and indicates that the 12th sound quality is specified. In addition, "minusan" and "で す" are interpreted by the HV driver 102, and the Japanese voices of "minusan" and "で す" are pronounced from the speaker 111. If the accented strings such as "み な さ ん" and "で す" contain accents (or strengths and weaknesses) and rhyme symbols indicating the pronunciation mode, the accented (or weakened) voices are pronounced.

In the user event "O0", for example, waveform data in which a voice pronounced as "鈴木" is sampled is registered. This user event "O0" is interpreted by the waveform reproducing player 104, so that the sound "鈴木" is pronounced from the speaker 111. In addition, in the user event "D20", for example, crowded short music phrase data is registered. This user event "D20" is interpreted by the phrase reproducing player 107, and the music which thrusts from the speaker 111 is pronounced. In this case, the reproduced voice becomes "Minasan 鈴木 で す" (when music phrases are reproduced), and waveform data is reproduced by the portion of "鈴木". The pronunciation of the sound by the reproduction of the waveform data is more natural in the reproduction of the seam in the pronunciation unit, compared to the sound reproduction reproduced by the speech synthesis in the pronunciation unit such as "Minana" or "で す". In addition, by reproducing a characteristic waveform by using the phrase "鈴木", it is possible to effectively reproduce the reproduced voice to the user. As described above, the HV script describes an event specifying playback of waveform data and music phrase data, so that the timing of reproduction of the waveform data and music phrase data can be arbitrarily specified. In addition, the setting regarding description of HV script is what is called a design matter, and is not limited to the above-mentioned description method.

Next, the operation of the audio / music reproducing apparatus according to the present embodiment will be described using the flowchart of FIG. First, a user creates an HV script by a text editor and registers it with the HV script player 101 (see step S101). At this time, if waveform data or music phrase data by user definition exists, the registration API 113 reads the waveform data or music phrase data from the RAM 112 for user data. The registration API 113 stores the waveform data in the waveform data RAM 105 and stores the music phrase data in the music phrase data RAM 108.

When the start instruction is made by the user (step S103), the HV script player 101 starts the analysis of the HV script (see step S102). The HV script player 101 determines whether an event starting with "D" or "O" is included in the HV script (step S104), and upon input of an event starting with "D" or "O". Next, it is determined whether the type is waveform data (step S105). When the type of the event is waveform data, the HV script player 101 instructs the waveform reproduction player 104 to process the same. The waveform reproducing player 104 reads the waveform data of the number following "D" or "O" from the waveform data RAM 105 and outputs it to the waveform reproducing unit 106 (step S106). The waveform regenerator 106 generates an audio signal based on this waveform data and outputs it to the speaker 111 via the adder 110 (step S107). As a result, the speaker 111 pronounces the corresponding voice.

If the type of event is not waveform data in step S105, the flow advances to step S108, and the HV script player 101 determines whether the type of event is music phrase data. When the event type is music phrase data, the HV script player 101 instructs the phrase playback player 107 to process the same. The phrase replay player 107 reads out the music phrase data of the number following "D" or "O" from the music phrase data RAM 108, and based on the time information in this music phrase data, The note information is output to the phrase sound source 109 (see step S109). The phrase sound source 109 generates a musical sound signal based on this note information, and outputs it to the speaker 111 via the adder 110 (refer to step S110). As a result, music is pronounced through the speaker 111. If it is determined in step S108 that the event type is not music phrase data, it is recognized that the event is a type of event that cannot be processed by the audio / music reproducing apparatus according to the present embodiment, and the flow advances to step S113.

In step S104, when an event starting with "D" or an event starting with "O" is not described in the HV script, the HV script player 101 instructs the HV driver 102 to perform the processing. The HV driver 102 converts the character string into a formant frame string using the synthesis dictionary data, and outputs it to the HV sound source 103 (see step S111). The HV sound source 103 generates a pronunciation signal based on this formant frame sequence and outputs it to the speaker 111 via the adder 110 (see step S112). As a result, the corresponding voice is pronounced from the speaker 111.

The HV script player 101 determines whether or not the analysis has been completed up to the last description of the HV script at the end of the processing according to the event (step S113). If the technique to be interpreted still remains, the flow returns to step S104. On the other hand, when the analysis of all the techniques of the HV script is completed, the audio / music reproduction processing shown in FIG. 13 ends.

In the case of "TJK12 MINAO0DESU。 D20" shown as a description example of the HV script according to the present embodiment, after the pronunciation of the waveform data defined by the event "O0" is finished, the following phrase "で す" needs to be pronounced. have. For example, when the HV script player 101 analyzes an event of waveform data (or music phrase data), the playback of the next event is suspended for a while and the waveform playback player 104 (or phrase playback player) is suspended. At the end of the pronunciation by (107)), the waveform reproduction player 104 outputs a signal indicating the end of the pronunciation to the HV script player 101.

In addition, when the simultaneous reproduction process by the HV driver 102, the waveform reproduction player 104, and the phrase reproduction player 107 is allowed, you may control these reproduction processes by description of an HV script. For example, when the HV script is described as "TJK12 MINAO03Desu.D20", the event that the predetermined silence period is provided by "" (space) and "3" following "O0" is shown. While the phrase &quot; 」木 &quot; shown by &quot; O0 &quot; is pronounced, the audio reproduced by the HV driver 102 is controlled to be silent. In addition, when the HV script is described as "TJK12 kun. D20 minus O0 3 desu.", The music designated by "D20" and the voice "minusan 鈴 wooden で。" are pronounced simultaneously.

14 is a block diagram showing an example of the configuration of a mobile telephone including the audio / music reproducing apparatus according to the present embodiment. Here, reference numeral 141 denotes a CPU that controls each unit of the cellular phone. Reference numeral 142 denotes an antenna for data transmission and reception. Reference numeral 143 denotes a communication unit which modulates the data for transmission and outputs it to the antenna 142 and demodulates the reception data received by the antenna 142. Reference numeral 144 denotes a voice processing unit, which converts voice data of a caller who is output from the communication unit 143 into a voice signal when the mobile phone is called, and outputs the voice signal to an ear speaker (or earphone, not shown). The voice signal input by (not shown) is converted into voice data and output to the communication unit 143.

Reference numeral 145 denotes a sound source, and has the same function as that of the HV sound source 103, the waveform reproducer 106, and the phrase sound source 109 shown in FIG. Reference numeral 146 denotes a speaker and pronounces a desired voice or musical sound. Reference numeral 147 denotes an operation unit operated by the user. Reference numeral 148 denotes a RAM that stores waveform data, music phrase data, and the like defined by the HV script and the user. Reference numeral 149 denotes a program that the CPU 141 executes, and a ROM that stores synthesis dictionary data, default waveform data, default music phrase data, and the like. Reference numeral 150 denotes a display unit, and displays the result of operation by the user, the state of the mobile phone, and the like on the screen. Reference numeral 151 denotes a vibrator, and generates a vibration upon receiving an instruction from the CPU 141 at the time of incoming call from the cellular phone. Each block described above is connected to each other via a bus B.

The mobile phone has a function of generating waveform data from voice, and sends the voice input by the microphone to the voice processing unit 144, converts the voice data into waveform data, and stores the waveform data in the RAM 148. In addition, when the music phrase data is downloaded from the WEB server by the communication unit 143, the music phrase data is stored in the RAM 148.

The CPU 141 is similar to the HV script player 101, the HV driver 102, the waveform reproducing player 104, and the phrase reproducing player 107 shown in FIG. 11, in accordance with the program stored in the ROM 149. FIG. Execute the processing of. The CPU 141 also analyzes the events described in the HV script read from the RAM 148. If the event indicates the pronunciation by speech synthesis, the CPU 141 reads out the synthesis dictionary data from the ROM 149 and converts the character string described in the HV script into a formant frame sequence to the sound source 145. Output

When the event indicates reproduction of waveform data, the CPU 141 reads waveform data of a number following "D" or "O" in the HV script from the RAM 148 or the ROM 149 to the sound source 145. Output When the event indicates the reproduction of music data, the CPU 141 reads the music phrase data of the number following "D" or "O" from the HV script from the RAM 148 or the ROM 149, and the music phrase is read. Based on the time information in the data, note information in the music phrase data is output to the sound source 145.

The sound source 145 generates a synthesized pronunciation signal based on the formant frame string output from the CPU 141 and outputs the synthesized pronunciation signal to the speaker 146. In addition, an audio signal is generated based on the waveform data output from the CPU 141 and output to the speaker 146. Also, a sound signal is generated based on the music phrase data output from the CPU 141 and output to the speaker 146. The speaker 146 appropriately pronounces a speech or musical sound based on the synthesized pronunciation signal, the speech signal, or the musical sound signal.

When the user operates the operation unit 147 to start the software corresponding to the text editing, the user can create an HV script while checking the contents displayed on the screen of the display unit 150, and the HV script thus created is RAM It can save to 148.

It is also possible to apply the HV script written by the user to the incoming melody. In this case, it is assumed that the use of the HV script when receiving a mobile phone is stored in advance in the RAM 148 as setting information. That is, when the communication unit 143 receives call information transmitted from another mobile phone or the like through the antenna 142, the communication unit 143 notifies the CPU 141 of the incoming call. Upon receipt of the notification of notification, the CPU 141 reads the setting information from the RAM 148, reads the HV script indicated by the setting information from the RAM 148, and starts the analysis. Processing thereafter is as described above. That is, according to the type of event described in the HV script, the speaker 146 pronounces a voice or a musical sound.

The user may also attach the HV script to an electronic mail and send it to another terminal. Further, the CPU 141 may interpret the body of the e-mail itself in the form of an HV script, receive an instruction from the user, and output the playback instruction of the HV script to the voice processing unit 144 according to the description in the e-mail. . In addition, it is not necessary to burden the CPU 141 with all the functions of the HV script player 101, the HV driver 102, the waveform play player 104, and the phrase play player 107. For example, the sound source 145 may bear any of the above functions.

In addition, the application target of the present embodiment need not be limited to a cellular phone, but is applied to, for example, a portable terminal such as a personal handyphone system (PHS) or a personal digital assistant (PDA). Voice and music reproduction as described above may be performed.

Further, as an application example of the present embodiment, by allowing a user to input HV scripts in a portable mobile terminal such as a mobile phone, the general user can not only form text for speech synthesis but also formally sampled waveform data and music phrase data. You can easily write HV scripts for playback. In addition, when the transmitting and receiving portable mobile terminal is provided with the voice and music reproducing apparatus according to the present embodiment, the user can operate the portable mobile terminal to send and receive HV scripts by attaching it to an e-mail. Thereby, the portable mobile terminal on the receiving side can appropriately reproduce not only speech synthesis texts but also shaped sampling data and music phrase data by the received electronic mail. In addition, playback of voices and music using the HV script can also be used as an incoming melody.

Next, the configuration and operation of the voice and music reproducing apparatus according to the third embodiment of the present invention will be described with reference to Figs. The third embodiment is a combination of the first and second embodiments described above, the middleware performs HV playback, waveform playback, and music phrase playback, and the sound source is based on these three types of data. By generating a synthesized pronunciation signal, a signal from three systems is synthesized and output to a speaker.

Here, FIG. 15 is a combination of some components of FIG. 11 based on the configuration of FIG. 1, and reference numerals 211 to 219 correspond to reference numerals 11 to 19 of FIG. 1, and reference numerals 303 to 313 indicate Correspond to reference numerals 103 to 113. That is, the user data RAM of FIG. 11 is connected to the middleware of FIG. 1 via the user data API, and in the middleware, the waveform reproduction player and the phrase reproduction player of FIG. 11 are respectively the waveform data RAM and the music phrase data RAM. It is connected to and added. In addition, the application software has a function as the HV script player of FIG. 11, and processes the processing to any one of an HV converter, a waveform playback player, and a music phrase playback player according to the type of events described in the HV script through the HV script middleware API. Instruct. In addition, the sound source has three functions of the HV sound source, the waveform generator and the music phrase sound source in Fig. 11, and each of these output signals is synthesized in an adder and pronounced as a speaker. In addition, since the operation | movement of each component shown in FIG. 15 is the same as that of the corresponding component shown in FIG. 1 and FIG. 11, the detailed description is abbreviate | omitted.

FIG. 16 is a flowchart showing the operation of the audio / music reproducing apparatus shown in FIG. This adds a part of the flowchart of FIG. 8 based on the flowchart of FIG. 13, and S211-S216 corresponds to S11-S16 of FIG. 8, and S304-S310, S312, S313 are FIG. Corresponds to symbols S104 to S110, S112, and S113. That is, when the determination result in step S104 in FIG. 13 is "NO", the same processing as in steps S13, S14, and S15 in FIG. 8 is executed, and the HV sound source reproduction processing is executed in the next step S312. In this manner, by inputting one HV script, it is possible to reproduce the audio by the HV sound source, the waveform data by the waveform reproduction player, and the music phrase based on the note information by the music phrase reproduction player. In addition, since the process of each step shown in FIG. 16 is the same as that of FIG. 8 and FIG. 13, detailed description is abbreviate | omitted.

Finally, the rhyme symbols used in the above embodiments will be described. For example, "Ha ^ 3Jima $ ri ^ Ma $ 5Shi> 10Ta" described in the HV script adds a predetermined innation to the language "HajiMariMashita" (that is, pronounced strings). In this case, "^", "$", ">", and the like correspond to rhyme symbols. A predetermined accent (accent) is added to the character following the rhyme symbol (the character following the digit if the numerical value is described immediately after the rhyme symbol).

Specifically, "^" means raising the pitch during pronunciation, "$" means lowering the pitch during pronunciation, and ">" means lowering the volume during pronunciation, and voice synthesis is performed accordingly. All. In addition, when a numerical value is described immediately after the rhyme symbol, the numerical value designates the change amount of the accent to add. For example, in the phrase "Ha ^ 3 じ ま", "は" is pronounced at the standard pitch and volume, indicating that "じ" is raised by the amount of "3" during the pronunciation. "ま" means to pronounce according to the raised pitch.

In this way, when a predetermined accent (or intonation) is added to a character included in the pronounced language, the above-described rhyme symbol (a numerical value representing the amount of change in tonation) is described immediately before the character. In addition, although the said rhyme symbol controls the pitch and volume during pronunciation, it is not limited to this, For example, the symbol which controls a sound quality and a speed can be used. By adding such a symbol to the HV script, it is possible to suitably express pronunciation aspects such as accents.

In addition, this invention does not need to be limited to the above-mentioned embodiment, A change etc. within the scope of the invention can be included in this invention.

Claims (15)

Storage means for storing composition dictionary data which preforms and stores formant frame data corresponding to a phonetic character representing a predetermined pronunciation unit in association with the phonetic character; Registration means for registering, as user dictionary data, user phrase data indicating separate formant frame data for use in place of the formant frame data corresponding to the phonetic characters stored in the synthesis dictionary data, according to a user's operation; Input script data including a character string composed of a plurality of phonetic characters and event data indicating replacement of formant frame data corresponding to at least some of the phonetic characters of the string; Input means to say, Interpret the script data, read formant frame data from the composite dictionary data based on phonetic characters other than the partial phonetic characters, and from the user dictionary data based on the event data and the character strings Speech synthesis means for reading phrase data and generating synthesized speech based on the read formant frame data and the read user phrase data Speech reproduction apparatus comprising a speech reproduction apparatus. delete The method of claim 1, Music reproducing means for reproducing music based on the music reproducing information, The input means includes music reproducing information for reproducing music, voice reproducing information including the script data, and the user phrase data, and music reproducing based on the music reproducing information and voice reproducing based on the voice reproducing information. A data exchange format, which is an information structure for reproducing synchronously, The music reproducing means reproduces the music reproducing information, And the speech synthesizing means is adapted to reproduce the speech reproduction information. delete A portable terminal device comprising the audio reproducing apparatus according to claim 1. First storage means for storing sound data, Second storage means for storing script data describing a character string composed of a pronunciation character representing a predetermined pronunciation unit, a rhyme symbol representing a pronunciation aspect of the character string, and event data instructing reproduction of the sound data; The script data is read sequentially from the second storage means, the pronunciation is instructed based on the character string and the rhyme code in the script data, and when the event data in the script data is read, the sound is read based on the event data. Playback instruction means for instructing playback of data; Synthesized pronunciation signal generation means for generating a synthesized pronunciation signal by performing voice synthesis based on the pronunciation instruction of the character string from the reproduction instruction means; Sound signal generation means for reading the sound data from the first storage means based on a playback instruction of the sound data from the playback instruction means and generating a sound signal based on the sound data; And a synthesized speech generating means for generating a synthesized speech based on the synthesized pronunciation signal and generating a sound based on the sound signal. The method of claim 6, And the sound data is waveform data generated by sampling a predetermined sound. The method of claim 6, And the sound data is music data including musical note information indicating a pitch and a volume of a sound to be pronounced. The method of claim 6, And said synthesized speech signal generating means stores formant control parameters that characterize the pronunciation of characters, and performs speech synthesis using formant control parameters corresponding to character strings in said script data. The method according to any one of claims 6 to 9, And the script data is described by a file composed of text data. The method of claim 6, Form synthesis dictionary data which preforms and stores formant frame data corresponding to a phonetic character representing a predetermined phonetic unit with the phonetic character, In accordance with a user's operation, register user phrase data indicating separate formant frame data for use in place of the formant frame data corresponding to the phonetic characters stored in the synthesis dictionary data, as user dictionary data, When the script data includes event data for instructing replacement of formant frame data corresponding to at least some of the pronunciation characters in the character string, the synthesized pronunciation signal generating means is configured to produce pronunciation characters other than the partial pronunciation characters. Read the formant frame data from the composite dictionary data based on the readout, and read the user phrase data from the user dictionary data based on the event data and the character string of the partial, read the formant frame data and read the And generating the synthesized pronunciation signal based on user phrase data. A portable terminal device comprising the audio reproducing apparatus according to any one of claims 6 to 9 and 11. A portable terminal device comprising the audio reproducing apparatus according to claim 3. delete A portable terminal device comprising the audio reproducing apparatus of claim 10.

Images