KR100994340B1 - Music contents production device using tts - Google Patents

Abstract

PURPOSE: A musical content composition device using text-to-speech is provided to easily create musical contents to be used in person or sold. CONSTITUTION: A musical content composition device using text-to-speech comprises a music reproduction information acquiring unit(110), a sentence construction analyzing unit(120), a pronunciation converting unit(130), an optimum phoneme selecting unit(140), a sound source selecting unit(150), a rhythm control unit(160), a voice converting unit(170), a tone converting unit(180), and a song and background music synthesizing unit. The music reproduction information acquiring unit acquires lyrics, singers, tracks, musical scales, sound lengths, bits, tempo, and musical effects for music reproduction. The sentence construction analyzing unit analyzes sentences of the lyrics obtained by the music reproduction information acquiring unit and changes into the form defined according to linguistic characteristics. The pronunciation converting unit changes the analyzed data into phonemes. The optimum phoneme selecting unit selects the optimum phonemes corresponding to the lyrics extracted through the sentence construction analyzing unit and the pronunciation converting unit. The sound source selecting unit selects a sound source corresponding to the phonemes selected by the optimum phoneme selecting unit based on singer information obtained by the music reproduction information acquiring unit. The rhythm control unit controls the length and pitch of the optimum phonemes combined according to the characteristics of the sentences of the lyrics. The voice converting unit matches the sentences of the lyrics to output voice according to the musical scales, sound lengths, bits, and tempo obtained by the music reproduction information acquiring unit. The tone converting unit matches tones to the voice to be reproduced according to the musical effects obtained by the music reproduction information acquiring unit. The song and background music synthesizing unit synthesizes the background music information obtained by the music reproduction information acquiring unit and the tones finally converted by the tone converting unit.

Description

Music contents production device using TTS.

The present invention relates to an apparatus for producing music contents using text speech synthesis, and more particularly, to synthesize a song synthesized according to arbitrary lyrics, scale, and length using text to speech (TTS). The present invention relates to a music content production apparatus using character speech synthesis that outputs or synthesizes songs corresponding to background music and lyrics.

Conventional speech synthesis technology has been used to be limited to the simple information transfer function, such as ARS (Auto Answering Service), voice guidance, navigation voice guidance by simply outputting the input text characters in the form of a dialogue.

Therefore, by using a technology that can reproduce all human voice functions in addition to a simple information transmission function, there is a demand for a text-to-speech synthesis technology that can be applied to various services such as songs, compositions, drama voice actors, and intelligent robots.

Accordingly, the present invention has been proposed in view of the problems of the prior art as described above, and an object of the present invention is to provide a device capable of uttering a voice representing a rhythm according to a scale when the arbitrary lyrics and the scale are input. It is to enable the general public to sing the quality of singing.

Another object of the present invention is to process the elements necessary for music, such as arbitrary lyrics, scale, musical length, music effects, background music settings, beat / tempo, and can be produced in the form of digital content, according to the characteristics of various languages The present invention provides a method for synthesizing lyrics and voice by analyzing corresponding texts, and a method for synthesizing speech for music.

In order to achieve the problem to be solved by the present invention,

Music content production apparatus using a character voice synthesis according to an embodiment of the present invention,

A music playback information acquisition unit for acquiring lyrics, singers, tracks, scales, musical lengths, beats, tempos, and music effects inputted for music reproduction;

A syntax analysis unit for analyzing the sentence of the lyrics obtained by the music reproduction information acquisition unit and converting the sentence into a form defined according to linguistic characteristics;

A pronunciation converter for converting the data analyzed by the syntax analyzer based on a phoneme;

An optimum phoneme selection unit for selecting an optimum phoneme according to a rule defined in advance from the optimum phoneme corresponding to the lyrics analyzed by the syntax analyzer and the pronunciation converter;

A sound source selection unit for acquiring the singer information acquired by the music reproduction information acquisition unit and selecting a sound source corresponding to the phoneme selected through the optimum phoneme selection unit from a sound source database;

A rhyme control unit for acquiring the optimal phoneme selected by the optimum phoneme selecting unit according to the sentence characteristics of the lyrics and controlling the length and pitch when combining the optimum phonemes;

A voice converter for acquiring the sentences of the lyrics synthesized by the rhyme control unit and matching the sentences of the lyrics acquired to be reproduced according to the scale, the length, the beat, and the tempo obtained by the music reproduction information acquisition unit;

And a tone conversion unit for acquiring a voice converted by the voice conversion unit and matching a tone to the converted voice to be reproduced according to a music effect obtained by the music reproduction information acquisition unit. Will be solved.

Music content production apparatus using the character voice synthesis according to the present invention having the above configuration and operation, so that anyone can easily produce music content can distribute the content created by individuals online, offline, ringtones, coloring in the mobile phone (RBT, Ring Back Tone), etc., can be used for music content application supplementary services, and can be used for music playback and voice guidance on various types of portable devices, and people in ARS (auto answering system) and navigation (map guidance device) It can provide a voice guidance service with a similar accent, and the artificial intelligence robot device to provide an effect that allows you to speak with a similar accent and sing.

In addition, it will provide a better effect to express the accent of the natural person that can replace the voice actor in the production of drama or animation content.

1 is a block diagram of an apparatus for producing music contents using character speech synthesis according to an embodiment of the present invention.
2 is a block diagram of an apparatus for producing music contents using character speech synthesis according to another embodiment of the present invention.
3 is a block diagram of a music reproduction information acquisition unit of the apparatus for producing music contents using character speech synthesis according to an embodiment of the present invention.
4 is a screen showing a music content production program using character speech synthesis according to an embodiment of the present invention.

Music content production apparatus using the character voice synthesis of the present invention for achieving the above object,

In the music content production apparatus using character speech synthesis,

A music playback information acquisition unit for acquiring lyrics, singers, tracks, scales, musical lengths, beats, tempos, and music effects inputted for music reproduction;

A syntax analysis unit for analyzing the sentence of the lyrics obtained by the music reproduction information acquisition unit and converting the sentence into a form defined according to linguistic characteristics;

A pronunciation converter for converting the data analyzed by the syntax analyzer based on a phoneme;

An optimum phoneme selection unit for selecting an optimum phoneme according to a rule defined in advance from the optimum phoneme corresponding to the lyrics analyzed by the syntax analyzer and the pronunciation converter;

A sound source selection unit for acquiring the singer information acquired by the music reproduction information acquisition unit and selecting a sound source corresponding to the phoneme selected through the optimum phoneme selection unit from a sound source database;

A rhyme control unit for acquiring the optimal phoneme selected by the optimum phoneme selecting unit according to the sentence characteristics of the lyrics and controlling the length and pitch when combining the optimum phonemes;

A voice converter for acquiring the sentences of the lyrics synthesized by the rhyme control unit and matching the sentences of the lyrics acquired to be reproduced according to the scale, the length, the beat, and the tempo obtained by the music reproduction information acquisition unit;

And a tone conversion unit for acquiring a voice converted by the voice conversion unit and matching a tone to the converted voice to be reproduced according to a music effect acquired by the music reproduction information acquisition unit. .

At this time, the music reproduction information acquisition unit,

Lyrics information acquisition unit for acquiring lyrics information,

A vocal effect acquisition unit for acquiring vocal effect information adjusted by a user,

And a singer information acquisition unit for acquiring the singer information.

On the other hand, according to another embodiment of the present invention the apparatus for producing music contents using character speech synthesis,

In the music content production apparatus using character speech synthesis,

A music playback information acquisition unit for acquiring lyrics, singers, tracks, scales, musical lengths, beats, tempos, and music effects inputted for music reproduction;

A syntax analysis unit for analyzing the sentence of the lyrics obtained by the music reproduction information acquisition unit and converting the sentence into a form defined according to linguistic characteristics;

A pronunciation converter for converting the data analyzed by the syntax analyzer based on a phoneme;

An optimum phoneme selection unit for selecting an optimum phoneme according to a rule defined in advance from the optimum phoneme corresponding to the lyrics analyzed by the syntax analyzer and the pronunciation converter;

A sound source selection unit for acquiring the singer information acquired by the music reproduction information acquisition unit and selecting a sound source corresponding to the phoneme selected through the optimum phoneme selection unit from a sound source database;

A rhyme control unit for acquiring the optimal phoneme selected by the optimum phoneme selecting unit according to the sentence characteristics of the lyrics and controlling the length and pitch when combining the optimum phonemes;

A voice converter for acquiring the sentences of the lyrics synthesized by the rhyme control unit and matching the sentences of the lyrics acquired to be reproduced according to the scale, the length, the beat, and the tempo obtained by the music reproduction information acquisition unit;

A tone conversion unit for acquiring a voice converted by the voice conversion unit and matching a tone to the converted voice to be reproduced according to a music effect obtained by the music reproduction information acquisition unit;

And a song and background music synthesis unit for synthesizing the background music information acquired by the music reproduction information acquisition unit and the tone tone finally converted by the tone conversion unit.

At this time, the music reproduction information acquisition unit,

Lyrics information acquisition unit for acquiring lyrics information,

A background music information acquisition unit for acquiring the selected background music sound source information among the background music sound sources stored in the sound source database;

A vocal effect acquisition unit for acquiring vocal effect information adjusted by a user,

And a singer information acquisition unit for acquiring the singer information.

In addition, according to an additional aspect of one embodiment of the present invention or another embodiment of the present invention further comprises a piano keyboard position acquisition unit for obtaining the piano keyboard position information selected by the user in the virtual piano musical instrument output on the screen do.

Hereinafter, an embodiment of a music content producing apparatus using character voice synthesis according to the present invention will be described in detail.

1 is a block diagram of an apparatus for producing music contents using character speech synthesis according to an embodiment of the present invention.

As shown in FIG. 1, an apparatus for producing music contents using character speech synthesis according to an embodiment of the present invention,

In the music content production apparatus using character speech synthesis,

A music reproduction information acquisition unit (110) for acquiring lyrics, singers, tracks, scales, musical lengths, beats, tempos, and music effects input for music reproduction;

A syntax analysis unit 120 for analyzing the sentence of the lyrics obtained by the music reproduction information acquisition unit and converting the sentence into a form defined according to linguistic characteristics;

A pronunciation converter 130 for converting the data analyzed by the syntax analyzer based on a phoneme;

An optimum phoneme selection unit 140 for selecting an optimal phoneme according to a rule defined in advance by selecting the optimum phoneme corresponding to the lyrics analyzed by the syntax analyzer and the pronunciation converter;

A sound source selection unit 150 for acquiring the singer information obtained by the music reproduction information acquisition unit and selecting a sound source corresponding to the phoneme selected through the optimum phoneme selection unit from a sound source database;

A rhyme control unit 160 for controlling the length and pitch when acquiring the optimal phoneme selected by the optimum phoneme selecting unit according to the sentence characteristics of the lyrics and combining the optimal phonemes;

A voice converter 170 for acquiring the sentences of the lyrics synthesized by the rhyme control unit and matching the sentences of the lyrics acquired to be reproduced according to the scale, the length, the beat, and the tempo obtained by the music reproduction information acquisition unit; ;

And a tone conversion unit 180 for acquiring a voice converted by the voice conversion unit and matching a tone to the converted voice to be reproduced according to a music effect obtained by the music reproduction information acquisition unit. It features.

The music reproduction information acquisition unit 110 acquires lyrics, singers, tracks, scales, sound lengths, beats, tempos, and music effects input for music reproduction.

That is, the music content production program is mounted on the apparatus of the present invention so that the operator can perform the music content using the character voice synthesis as shown in FIG.

The lyrics, singer, track, scale, musical length, beat, tempo, music effect information, etc. are stored and managed in the music information database 200, the music playback information with reference to the information required for the music playback selected by the user The acquiring unit acquires the corresponding information stored in the music information database.

The production program is output to the user's monitor so that the user can select various operation modes required for the production of music contents. When the effect, the creation method, etc. are selected, the selected information is acquired by the music reproduction information acquisition unit 110.

In this case, the sentence of the lyrics obtained by the music reproduction information acquisition unit is analyzed by the syntax analysis unit 120 and converted into a form defined according to linguistic characteristics.

The linguistic characteristics of the Korean language is given by a syntax, and include an object, a verb, a survey, an adverb, and the like. The linguistic characteristics are defined as linguistic characteristics, and all languages such as English and Japanese have such characteristics.

The above-defined form means to be divided into morphemes of language, and morphemes are the smallest units having meanings in the language.

For example, the sentence 'Donghaemul and Baekdusanyi' is morphologically divided into 'Donghaemul' + 'and' + 'Baekdusan' + '이'.

After classifying the morphemes, the sentence components are analyzed, for example, 'donghaemul' = noun, 'and' = survey, 'baekdusan' = noun, 'yi' = survey, such as noun, survey, adverb, adjective. Analyze sentence components using verbs and verbs.

In other words, if the selected lyrics are Korean, they are converted to the defined form according to Korean characteristics.

The data analyzed by the parser is transmitted from the pronunciation converter 130 and converted to phoneme based, and corresponds to the lyrics analyzed by the parser and the phonetic converter through the optimal phoneme selecting unit 140. The optimal phonemes are selected according to the predefined rules.

The pronunciation converting unit converts the phoneme based phoneme into a phonetic form according to a Hangul reading rule.

For example, 'Donghaemul and Baekdusanyi' will be expressed as 'Donghaemulgak Baektu Sani', and if divided into phonemes, 'Daehaemulwa' is 'do + Ong + Ohae + Um + water + Ulga' and It will be converted as well.

The optimum phoneme selection unit 140, when the analyzed lyrics are Donghae, the best phoneme is, for example, Do, Ong, Ohae, caress, water, Ulga and so on to select this.

The sound source selection unit 150 obtains the singer information acquired by the music reproduction information acquisition unit and selects a sound source of the acquired singer information from a sound source database as a sound source corresponding to the phoneme selected through the optimum phoneme selection unit.

That is, when the singer is selected as the girl's generation, the sound source corresponding to the girl's generation is selected from the sound source DB.

Since track information may be provided in addition to the singer information, if the user selects a track other than the singer, the corresponding track information may be provided.

The rhyme control unit 160 obtains the optimal phoneme selected by the optimum phoneme selection unit according to the sentence characteristics of the lyrics, and controls length and pitch when combining the optimal phonemes.

The sentence characteristic refers to a law applied when converting a sentence such as a soft-sounding or palatalization into a pronunciation, that is, a language rule in which an expression symbol represented by a character and a pronunciation symbol are different.

The length means a note length corresponding to the lyrics, that is, a length of 1, 2, 3 beats, and the pitch means a scale of the lyrics, that is, a pitch defined in music such as 'Doremi Pasolasido'.

In other words, it plays a role of controlling length and pitch when combining phonemes to synthesize natural utterance according to the characteristics of a sentence.

The voice converter 170 acquires a sentence of the lyrics synthesized by the rhyme control unit to match the sentence of the lyrics obtained to be reproduced according to the scale, the length, the beat, the tempo obtained by the music reproduction information acquisition unit It will play a role.

That is, the sound source corresponding to the lyrics is converted to a voice according to the scale, length, beat, and tempo. For example, the sound source corresponding to 'dong' is beaten on a scale (pitch) called 'sol'. This plays the sound source at 120 tempos, in beats of 4/4 beats.

The pitch refers to the height of the sound, and the present invention provides a virtual piano musical instrument function so that the user can easily specify the height of the sound.

The note length means the length of the note, and the note length is provided to facilitate editing of the note length like the music score.

The notes provided by default are quarter notes (1), half notes (1/2), quarter notes (1/4), eighth notes (1/8), sixteenth notes (1/16), 32 The quarter note (1/32) and the 64-note note (1/64).

The beat is a unit of beat in music, and there are 1/2 beat, 1/4 beat, and 1/8 beat.

    The number corresponding to the denominator is (1,2,4,8,16,32,64) and the number corresponding to the numerator is (1 to 256).

The tempo (Tempo) means the music progression speed of music, and usually provides a number (20 ~ 300), the smaller the number is the slower speed, the larger the number means the faster speed.

Usually, the speed of one beat is 120.

The tone converting unit 180 acquires a voice converted by the voice converting unit and matches a tone to the converted voice to be reproduced according to a vocal effect or a creative method obtained by the music reproduction information obtaining unit. It will play a role.

For example, the sound source ‘동’ is used to change the tone by giving music effects such as vibration and attack.

The music effect and the creation method is to provide a function for maximizing the musical effect, the music effect is a function to support the natural voice of the person is to convert the tone as follows.

As shown in FIG. 4, the production program includes VEL (velocity), DYN (dynamics), BRE (Breathiness), BRI (Brightness), CLE (Clearness), OPE (Opening), GEN (Gender Factor), and POR (Portamento Timing). ), PIT (Pitch Bend), PBS (Pitch Bend Sensitivity), VIB (Vibration) and so on.

As the VEL (velocity) is increased as the attack, the consonant becomes shorter as the consonant becomes shorter, and the DYN (dynamics) is the strength and weakness of controlling the dynamics of the singer (sound and softness).

The BRE (Breathiness) is to add a breath when the value is high, the BRI (Brightness) is to increase or decrease the frequency components of the sound is high and bright and low to provide a dim and mild sound.

The CLE (Clearness) is similar to BRI, but the principle is different. In other words, high value provides sharp and clear sound and low value provides low and heavy sound.

The OPE (Opening) simulates the appearance that the tone is changed by the opening state of the mouth, and provides high and clear and low-noticeable characteristics.

The GEN (Gender Factor) is a broad variation of the singer's character, which provides a high masculine and low feminine feeling.

The POR (Portamento Timing) adjusts the point at which the pitch changes, and the PIT (Pitch Bend) adjusts the EQ band for the pitch, and the PBS (Pitch Bend Sensitivity) adjusts the sensitivity or sensitivity to the pitch adjustment. The VIB (Vibration) is to perform the function of adjusting the shaking of the sound.

Chang means the way of singing a person's song and implements various methods by processing the vocal sound sources such as vocal music effects.

For example, they offer singing techniques such as female voices, male voices, child voices, robot voices, pops, classics, and breaks.

2 is a block diagram of an apparatus for producing music contents using character speech synthesis according to another embodiment of the present invention.

As shown in FIG. 2, the apparatus further includes a song and background music synthesis unit 190 for synthesizing the background music information acquired by the music reproduction information acquisition unit and the tone tone finally converted by the tone conversion unit. do.

For example, when playing a sound source called 'Donghaemul and Baekdusanyi', the background music of the song (usually played by a musical instrument) is synthesized.

That is, the music of the completed form is output by synthesizing the background music with the converted final tone.

3 is a block diagram of a music reproduction information acquisition unit of the apparatus for producing music contents using character speech synthesis according to an embodiment of the present invention.

As shown in FIG. 3, the music reproduction information acquisition unit 110,

Lyrics information acquisition unit 111 for acquiring lyrics information,

A background music information acquisition unit 112 for acquiring the selected background music sound source information among the background music sound sources stored in the sound source database;

A vocal effect acquisition unit 113 for acquiring vocal effect information adjusted by a user;

And a singer information acquisition unit 114 for acquiring the singer information.

In addition, according to an additional aspect it may be configured to further include a piano keyboard position acquisition unit (not shown) for obtaining the piano keyboard position information selected by the user in the virtual piano musical instrument output on the screen.

The piano keyboard position information is provided in advance by defining a frequency corresponding to the pitch (pitch) of each key corresponding to the piano musical instrument.

4 is a screen showing a music content production program using character speech synthesis according to an embodiment of the present invention.

As shown in FIG. 4, a lyrics editing area 410 in which a user can edit lyrics, a background music editing area 420 in which background music can be edited, and a virtual piano musical instrument area in which a user manipulates a piano keyboard 430, the vocal effect editing area 440 in which the user can edit vocal effects, the singer setting area 450 in which the singer or track can be edited, the file, edit, audio, view, task, track, When outputting the setting area 460 to select the lyrics, settings, methods, help, etc. on the screen, the user will perform the desired edit.

The lyrics editing area 410 may input a minimum unit of a language (syllable), and display the sound of each syllable and display a phonetic symbol.

Each syllable will have the properties of Pitch and Length.

The background music editing area 420 can input and edit conventional sound sources such as WAV and MP3.

The virtual piano musical instrument region 430 provides a function corresponding to a piano musical instrument, so that a sound suitable for each piano keyboard position can be reproduced.

The singer setting area 450 may select a singer sound source corresponding to a vocal and provide a function of editing various tracks to perform a function of singing by several singers.

In the setting area 460, a setting method for setting various singing techniques, an editing basic unit note, an editing screen option, and the like can be set.

Through the configuration and operation as described above, anyone can easily produce music contents, so that personally-created contents can be distributed online and offline, and the application of music contents such as ringtones and coloring (RBT, Ring Back Tone) in mobile phones is added. It can be used for service, music playback and voice guidance on various types of portable devices, ARS (automatic answering system), navigation (map guidance device) can provide voice guidance service with human-like accent, The artificial intelligence robot device provides the effect of speaking and singing in a human-like accent.

Those skilled in the art to which the present invention pertains as described above may understand that the present invention may be implemented in other specific forms without changing the technical spirit or essential features of the present invention. Therefore, the above-described embodiments are to be understood as illustrative in all respects and not restrictive.

The scope of the invention is indicated by the following claims rather than the above description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the invention. do.

110: music playback information acquisition unit
120: parser
130: pronunciation converter
140: optimal phone selection unit
150: sound source selection unit
160: rhyme control unit
170: voice conversion unit
180: tone converter

Claims (5)

delete In the music content production apparatus using character speech synthesis,
A music playback information acquisition unit for acquiring lyrics, singers, tracks, scales, musical lengths, beats, tempos, and music effects inputted for music reproduction;
A syntax analysis unit for analyzing the sentence of the lyrics obtained by the music reproduction information acquisition unit and converting the sentence into a form defined according to linguistic characteristics;
A pronunciation converter for converting the data analyzed by the syntax analyzer based on a phoneme;
An optimum phoneme selection unit for selecting an optimum phoneme according to a rule defined in advance from the optimum phoneme corresponding to the lyrics analyzed by the syntax analyzer and the pronunciation converter;
A sound source selection unit for acquiring the singer information acquired by the music reproduction information acquisition unit and selecting a sound source corresponding to the phoneme selected through the optimum phoneme selection unit from a sound source database;
A rhyme control unit for acquiring the optimal phoneme selected by the optimum phoneme selecting unit according to the sentence characteristics of the lyrics and controlling the length and pitch when combining the optimum phonemes;
A voice converter for acquiring the sentences of the lyrics synthesized by the rhyme control unit and matching the sentences of the lyrics acquired to be reproduced according to the scale, the length, the beat, and the tempo obtained by the music reproduction information acquisition unit;
A tone conversion unit for acquiring a voice converted by the voice conversion unit and matching a tone to the converted voice to be reproduced according to a music effect obtained by the music reproduction information acquisition unit;
Music using a character voice synthesis, comprising: a song and a background music synthesis unit for synthesizing the background music information acquired by the music reproduction information acquisition unit and the tone tone finally converted by the tone conversion unit; Content production device.
delete The method of claim 2,
The music reproduction information acquisition unit,
Lyrics information acquisition unit for acquiring lyrics information,
A background music information acquisition unit for acquiring the selected background music sound source information among the background music sound sources stored in the sound source database;
A vocal effect acquisition unit for acquiring vocal effect information adjusted by a user,
Apparatus for producing music contents using character speech synthesis, comprising: a singer information acquisition unit for acquiring singer information.
The method of claim 4, wherein
And a piano keyboard position acquiring unit for acquiring piano keyboard position information selected by a user in a virtual piano musical instrument output on the screen.

Images