JP6281447B2 - Speech synthesis apparatus and speech synthesis system - Google Patents

Description

  The present invention relates to a technique for notifying success or failure of realization of a singing technique.

2. Description of the Related Art Conventionally, a karaoke apparatus that plays a music piece according to music data that represents the pitch and length of each of the constituent sounds that make up the music piece is known (see Patent Document 1).
In the karaoke apparatus described in Patent Literature 1, a specific sound is obtained by subtracting a reference pitch that is a pitch of a constituent sound from a singing pitch specified according to a result of sequential frequency analysis of voices input during performance of a musical piece. Deriving the height difference. At the same time, in the karaoke apparatus described in Patent Document 1, an auxiliary sound having a pitch obtained by shifting the singing pitch by twice the specific pitch difference so as to be symmetrical with the singing pitch with the reference pitch as an axis. Output. At this time, the volume of the auxiliary sound is controlled to be larger as the specific pitch difference is larger, and to be “0 (that is, muted state)” if the specific pitch difference is “0”.

JP 2010-2331070 A

  The user of the karaoke apparatus described in Patent Document 1 can recognize that there is a difference between the singing pitch and the reference pitch when the auxiliary sound is output. Then, the pitch of the singing sound can be brought close to the reference pitch by pulling the singing sound by the auxiliary sound.

  However, in the karaoke apparatus described in Patent Document 1, it is impossible for the user to recognize whether or not the user has realized the singing technique to be used in the music, and the singing technique is made to the user. There is a problem that it cannot be realized.

  That is, in the conventional technique, there is a problem that the user cannot recognize the success or failure of realizing the singing technique to be used in the music, and the user cannot realize the singing technique.

  Then, an object of this invention is to provide the technique which makes a user recognize the success or failure of realization of a singing technique, and makes a user realize the singing technique.

The present invention made to achieve the above object relates to a speech synthesizer that generates and outputs an exemplary vocal, which is a voice of singing music, by speech synthesis.
In the present invention, a specified piece of music is designated as a specific piece of music having a plurality of notes composed of combinations of pitches and note values and lyrics are assigned to at least some of the plurality of notes. A note that constitutes a specific musical piece and that is assigned a lyrics is defined as a constituent note.

The speech synthesizer of the present invention includes feature data acquisition means, feature amount calculation means, comparison means, and speech synthesis means.
Among these, the feature data acquisition means includes a technique feature amount representing a feature amount of a specific technique that is a singing technique used by each designated musical note by a designated singer that is a designated singer, and the pitch, pitch value, and Singing feature data associated with each note property that is a combination of lyrics assigned to the constituent note is acquired. The feature amount calculation means analyzes the voice data input during the performance of the specific music piece, and calculates a singing feature amount that represents the feature amount of the singing skill at each of the constituent notes expressed by the voice data.

  Furthermore, the comparison unit compares the skill feature amount included in the singing feature data acquired by the feature data acquisition unit and the singing feature amount calculated by the feature amount calculation unit for each constituent note corresponding to each other. Then, as a result of the comparison by the comparison means, the speech synthesis means, if the skill difference that is the difference between the skill feature quantity and the singing feature quantity does not satisfy the prescribed condition specified in advance, An exemplary vocal for the target note is synthesized and output so that the technical difference in the target note, which has the same note property and is different from the constituent note, satisfies the specified condition.

  According to such a speech synthesizer, for a constituent note (that is, a target note) that has the same note property as the note property of the constituent note whose skill difference does not satisfy the specified condition, and is different from the constituent note. Can synthesize and output model vocals.

As a result, the user of the speech synthesizer can recognize whether or not the user himself can realize the singing technique used when the designated singer sings the specific music piece.
Moreover, according to the speech synthesizer of the present invention, the exemplary vocal for the target note is speech-synthesized so that the technical difference satisfies the specified condition. And, the user who sang by listening to this model vocal is pulled by the model vocal so that the singing skill when singing is brought closer to the singing technique used when the designated singer sings. Can do.

According to such a speech synthesizer, the user's own singing can be brought close to the singing mode of the designated singer.
For these reasons, according to the speech synthesizer of the present invention, the user can recognize the success or failure of realizing the singing technique, and the user can realize the singing technique.

  The singing technique of the present invention may include “vibrato”. In this case, as a result of the comparison by the comparison means, the speech synthesizing means of the present invention has a first prescribed threshold value in which the difference between the vibrato feature quantity in the singing feature quantity and the vibrato feature quantity in the technical feature quantity is defined in advance. If it exceeds, it is sufficient that the skill difference does not satisfy the prescribed condition.

According to such a speech synthesizer, the user can recognize the success or failure of “vibrato” as a singing technique.
Furthermore, the speech synthesizing means of the present invention synthesizes speech so that the difference between the highest frequency value and the lowest frequency value is increased in the exemplary vocal for the target note if the skill difference exceeds the first specified threshold. May be output.

  3. The speech synthesizer according to claim 2, wherein the skill difference does not satisfy the prescribed condition means that the designated singer sings the constituent note using “vibrato” but uses it for the constituent note. This is a case where the person could not sing using “Vibrato”. In this case, according to the speech synthesizer described in claim 3, it is possible to deepen the “vibrato” in the exemplary vocal at the target note.

By outputting such an exemplary vocal, it is easy to realize “vibrato” when the user sings the target note.
In addition, the singing technique of the present invention may include “shaking”. “Chicking” is a singing technique that sings continuously while changing the utterance pitch for a group of notes including two notes having different pitches along the time axis.

  In this case, as a result of the comparison by the comparison means, the speech synthesis means may be configured such that the difference between the screaming feature value in the singing feature value and the screaming feature value in the skill feature value exceeds a second prescribed threshold value defined in advance. For example, it is sufficient that the skill difference does not satisfy the prescribed condition.

According to such a speech synthesizer, it is possible to make the user recognize the success or failure of “shakuri” as a singing technique.
Further, the speech synthesis means of the present invention, if the skill difference exceeds the second specified threshold value, the note after the time axis in the note group is set as the target note, and the pitch of the model vocal is set to the pitch of the target note. Speech synthesis may be performed and output so that the pitch transition change rate at which the pitch transitions becomes faster.

  In the speech synthesizer according to claim 4, the case where the technical difference does not satisfy the prescribed condition means that the user sings "shrimp" even though the designated singer sang the constituent notes using "shrimp". This is the case when it could not be realized. In this case, according to the speech synthesizer, the pitch transition change speed at which the pitch of the model vocal transitions to the pitch of the target note can be increased.

By synthesizing and outputting such model vocals, it is possible to easily realize “scribbling” when the user sings the target note.
Further, the feature data acquisition means of the present invention includes a step of extracting vocal data representing a vocal sound from music data including a vocal sound sung by a designated singer, and score data comprising a plurality of notes constituting the music. Based on the obtaining step, each note constituting the obtained musical score data, and the vocal data, the note vocal data which is the section of the vocal data corresponding to each note constituting the singing melody in the music is specified, The step of determining the technical feature amount of each note vocal data and the target musical score data representing the score of the specific music are obtained, and the technical feature of the note whose note property matches the constituent note included in the acquired target musical score data Singing generated by executing the step of generating singing feature data by associating quantities Features data may be acquired.

By performing each of the steps described above, the singing feature data can be reliably generated. According to the speech synthesizer of the present invention, singing feature data can be acquired.
The present invention is a speech synthesis system that generates and outputs an exemplary vocal that is a voice of singing music by speech synthesis, and includes a feature data acquisition unit, a feature amount calculation unit, a comparison unit, and a speech synthesis unit. It may be made as a speech synthesis system provided with.

  Even with such a system, the same effect as that of the first aspect of the invention can be obtained.

It is a block diagram which shows schematic structure of a karaoke system. It is a flowchart which shows the process sequence of a technique feature production | generation process. It is explanatory drawing explaining the outline | summary of technical feature data. It is a flowchart which shows the process sequence of a song feature production | generation process. It is explanatory drawing explaining the outline | summary of singing characteristic data. It is a flowchart which shows the process sequence of a karaoke performance process. It is a figure which shows the outline | summary of the model vocal output in a karaoke performance process. It is a figure which shows the outline | summary of the model vocal output in a karaoke performance process.

Embodiments of the present invention will be described below with reference to the drawings.
<System configuration>
The karaoke system 1 shown in FIG. 1 plays a specific music that is a music specified by the user, and a singing technique used when a specified singer that is a singer specified by the user sings the specific music, and the specific music as a user. Is a system that informs the deviation from the singing technique used when singing with an exemplary vocal generated by speech synthesis.

Such a karaoke system 1 includes an information processing device 2, an information processing server 10, and a karaoke device 30.
The information processing device 2 calculates the technical feature data SF based on the music data WD and the MIDI music MD prepared for each music. The technical feature data SF referred to here is data representing features in the way the singer sings. The singer here includes a professional singer and an amateur singer.

  In the information processing server 10, singing feature data MS representing the feature amount of the singing skill when the designated singer sings the specific music based on the technical feature data SF and the MIDI music MD calculated by the information processing device 2. Generate.

The karaoke apparatus 30 plays a specific musical piece according to the MIDI musical piece MD stored in the information processing server 10, and generates and outputs an exemplary vocal based on the singing feature data MS generated by the information processing server 10.
<Music data>
Next, the music data WD is prepared in advance for each specific music, and includes music management information in which information related to the music is described, and master waveform data representing the performance sound of the music. The music management information includes music identification information for identifying music (hereinafter referred to as music ID) and singer identification information for identifying the singer who sang the music (hereinafter referred to as singer ID).

  The master waveform data of this embodiment is audio data including performance sounds of a plurality of musical instruments and vocal sounds sung by a singer. The audio data may be data constituted by an audio file in an uncompressed audio file format, or data constituted by an audio file in an audio compression format.

  In the following, voice waveform data representing the performance sound of the musical instrument included in the master waveform data is referred to as accompaniment data, and voice waveform data representing the vocal sound included in the master waveform data is referred to as vocal data.

Musical instrument performance sounds included in the accompaniment data of the present embodiment include percussion instrument (eg, drum, drum, cymbal, etc.) performance sounds, stringed instrument (eg, guitar, bass, etc.) performance sounds, percussion instrument (eg, piano) ) And wind instruments (eg, trumpet, clarinet, etc.).
<MIDI music>
The MIDI music MD is prepared in advance for each music, and has music management information, performance data, and lyrics data.

Among these, the music management information includes a music ID and a singer ID.
The performance data is data representing the score of one piece of music according to the well-known MIDI (Musical Instrument Digital Interface) standard. This performance data has at least a music ID and a music score track representing a music score for each musical instrument used in the music.

The musical score track defines at least the pitch (so-called note number) and the period during which the MIDI sound source outputs the performance sound (hereinafter referred to as the note length) for each performance sound output from the MIDI sound source. Has been. The note length in the score track is the performance start timing (so-called note-on timing) indicating the time from the start of the performance of the music until the output of the performance sound and the music until the output of the performance sound ends. Performance end timing (so-called note-off timing) representing the time from the start of the performance.

  That is, in the score track, one note NO is defined by the note number and the note length represented by the note-on timing and note-off timing. The musical score track functions as one musical score by arranging note NO in the order of performance. Note that the musical score track is prepared for each instrument such as a keyboard instrument, a stringed instrument, a percussion instrument, and a wind instrument, for example. Among these, in this embodiment, a specific musical instrument (for example, vibraphone) is defined as a musical instrument responsible for singing melody in music.

  On the other hand, the lyrics data is data relating to the lyrics of the music, and includes lyrics telop data and lyrics output data. The lyrics telop data represents characters that constitute the lyrics of the music (hereinafter referred to as lyrics component characters). The lyrics output data is data in which a timing correspondence relationship that associates the lyrics output timing, which is the output timing of the lyrics constituent characters, with the performance of the performance data is defined.

Specifically, in the timing correspondence relationship in the present embodiment, the timing for starting the output of the lyrics telop data is associated with the timing for starting the performance of the performance data. Furthermore, in the timing correspondence relationship, the lyrics output timing of each lyrics constituent character along the time axis of the music is defined by the elapsed time from the performance start of the performance data. As a result, the note NO of each performance sound defined in the score track is associated with each of the lyrics constituent characters.
<Information processing device>
The information processing apparatus 2 is a known information processing apparatus (for example, a personal computer) including an input receiving unit 3, an external output unit 4, a storage unit 5, and a control unit 6.

  The input receiving unit 3 is an input device that receives input of information and commands from the outside. The input device here is, for example, a key or switch, a reading drive for reading data stored in a portable storage medium (for example, CD, DVD, flash memory), or a communication port for acquiring information via a communication network. Etc. The external output unit 4 is an output device that outputs information to the outside. Here, the output device is a writing drive that writes data to a portable storage medium, a communication port that outputs information to a communication network, or the like.

  The storage unit 5 is a known storage device configured to be able to read and write stored contents. The storage unit 5 stores at least one piece of music data WD and at least one MIDI piece of music MD in association with each common piece of music.

  The control unit 6 is a known control device that is configured around a known microcomputer including a ROM 7, a RAM 8, and a CPU 9. The ROM 7 stores processing programs and data that need to retain stored contents even when the power is turned off. The RAM 8 temporarily stores processing programs and data. The CPU 9 executes each process according to a processing program stored in the ROM 7 or RAM 8.

The ROM 7 of the present embodiment stores a processing program for the control unit 6 to execute the technical feature generation processing. The skill feature generation process is a process for generating skill feature data SF based on the music data WD and the MIDI music MD stored in the storage unit 5.
<Information processing server>
The information processing server 10 includes a communication unit 12, a storage unit 14, and a control unit 16.

  Among these, the communication unit 12 performs communication between the information processing server 10 and the outside via a communication network. That is, the information processing server 10 is connected to the karaoke apparatus 30 via a communication network. The communication network referred to here may be a wired communication network or a wireless communication network.

  The storage unit 14 is a known storage device configured to be able to read and write stored contents. The storage unit 14 stores at least one MIDI music piece MD. 1 is an identifier for identifying the MIDI music piece MD stored in the storage unit 14 of the information processing server 10, and is a natural number of 1 or more. Furthermore, the technical feature data SF generated by the information processing apparatus 2 executing the data generation process is stored in the storage unit 14. 1 is an identifier for identifying the technical feature data SF stored in the storage unit 14 of the information processing server 10, and is a natural number of 1 or more.

  The control unit 16 is a known control device that is configured around a known microcomputer including a ROM 18, a RAM 20, and a CPU 22. The ROM 18 stores processing programs and data that need to retain stored contents even when the power is turned off. The RAM 20 temporarily stores processing programs and data. The CPU 22 executes each process according to a processing program stored in the ROM 18 or the RAM 20.

The ROM 18 of the control unit 16 stores a processing program for the control unit 16 to execute the singing feature generation process. The singing feature generation process is a singing that represents the characteristics of the singing technique that is displayed when a designated singer who has designated a song designated by the user of the karaoke device 30 (that is, a specific song) as a model person sings. This is processing for generating feature data MS. The designated singer may be the original (original) singer who sang the specific music, or may be a singer different from the original singer who sang the specific music.
<Karaoke equipment>
The karaoke apparatus 30 includes a communication unit 32, an input reception unit 34, a music playback unit 36, a storage unit 38, an audio control unit 40, a video control unit 46, and a control unit 50.

  In the communication unit 32, the karaoke apparatus 30 communicates with the outside via a communication network. The input receiving unit 34 is an input device that receives input of information and commands in accordance with external operations. Here, the input device is, for example, a key, a switch, a reception unit of a remote controller, or the like. The music playback unit 36 performs a music performance based on the MIDI music MD downloaded from the information processing server 10.

  The music reproducing unit 36 is, for example, a MIDI sound source. The voice control unit 40 is a device that controls voice input / output, and includes an output unit 42 and a microphone input unit 44. A microphone 62 is connected to the microphone input unit 44. As a result, the microphone input unit 44 acquires the sound input via the microphone 62. A speaker 60 is connected to the output unit 42. The output unit 42 outputs the sound source signal of the music reproduced by the music reproducing unit 36 and the sound source signal of the singing sound from the microphone input unit 44 to the speaker 60. The speaker 60 outputs the sound source signal output from the output unit 42 instead of sound.

  The video control unit 46 outputs a video or an image based on the video data sent from the control unit 50. The video control unit 46 is connected to a display unit 64 that displays video or images.

  The control unit 50 is configured around a known computer having at least a ROM 52, a RAM 54, and a CPU 56. The ROM 52 stores processing programs and data that need to retain stored contents even when the power is turned off. The RAM 54 temporarily stores processing programs and data. The CPU 56 executes each process according to a processing program stored in the ROM 52 or the RAM 54.

The ROM 52 of the control unit 50 stores a processing program for the control unit 50 to execute karaoke performance processing. In the karaoke performance process, a specific musical piece designated by the user is played, and based on the singing characteristic data MS, an exemplary vocal that is a singing voice reproducing the characteristic of the singing method of the designated singer is generated by voice generation and output. It is processing.
<Technical feature generation processing>
Next, a technique feature generation process executed by the control unit 6 of the information processing apparatus 2 will be described.

This technical feature generation process is activated at the timing when an activation command for activating a processing program is input via the input receiving unit 3 of the information processing apparatus 2.
2 is activated, first, the control unit 6 obtains a singer ID input via an input receiving unit (not shown) of the information processing apparatus 2 (S105). . Subsequently, the control unit 6 acquires one piece of music data WD including the singer ID acquired in S105 from all the music data WD stored in the storage unit 5 of the information processing apparatus 2 (S110). .

  Further, in the technique feature generation process, the control unit 6 associates the same music ID with the music data WD acquired in S110 out of all the MIDI music MD stored in the storage unit 5 of the information processing apparatus 2. The obtained MIDI music piece MD is acquired (S120). That is, in S110 and S120, the control unit 6 acquires music data WD and MIDI music MD for the same music.

  Subsequently, the control unit 6 uses the MIDI music MD (hereinafter referred to as “hereinafter referred to as MIDI music”) acquired in S120 during the reproduction time of each sound corresponding to each note in the music data WD (hereinafter referred to as “acquired music data”) acquired in S110. The acquisition MIDI is adjusted so that the performance timings of the notes constituting the “acquisition MIDI”) match (S130). As a technique for adjusting the acquired MIDI in S130, it is conceivable to use a known technique (for example, the technique described in Japanese Patent No. 5310679). Specifically, in the method described in Japanese Patent No. 5310679, the control unit 6 renders the acquired MIDI, and both the rendering result of the acquired MIDI and the master waveform data of the acquired music data are spectral data in a predetermined time unit. Convert to And the performance start timing and performance end timing of each performance sound are corrected so that the time on both spectrum data may synchronize. Note that DP matching may be used when adjusting the time on the spectrum data so as to be synchronized.

  In the technical feature generation process, the control unit 6 acquires master waveform data included in the acquired music data WD (S140). Subsequently, the control unit 6 separates and extracts vocal data and accompaniment data from the master disk waveform data acquired in S140 (S150). As a method for separating accompaniment data and vocal data in S150, there is a method using a pitch and a harmonic component estimated using a known method (for example, “PreFEst” described in JP-A-2008-134606). Conceivable. Note that PreFEst is a technique for estimating the pitch of a vocal (that is, the fundamental frequency) and the magnitude of a harmonic component by regarding the most prevalent voice waveform in the master waveform data as vocal data.

Further, the control unit 6 performs the musical note vocal Vo (a, i) based on the MIDI music MD (hereinafter referred to as “adjusted MIDI”) whose time has been adjusted in S130 and the vocal data extracted in S150. ) Is specified (S160). The note vocal Vo (a, i) is a section corresponding to each note NO (a, i) constituting the singing melody in the vocal data. In S160, the control unit 6 collates the vocal data extracted in S150 with the performance start timing nnt (a, i) and performance end timing nft (a, i) in the adjusted MIDI, so that the note vocal Vo ( a, i) are specified. In addition, the code | symbol a is a code | symbol which identifies a music, and the code | symbol i is a code | symbol which identifies the note NO of the song melody in a music.

  Further, in the technique feature generation process, the control unit 6 determines a technique feature quantity S (a, i) representing the feature quantity of the singing technique at each note vocal Vo (a, i) (S170). The singing skills mentioned here include at least “vibrato” and “shrimp”. Note that “chucking” is a technique of singing continuously while changing the utterance pitch for a group of notes including two notes having different pitches along the time axis.

  Among these, in calculating the technical feature amount (hereinafter referred to as “vibrato feature amount”) vib for “vibrato”, the control unit 6 first performs frequency analysis (DFT) for each of the note vocals Vo (a, i). carry out. And the control part 6 calculates the vibrato feature-value vib according to following (1) Formula.

vib (a, i) = vib_per (a, i) × vip_dep (a, i) (1)
However, vib_per (a, i) in the above equation (1) is an index representing the protruding accuracy of the spectrum peak in each note vocal Vo (a, i). This vib_per may be obtained by dividing the peak value of the frequency analysis result (that is, the amplitude spectrum) by the average value of the frequency analysis result. Further, vip_dep in the above equation (1) is a standard deviation of each note vocal Vo (a, i).

  In calculating the skill feature amount (hereinafter referred to as “shackle feature amount”) rise (a, i) for “shrimp”, the control unit 6 first calculates a differential change obtained by differentiating the pitch time change of vocal data. To do. Subsequently, the control unit 6 specifies the timing at which the differential change becomes a positive value along the time axis before the performance start timing nnt (a, i) of each note NO (a, i). Further, the control unit 6 obtains the cross-correlation value between the pitch time change of the vocal data and the predefined exemplary curve in the section from the specified timing to the performance start timing nnt (a, i), and the scribing feature amount. Calculate as rise (a, i).

  Further, in the technical feature generation process, the control unit 6 calculates the speech parameter P (a, i) of each note vocal Vo (a, i) (S180). The audio parameter P derived in S180 of the present embodiment includes at least the fundamental frequency (f0), the mel frequency cepstrum (MFCC), power, and each time difference. Since these fundamental frequency, MFCC, and power deriving methods are well known, detailed description thereof is omitted here. For example, if the fundamental frequency is used, autocorrelation along the time axis of the note vocal Vo, note vocal What is necessary is just to derive | lead-out using methods, such as the autocorrelation of the frequency spectrum of Vo, or a cepstrum method. In the case of MFCC, the result of frequency analysis (for example, FFT) for each time analysis window by applying a time analysis window to the note vocal Vo, and the result of logarithmizing the size for each frequency, Furthermore, it may be derived by frequency analysis. The power may be derived by integrating the result of squaring the amplitude by applying a time analysis window to the note vocal Vo and integrating it in the time direction.

Furthermore, in the technical feature generation process, the control unit 6 specifies the note property p (a, i) of the note NO (a, i) corresponding to each note vocal Vo (a, i) (S190). In S190 of the present embodiment, specifically, the control unit 6 extracts information on each note NO (a, i) defined in the acquired MIDI from the acquired MIDI as a note property p (a, i). To identify.

The note property p (a, i) mentioned here includes a specific note attribute, a front note attribute, and a rear note attribute.
The specific note attribute is information representing the attribute of the note NO (a, i). The specific note attributes include the scale (pitch) of note NO (a, i), note length, syllable of lyrics, and vowel of lyrics. The previous note attribute is information representing the attribute of the note NO (a, i-1) immediately preceding the note NO (a, i) (hereinafter referred to as the previous note) along the time axis. The preceding note attributes include the scale (pitch), note length, syllable of the previous note NO (a, i), and the preceding note NO (a, i-1) and note NO (a, i). Including the length of time between them (ie, the silent period).

  Furthermore, the post note attribute is information representing the attribute of the note NO (a, i + 1) immediately after the specific note NO (a, i) along the time axis (hereinafter referred to as a post note). The subsequent note attributes include a scale (pitch), a note length, a syllable of the lyrics, and a time length between the note NO (a, i) and the subsequent note NO (a, i + 1) (that is, a silent period). Including.

Note that the note length and the time length between notes in the note property p (a, i) may be quantized to a predetermined class.
In the technique feature generation process, the control unit 6 subsequently uses the note property p (a, i) of each note NO (a, i) specified in S190 to obtain the technique feature amount of the corresponding note NO (a, i). Corresponding to S (a, i) and speech parameter P (a, i) (S200).

  Furthermore, in the technical feature generation process, has the control unit 6 completed the steps from S110 to S200 for all the songs that satisfy the preset setting condition among the songs associated with the singer ID acquired in S105? It is determined whether or not (S210). The setting condition referred to here is that the singer ID acquired in S105 is associated, and both the music data WD and the MIDI music MD are prepared.

  As a result of the determination in S210, if the steps from S110 to S200 have not been completed for all the music pieces satisfying the setting condition (S210: NO), the control unit 6 returns the technique feature generation process to S110. Then, the control unit 6 acquires new song data WD from the song data WD associated with the singer ID designated in S105 (S110), and further, the MIDI song corresponding to the song data WD. The MD is acquired (S120), and the steps from S130 to S210 are executed.

  On the other hand, as a result of the determination in S210, if the steps from S110 to S200 have been completed for all songs (S210: YES), the control unit 6 shifts the technique feature generation process to S220.

In S220, the control unit 6 calculates a representative value for each of the skill feature S and the speech parameter P for each of the associated note properties p.
That is, in S220 of the present embodiment, the control unit 6 performs the technical feature amount S and the note feature S in the note NO having the common attributes for each of the note NO having the same specific note attribute, the preceding note attribute, and the subsequent note attribute. The arithmetic average of each of the speech parameters P is calculated as a representative value of the skill feature amount S. That is, in S220, the representative value of the vibrato feature amount vib and the representative value of the sneezing feature amount rise are calculated for each note property p as the representative values of the skill feature amount S. In S220, as the representative value of the voice parameter P, the representative value of the fundamental frequency, the representative value of the MFCC, and the representative value of the power are calculated for each note property p.

  The arithmetic mean calculated as the representative value in S220 is performed within the range of all the music pieces for which the skill feature amount S (a, i) and the speech parameter P (a, i) are calculated. The representative value calculated in S220 is not limited to the arithmetic mean result, and may be a median value or a mode value.

  Subsequently, the control unit 6 generates the technical feature data SF by associating the representative value of the technical feature amount S calculated in S220 and the representative value of the voice parameter P with the corresponding note property p and singer ID. Is stored in the storage unit 5 (S230).

Then, the control part 6 complete | finishes this technical feature generation process.
That is, in the skill feature generation process, skill feature data SF representing features in the way of singing a singer is generated for each singer. As shown in FIG. 3, the skill feature data SF generated by the skill feature generation process includes a singer ID, a representative value of the skill feature amount S in the common note property p, and a representative value of the speech parameter P. , And the common note property p.

Note that the technical feature data SF generated when the control unit 6 of the information processing device 2 executes the technical feature generation processing is stored in the storage unit 14 of the information processing server 10 using a portable storage medium. good. When the information processing device 2 and the information processing server 10 are connected via a communication network, the technical feature data SF stored in the storage unit 5 of the information processing device 2 is transferred via the communication network. Thus, the information may be stored in the storage unit 14 of the information processing server 10.
<Singing feature generation processing>
Next, singing feature generation processing executed by the control unit 16 of the information processing server 10 will be described.

  When the singing feature generation process shown in FIG. 4 is activated, the control unit 16 acquires one MIDI music MD from all the MIDI music MD stored in the storage unit 14 (S310). One MIDI musical piece MD acquired by the control unit 16 in S310 may be a MIDI musical piece MD corresponding to the musical piece designated in S510 (details will be described later) of the karaoke performance processing. It may be a MIDI music MD corresponding to a music specified via an input device (not shown) connected to.

  Subsequently, the control unit 16 analyzes the MIDI musical piece MD acquired in S310 and specifies the musical note property p (b, i) of each musical note NO (b, i) constituting the song melody in the MIDI musical piece MD. (S320). The code b here is a code for identifying the music corresponding to the MIDI music MD acquired in S310.

  In the singing feature generation process, the control unit 16 acquires one skill feature data SF from all the skill feature data SF stored in the storage unit 14 (S330). One skill feature data SF acquired by the control unit 16 in S330 is the skill feature data SF corresponding to the singer (that is, the designated singer) designated in S520 (details will be described later) of the karaoke performance process. Alternatively, it may be technical feature data SF corresponding to a singer designated through an input device (not shown) connected to the information processing server 10.

Subsequently, the control unit 16 generates singing feature data MS (S340). The generation of the singing feature data MS in S340 is performed using the skill feature amount S and the speech parameters included in the skill feature data SF acquired in S330 for each note NO (b, i) specified by the analysis in S320. This is realized by assigning a representative value of the technical feature quantity S and a representative value of the speech parameter P that satisfy certain conditions. The specific condition mentioned here is that the note property p that matches the note property p (b, i) in each note NO (b, i) is associated.

  By the way, there may be a case where the technical feature quantity S and the voice parameter P satisfying specific conditions do not exist in the technical feature quantity S and the voice parameter P. In S340 in this case, the control unit 16 obtains the representative values of the technical feature amount S and the speech parameter P associated with the note property p adjacent to the note property p (b, i) of the note NO (b, i). And assigning to the note NO (b, i). The adjacent note property p referred to here is, for example, that the scale is different by one note, the note length is different by one rank, or the like. In addition to the above method, the control unit 16 has the technical feature S corresponding to the two adjacent notes NO (b, i−1) and NO (b, i + 1) on the two axes of time and frequency. A value obtained by averaging the representative values of the voice parameter P may be assigned to the note NO (b, i).

  Further, in S340, the control unit 16 adds the song ID included in the MIDI song MD acquired in S310 to the generated song feature data MS and the singer ID included in the skill feature data SF acquired in S330 (ie, The designated singer ID) is assigned and stored in the storage unit 14.

Then, this singing characteristic production | generation process is complete | finished.
That is, in the singing feature generation process, as shown in FIG. 5, each note NO (b, i) constituting the singing melody in the MIDI music piece MD has a note common to the note property p of the note NO (b, i). A representative value of the skill feature amount S and a representative value of the voice parameter P to which the property p is associated are assigned. Thereby, the control part 16 of the information processing server 10 produces | generates song characteristic data MS.
<Karaoke performance processing>
Next, the karaoke performance process which the control part 50 of the karaoke apparatus 30 performs is demonstrated.

The karaoke performance process is activated when a command for activating a processing program for executing the karaoke performance process is input.
In the karaoke performance process shown in FIG. 6, when activated, the control unit 50 first selects the MIDI music MD corresponding to the music (ie, specific music) designated via the input receiving unit 34 as the information processing server 10. (S510). Subsequently, the control unit 50 sings a specific music piece by a designated singer designated via the input receiving unit 34 from all the singing feature data MS stored in the storage unit 14 of the information processing server 10. The singing feature data MS representing the singing technique is acquired (S520).

  Further, in the karaoke performance process, the control unit 50 specifies the technical note NO based on the MIDI music MD acquired in S510 and the singing feature data MS acquired in S520 (S530). The technical note NO referred to here is a musical note on a specific musical piece in which the singing skill is used when the designated singer sings the specific musical piece. Specifically, in the present embodiment, each note NO (c, i) associated with the skill feature amount S equal to or greater than a predetermined threshold value may be specified as the skill note NO (c, i). In other words, in the note NO (c, i) of the MIDI music piece MD, if the “vibrato feature amount vib” corresponding to the note NO (c, i) is equal to or greater than the first threshold, the designated singer uses “vibrato”. It is specified that the note is NO (c, i). In addition, in the note NO (c, i) of the singing feature data MS, if the “shrimp feature amount rise” corresponding to the note NO (c, i) is equal to or greater than the second threshold, the designated singer uses “shrimp”. It is specified that it is a technical note NO (c, i).

The code c here is a code for identifying the music corresponding to the MIDI music MD acquired in S510.
Furthermore, in S530, the control part 50 sets the skill information showing the singing skill which should be used by the model vocal in each skill note NO (c, i) to an initial value. The initial value referred to here is the skill feature amount S of each note NO represented in the singing feature data MS.

  Subsequently, the control unit 50 starts playing the MIDI music piece MD acquired in S510, and displays various information including lyrics on the display unit 64 (S540). Specifically, in the performance of the MIDI music MD in S540, the control unit 50 sequentially outputs the MIDI music MD to the music playback unit 36 along the time axis. The music reproducing unit 36 that has acquired the MIDI music MD performs the music. Then, the sound source signal of the music played by the music playback unit 36 is output to the speaker 60 via the output unit 42. Then, the speaker 60 outputs the sound source signal instead of sound.

  In S540, the control unit 50 outputs image signals representing various types of information to the video control unit 46. The video control unit 46 that has acquired the image signal displays various information on the display unit 64 in synchronization with the performance of the specific music in the music playback unit 36. In addition, in the various information displayed on the display unit 64, whether or not the user of the karaoke apparatus 30 can reproduce the singing technique in each technical note NO (c, i) in addition to the lyrics in each musical note NO of the specific music. Success / failure information indicating

  Further, in the karaoke performance process, the control unit 50 generates a model vocal by voice synthesis (S550). The speech synthesis in S570 is realized by a well-known formant synthesis. The exemplary vocal in the present embodiment is generated by performing voice synthesis by adjusting the voice parameter P so that the voice quality and singing skill of the designated singer are reproduced.

  Then, the control unit 50 outputs the exemplary vocal generated by performing the voice synthesis in S550 to the output unit 42 (S560). The output unit 42 emits an exemplary vocal from the speaker 60.

  Subsequently, in the karaoke performance process, the control unit 50 acquires the voice input through the microphone 62 and the microphone input unit 44 as singing voice data (S570). And the control part 50 memorize | stores the singing voice data acquired in S530 in the memory | storage part 38 (S580).

  Subsequently, in the karaoke performance process, the control unit 50 obtains the note singing data Vos (c, i) from the singing voice data along the time axis in the music based on the singing voice data stored in the storage unit 38. Extract (S590). The note singing data Vos mentioned here is a singing waveform obtained by singing the current note NO (c, i). The musical note singing data may be specified by, for example, replacing “vocal data” with “singing voice data” and using the same technique as S160 in the technique feature generation processing.

  Subsequently, in the karaoke performance process, the control unit 50 calculates a singing feature amount SS (c, i) representing a feature amount of the singing technique in each note singing data Vos (c, i) (S600). The singing feature amount SS (c, i) mentioned here includes a singing voice vibrato feature amount Vvib (c, i) and a singing voice chatting feature amount Vrise (c, i).

  Among these, the singing voice vibrato feature value Vvib (c, i) is a singing skill amount for “vibrato” in the note singing data Vos (c, i). The singing voice scribbling feature amount Vrise (c, i) is a singing skill amount for “scribbing” in the note singing data Vos (c, i). The singing voice vibrato feature value Vvib (c, i) and the singing voice chatter feature value Vrise (c, i) are calculated by changing “vocal data” to “singing voice data” and “note vocal” to “note singing data”. And the same technique as S170 in the skillful feature generation process may be used.

  Subsequently, in the karaoke performance process, the control unit 50 includes the skill feature amount S (c, i) included in the song feature data acquired in S520 and the song feature amount SS (c, i) calculated in S600. Are compared for the current note NO (c, i) (S610). Then, as a result of the comparison in S610, the control unit 50 determines that the skill difference, which is the difference between the skill feature amount S (c, i) and the singing feature amount SS (c, i), satisfies a prescribed condition. It is determined whether or not (S610). The prescribed conditions referred to here are conditions indicating that the user can reproduce the singing technique used by the designated singer. As the defining condition, it is conceivable that the skill difference is within a predetermined threshold range.

  For example, in the technical note NO (c, i), the absolute value of the result of subtracting the singing voice vibrato feature amount Vvib (c, i) from the vibrato feature amount vib (c, i) is less than the first prescribed threshold value defined in advance. If so, it may be determined that the skill difference satisfies the prescribed condition, that is, the user cannot reproduce the “vibrato” used by the designated singer. In addition, in the technical note NO (c, i), the absolute value of the result of subtracting the singing voice chatting feature value Vrise (c, i) from the scribbling feature value rise (c, i) is less than a second prescribed threshold value defined in advance. If so, it may be determined that the skill difference satisfies the prescribed condition, that is, the user cannot reproduce the “shrunk” used by the designated singer.

As a result of the determination in S610, if the skill difference satisfies the specified condition (S620: YES), the control unit 50 shifts the karaoke performance process to S630.
In S630, the control unit 50 displays success / failure information for the current note NO, and sets the synthesis mode of the exemplary vocal at the target note NO corresponding to the current note NO to the standard mode. Note that the target note NO is the current note NO (c, i along the time axis in a specific musical piece among the note NOs having the same note property p as the note property p of the current note NO (c, i). ) Is a note NO that appears after).

  That is, in S630, when the designated singer uses “vibrato” or “shrimp” in the current note NO, if the user can reproduce “vibrato” or “shrimp”, “vibrato” Or “success” is displayed as success / failure information, which means that the user can reproduce “suck”. The success icon here is, for example, a double circle. In S630, when the designated singer uses “vibrato” or “shrimp” in the current note NO, if the user can reproduce “vibrato” or “shrimp”, the current note NO (c, i) is set. Maintaining the singing skills with the exemplary vocals in all the corresponding target note NO at the initial value is executed by setting the synthesis mode to the standard mode.

Thereafter, the control unit 50 shifts the karaoke performance process to S660 described later in detail.
On the other hand, as a result of the comparison in S610, if the technical difference does not satisfy the prescribed condition (S620: NO), the control unit 50 shifts the karaoke performance process to S640. In S640, the control unit 50 determines whether the difference between the skill feature amount S (c, i) and the singing feature amount SS (c, i) represents singer emphasis or non-reproduction of the singer. To do. The singer emphasis mentioned here is a state where the user sings too much with emphasis on the singing technique with respect to the singing technique used by the designated singer. On the other hand, non-reproduction of the singer is a state in which the user cannot reproduce the singing technique used by the designated singer.

In the present embodiment, when the current note NO is a note NO other than the skillful note and the singing feature amount SS (c, i) indicates that the user has performed the singing skill, the singer Judged to be emphasis. Specifically, if the result of subtracting the singer feature amount SS from the skill feature amount S is a negative value and the absolute value thereof exceeds a predetermined first threshold value, singer emphasis is provided. It is determined that

  On the other hand, in the present embodiment, if the current note NO is a skill note and the singing feature amount SS (c, i) does not indicate that the user has performed the singing technique, the singer is not emphasized. It is determined that it is a non-reproduced singer. Specifically, if the result of subtracting the singer feature amount SS from the skill feature amount S is a positive value and the absolute value thereof exceeds a second prescribed threshold value defined in advance, the singers Judged to be a reproduction.

  If the result of determination in S640 is singer emphasis (S640: YES), the control unit 50 displays success / failure information for the current note NO and an example of the target note NO corresponding to the current note NO. The vocal synthesis mode is set as the suppression mode (S650).

  That is, in S650, if the designated singer does not use a singing technique such as “vibrato” or “shack” in the current note NO, and if the user is singing using “vibrato” or “shack”, the current note NO On the other hand, a suppression icon indicating that “vibrato” or “sneezing” is unnecessary is displayed as success / failure information. Moreover, in S650, maintaining the singing technique by the model vocal with all the target note NO corresponding to the present note NO at an initial value is performed as setting a synthetic | combination aspect to a suppression aspect.

Thereafter, the control unit 50 shifts the karaoke performance process to S670.
On the other hand, as a result of the determination in S640, if the singer is not emphasized (that is, if the singer is not reproduced) (S640: NO), the control unit 50 displays the success / failure information for the current note NO and the current time The skill information on the exemplary vocal at the target note NO corresponding to the note NO is set as an emphasis mode (S660).

  That is, in S660, as shown in FIG. 7, when the designated singer uses a singing technique such as “vibrato” or “shrimp” in the current note NO, the user sings using “vibrato” or “shrimp”. If not, a first emphasis icon indicating that “vibrato” or “suckling” should be used for the current note NO is displayed. Here, the first emphasis icon is, for example, “x mark”.

  Also, in S660, the adjustment of the speech parameter P so as to emphasize the singing skill in the exemplary vocals for all the target notes NO corresponding to the current note NO is performed by synthesizing the exemplary vocals for the target notes NO. This is performed by setting the aspect to the emphasized aspect.

  Specifically, in S660, if the singing technique used by the designated singer at the current note NO is “vibrato”, the control unit 50 performs the target note corresponding to the current note NO as shown in FIG. The voice parameter P is adjusted so that the difference between the maximum value of the frequency and the minimum value of the frequency becomes large in the “vibrato” with the exemplary vocal in NO. On the other hand, if the singing technique used by the designated singer in the current note NO is “Sharukuri”, the control unit 50 is an exemplary vocal in the target note NO corresponding to the current note NO as shown in FIG. In “Shacking”, the voice parameter P is adjusted so that the pitch transition change rate at which the pitch of the model vocal transitions to the pitch of the target note NO is faster than the initial value.

Thereafter, the control unit 50 shifts the karaoke performance process to S670.
In S670, the control unit 50 subsequently determines whether or not the performance of the specific music has ended. If the result of the determination in S670 is that the performance of the specific music has not ended (S670: NO), the control unit 50 returns the karaoke performance processing to S540. In S540, the control unit 50 plays the continuation of the specific music piece and displays various information. In subsequent S550, if the current note NO (c, i) is the target note NO, the control unit 50 performs S630, S650,
The exemplary vocal is synthesized according to the synthesis mode set in any one of S660. Thereafter, the steps from S570 to S670 are repeated.

On the other hand, if the result of determination in S670 is that the performance of the specific music has been completed (S670: YES), the control unit 50 ends the karaoke performance processing.
That is, in the karaoke performance processing of the present embodiment, the voice data input during the performance of the specific music is analyzed, and the characteristic amount of the singing technique is expressed by each note (component note) NO expressed by the voice data. The singing feature amount SS is calculated. The calculated singing feature amount S and the skill feature amount S included in the singing feature data MS are compared for each note NO corresponding to each other. As a result of comparison, if the skill difference that is the difference between the skill feature quantity S and the singing feature quantity SS does not satisfy the prescribed condition, a note property p that is the same as the note property p of the current note NO is obtained. An exemplary vocal is generated by speech synthesis and output so that the technical difference satisfies the specified condition in the target note NO that is a note NO that is different from the current note NO.

  In the voice synthesis, if the singing technique used by the designated singer at the current note NO is “vibrato”, the maximum frequency and the frequency at the “vibrato” at the model vocal corresponding to the current note NO are “vibrato”. A model vocal is generated and output so that the difference from the minimum value of the value increases. On the other hand, if the singing technique used by the designated singer at the current note NO is “Sharukuri”, the model vocal at the pitch of the target note NO at “Sharukuri” in the model vocal at the target note NO corresponding to the current note NO. An exemplary vocal is generated by voice synthesis and output so that the pitch transition change speed at which the pitch of the voice changes becomes faster than the initial value.

In addition, the karaoke system 1 of this embodiment is an example of the speech synthesis system described in the scope of claims, and the karaoke apparatus 30 is an example of the speech synthesis apparatus described in the scope of claims.
[Effect of the embodiment]
As described above, according to the karaoke performance process, if the technical difference in the current note NO (c, i) does not satisfy the specified condition, the model in the target note NO corresponding to the current note NO (c, i). Voice synthesis is performed so that the technical difference satisfies the specified condition.

As a result, the user of the speech synthesizer can recognize whether or not the user himself can realize the singing technique used when the designated singer sings the specific music piece.
Moreover, in the karaoke performance process, if the user cannot reproduce the singing technique used by the designated singer, the skill difference in the target note NO corresponding to the current note NO (c, i) is determined as the model vocal generated by speech synthesis. The specified conditions are met.

  And the user of the karaoke apparatus 30 who sang and listened to this model vocal is when the designated singer sings the singing skill when he sang because the singing voice when the user sings is pulled by the model vocal. It can be close to the singing technique used.

  According to such a karaoke apparatus 30, the user's own singing can be brought close to the singing mode of the designated singer. From these things, according to the karaoke apparatus 30, the user can recognize the success or failure of realization of the singing technique, and the user can realize the singing technique.

In particular, in the karaoke performance process, when the designated singer uses “vibrato” in the current note NO and the user does not sing using “vibrato”, the exemplary vocal in the target note NO corresponding to the current note NO. In “Vibrato”, the audio parameter P is adjusted so that the difference between the maximum frequency value and the minimum frequency value becomes large. Thereby, the “vibrato” in the exemplary vocal in the target note output from the karaoke device 30 can be deepened.

By outputting such an exemplary vocal, it is easy to realize “vibrato” when the user sings the target note.
Also, in the karaoke performance process, when the designated singer uses “shakuri” in the current note NO and the user does not sing using “shakuri”, the exemplary vocal in the target note NO corresponding to the current note NO. In the “shacking”, the voice parameter P is adjusted so that the pitch transition change speed at which the pitch of the model vocal shifts to the pitch of the target note NO is faster than the initial value. Thereby, the pitch transition change speed of the “shrimp” in the exemplary vocal with the target note output from the karaoke device 30 can be increased.

By synthesizing and outputting such an exemplary vocal, when the user sings the target note NO, it is possible to easily realize “shrimp”.
[Other Embodiments]
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

  Moreover, in the said embodiment, although the information processing server 10 performed the song feature production | generation process, the apparatus which performs a song feature production | generation process is not restricted to the information processing server 10. FIG. In other words, the information processing device 2 or the karaoke device 30 may be the device that executes the singing feature generation process.

  In addition, the aspect which abbreviate | omitted a part of structure of the said embodiment is also embodiment of this invention. Further, an aspect configured by appropriately combining the above embodiment and the modification is also an embodiment of the present invention. Moreover, all the aspects which can be considered in the limit which does not deviate from the essence of the invention specified by the wording described in the claims are the embodiments of the present invention.

In addition to the above-described speech synthesizer, the present invention can be realized in various forms such as a program executed by a computer to realize speech synthesis and a speech synthesis method.
[Correspondence between Embodiment and Claims]
Finally, the relationship between the description of the above embodiment and the description of the scope of claims will be described.

  The function realized by executing S520 of the karaoke performance processing is an example of the feature data acquisition means described in the claims, and the function realized by executing S560 to S600 is claimed. It is an example of the feature-value calculation means described in the range. Further, the function obtained by executing S610 of the karaoke performance processing is an example of the comparison means described in the claims, and the function obtained by executing S620 to S660, S540, and S550 is the voice. It is an example of a synthetic | combination means.

  DESCRIPTION OF SYMBOLS 1 ... Speech synthesis system 2 ... Information processing device 3 ... Input reception part 4 ... External output part 5, 14, 38 ... Memory | storage part 6, 16, 50 ... Control part 7, 18, 52 ... ROM 8, 20, 54 ... RAM 9, 22, 56 ... CPU 10 ... Information processing server 12, 32 ... Communication unit 30 ... Karaoke device 34 ... Input reception unit 36 ... Music playback unit 40 ... Audio control unit 42 ... Output unit 44 ... Microphone input unit 46 ... Video control Unit 60 ... Speaker 62 ... Microphone 64 ... Display unit

Claims (8)

A voice synthesizer that generates and outputs an exemplary vocal that is the voice of singing a song by voice synthesis,
Among the music pieces having a plurality of notes composed of combinations of pitches and note values, and lyrics are assigned to at least some of the plurality of notes, one designated music piece is designated as a specific music piece, and the specific music piece A note to which the lyrics are assigned as a constituent note,
A skill feature amount representing a feature amount of a specific skill, which is a singing skill used by each designated note by a designated singer who is a designated singer, is assigned to a pitch, a note value, and the relevant note. Characteristic data acquisition means for acquiring singing characteristic data associated with each note property that is a combination of lyrics;
Analyzing voice data input during the performance of the specific music, a feature quantity calculation means for calculating a singing feature quantity representing a singing skill quantity in each of the constituent notes expressed in the voice data;
Comparison means for comparing the skill feature amount included in the singing feature data acquired by the feature data acquisition means and the singing feature amount calculated by the feature amount calculation means for each constituent note corresponding to each other;
As a result of the comparison by the comparison means, if the skill difference that is the difference between the skill feature quantity and the singing feature quantity does not satisfy the prescribed condition specified in advance, the note property that is the same as the note property of the constituent note Voice synthesis means for synthesizing and outputting an exemplary vocal for the target note such that the technical difference in the target note that is a constituent note different from the constituent note satisfies the specified condition. Prepared ,
The singing technique includes vibrato,
The speech synthesis means
If the difference between the vibrato feature amount in the singing feature amount and the vibrato feature amount in the skill feature amount exceeds a predetermined first threshold range as a result of the comparison by the comparison means, the skill is determined. It is assumed that the difference does not satisfy the specified conditions
A speech synthesizer characterized by the above. The speech synthesis means
If the skill difference exceeds the first threshold range, voice synthesis is performed so that the difference between the maximum frequency value and the minimum frequency value is increased in the exemplary vocal for the target note. The speech synthesizer according to claim 1 . The singing technique includes a squeak that continuously sings while changing the utterance pitch with respect to a group of notes including two notes having continuous pitches along the time axis,
The speech synthesis means
If the difference between the screaming feature value in the singing feature value and the screaming feature value in the skill feature value exceeds a predetermined second threshold range as a result of the comparison by the comparison means, the skill difference The speech synthesizer according to claim 1 or 2 , wherein the predetermined condition is not satisfied. A voice synthesizer that generates and outputs an exemplary vocal that is the voice of singing a song by voice synthesis,
Among the music pieces having a plurality of notes composed of combinations of pitches and note values, and lyrics are assigned to at least some of the plurality of notes, one designated music piece is designated as a specific music piece, and the specific music piece A note to which the lyrics are assigned as a constituent note,
A skill feature amount representing a feature amount of a specific skill, which is a singing skill used by each designated note by a designated singer who is a designated singer, is assigned to a pitch, a note value, and the relevant note. Characteristic data acquisition means for acquiring singing characteristic data associated with each note property that is a combination of lyrics;
Analyzing voice data input during the performance of the specific music, a feature quantity calculation means for calculating a singing feature quantity representing a singing skill quantity in each of the constituent notes expressed in the voice data;
Comparison means for comparing the skill feature amount included in the singing feature data acquired by the feature data acquisition means and the singing feature amount calculated by the feature amount calculation means for each constituent note corresponding to each other;
As a result of the comparison by the comparison means, if the skill difference that is the difference between the skill feature quantity and the singing feature quantity does not satisfy the prescribed condition specified in advance, the note property that is the same as the note property of the constituent note Voice synthesis means for synthesizing and outputting an exemplary vocal for the target note such that the technical difference in the target note that is a constituent note different from the constituent note satisfies the specified condition. Prepared ,
The singing technique includes a squeak that continuously sings while changing the utterance pitch with respect to a group of notes including two notes having continuous pitches along the time axis,
The speech synthesis means
If the difference between the screaming feature value in the singing feature value and the screaming feature value in the skill feature value exceeds a predetermined second threshold range as a result of the comparison by the comparison means, the skill difference Does not meet the above requirements
A speech synthesizer characterized by the above. The speech synthesis means
If the skill difference exceeds the second threshold range, the note after the time axis in the note group is set as the target note, and the pitch of the model vocal shifts to the pitch of the target note. The speech synthesizer according to claim 3 or 4, wherein speech synthesis is performed so that a pitch transition change speed becomes faster. The feature data acquisition means includes
Extracting vocal data representing the vocal sound from music data including the vocal sound sung by the designated singer;
Obtaining musical score data composed of a plurality of notes constituting the music;
Based on each note constituting the obtained musical score data and the vocal data, the note vocal data which is a section of the vocal data corresponding to each note constituting the melody of the song is specified, and each note Determining the skill features in vocal data;
The singing feature data is obtained by acquiring target musical score data representing the score of the specific musical piece, and associating a technical feature amount in a note having the same note property with the constituent note included in the acquired target musical score data. The voice synthesizer according to any one of claims 1 to 5, wherein the singing characteristic data generated by executing the step is generated. A voice synthesizer that generates and outputs an exemplary vocal that is the voice of singing a song by voice synthesis,
Among the music pieces having a plurality of notes composed of combinations of pitches and note values, and lyrics are assigned to at least some of the plurality of notes, one designated music piece is designated as a specific music piece, and the specific music piece A note to which the lyrics are assigned as a constituent note,
A skill feature amount representing a feature amount of a specific skill, which is a singing skill used by each designated note by a designated singer who is a designated singer, is assigned to a pitch, a note value, and the relevant note. Characteristic data acquisition means for acquiring singing characteristic data associated with each note property that is a combination of lyrics;
Analyzing voice data input during the performance of the specific music, a feature quantity calculation means for calculating a singing feature quantity representing a singing skill quantity in each of the constituent notes expressed in the voice data;
Comparison means for comparing the skill feature amount included in the singing feature data acquired by the feature data acquisition means and the singing feature amount calculated by the feature amount calculation means for each constituent note corresponding to each other;
As a result of the comparison by the comparison means, if the skill difference that is the difference between the skill feature quantity and the singing feature quantity does not satisfy the prescribed condition specified in advance, the note property that is the same as the note property of the constituent note Voice synthesis means for synthesizing and outputting an exemplary vocal for the target note such that the technical difference in the target note that is a constituent note different from the constituent note satisfies the specified condition. Prepared ,
The feature data acquisition means includes
Extracting vocal data representing the vocal sound from music data including the vocal sound sung by the designated singer;
Obtaining musical score data composed of a plurality of notes constituting the music;
Based on each note constituting the obtained musical score data and the vocal data, the note vocal data which is a section of the vocal data corresponding to each note constituting the melody of the song is specified, and each note Determining the skill features in vocal data;
The singing feature data is obtained by acquiring target musical score data representing the score of the specific musical piece, and associating a technical feature amount in a note having the same note property with the constituent note included in the acquired target musical score data. Generating step and
The singing feature data generated by executing
A speech synthesizer characterized by the above. A voice synthesis system that generates and outputs an exemplary vocal that is the voice of singing a song by voice synthesis,
Among the music pieces having a plurality of notes composed of combinations of pitches and note values, and lyrics are assigned to at least some of the plurality of notes, one designated music piece is designated as a specific music piece, and the specific music piece A note to which the lyrics are assigned as a constituent note,
A skill feature amount representing a feature amount of a specific skill, which is a singing skill used by each designated note by a designated singer who is a designated singer, is assigned to a pitch, a note value, and the relevant note. Characteristic data acquisition means for acquiring singing characteristic data associated with each note property that is a combination of lyrics;
Analyzing voice data input during the performance of the specific music, a feature quantity calculation means for calculating a singing feature quantity representing a singing skill quantity in each of the constituent notes expressed in the voice data;
Comparison means for comparing the skill feature amount included in the singing feature data acquired by the feature data acquisition means and the singing feature amount calculated by the feature amount calculation means for each constituent note corresponding to each other;
As a result of the comparison by the comparison means, if the skill difference that is the difference between the skill feature quantity and the singing feature quantity does not satisfy the prescribed condition specified in advance, the note property that is the same as the note property of the constituent note Voice synthesis means for synthesizing and outputting an exemplary vocal for the target note such that the technical difference in the target note that is a constituent note different from the constituent note satisfies the specified condition. Prepared ,
The singing technique includes vibrato,
The speech synthesis means
If the difference between the vibrato feature amount in the singing feature amount and the vibrato feature amount in the skill feature amount exceeds a predetermined first threshold range as a result of the comparison by the comparison means, the skill is determined. A speech synthesis system characterized in that a difference does not satisfy the prescribed condition .