JP2015072432A - Karaoke device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce unpleasantness given to a user to the minimum by making a delay of the synthetic sound such as a guide vocal for the karaoke performance as small as possible when restored to an ordinary reproduction state from a so-called trick play such as a temporary stop, a fast-forward operation, and a rewinding operation.SOLUTION: When finishing a trick play such as a temporary stop, a performance speed of the karaoke musical piece is restored to an original speed, and a generation of a synthetic sound and an output of the synthetic sound are performed according to a synthetic sound synchronous timing by setting the synthetic sound synchronous timing which is a timing to allow an output of the synthetic sound to synchronize with a performance of the karaoke musical piece. When the generation of the synthetic sound is not made in time for the synthetic sound synchronous timing, a digital sound signal is generated prior to the generation of the synthetic sound, and after finishing the generation of the digital sound signal, with the generation of the synthetic sound concentrated, the generation of the synthetic sound is performed by a speed having a value larger than the performance speed of the karaoke musical piece until the generation of the synthetic sound catches up to the performance of the karaoke musical piece.

Description

  The present invention relates to a karaoke apparatus that performs karaoke music based on MIDI data and outputs a synthesized voice generated by voice synthesis based on compressed voice data in accordance with the performance of the karaoke music.

  Patent Document 1 describes a karaoke apparatus having a guide vocal function that supports a singing by outputting a guide vocal that is synthesized based on voice data of a singer during karaoke performance of karaoke music.

JP 2009-244789 A

  However, the technique described in Patent Document 1 has the following problems. That is, with respect to the speech synthesis performed to generate the guide vocal as described above, since the processing load on the processing device such as the CPU provided in the karaoke device is large, from so-called trick play such as pause, fast forward, and rewind. When returning to the normal playback state, there is a problem that voice synthesis is not in time for karaoke performance, and the user feels uncomfortable. Note that the higher the density of the number of pronunciations included in the speech data, the greater the processing load on the processing device for speech synthesis, and the delay of the synthesized speech with respect to the karaoke performance becomes significant.

  The present invention has been made in view of such problems, and its purpose is to provide a guide to karaoke performance when returning from a so-called trick play such as pause, fast forward, and rewind to a normal playback state. It is an object of the present invention to provide a technique for minimizing the delay of synthesized speech such as vocals and minimizing discomfort to the user.

  The karaoke apparatus according to claim 1, which has been made to solve the above-described problem, is based on music data, music performance means for performing karaoke music at a predetermined performance speed, and voice synthesis based on voice synthesis data. Voice synthesis processing means for generating a synthesized voice for singing the karaoke music, and outputting the generated synthesized voice at the same value as the performance speed of the karaoke music along with the performance of the karaoke music by the music performance means; Karaoke music performance by the means, generation of synthesized speech by the voice synthesis processing means, control means for controlling the output of the synthesized voice by the voice synthesis processing means, and temporarily changing the performance speed of the karaoke music by the music performance means Instruction accepting means for accepting a change instruction to the effect or a release instruction to cancel the change.

  When the instruction accepting means accepts the change instruction, the control means changes the performance speed of the karaoke music by the music playing means, and also generates synthesized speech and the speech synthesis process by the speech synthesis processing means. The output of the synthesized speech by the means is stopped.

  Thereafter, when the instruction receiving means receives the release instruction, the control means restores the performance speed of the karaoke music by the music performance means and outputs the synthesized speech by the voice synthesis processing means. Synthetic voice synchronization timing, which is a timing to synchronize with the performance of the karaoke music by the music performance means, is set, generation of synthesized voice by the voice synthesis processing means and synthesis voice by the voice synthesis processing means in accordance with the synthetic voice synchronization timing Is output.

  In addition, when the synthesized speech generation by the speech synthesis processing means is not in time for the synthesized speech synchronization timing, the control means generates the synthesized speech by the speech synthesis processing means for the karaoke music piece by the music performance means. The synthesized speech is generated by the speech synthesis processing means at a speed larger than the performance speed of the karaoke piece until the performance is caught up.

  According to the karaoke apparatus of the present invention configured in this way, when returning to a normal playback state from so-called trick play such as pause, fast forward, and rewind, the delay of synthesized speech such as guide vocals with respect to karaoke performance is reduced. It can be made as small as possible to minimize discomfort for the user.

  A karaoke apparatus according to claim 2 is the karaoke apparatus according to claim 1, wherein a digital audio signal of karaoke music is generated on the basis of the compressed audio data, and the generated digital audio signal is converted into a karaoke music by the music performance means. A live sound reproduction processing means that outputs the performance at the same speed as the performance speed of the karaoke music piece, and the control means generates a digital audio signal by the raw sound reproduction processing means, and a digital audio signal by the raw sound reproduction processing means Control the output of.

  When the instruction accepting unit accepts the change instruction, the control unit stops the generation of the digital audio signal by the raw sound reproduction processing unit and the output of the digital audio signal by the raw sound reproduction processing unit. When the instruction accepting unit accepts the release instruction, the control unit synchronizes the output of the digital audio signal by the live sound reproduction processing unit with the performance of the karaoke music by the music playing unit, and the audio signal synchronization timing. And generating the digital audio signal by the raw sound reproduction processing means and outputting the digital audio signal by the raw sound reproduction processing means in synchronization with the audio signal synchronization timing. Set the signal synchronization timing at a time delayed by a predetermined time from the set time. It is way.

  According to the karaoke apparatus of the present invention configured as described above, when the synthesized speech generation by the speech synthesis processing unit is not in time for the synthesized speech synchronization timing, the digital sound signal is generated by the raw sound reproduction processing unit. The generation of the digital audio signal is started in advance, and after the end of the generation of the digital audio signal, the synthesized speech by the speech synthesis processing unit is focused on the generation of the synthesized speech by the speech synthesis processing unit. Is quick to catch up with the performance of karaoke music by the music playing means. Therefore, it is possible to minimize the delay of the synthesized voice such as the guide vocal with respect to the karaoke performance, and to minimize the discomfort given to the user.

  The karaoke apparatus according to claim 3 is the karaoke apparatus according to claim 1 or 2, wherein the control means is unable to generate the synthesized voice by the voice synthesis processing means in time for the synthesized voice synchronization timing. In this case, the output level of the synthesized speech by the speech synthesis processing means is reduced until the generation of the synthesized speech by the speech synthesis processing means catches up with the performance of the karaoke music by the music performance means.

  According to the karaoke apparatus of the present invention configured in this way, when returning to a normal playback state from so-called trick play such as pause, fast forward, and rewind, the delay of synthesized speech such as guide vocals with respect to karaoke performance is reduced. The user can be prevented from feeling as much as possible, and the discomfort given to the user can be minimized.

  Further, according to the present invention, as shown in claim 4, the karaoke music is processed by music performance processing for performing karaoke music at a predetermined performance speed based on music data, and by voice synthesis based on voice synthesis data. A voice synthesis process for generating a synthesized voice to sing, outputting the generated synthesized voice at the same value as the performance speed of the karaoke music together with the performance of the karaoke music, and temporarily changing the performance speed of the karaoke music An instruction reception process for receiving a change instruction or a release instruction for canceling the change, and when the change instruction is received, the performance speed of the karaoke song is changed, and the generation of the synthesized voice and the generation of the synthesized voice When the output is stopped and then the release instruction is accepted, the performance speed of the karaoke music is restored and the output of the synthesized speech is Synthetic voice synchronization timing, which is a timing to synchronize with the performance of Laoke music, is set, and the synthetic voice is generated and the synthetic voice is output in accordance with the synthetic voice synchronization timing. A control process for generating the synthesized speech at a speed greater than the performance speed of the karaoke song until the synthesized speech generation catches up with the performance of the karaoke song when the synchronization timing is not met. It can also be realized as a program for execution.

It is a block diagram which shows schematic structure of the karaoke apparatus 1 of embodiment. It is a flowchart which shows the procedure of an input reception process. It is a flowchart which shows the procedure of a hard MIDI performance process. It is a flowchart which shows the procedure of a live sound performance process. It is a flowchart which shows the procedure of a soft MIDI performance process. It is a flowchart which shows the procedure of a singing voice synthetic | combination reproduction | regeneration process. It is explanatory drawing explaining the various processes which the main control part 10 performs. It is explanatory drawing explaining the various processes which the main control part 10 performs.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[1. Description of configuration of karaoke apparatus 1]
As shown in FIG. 1, the karaoke apparatus 1 includes a main control unit 10, a hard disk drive (HDD) 20, a sound processing module 30, an operation processing unit 40, an operation unit 41, a communication interface unit 42, a microphone amplifier 43, an analog-digital A conversion unit (ADC) 44, a video processing unit 45, and the like are provided. In addition, a monitor 50, a speaker amplifier 51, a speaker 52, and a microphone 53 are connected to the karaoke apparatus 1.

[1.1. Description of Main Control Unit 10 and HDD 20]
The main control unit 10 is an information processing device that controls the entire karaoke apparatus 1. The main control unit 10 is configured mainly with a CPU, ROM, RAM, I / O, and the like, and executes various processes based on programs and data read from the HDD 20.

  The HDD 20 is a karaoke database in which many pieces of music data indicating karaoke accompaniment contents and lyrics data indicating lyrics, system programs and application programs for controlling the operation of the karaoke apparatus 1, data used for reproducing various information contents, and the like. The various data are stored. The music data stored in the HDD 20 includes performance data (MIDI data) of accompaniment music generated by the MIDI standard, compressed audio data obtained by converting the accompaniment raw sound into digital data by a predetermined compressed audio method, and musical score data for guide vocals And speech synthesis data comprising speech synthesis parameters and phoneme data.

  The main control unit 10 includes a live sound reproduction processing unit 11, a MIDI sequencer 12, a MIDI performance control unit 13, a software MIDI sound source unit 14, and a voice synthesis processing unit 15 as a functional configuration related to the performance of karaoke music. These configurations are realized by a program executed by the main control unit 10.

  Among these, the live sound reproduction processing unit 11 performs so-called live sound reproduction in which the music data of the compressed audio data read from the HDD 20 is decoded to generate karaoke music and output at the same value as the performance speed of the karaoke music. Music playback function. The digital audio signal of the music output from the raw sound reproduction processing unit 11 is input to the key control unit 31 of the audio processing module 30.

  The MIDI sequencer 12 is a software sequencer for causing a MIDI sound source 32 mounted on the audio processing module 30 to play a musical piece based on MIDI data. The MIDI sequencer 12 sends performance control data (MIDI message) to the MIDI sound source 32 of the sound processing module 30 based on the MIDI data read from the HDD 20, and causes the MIDI sound source 32 to generate musical tones. In the present embodiment, the generated musical sound is PCM data of musical sound.

  The MIDI performance control unit 13 is a software sequencer for causing the software MIDI tone generator unit 14 of the main control unit 10 to play karaoke music at a predetermined performance speed based on MIDI data.

  The software MIDI sound source unit 14 is software that simulates a MIDI sound source by a program executed by the CPU of the main control unit 10 and outputs a digital sound signal (such as PCM data) of musical sound based on MIDI data.

  The voice synthesis processing unit 15 generates an ideal singing voice for a song based on the voice synthesis data composed of the score data for the guide vocal, the parameters for voice synthesis, and the phoneme data, and the voice of the user who provided the phoneme data. Is a software sequencer that includes a speech synthesis engine for generating synthesized speech synthesized so as to be realized by the above and outputting it at the same value as the performance speed of the karaoke music.

  For example, Japanese Patent Application Laid-Open No. 09-90966, Japanese Patent Application Laid-Open No. 2013-114131, and Japanese Patent Application Laid-Open No. 2013-114131 are techniques for generating a singing voice by a voice synthesis engine from musical score data, speech synthesis parameters, phoneme data and waveform data. Since a plurality of prior arts such as -190473 are disclosed and already known, they will not be described in detail here.

  In the karaoke apparatus 1 of the present embodiment, a live music reproduction based on compressed audio data, a hardware MIDI sound source, a software MIDI sound source, and a synthetic sound reproduction based on data for speech synthesis are simultaneously used to play a music piece using a plurality of sound sources. Assumes to do. Therefore, the main control unit 10 performs the processes of the raw sound reproduction processing unit 11, the MIDI sequencer 12, the MIDI performance control unit 13, and the voice synthesis processing unit 15 in parallel.

  The main control unit 10 corresponds to music performance means, speech synthesis processing means, raw sound reproduction processing means, control means, and instruction receiving means. The processing performed by the main control unit 10 corresponds to music performance processing, speech synthesis processing, instruction reception processing, and control processing.

[1.2. Explanation of Configuration of Audio Processing Module 30]
The sound processing module 30 performs various digital sound processing on the input signal corresponding to each sound source from the main control unit 10 and the digital sound signal of the singing sound input through the microphone 53 connected to the karaoke apparatus 1. It is a control device. The audio processing module 30 includes a key control unit 31, a MIDI sound source 32, a mute switch unit 33, an effect unit 34, a mixer 35, a digital / analog conversion unit (DAC) 36, and a mute switch unit 37.

  The key control unit 31 adjusts the pitch of the performance sound key set by the user via the operation unit 41 or the like with respect to the digital sound signal of the live sound output from the live sound playback processing unit 11 of the main control unit 10. At the same time, the pitch of the reproduced sound is changed and output to the mixer 35. The key sequence of the hardware MIDI tone generator and the software MIDI tone generator is adjusted by the MIDI sequencer 12 and the MIDI performance controller 13 of the main controller 10.

  The MIDI sound source 32 generates a digital audio signal of musical tone based on the MIDI message sent from the MIDI sequencer 12 and outputs it to the mixer 35. The function of the MIDI sound source 32 is the same as that of a well-known hardware MIDI sound source mounted on a karaoke apparatus or the like.

  The mute switch unit 33 is a switch in the middle of the output path of the digital audio signal output from the software MIDI tone generator unit 14 of the main control unit 10. The mute switch unit 33 turns on / off the input of the digital audio signal from the software MIDI tone generator unit 14 to the mixer 35 in accordance with the control from the MIDI performance control unit 13.

The effect unit 34 adds an acoustic effect such as echo to the digital audio signal of the microphone input sound converted by the ADC 44, and outputs it to the mixer 35.
The mixer 35 adjusts the balance of the volume and tone of the digital audio signals of a plurality of sound sources input from the key control unit 31, the MIDI sound source 32, the mute switch unit 33, the effect unit 34, and the mute switch unit 37. Mixing is performed, and the mixed digital audio signal is output to the DAC 36.

  The DAC 36 converts the digital audio signal generated by mixing by the mixer 35 into an analog audio signal. The analog audio signal converted by the DAC 36 is amplified by the speaker amplifier 51 connected to the karaoke apparatus 1 and emitted from the speaker 52 connected to the speaker amplifier 51.

  The mute switch unit 37 is a switch in the middle of the output path of the synthesized speech output from the speech synthesis processing unit 15 of the main control unit 10. The mute switch unit 37 turns on and off the input of the synthesized speech from the speech synthesis processing unit 15 to the mixer 35 in accordance with the control from the speech synthesis processing unit 15.

[1.3. Description of other configurations of karaoke apparatus 1]
The operation unit 41 is an operation for designating a karaoke piece to be played, an operation for changing the pitch of a key during the performance, an operation for temporarily changing the performance speed (change instruction), and an operation for canceling the temporary change of the performance speed (cancellation) This is an input device for performing various operations such as instructions. The operation for temporarily changing the performance speed includes so-called trick play such as pause, fast forward, and rewind. The operation processing unit 40 processes a signal from the operation unit 41 and inputs it to the main control unit 10. The communication interface unit 42 is configured to connect the karaoke device 1 to the LAN 100 and perform communication with other karaoke devices, a remote control terminal, and an information processing device such as an external server.

The microphone amplifier 43 is an amplifier that amplifies the singer's voice (analog voice signal) input from the microphone 53 connected to the karaoke apparatus 1.
The ADC 44 is a converter that converts an analog audio signal from the microphone amplifier 43 into a digital audio signal.

  The video processing unit 45 is a video processing device including a graphic engine that converts image information into a video image and a decoder that reproduces compressed video data. The image information visualized by the video processing unit 45 is displayed on the monitor 50 connected to the karaoke apparatus 1.

In addition, since the other function and structure in the karaoke apparatus 1 are based on a well-known technique, detailed description here is abbreviate | omitted.
[2. Description of various processes of main control unit 10]
Next, various processes executed by the main control unit 10 will be described in order below. In these various processes, a case where a fast-forward operation input is accepted will be described as an example. However, not only a fast-forward operation input but also other trick play operation inputs such as rewind and pause are accepted. It is.

[2.1. Explanation of input reception process]
The procedure of the input reception process executed by the main control unit 10 will be described with reference to the flowchart of FIG. This process is executed in parallel with other processes executed by the main control unit 10.

  In the first step S110, it is determined whether or not there is an input (fast-forward input) for executing fast-forwarding to the operation unit 41. When it is determined that there is a fast-forward input (S110: YES), the process proceeds to S120. On the other hand, if it is determined that there is no fast-forward input (S110: NO), the process proceeds to S130.

  In S120, notification is made that a fast-forward input has been made in the hard MIDI performance processing, live sound performance processing, soft MIDI performance processing, and singing voice synthesis reproduction processing. Thereafter, the process proceeds to S110.

  In S <b> 130, it is determined whether or not an input for canceling fast-forwarding (fast-forwarding cancellation) has been made to the operation unit 41. If it is determined that fast-forwarding has been canceled (S130: YES), the process proceeds to S140. On the other hand, if it is determined that the fast-forwarding has not been canceled (S130: NO), the process proceeds to S110 in order to wait until the fast-forwarding input is canceled.

  In S140, a return seek position for soft MIDI performance is calculated according to the number of raw sounds (the number of simultaneous output of digital audio signals of music by the raw sound reproduction processing unit 11). Note that the return seek position for soft MIDI performance refers to the timing at which the output of the digital sound signal of the musical sound based on the MIDI data by the software MIDI tone generator unit 14 is synchronized with the musical performance (karaoke music performance) by the MIDI sound source 32. It is. Here, the return seek position for soft MIDI performance is calculated as a position delayed from the current performance position by a value obtained by multiplying the number of raw sounds by a predetermined value set in advance through experiments or the like. Thereafter, the process proceeds to S150.

  In S150, based on the return seek position for soft MIDI performance calculated in S140, a return seek position (audio signal synchronization timing) for live sound performance is calculated. The return seek position for the live sound performance is a timing at which the output of the digital audio signal of the music by the live sound reproduction processing unit 11 is synchronized with the performance of the musical sound by the MIDI sound source 32 (the performance of the karaoke music). Here, the return seek position for live sound performance is calculated as a position delayed from the return seek position for soft MIDI performance by a predetermined value set in advance through experiments or the like. Thereafter, the process proceeds to S160.

  In S160, based on the return seek position for soft MIDI performance, the return seek position (synthetic voice synchronization timing) for singing synthesized performance is calculated. Note that the return seek position for the singing synthesis performance is a timing at which the output of the synthesized speech by the speech synthesis processing unit 15 is synchronized with the musical performance (the performance of karaoke music) by the MIDI sound source 32. Here, the return seek position for the singing synthesis performance is calculated as a position delayed from the return seek position for the soft MIDI performance by a predetermined value set in advance through experiments or the like. Note that the return seek position for the singing synthesis performance is set to a time delayed by a predetermined time from the time when the return seek position for the live sound performance is set. Thereafter, the process proceeds to S170.

  In S170, the hard MIDI performance process, the live sound performance process, the soft MIDI performance process, and the singing voice synthesis / playback process are notified that the fast-forwarding has been canceled, and the return seek position for the soft MIDI performance calculated in S140. Is transmitted to the soft MIDI performance process, the return seek position for the live sound performance calculated in S150 is notified to the live sound performance process, and the return seek position for the song synthesis performance calculated in S160 is notified to the singing voice synthesis reproduction process. Thereafter, in order to prepare for the next fast-forward input, the process proceeds to S110.

[2.2. Explanation of hard MIDI performance processing]
Next, the procedure of the hard MIDI performance process executed by the main control unit 10 will be described with reference to the flowchart of FIG. This process is executed in parallel with other processes executed by the main control unit 10.

  In the first step S210, it is determined whether or not a fast-forward input notification has been received from the input reception process. If it is determined that there is a notification of fast-forward input (S210: YES), the performance speed of the karaoke music is changed to the fast-forward speed, and the process proceeds to S220. On the other hand, when it is determined that there is no notification of fast-forward input (S210: NO), the process proceeds to S240.

  In S220, it is determined whether or not there is a notification of canceling fast-forward from the input reception process. When it is determined that there is a notification of canceling fast-forwarding (S220: YES), the process proceeds to S240. On the other hand, if it is determined that there is no fast-forwarding cancellation notification (S220: NO), the process proceeds to S230.

  In S230, a fast-forward performance is performed in which a musical tone performance (karaoke music performance) by the MIDI sound source 32 is performed at a performance speed at the time of fast-forward. Thereafter, in order to prepare for canceling fast-forwarding, the process proceeds to S210.

  In S240, live sound performance is performed by the live sound performance processing, specifically, output of data from the live sound reproduction unit 11, output of data from the software MIDI sound source unit 11, or output of data from the singing voice synthesis unit 15 is performed. Determine if it is done. When it is determined that a live sound performance is not being performed (S240: NO), the process proceeds to S250. On the other hand, if it is determined that a live sound performance is being performed (S240: YES), the process proceeds to S260.

  A process for generating each data in order to perform data output from the raw sound reproduction unit 11, data output from the software MIDI sound source unit 11, or data output from the singing voice synthesis unit 15 will be described later.

In S250, a constant speed performance is performed in which a musical tone performance (karaoke music performance) by the MIDI sound source 32 is performed at a performance speed at the time of reproduction. Thereafter, the process proceeds to S210.
In S260, a constant speed performance in which a musical sound performance (karaoke music performance) is played by the MIDI sound source 32 at a performance speed at the time of reproduction is executed in synchronization with a live sound performance performed by the live sound performance processing. Thereafter, the process proceeds to S210.

[2.3. Explanation of live sound performance processing]
Next, the procedure of the live sound performance process executed by the main control unit 10 will be described with reference to the flowchart of FIG. This process is executed in parallel with other processes executed by the main control unit 10.

  In the first step S310, it is determined whether or not a fast-forward input notification has been received from the input reception process. If it is determined that there is a fast-forward input notification (S310: YES), the live sound reproduction is stopped and the process proceeds to S320. On the other hand, if it is determined that there is no notification of fast-forward input (S310: NO), the process proceeds to S350.

  In S320, it is determined whether or not a notification of canceling fast-forwarding has been received from the input reception process. If it is determined that there is no notification of canceling fast-forwarding (S320: NO), S320 is executed again to wait until notification of canceling fast-forwarding is made. On the other hand, when it is determined that there is a notification of canceling fast-forwarding (S320: YES), the process proceeds to S330.

In S330, a return seek position for live sound performance is received from the input reception process. Thereafter, the process proceeds to S340.
In S340, the decoding from the return seek position for the live sound performance received in S330 is prepared. This is because, in an ideal performance operation, a performance sound is generated as soon as the performance is started (restarted) (see FIG. 7A), but in the case of a live sound, it is actually immediately after the performance is started (restarted). This is because there is no performance sound and there is a time lag (see FIG. 7B). Thereafter, the process proceeds to S350.

  In S350, when moving from S310, raw sound data (composition data of compressed audio data) is sequentially decoded from the current performance position to generate PCM (Pulse Code Modulation) data. When moving from S340, the raw sound is generated. The raw sound data (composition data of compressed audio data) is decoded from the return seek position for performance, and PCM data is generated. Thereafter, the process proceeds to S360.

In S360, the generated data (PCM data generated by sequentially decoding the raw sound data in S350) is passed to the audio processing module 30. Thereafter, the process proceeds to S310.
[2.4. Explanation of soft MIDI performance processing]
Next, the procedure of the soft MIDI performance process executed by the main control unit 10 will be described with reference to the flowchart of FIG. This process is executed in parallel with other processes executed by the main control unit 10.

  In the first step S410, it is determined whether or not a fast-forward input notification has been received from the input reception process. If it is determined that there is a notification of fast-forward input (S410: YES), the performance speed of the karaoke music is changed to the fast-forward speed, and the process proceeds to S420. On the other hand, if it is determined that there is no notification of fast-forward input (S410: NO), the process proceeds to S450.

  In S420, it is determined whether or not there is a notification of canceling fast-forwarding from the input reception process. If it is determined that there is no notification of canceling fast-forwarding (S420: NO), S420 is executed again to wait until notification of canceling fast-forwarding is made. On the other hand, when it is determined that there is a notification of canceling fast-forwarding (S420: YES), the process proceeds to S430.

In S430, a return seek position for soft MIDI performance is received from the input reception process. Thereafter, the process proceeds to S440.
In S440, interpretation / rendering from the return seek position for soft MIDI performance received in S430 is prepared. Thereafter, the process proceeds to S450.

  In S450, when moving from S410, musical tone PCM data generated by software MIDI generated by interpreting and rendering MIDI data from the current performance position is generated. On the other hand, when moving from S440, software is generated. Musical tone PCM data generated by software MIDI generated by interpreting and rendering MIDI data from the return seek position for MIDI performance is generated. Thereafter, the process proceeds to S460.

  In S460, the generated data is passed to the audio processing module 30 (PCM data generated by interpreting the MIDI data in S450 and rendering). Thereafter, the process proceeds to S410.

[2.5. Explanation of singing voice synthesis playback process]
Next, the procedure of the singing voice synthesis and reproduction process executed by the main control unit 10 will be described with reference to the flowchart of FIG. This process is executed in parallel with other processes executed by the main control unit 10.

  In first step S510, it is determined whether or not there is a notification of fast-forward input from the input reception process. If it is determined that a fast-forward input has been notified (S510: YES), the reproduction of the synthesized speech is stopped, and the process proceeds to S520. On the other hand, if it is determined that there is no notification of fast-forward input (S510: NO), the process proceeds to S560.

  In S520, it is determined whether or not there is a notification of fast-forward cancellation from the input reception process. If it is determined that there is no notification of canceling fast-forwarding (S520: NO), S520 is executed again in order to wait until notification of canceling fast-forwarding is made. On the other hand, when it is determined that there is a notification of canceling fast-forwarding (S520: YES), the process proceeds to S530.

In S530, a return seek position for singing voice synthesis is received from the input reception process. Thereafter, the process proceeds to S540.
In S540, singing voice synthesis from the return seek position for singing voice synthesis received in S530 is prepared. This is because, in an ideal performance operation, a performance sound is generated as soon as the performance is started (resumed) (see FIG. 7A). However, in the case of speech synthesis, the performance is actually performed as in the case of the live sound. This is because no performance sound is generated immediately after starting (resuming), and there is a time lag (see FIG. 7B). Thereafter, the process proceeds to S550.

  In S550, it is determined whether or not it is delayed from the overall progress. Specifically, the performance position of the karaoke music is compared with the playback position of the synthesized voice, and it is determined whether or not the playback position of the synthesized voice is delayed from the performance position of the karaoke music. If it is determined that there is no delay from the overall progress (S550: NO), the process proceeds to S560. On the other hand, when it is determined that the overall progress is delayed (S550: YES), the process proceeds to S580.

  If it is determined that the process is delayed from the overall progress as described above (S550: YES), the mute switch unit 37 of the audio processing module 30 is turned on. Details of the on / off control of the mute switch unit 37 will be described later.

  In S560, in the case of shifting from S510, the score / lyric data (speech synthesis data) is interpreted from the current performance position, and the singing voice is converted at a constant speed (karaoke performance) from the parameters for speech synthesis and the phoneme data. (Same as the performance speed of). Similarly, in the case of the transition from S550, the score / lyric data (speech synthesis data) is interpreted from the return seek position for singing speech synthesis, and the singing voice is obtained from the speech synthesis parameters and the phoneme data. Synthesize at constant speed (same as the performance speed of karaoke performance). Thereafter, the process proceeds to S570.

  In S570, the generated data (which is PCM data of singing voice generated by synthesizing the singing voice at the same speed by interpreting the score / lyric data in S560) is passed to the voice processing module 30. Thereafter, the process proceeds to S510.

  In S580, the score / lyric data (speech synthesis data) is interpreted, and the singing voice is synthesized at the fastest speed (at a speed larger than the performance speed of the karaoke performance), and the synthesized singing voice generation data is generated. Is generated and output, and the generated data is passed to the voice processing module. Thereafter, the process proceeds to S550.

  At this time, in the singing voice synthesis and reproduction process, the main control unit 10 performs a period of delay from the overall progress, that is, a period of a loop in which a positive determination is made in the process of S550 and the process proceeds to S550 again. An instruction to turn on the mute of the mute switch unit 37 is given to the processing module 30. For this reason, the volume of the synthesized speech is muted or reduced during this period.

  In addition, the main control unit 10 makes a mute switch unit 37 to the voice processing module 30 when a negative determination is made in the process of S550 in the singing voice synthesizing / reproducing process and it is not delayed from the overall progress. Change the mute from on to off and give instructions to unmute. At this time, input of the generated data to the mixer 35 is resumed. Then, the input volume of the synthesized voice to the mixer 35 is restored from mute or eliminated.

[3. Description of processing executed by voice processing module 30]
Next, processing of the voice processing module 30 that outputs karaoke music including voice synthesized voices will be described. That is, in the audio processing module 30, when a control signal for turning on / off the mute is input from the main control unit 10 to the mute switch units 33 and 37, input of each generated data to the mixer 35 is controlled based on the control signal. Each generated data is mixed by the mixer 35 and output to the output device (DAC 36). Then, it is converted into an audible music signal by the output device (DAC 36) and input to the speaker amplifier 51, whereby a karaoke piece including the synthesized voice is output from the speaker 52.

[4. effect]
Thus, according to the karaoke apparatus 1 of this embodiment, it has the following effects by performing the various processes described above.

  That is, when a change instruction (trick play execution instruction) for temporarily changing the performance speed of karaoke music is received, the performance speed of the karaoke music is changed, and the generation of the synthesized voice and the output of the synthesized voice are stopped. After that, upon receiving a cancel instruction (trick play end instruction) to cancel the change of the performance speed of the karaoke music, the performance speed of the karaoke music is returned to the original and the output of the synthesized voice is sent to the performance of the karaoke music. (Synthetic voice synchronization timing) is set, and synthetic voice is generated and synthesized voice is output in synchronism with the synthetic voice synchronization timing (see FIG. 8A, pause is exemplified as trick play). When the synthesized voice is not generated in time for the synthesized voice synchronization timing, the digital voice signal is generated in advance, and after the digital voice signal is generated, the synthesized voice is focused on. Generation is performed at a speed larger than the performance speed of the karaoke music and the output level of the synthesized voice is muted (see FIG. 8B, paused as trick play). Ex)

  Therefore, according to the karaoke apparatus 1 of the present embodiment, when returning to a normal playback state from so-called trick play such as pause, fast forward, and rewind, the delay of synthesized speech such as guide vocals for karaoke performance is minimized. In addition, it is possible to prevent the user from feeling the delay of the synthesized speech as much as possible, and to minimize discomfort given to the user.

[5. Other Embodiments]
As mentioned above, although one Embodiment of this invention was described, this invention is not limited to the said embodiment, It is possible to implement in the following various aspects.

  In the above embodiment, when the synthesized speech is not generated in time for the synthesized speech synchronization timing, the synthesized speech output level is muted until the synthesized speech generation catches up with the performance of the karaoke song. However, the output level of the synthesized voice may be reduced until the generation of the synthesized voice catches up with the performance of the karaoke music.

  Even with this configuration, it is possible to prevent the user from feeling the delay of the synthesized speech as much as possible, and to minimize the discomfort given to the user.

DESCRIPTION OF SYMBOLS 1 ... Karaoke apparatus, 10 ... Main control part, 11 ... Raw sound reproduction process part, 12 ... MIDI sequencer, 13 ... MIDI performance control part, 14 ... Software MIDI sound source part, 15 ... Speech synthesis process part, 20 ... HDD, 30 ... Audio processing module 31 ... Key control unit 32 ... MIDI sound source 33 ... Mute switch unit 34 ... Effect unit 35 ... Mixer 36 ... DAC 37 ... Mute switch unit 40 ... Operation processing unit 41 ... Operation unit 42 ... Communication interface unit, 43 ... Microphone amplifier, 44 ... ADC, 45 ... Video processing unit, 50 ... Monitor, 51 ... Speaker amplifier, 52 ... Speaker, 53 ... Microphone, 100 ... LAN.

Claims (4)

Music performance means for performing karaoke music performance at a predetermined performance speed based on the music data;
A synthesized voice for singing the karaoke music is generated by voice synthesis based on the voice synthesis data, and the generated synthesized voice is played at the same value as the performance speed of the karaoke music along with the performance of the karaoke music by the music playing means. Speech synthesis processing means for outputting;
Control means for controlling performance of karaoke music by the music performance means, generation of synthesized speech by the speech synthesis processing means, and output of synthesized speech by the voice synthesis processing means;
An instruction receiving means for receiving a change instruction for temporarily changing the performance speed of the karaoke music by the music playing means or a release instruction for releasing the change;
With
The control means includes
When the instruction accepting means accepts the change instruction, it changes the performance speed of the karaoke music by the music playing means, generates synthesized speech by the speech synthesis processing means, and generates synthesized speech by the speech synthesis processing means. Stop the output,
Thereafter, when the instruction receiving means receives the release instruction, the performance speed of the karaoke music by the music performance means is restored, and the output of the synthesized voice by the voice synthesis processing means is output to the karaoke by the music performance means. Set a synthesized voice synchronization timing that is a timing to synchronize with the performance of the music, generate a synthesized voice by the voice synthesis processing means in accordance with the synthesized voice synchronization timing, and output a synthesized voice by the voice synthesis processing means,
If the synthesis of the synthesized speech by the speech synthesis processing means is not in time for the synthesized speech synchronization timing, the speech synthesis is performed until the generation of the synthesized speech by the speech synthesis processing means catches up with the performance of the karaoke music by the music performance means. A karaoke apparatus characterized in that the synthesized voice is generated by the processing means at a speed greater than the performance speed of the karaoke music piece. The karaoke apparatus according to claim 1,
further,
Live sound reproduction processing means for generating a digital audio signal of karaoke music based on the compressed audio data, and outputting the generated digital audio signal at the same value as the performance speed of the karaoke music along with the performance of the karaoke music by the music performance means. Prepared,
The control means includes
Controlling the generation of the digital audio signal by the raw sound reproduction processing means, and the output of the digital audio signal by the raw sound reproduction processing means,
When the instruction receiving unit receives the change instruction, the digital sound signal generation by the raw sound reproduction processing unit and the output of the digital sound signal by the raw sound reproduction processing unit are stopped,
Thereafter, when the instruction receiving unit receives the release instruction, an audio signal synchronization timing is set which is a timing for synchronizing the output of the digital audio signal by the raw sound reproduction processing unit with the performance of the karaoke music by the music playing unit. Then, in accordance with the audio signal synchronization timing, generation of the digital audio signal by the raw sound reproduction processing unit and output of the digital audio signal by the raw sound reproduction processing unit,
The synthesized voice synchronization timing is set to a time delayed by a predetermined time from the time when the voice signal synchronization timing is set. In the karaoke apparatus according to claim 1 or 2,
When the synthesized speech generation by the speech synthesis processing unit is not in time for the synthesized speech synchronization timing, the control unit generates the synthesized speech by the speech synthesis processing unit for the performance of the karaoke music by the music performance unit. A karaoke apparatus characterized in that the output level of synthesized speech by the speech synthesis processing means is reduced until it catches up. A music performance process for performing karaoke music at a predetermined performance speed based on the music data;
Voice synthesis that generates synthesized voice for singing the karaoke music by voice synthesis based on the data for voice synthesis, and outputs the generated synthesized voice at the same value as the performance speed of the karaoke music along with the performance of the karaoke music. Processing,
An instruction reception process for receiving a change instruction for temporarily changing the performance speed of the karaoke music piece or a release instruction for releasing the change;
When the change instruction is received, the performance speed of the karaoke piece is changed, and the generation of the synthesized voice and the output of the synthesized voice are stopped. Returning the performance speed of the music, setting a synthesized voice synchronization timing that is a timing to synchronize the output of the synthesized voice with the performance of the karaoke song, and generating the synthesized voice according to the synthesized voice synchronization timing and When the synthesized speech is output and the synthesized speech is not generated in time for the synthesized speech synchronization timing, the synthesized speech is generated until the synthesized speech catches up with the performance of the karaoke song. Control processing performed at a speed greater than the performance speed of
A program that causes a computer to execute.