JP5244739B2 - SINGING VOICE REPRODUCTION DEVICE, SINGING VOICE REPRODUCTION METHOD, AND COMPUTER PROGRAM - Google Patents

Description

  The present invention relates to a singing voice reproducing apparatus that performs evaluation on singing voice data, a singing voice reproducing method, and a computer program that causes a computer to function as the singing voice reproducing apparatus.

A karaoke apparatus having a function of displaying a score of singing at the end of singing has been put into practical use.
On the other hand, Patent Document 1 discloses a karaoke apparatus that performs singing guidance by causing a singer to visually recognize the strength of karaoke music. The karaoke apparatus according to Patent Document 1 displays lyrics data of karaoke songs and voice waveform data of an exemplary song on a display unit, and a color corresponding to each amplitude range of the voice waveform data, for example, from the low amplitude side. The color-coded display is made in order of blue, green, yellow, and red. The singer can instantly recognize that it is sufficient to sing in a voice waveform portion displayed in red and to utter weakly in a voice waveform portion displayed in blue. Become.

  Further, in Patent Document 2, the comparison unit compares the pitch and volume of the singing voice and the guide melody to determine pitch difference data and volume difference data, and displays the contents of each difference data in real time. A karaoke device is disclosed that allows a person to know exactly the state of his singing.

JP 2007-264060 A JP 2002-162978 A

  However, in the prior art, the voice waveform related to the singing voice data and the melody of the model guide melody cannot be displayed in an easy-to-view and comparable manner. It was not easy to grasp how much the pitch with the melody was shifted and how much the rhythm was shifted.

  The present invention has been made in view of such circumstances, and an exemplary melody block is superimposed on a line score image, and a voice waveform expressing the melody of a singing voice is superimposed on the line score and the melody block. In particular, the voice waveform part superimposed on the melody block and the voice waveform part superimposed on the line score are overlapped in different waveform line types or superposition modes, thereby clearly singing voice and model melody. It is an object of the present invention to provide a singing voice reproducing device, a singing voice reproducing method, and a computer program that allows a computer to function as the singing voice reproducing device, enabling objective and visual comparison.

  The singing voice reproducing apparatus according to the present invention includes a reproducing means for reproducing karaoke song data, an acquiring means for acquiring voice data of a song performed while the reproducing means reproduces karaoke song data, In a singing voice reproducing apparatus comprising a reproducing means for reproducing the voice data acquired by the acquiring means, a storage means for storing singing melody data indicating the melody of the sound to be sung in accordance with the karaoke song data, and a melody Based on line music image generation means for generating a line music image for expression, and song melody data stored in the storage means, the length in the time axis direction of the line music is the length of the sound to be sung. A melody block superimposing unit that superimposes a melody block having a width corresponding to the melody block at a position corresponding to the pitch of the sound to be sung on the line music, and a melody of the sound related to the audio data acquired by the acquisition unit. Speech waveform superimposing means for superimposing the expressed speech waveform on the line music and the melody block, the speech waveform superimposing means comprising: a speech waveform portion superimposed on the melody block; and a speech superimposed on the line music. The waveform portion is characterized by being overlapped in a different waveform line type or superposition mode.

  The singing voice reproducing device according to the present invention stores vibrato data indicating the timing of singing by vibrato according to the karaoke song data, and the vibrato data, and based on the vibrato data, Vibrato speech waveform superimposing means for superimposing a vibrato speech waveform having a size indicated by vibrato data on the melody block and the line score, the vibrato speech waveform superimposing means comprising: A voice waveform expressing a melody and a vibrato voice waveform are superimposed in different waveform line types or superposition modes.

  In the singing voice reproducing apparatus according to the present invention, the voice waveform superimposing means superimposes the voice waveform of the time to be sung by vibrato on the line score and the melody block in the same waveform line type or superposition mode. It is characterized by that.

  The singing voice reproducing apparatus according to the present invention stores lyric data indicating lyrics to be sung according to the karaoke song data, and a melody block displaying lyrics based on the lyric data and singing melody data. Means for generating is provided.

  A singing voice reproducing method according to the present invention reproduces karaoke song data, acquires voice data of a song performed while the karaoke song data is being reproduced, and reproduces the obtained voice data. And generating a line score image for expressing a melody, and the time axis dimension of the line score based on the song melody data indicating the melody of the sound to be sung in accordance with the karaoke song data. A step of superimposing a melody block having a width corresponding to the length of the sound to be sung at a position according to the pitch of the sound to be sung on the line score, and a melody of the sound related to the acquired voice data. A speech waveform superimposing step of superimposing the expressed speech waveform on the line score and the melody block, the speech waveform superimposing step comprising: a speech waveform portion superimposed on the melody block; and the line A speech waveform portion superimposed above, characterized by superimposing a different waveform line species or superimposed manner.

  A computer program according to the present invention is a computer program for causing a computer to reproduce audio data performed while karaoke song data is being reproduced, and causing the computer to generate a line score image for expressing a melody. And, based on the singing melody data indicating the melody of the sound to be sung in accordance with the karaoke song data, the time axis dimension in the line score is a width corresponding to the length of the sound to be sung. A step of superimposing a melody block at a position on the line score corresponding to the pitch of the sound to be sung; and a computer expressing a voice waveform expressing the melody of the sound related to the acquired voice data as the line score and the melody. A speech waveform superimposing step to superimpose on the block, and the speech waveform superimposing step sends the melody block to the computer. A speech waveform portion superimposed on, and a voice waveform portion superimposed on the line score, characterized by superimposing a different waveform line species or superimposed manner.

In the present invention, the storage means stores singing melody data indicating the melody of the sound that serves as an example to be sung in accordance with the karaoke song data. Then, a line music image is generated, and a melody block expressing the melody is superimposed on the line music image. Also, a speech waveform expressing the melody of the sound related to the acquired singing speech data is superimposed on the line score and the melody block. In particular, the speech waveform portion superimposed on the melody block and the speech waveform portion superimposed on the line score are superimposed in different waveform line types or superposition modes.
Therefore, the singing voice and the model melody can be clearly compared objectively and visually.

In the present invention, based on the vibrato data, a vibrato speech waveform having a size indicated by the vibrato data is superimposed on the melody block and the line score. In particular, the speech waveform representing the melody of the sound related to the speech data and the vibrato speech waveform are superimposed in different waveform line types or superposition modes.
Therefore, the voice waveform of the singing voice sung with vibrato and the exemplary vibrato voice waveform can be clearly compared objectively and visually.
The size of the vibrato means information characterizing the vibrato, for example, information such as pitch fluctuation and periodicity.

  In the present invention, the voice waveform of the singing that should be sung with vibrato is superimposed on the line score and the melody block in the same waveform line type or superposition mode. Therefore, the voice waveform related to the vibrato is not changed depending on whether it is inside or outside the block. That is, the vibrato singing position is configured not to instruct the singer to deviate the pitch.

  In the present invention, lyrics are displayed on the melody block. Therefore, it becomes possible to grasp the melody and the lyrics at a glance.

  According to the present invention, a singing voice and an exemplary melody can be clearly compared objectively and visually.

It is a block diagram which shows typically the example of 1 structure of the karaoke apparatus which concerns on this Embodiment. It is explanatory drawing which shows notionally an example of the record layout of music DB. It is explanatory drawing which shows notionally an example of the record layout of evaluation result log | history DB. It is explanatory drawing which shows notionally an example of the record layout of model sound DB. It is a flowchart which shows the process sequence of the control part which concerns on singing practice. It is the schematic diagram which showed an example of the singer reception screen and the practice menu screen. It is the schematic diagram which showed an example of the instructional animation. It is a flowchart which shows the process sequence of the subroutine which concerns on singing practice. It is the schematic diagram which showed an example of the singing practice screen. It is a flowchart which shows the process sequence of the subroutine which concerns on a song voice data confirmation process. It is the schematic diagram which showed an example of the confirmation screen of song voice data. It is a flowchart which shows the process sequence of the subroutine which concerns on song evaluation analysis. It is a flowchart which shows the process sequence of the subroutine which concerns on song evaluation analysis. It is the schematic diagram which showed an example of the evaluation result display screen. It is the schematic diagram which showed an example of the evaluation result display screen. It is the schematic diagram which showed an example of the evaluation result display screen. It is the schematic diagram which showed an example of the evaluation result display screen. It is a schematic diagram which shows an example of a singing range measurement screen.

  The karaoke apparatus (singing voice reproducing apparatus) according to the present embodiment of the present invention has a function of evaluating a singing performed by a singer and storing evaluation results for each of a plurality of singers. By giving a graph that can be used to compare the evaluation results for other songs with the evaluation results for other singers, one singer is motivated to improve singing skills, enabling effective singing training. Is.

Hereinafter, the present invention will be described in detail with reference to the drawings illustrating embodiments thereof.
FIG. 1 is a block diagram schematically showing a configuration example of the karaoke apparatus 1 according to the present embodiment. The karaoke apparatus 1 is realized by a home video game computer, and includes a control unit 10 that controls the operation of each component of the karaoke apparatus 1, for example, a CPU (Central Processing Unit). The control unit 10 includes a ROM 11, a RAM 12, an external storage device 13, a video output unit 14, an internal storage device 15, an audio output unit 16, an audio input unit 17, a remote operation signal receiving unit 18, a communication unit 19, and a bus. A clock unit 20 is connected. Needless to say, the karaoke apparatus 1 may be configured as a dedicated machine.

  The control unit 10 reads out and executes the computer program 3 recorded in the ROM 11 and the internal storage device 15 described later to the RAM 12, thereby implementing the later-described singing voice reproduction method according to the present invention, and operates as the karaoke device 1. . In addition, although the example in which the control part 10 performs song evaluation processing, audio | voice reproduction | regeneration processing, etc. is demonstrated, it cannot be overemphasized and you may comprise so that various processes may be implement | achieved by hardware, such as a song evaluation circuit and a MIDI sound source. .

  The ROM 11 is a non-volatile memory such as a mask ROM or EEPROM that stores a control program necessary for the operation of the computer.

  The RAM 12 is a volatile memory such as a DRAM or SRAM that temporarily stores various data generated when the arithmetic processing of the control unit 10 is executed.

  The external storage device 13 is a DVD (Digital Versatile Disc) -ROM, a BD (Blu-ray Disc), a CD (Compact Disc) -ROM, or the like on which the computer program 3 according to Embodiment 1 of the present invention is recorded in a computer-readable manner. The optical disc drive reads the computer program 3 and other various data from the recording medium 2. The optical disk drive is an example of the external storage device 13, and the external storage device 13 may be configured by a device that reads and writes data from a magnetic disk such as a hard disk or a flexible disk, a magneto-optical disk, a semiconductor memory, or the like.

The internal storage device 15 is a hard disk, a nonvolatile semiconductor memory, or the like that stores a computer program 3 for causing a computer to function as the karaoke device 1. The computer program 3 according to the present invention may be downloaded from an external computer (not shown) connected to the communication network N.
The internal storage device 15 stores a music DB (Data Base) 15a, a guidance comment DB 15b, an evaluation result history DB 15c, a model sound DB 15d, and a instructional video DB 15e. Details of the various databases will be described later.

  The video output unit 14 converts image data for displaying various menu screens, lyrics images, and singing evaluation results given by the control unit 10 into video signals that can be displayed on the display unit 4, and converts the video signals. It is a circuit that outputs to the display unit 4. The display unit 4 is, for example, a liquid crystal display, an organic EL display, a CRT, or the like.

  The audio output unit 16 is a circuit that converts the audio data given by the control unit 10 into an audio signal that can be output by the speaker 5 and outputs the audio signal to the speaker 5. Needless to say, the audio output unit 16 and the audio input unit 17 may be configured by one connector, for example, an HDMI connector.

  The audio input unit 17 is, for example, a USB port to which audio data is input, and the microphone 6 is connected via a USB cable. The microphone 6 collects a song performed by a singer and converts it into an analog audio signal, amplifies and AD converts the audio signal, and outputs the AD-converted digital audio data to the audio output unit 16. . Hereinafter, the sound data of the singing performed by the singer is referred to as singing sound data.

  The remote operation signal reception unit 18 is a circuit that receives a remote operation signal transmitted from the remote controller 7 for operating the operation of the karaoke apparatus 1 and gives the control unit 10 data related to the received remote operation signal.

  The communication unit 19 is an interface for transmitting and receiving karaoke song data and data related to various services to and from an external karaoke server 8 connected via a wired or wireless communication network N. Transmission / reception of information is controlled by the control unit 10.

  The clock unit 20 is a circuit that counts the date and time when the singing evaluation is performed, and gives a timing result to the control unit 10 in response to a request from the control unit 10.

  FIG. 2 is an explanatory diagram conceptually showing an example of the record layout of the music DB 15a. The table (file) of the music DB 15a has a plurality of columns (fields), for example, a “music ID” column, a “practice content” column, a “level” column, a “music data” column, a “technical comment data” column, a “model sound”. Each row (record) has information corresponding to each column.

The “music ID” column stores information unique to a plurality of different practice music data.
The “practice content” column stores the content of singing practice by each piece of music data, for example, data indicating the type of utterance, back voice, rhythm, vibrato, and the like.
The “level” column stores data indicating the difficulty level of practice content, for example, data indicating level 1 to level 4 and the like.

The “music data” column is further divided into a plurality of items, and stores accompaniment data for singing practice (karaoke music data), lyrics data, and main melody data. The structure of the music data is not particularly limited, and the accompaniment data, the lyrics data, and the main melody data may be configured as a single music file, or may be configured as separate files and managed in association with each file. May be.
The accompaniment data is, for example, MIDI (Musical Instrument Digital Interface) data, which specifies data on / off of sound, pitch data specifying pitch, volume data (velocity) specifying volume, instrument control data, sound Is composed of timbre data for designating a musical instrument that outputs a sound, time data (delta time) for designating a time interval of each sound reproduced in time series.
The main melody data indicates the melody of the sound to be sung in accordance with the accompaniment, that is, the data indicating the on / off of the sound that constitutes the main melody, the pitch data, the volume data, and the time interval of each sound of the main melody that is played back in time series. Time data. Instead of the time data indicating the time interval, data indicating the elapsed time from the start time of the accompaniment data may be used. The main melody data includes vibrato time to be sung in accordance with the accompaniment, vibrato data indicating the pitch, amplitude and pitch of the pitch, and back sound data indicating the time to sing in the back voice. The vibrato data may be composed of data different from the main melody data. In addition, the main melody data does not necessarily have to be complicated. For example, when the purpose is to focus on rhythmic feeling, the main melody data is simple enough to turn on and off a sound with a fixed pitch. Also good.

The “technique comment data” column stores character data for commenting a technique to be used for singing according to the accompaniment and time data indicating a timing at which the technique should be adopted.
The “model sound data” column stores model sound data as a model to sing along with the accompaniment. The exemplary sound data is, for example, MIDI data, PCM (pulse code modulation) data, or the like.
The various data described above may be stored in advance in the internal storage device 15 or may be acquired from the karaoke server 8 through the communication network N.

  The instruction comment DB 15b stores an evaluation result storing a typical evaluation result pattern and instruction comment data storing advice and instruction contents for each evaluation result in association with each other.

  FIG. 3 is an explanatory diagram conceptually illustrating an example of a record layout of the evaluation result history DB 15c. The table of the evaluation result history DB 15c includes a plurality of columns, for example, an “evaluation ID” column, a “singer ID” column, an “evaluation result” column, an “evaluation date / time” column, a “publication availability” column, and the like. Each row has information corresponding to each column.

The “evaluation ID” column stores unique information for identifying an evaluation result for each of a plurality of singing voice data.
The “singer ID” column stores information for identifying a singer evaluated for singing.
The “evaluation result” column is further divided into a plurality of items, each of which stores evaluations related to utterance, back voice, rhythm, and vibrato. Each evaluation is managed with a maximum score of 100, but needless to say, it is not limited to this.
The “evaluation date and time” column stores the date and time when the singing evaluation was performed.
The “open / disallowed” column stores data indicating whether or not the singing evaluation may be disclosed. The availability of the release is specified by each singer. More specifically, the singer can designate whether or not the release can be performed by operating the remote controller 7, and the control unit 10 of the karaoke apparatus 1 can be released by the remote operation signal receiving unit 18 and the remote control 7. Acceptability is accepted and the accepted availability is stored in the “Availability” column. Note that whether or not to disclose can be specified in units of evaluation results, but may be specified in units of singers.

FIG. 4 is an explanatory diagram conceptually showing an example of the record layout of the model sound DB 15d. The table of the model sound DB 15d includes a plurality of columns, for example, a “model sound ID” column, an “evaluation result” column, a “model sound data” column, and the like, and each row has information corresponding to each column.
The “model sound ID” column stores unique information for identifying each of the plurality of model sound data.
The “evaluation result” column stores a representative evaluation result pattern. For example, 5 × 5 × 5 × 5 = 625 possible evaluation results are stored when 20 points, 40 points, 60 points, 80 points, or 100 points are given as evaluation points regarding utterance, back voice, rhythm, and vibrato. .
The “model sound data” column stores model sound data for confirming the evaluation result corresponding to each evaluation result, that is, voice data to which the evaluation result should be given when singing evaluation is performed.

  The instructional video DB 15e stores a music ID in association with instructional video data that teaches a singing practice method, singing technique, and the like using the music data related to the music ID.

FIG. 5 is a flowchart showing a processing procedure of the control unit 10 related to singing practice, FIG. 6 is a schematic diagram showing an example of a singer reception screen and a practice menu screen, and FIG. 7 is a schematic diagram showing an example of a teaching video. FIG. The control unit 10 reads the singer reception screen data from the internal storage device 15, and outputs the read singer reception screen data to the display unit 4 through the video output unit 14, whereby the singer reception screen is displayed on the display unit 4. It is displayed (step S11). Since the processing method for displaying an image on the display unit 4 is the same as the processing method described in step S11, details thereof will be omitted below.
Next, the control unit 10 receives one singer who performs singing practice through the remote operation signal receiving unit 18 (step S12).

  FIG. 6A is a schematic diagram illustrating an example of a singer reception screen. The singer reception screen displays face images corresponding to a plurality of singers side by side, and the singer selects a face image using the remote controller 7 to select one singer performing singing practice. You can choose.

  Subsequently, the control part 10 reads practice menu screen data, and displays the practice menu screen on the display part 4 (step S13).

  FIG. 6B is a schematic diagram illustrating an example of a practice menu screen. The practice menu screen has characters of “teaching video viewing mode”, “daily practice mode”, “verification mode”, and “appropriate range measurement mode” juxtaposed, and the singer uses the remote control 7 to select either mode. Can be selected. “Teaching video viewing mode” is a mode for playing back a teaching video that teaches singing practice, singing techniques, and the like. “Daily practice mode” is a mode in which singing practice is performed according to practice content and level. The “verification mode” is a mode for evaluating the singing technique of the singer. The “appropriate sound range measurement mode” is a mode for measuring the vocal range of the singer.

  Next, the control unit 10 monitors the operation state of the remote controller 7 through the remote operation signal receiving unit 18 and determines whether or not the instructional video viewing mode is selected (step S14). If it is determined that the instructional video viewing mode has been selected (step S14: YES), the control unit 10 reads out the instructional video data from the internal storage device 15, and reproduces the read out instructional video data as shown in FIG. 7 (step S14). S15). More specifically, the control unit 10 displays a plurality of practice contents and levels on the display unit 4 and accepts one practice content and level. Then, the control unit 10 is configured to search the instructional video data corresponding to the accepted practice content and level in the instructional DB, read out the searched instructional video data from the internal storage device 15, and reproduce it.

  When the process of step S15 is completed or when it is determined that the instructional video viewing mode is not selected (step S14: NO), the control unit 10 monitors the operation state of the remote controller 7 and the daily practice mode is selected. It is determined whether or not (step S16). When it determines with the daily practice mode having been selected (step S16: YES), the control part 10 calls a subroutine, and performs the process which concerns on a singing practice (step S17).

  FIG. 8 is a flowchart showing a processing procedure of a subroutine related to singing practice, FIG. 9 is a schematic diagram showing an example of a singing practice screen and the like, and FIG. 11 is a schematic diagram showing an example of a confirmation screen for singing voice data. is there. When the subroutine related to singing practice is called in step S17, the control unit 10 reads the practice content selection screen data from the internal storage device 15 and causes the display unit 4 to display the practice content selection screen. Is received (step S31).

FIG. 9A is a schematic diagram illustrating an example of a practice content selection screen. On the practice content selection screen, practice content such as “speech”, “back voice”, “rhythm” and “vibrato” are displayed in a vertical direction, and levels “1” and “2” indicating the difficulty of each practice content are displayed. ”,“ 3 ”and“ 4 ”are displayed side by side in the horizontal direction. The numbers indicating each level are enclosed in squares, the practice content and levels enclosed in white squares indicate selectable items, and the practice content and levels enclosed in hatched squares are Indicates items that cannot be selected. In the first singing practice, you can only select the practice content of level "1", but if the evaluation result for the singing performed in the past is good, the items of upper level "2", "3" and "4" are sequentially It becomes possible to select. A thick line surrounding the cell is a cursor, which indicates that the exercise content and level corresponding to the cell, for example, the exercise content “speech” and level “1” are selected.
The singer can move the cursor by operating the remote controller 7 to select a desired practice content and level, and the control unit 10 can select the selected practice content and level through the remote operation signal receiving unit 18. Accept.

Next, as shown in FIG. 9B, the control unit 10 generates line music image data representing a line music image 91a formed by displaying five parallel lines in the horizontal direction, and the line music image 91a is generated. It is displayed on the display unit 4 (step S32). And the control part 10 reads the music data according to the practice content and level selected by step S31 from the internal memory 15, and based on the read music data, especially main melody data and lyrics data, the melody block 91b. 91c and the melody blocks 91b and 91c are superimposed on the line music image 91a (step S33). The melody block 91b is a rectangular image in which the pitch direction dimension in the line score is a predetermined length, and the dimension in the time axis direction in the line score is a width corresponding to the length of the sound to be sung. The melody block 91c is an image showing a part that should be particularly conscious of rhythm practice, and a cross mark is drawn inside a white circle image. The melody block 91c has a width corresponding to a predetermined length of a pitch in a line score and a length in a time axis direction of the line score corresponding to a predetermined length of a sound to be sung in rhythm practice. The melody blocks 91b and 91c are superimposed at positions corresponding to the pitch and playback time of the main melody. The predetermined length may be a length that can distinguish between the line score and the melody blocks 91b and 91c. Lyrics are displayed on the melody block 91b.
Moreover, the control part 10 superimposes the accompaniment progress position display line 91d which showed the progress position of the accompaniment on the line score image 91a. Further, based on the technique comment data, the control unit 10 superimposes the technique comment on the outside of the line score image 91a, particularly on the upper side, and superimposes a musical scale or the like on the lower side.

  In FIG. 9, the lyrics are not displayed on the melody block 91c. However, the lyrics may be displayed on the melody block 91c if necessary. In addition, two types of melody blocks 91b and 91c are displayed together in the line score image 91a. However, when the rhythm practice is focused, only the melody block 91c is displayed, and utterances and backs other than the rhythm are displayed. When the practice such as the above is performed with emphasis, only the melody block 91b may be displayed. Further, the width of the melody block 91b is configured to be shortened according to the length of the sound to be sung, but the melody block 91b is configured by one or a plurality of small blocks juxtaposed in the horizontal direction and should be sung. You may make it increase / decrease the number of the small blocks juxtaposed according to the length of a sound. Furthermore, although the circular melody block 91c is illustrated, it may be configured such that the lateral dimension is increased or decreased according to the length of the sound to be sung with rhythm. Further, although the melody blocks 91b and 91c are illustrated as rectangles and circles, it is needless to say that the melody blocks 91b and 91c are not limited to these shapes. It may be adopted.

  Next, the control unit 10 reads out the model sound data corresponding to the practice content and level received in step S31 from the internal storage device 15, and as shown in FIG. 9B, accompaniment data corresponding to the practice content and level. Then, the model sound data is reproduced (step S34). By the processing in step S34, the singer can confirm the practice content.

  Next, as shown in FIG. 9C, the control unit 10 acquires voice data of the song performed by the singer through the voice data input unit while reproducing the accompaniment data (step S35). The acquired audio data is written into the internal storage device 15 or the RAM 12.

  When the singing practice is finished, that is, when the reproduction of the accompaniment data is finished, the control unit 10 calls a subroutine to execute a process related to the singing voice data confirmation (step S36), and finishes the process concerning the singing practice. According to the process of step S36, the singer can check his / her singing voice. In addition, according to a singer's instruction | indication, you may make it reproduce | regenerate song voice data in multiple times.

  FIG. 10 is a flowchart showing a processing procedure of a subroutine related to the singing voice data confirmation processing. When a subroutine related to the singing voice data confirmation process is called in step S36, the control unit 10 displays a line formed by displaying five parallel lines in the horizontal direction as shown in FIGS. 11 (a) and 11 (b). Line music image data representing a music score image is generated, and the line music image is displayed on the display unit 4 (step S41).

And the control part 10 reads the music data according to the practice content and level selected by step S31 from the internal memory 15, and based on the read music data, especially main melody data and lyrics data, the melody block 91b. , 91c are generated, and the melody blocks 91b, 91c are superimposed on the line music image (step S42). The details of the melody blocks 91b and 91c are the same as described above.
Moreover, the control part 10 superimposes the accompaniment progress position display line 91d which showed the progress position of the accompaniment on a line music image. Further, based on the technique comment data, the control unit 10 superimposes a technique comment on the outside of the line music image, particularly on the upper side, and superimposes a musical scale or the like on the lower side.

  Next, the control unit 10 reproduces the singing voice data acquired in step S35 (step S43). And the control part 10 changes the color of the melody blocks 91b and 91c of the reproduced | regenerated location to a color different from the color of the melody blocks 91b and 91c of an unreproduced location (step S44). In the melody blocks 91b and 91c, hatched portions indicate regenerated locations.

  Next, the control unit 10 determines whether or not the current accompaniment progress position is a vibrato position to be sung by vibrato based on the timing indicated by the music data, particularly vibrato data (step S45). When it determines with it being a vibrato position (step S45: YES), as shown in FIG.11 (b), the control part 10 produces | generates the audio | voice waveform 91g of the same aspect inside and outside the melody block 91b based on song audio | voice data. The voice waveform 91g is generated and superimposed on the line score image and the melody image (step S46). The voice waveform 91g is a waveform that represents the frequency of the singing voice, that is, the pitch, and is high when the position on the line music image is above, and low when it is below. Moreover, the aspect of the audio | voice waveform 91g is a blue thick line, for example.

  And the control part 10 produces | generates the vibrato audio | voice waveform 91f based on the fluctuation width and periodicity which vibrato data show in the aspect different from the audio | voice waveform 91g, and this vibrato audio | voice waveform 91f is made into a line score image and the melody block 91b. Superimpose on it (step S47). The form of the vibrato audio waveform 91f is, for example, a red broken line. Instead of the vibrato speech waveform 91f, a melody block in which the upper side and the lower side are formed in a wave shape, for example, a sine wave shape, may be superposed on the line music image. The amplitude and periodicity of the upper and lower wave shapes of the melody block may be changed according to the amplitude and periodicity of the vibrato.

  According to the process of step S46 and step S47, the singer can compare the voice waveform 91g of the vibrato-sung portion of the singing voice data with the exemplary vibrato voice waveform 91f clearly and intuitively. In particular, although the vibrato speech waveform 91f is expressed by a broken line, the singer can normally recognize that the waveform portion expressed by the broken line is an intuitively exemplary speech waveform. In addition, as will be described later, in a portion other than the vibrato position, the voice waveform 91e on the melody blocks 91b and 91c is represented by a blue thin line, so the singer can perform the voice waveform 91g represented by a blue thick line. Can be intuitively recognized as a voice waveform of a singer.

  When it determines with it not being a vibrato position by step S45 (step S45: NO), as shown to FIG. 11 (a), (b), the control part 10 is based on song audio | speech data 91b, 91c The voice waveform 91e having a different aspect is generated inside and outside the voice waveform 91e, and the voice waveform 91e is superimposed on the line music image and the melody image (step S48). The audio waveform 91e is a blue thin line on the melody blocks 91b and 91c, and a red thin line outside the melody blocks 91b and 91c.

  When the process of step S47 or step S48 is completed, the control unit 10 monitors the operation state of the remote controller and determines whether or not to end the singing voice data confirmation process (step S49). When it determines with not complete | finishing the confirmation process of song voice data (step S49: NO), the control part 10 returns a process to step S42. When it determines with having completed the confirmation process of singing voice data (step S49: YES), the control part 10 complete | finishes a singing voice data confirmation process.

  Note that the line types and superposition modes of the audio waveforms 91e and 91g and the vibrato audio waveform 91f are merely examples, and color elements such as the thickness, hue, saturation, and brightness of a line segment expressing the audio waveform, a broken line, a chain line, and one point You may make it express in any one of aspects, such as a chain line, a superimposition aspect, such as blinking regularly or superimposing regularly, or combining each aspect.

  As shown in FIG. 5, when the process related to singing practice is completed, the control unit 10 then calls a subroutine and executes the process related to singing evaluation analysis (step S18).

  12 and 13 are flowcharts showing a processing procedure of a subroutine related to singing evaluation analysis, and FIGS. 14 to 17 are schematic views showing an example of an evaluation result display screen.

  The control unit 10 performs singing evaluation based on the singing voice data and the music data (step S51). Singing evaluation evaluates utterance, rhythm, etc. by comparing pitch, volume, rhythm, etc. in singing voice and main melody based on main melody data and singing voice data. In addition, when it is time to vibrato, it is related to vibrato by determining whether or not the pitch of the singing voice is moving up and down within a specific pitch range within a specific frequency range based on the vibrato data. Evaluate. The specific pitch range is determined by the vibrato amplitude indicated by the vibrato data, and the specific frequency range is specified by the vibrato periodicity indicated by the vibrato. Further, based on the back voice data, evaluation on the back voice is performed based on whether the pitch and the volume are stable at the time when the voice should be sung.

  Then, the control unit 10 stores the evaluation result obtained in step S51 in the internal storage device 15 (step S52). Specifically, the control unit 10 stores the singer ID that identifies the singer accepted in step S12 in the “singer” ID column of the evaluation result history DB 15c, and “speak” in the “evaluation result” column. , The evaluation results regarding “back voice”, “rhythm” and “vibrato” are stored. In addition, the control unit 10 stores the date and time that the clock unit 20 is measuring in the “evaluation date and time” column.

Subsequently, the control part 10 produces | generates the score announcement image data showing the comprehensive evaluation score based on an evaluation result, and the instruction | indication comment corresponding to an evaluation result, and displays the score announcement image on the display part 4 (step S53).
FIG. 14A is a schematic diagram showing an example of a scoring announcement image. The scoring announcement image is an image that displays the letters “scoring announcement”, a comprehensive evaluation score, and a guidance comment on the evaluation result. For example, the control unit 10 converts the total score of “speech”, “back voice”, “rhythm”, and “vibrato” into a full score of 100 and calculates it as a total evaluation score. Moreover, the control part 10 searches the instruction | indication comment data corresponding to this evaluation result from instruction | indication comment DB15b based on the evaluation result performed by step S51. And the control part 10 produces | generates the score announcement image containing a comprehensive evaluation score and guidance comment data.

Next, the control unit 10 has an analysis method selection tab, generates training result image data for showing the details of the evaluation result, and displays the training result screen on the display unit 4 (step S54).
FIG. 14B is a schematic diagram illustrating an example of a training result screen. The training result screen is an image in which three sheets having tabs are superimposed. The first sheet has a tab on which characters of “individual” are displayed, and displays the evaluation result of the individual on a radar chart. The second sheet has a tab on which the characters “history” are displayed, and as shown in FIG. 17, displays a history of individual evaluation results. The third sheet has a tab on which “comparison” characters are displayed. As shown in FIG. 16, the evaluation result for one singer's song and the evaluation result for another singer's song are displayed on the radar. It is displayed superimposed on the chart.

Next, the control unit 10 determines whether or not a personal tab has been selected by monitoring the operation state of the remote controller 7 (step S55). In the initial state, the personal tab is selected. When it determines with the personal tab having been selected (step S55: YES), the control part 10 is shown in the chart showing the evaluation result with respect to the singing voice data of one singer received at step S12, for example, FIG.14 (b). Such a radar chart is generated and displayed on the display unit 4 (step S56). Items displayed on the radar chart are “speech”, “back”, “rhythm”, and “vibrato”, as in the evaluation result items.
The radar chart is an example of a chart that displays the evaluation result, and the type of the graph is not particularly limited. For example, the evaluation result may be expressed as a bar graph, a pie graph, or a scatter diagram.

  Next, the control unit 10 monitors the operation state of the remote controller 7, and determines whether or not there is an instruction to change the evaluation result (step S57). For example, the singer can change the evaluation result by dragging the line or plot of the evaluation result displayed on the radar chart. When it is determined that there is an instruction to change the evaluation result (step S57: YES), the control unit 10 changes the evaluation result displayed on the radar chart as indicated by a rhombus drawn with a thick line in FIG. (Step S58). In addition, the white arrow has shown the item of the changed evaluation result.

  When the process of step S58 is completed, or when it is determined in step S57 that there is no instruction to change the evaluation result (step S57: NO), the control unit 10 monitors the operation state of the remote controller 7 and instructs to reproduce the model sound. It is determined whether or not there has been (step S59). For example, the singer can instruct the reproduction of the model sound by clicking on the radar chart. If it is determined that there is an instruction to reproduce the model sound (step S59: YES), the control unit 10 specifies the model sound corresponding to the evaluation result from the model sound DB 15d based on the evaluation result obtained in step S51. (Step S60), the identified model sound data is read and reproduced (Step S61).

When the process of step S61 is completed, when it is determined that there is no instruction to reproduce the model sound (step S59: NO), or when it is determined that the personal tab is not selected in step S55 (step S55: NO), The control unit 10 monitors the operation state of the remote controller 7 and determines whether or not a comparison tab has been selected (step S62). When it determines with the comparison tab having been selected (step S62: YES), the control part 10 monitors the operation state of the remote control 7, and receives the singer of comparison object (step S63). And the control part 10 is shown in the chart which displayed the evaluation result with respect to the song of the singer received by step S12, and the evaluation result with respect to the song of the singer received in step S63 in a superimposed or parallel manner, for example, FIG. Such a radar chart is generated, and the generated radar chart is displayed on the display unit 4 (step S64).
In FIG. 16, the rhombus drawn with a solid line indicates the evaluation result for the singing of the singer accepted in step S12, and the rhombus drawn with a broken line indicates the evaluation result for the singing of the singer accepted in step S63. Show.
In FIG. 16, the evaluation results are displayed in a superimposed manner, but the evaluation results may be displayed in parallel if the evaluation results can be compared. For example, the evaluation results may be displayed in a bar graph in parallel.

  Even when the comparison tab is selected, the same processing as in steps S57 to S61 described above may be executed to analyze the evaluation result. In this case, it is preferable that the model sound data corresponding to the evaluation result of the other person is reproduced.

When the process of step S64 is completed or when it is determined that the comparison tab is not selected (step S62: NO), the control unit 10 monitors the operation state of the remote controller 7 and determines whether the history tab is selected. Is determined (step S65). If it is determined that the history tab has been selected (step S65: YES), a chart displaying the history of the evaluation results of the singer accepted in step S12, for example, a line graph as shown in FIG. It is displayed (step S66).
The horizontal axis of the line graph is the date and time when the singing evaluation was performed, and the vertical axis is the score indicating the evaluation result. Line graphs plotted with rhombus marks, square marks, triangle marks, and cross marks indicate the history of evaluation results for “voice”, “back voice”, “rhythm”, and “vibrato”, respectively.

  When the process of step S66 is completed, or when it is determined that the history tab is not selected (step S65: NO), the control unit 10 monitors the operation state of the remote controller 7 and determines whether or not to end the analysis. Determination is made (step S67). When it determines with not complete | finishing an analysis (step S67: NO), the control part 10 returns a process to step S55. When it determines with complete | finishing an analysis (step S67: YES), the control part 10 complete | finishes the process which concerns on song evaluation analysis.

As shown in FIG. 5, when the process related to the singing evaluation analysis is completed, or when it is determined that the daily practice mode is not selected (step S <b> 16: NO), the control unit 10 monitors the operation state of the remote controller 7. It is then determined whether or not the verification mode has been selected (step S19). When it determines with the verification mode having been selected (step S19: YES), the control part 10 test | inspects a singer's singing technique (step S20).
The verification of the singing technique is performed by reproducing accompaniment data that can verify the singing technique such as vocalization, back voice, rhythm, and vibrato, acquiring singing voice data, and evaluating the acquired singing voice data. The singing evaluation analysis shown in FIGS. 12 to 17 may also be performed on the evaluation results obtained at the time of the test.

  When the verification process is completed or when it is determined that the verification mode is not selected (step S19: NO), the control unit 10 monitors the operation state of the remote controller 7 and determines whether or not the appropriate range measurement mode is selected. Is determined (step S21). When it determines with the appropriate sound range measurement mode having been selected (step S21: YES), the control part 10 performs the process which concerns on the appropriate sound range measurement which analyzes a different voice range for every singer (step S22).

FIG. 18 is a schematic diagram illustrating an example of a singing range measurement screen. FIG. 18A is a singer's high-pitched sound range measurement screen, and FIG. 18B is a singer's bass measurement screen. The control unit 10 displays on the display unit 4 that the singer's range is measured before singing practice. Next, as shown in FIG. 18A, the control unit 10 superimposes a pitch guideline that changes from a low tone to a high tone in the time axis traveling direction (the right direction in FIG. 18) on the line music image 91a. The generated image data is generated and displayed on the display unit 4. The pitch guideline has a color that changes stepwise so that the lower the tone, the colder the color, and the higher the tone, the warmer the color. And the control part 10 acquires audio | voice data through the audio | voice input part 17, measures the pitch of the acquired audio | voice data, and superimposes a note image on the pitch guideline corresponding to this pitch. Furthermore, the guideline corresponding to the pitch obtained by the measurement is changed to an image traced with a fluorescent color, a thick line, or the like. When the input of audio data is stopped or when a predetermined time has elapsed, the highest measured pitch is stored as the upper limit of the range.
Similarly, as shown in FIG. 18B, the control unit 10 superimposes a pitch guideline that changes from a high tone to a low tone in the time axis progression direction (right direction in FIG. 18) on the line music image 91a. The generated image data is generated and displayed on the display unit 4. And the control part 10 acquires audio | voice data through the audio | voice input part 17, measures the pitch of the acquired audio | voice data, and superimposes a note image on the pitch guideline corresponding to this pitch. When the input of audio data is stopped or when a predetermined time has elapsed, the measured lowest pitch is stored as the lower limit of the range.

  When the process of step S22 is completed, or when it is determined that the appropriate sound range measurement mode is not selected (step S21: NO), the control unit 10 determines whether or not to finish the singing practice (step S23). When it determines with not finishing singing practice (step S23: NO), the control part 10 returns a process to step S14. When it determines with finishing singing practice (step S23: YES), the control part 10 finishes the process which concerns on singing practice.

  In the present embodiment, the melody blocks 91b and 91c and the singing voice waveform are superimposed on the line score image, so that the singing voice and the model melody are clear and objective. And can be visually contrasted.

  In addition, the voice waveform of the singing voice sung with vibrato and the vibrato voice waveform 91f as an example can be clearly and objectively and visually compared, and the quality of the own vibrato can be grasped.

  Furthermore, the voice waveform of the occasional song to be sung with vibrato is superimposed on the line score and the melody block 91b in the same waveform line type or superposition mode. Therefore, at the vibrato singing position, even if the pitch deviates from the melody block 91b, that effect is not displayed, and misunderstanding of the singer can be prevented.

  Furthermore, since the lyrics are displayed on the melody block 91b, the melody and lyrics can be grasped at a glance.

Furthermore, after the singing practice, the singing voice data is subjected to singing evaluation, and the evaluation result is displayed on the radar chart. Therefore, the singer can evaluate the evaluation result of his singing objectively using the radar chart. Analysis.
In particular, because it is a configuration that displays a radar chart so that the evaluation result for one's singing and the evaluation result for another singing's singing can be compared, the motivation for improving the singing technique is one by comparing with the other's singing Gives singers and enables effective singing training.
For example, if one singer wants to be able to sing like other singers who have better singing skills than himself, the singing can be compared with the other singer's evaluation results. Whether the technology is excellent or how much the singing technology should be improved, that is, the distance between the current singing technology and the singing technology at the destination point can be clearly and objectively and visually understood. It is possible to clarify the goal and direction of technological improvement.

  In addition, since the model sound data corresponding to the evaluation result is played back, there is no embarrassment with respect to the user's own voice compared to the case of playing the user's own singing voice. It becomes possible to analyze the singing technique.

  Furthermore, since it is configured to change the evaluation result and to reproduce the model sound data corresponding to the evaluation result after the change, the singing voice after the improvement of the singing technique can be confirmed, and the singing technique is improved. Can be more effectively motivated.

  Furthermore, since the history of evaluation results can be displayed in a line graph, the effect of singing practice can be clearly, objectively and visually grasped.

  The embodiment disclosed this time is to be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the meanings described above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 Karaoke device (singing voice playback device)
2 Recording medium 3 Computer program 4 Display unit 5 Speaker 6 Microphone 8 Karaoke server 10 Control unit 13 External storage device 12 RAM
14 Video output unit 15 Internal storage device 15a Music DB
15b Guidance comment DB
15c Evaluation result history DB
15d model sound DB
15e instructional video DB
16 audio output unit 17 audio input unit 18 remote control signal receiving unit 19 communication unit 91a line music image 91b, 91c melody block 91d accompaniment progress position display line 91e, 91g audio waveform 91f vibrato audio waveform

Claims (6)

Reproduction means for reproducing karaoke song data, acquisition means for obtaining voice data of a song performed while the reproduction means reproduces karaoke song data, and reproduction of the voice data obtained by the acquisition means A singing voice reproducing device comprising:
Storage means for storing song melody data indicating the melody of the sound to be sung in accordance with the karaoke song data;
A line score image generating means for generating a line score image for expressing a melody;
Based on the singing melody data stored in the storage means, a melody block whose dimension in the time axis direction in the line is a width corresponding to the length of the sound to be sung should be sung on the line Melody block superimposing means for superimposing at a position corresponding to the pitch of the sound,
A speech waveform superimposing unit that superimposes a speech waveform expressing the melody of the sound related to the speech data acquired by the acquisition unit on the line score and the melody block;
The speech waveform superimposing means includes:
A singing voice reproducing apparatus, wherein a speech waveform portion superimposed on the melody block and a speech waveform portion superimposed on the line score are superimposed in different waveform line types or superposition modes. Vibrato data indicating the time to sing and vibrato according to the karaoke song data, and the vibrato data indicating the size of the vibrato,
Vibrato speech waveform superimposing means for superimposing a vibrato speech waveform having a size indicated by the vibrato data on the melody block and the line score based on the vibrato data;
The vibrato speech waveform superimposing means comprises:
The voice waveform expressing the melody of the sound related to the voice data acquired by the acquisition means and the vibrato voice waveform are superimposed in different waveform line types or superposition modes. Singing voice playback device. The speech waveform superimposing means includes
The voice sound reproduction of the singing voice according to claim 2, wherein a voice waveform of time to sing with vibrato is superimposed on the line score and the melody block in the same waveform line type or superposition mode. apparatus. Stores lyrics data indicating lyrics to be sung in accordance with the karaoke song data,
The singing voice reproducing device according to any one of claims 1 to 3, further comprising means for generating a melody block displaying lyrics based on the lyric data and singing melody data. In the singing voice reproduction method of reproducing karaoke song data, obtaining voice data of a song performed while the karaoke song data is being reproduced, and reproducing the obtained voice data,
Generating an image of a line score for expressing a melody;
Based on the singing melody data indicating the melody of the sound to be sung in accordance with the karaoke song data, a melody block whose dimension in the time axis direction in the line music is a width corresponding to the length of the sound to be sung, Superimposing it on a position according to the pitch of the sound to be sung on the line music;
A speech waveform superimposing step of superimposing a speech waveform expressing the melody of the sound related to the acquired speech data on the line score and the melody block;
The speech waveform superimposing step includes:
A singing voice reproduction method characterized by superimposing a speech waveform portion superimposed on the melody block and a speech waveform portion superimposed on the line score in different waveform line types or superposition modes. In a computer program for causing a computer to reproduce audio data performed while karaoke song data is being reproduced,
Causing the computer to generate a line score image for expressing the melody;
Based on the singing melody data indicating the melody of the sound to be sung in accordance with the karaoke song data on the computer, the melody whose dimension in the time axis direction in the line score corresponds to the length of the sound to be sung Superimposing a block on the line score at a position corresponding to the pitch of the sound to be sung;
A speech waveform superimposing step for superimposing a speech waveform expressing a melody of sound related to the acquired speech data on the line score and the melody block on the computer;
The speech waveform superimposing step includes:
A computer program characterized by causing a computer to superimpose a speech waveform portion superimposed on the melody block and a speech waveform portion superimposed on the line score in different waveform line types or superposition modes.

Images