JP2016126135A - Karaoke device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a karaoke device for supporting singing so that food vocalization is promoted, by making a user be conscious of a motion of a jaw during singing.SOLUTION: An opening/closing motion state according to a jaw motion of a singing person is identified, from a signal which is received at detection motion identification means 32 and transmitted from a detection device during musical performance of a music. Then syllable specification means 33 and reference specification means 34 for forming reference information processing means specify reference data for estimating the opening/closing motion state according to a jaw motion of the singing person according to a syllable of text data of the music which the singing person sings, from a monosyllable table 54, and two continuous syllable tables 55 in which the syllable and jaw opening/closing motion state are set in advance. Then comparison determination processing means 35 compares the jaw opening/closing motion state identified by the detection motion identification means 32 and the reference data specified by the reference information processing means at a singing timing corresponding to each text letter, then, displays a comparison result evaluation.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a karaoke apparatus having an utterance support function in karaoke singing.

  In recent years, when a user sings along with the performance of a karaoke device installed in a karaoke room, it is recommended to sing with a wide open mouth for good speech and beauty and health. Yes. In karaoke singing, opening the mouth extremely wide may increase the burden on the temporomandibular joint and cause harm, so do not be conscious of opening the mouth greatly, but relax the joint of the lower jaw and smoothly It is thought that it is necessary to be conscious of moving. On the other hand, when singing with a well-known member, unlike the case of singing under the leadership, there is no particular concern about the movement of the chin, and the microphone signal amplification function is much lower. There is a strong tendency to sing without moving the jawbone, and it is desirable to provide karaoke content that makes it conscious that the joint of the lower jaw can be moved smoothly by relaxing the joint of the lower jaw.

  Conventionally, as a technique for detecting the movement of the jaw, for example, as in Patent Document 1, the pressure change in the ear canal caused by the movement of the muscle caused by mastication is sensed and measured by a pressure sensor inserted in the ear canal, and the pressure is measured. A technique for displaying the number of changes as the number of mastications is known. Further, as in Patent Document 2, the base part is brought into close contact with the human body, the detection means is continuously located on the base part, the movement derived from mastication is detected, the detection signal is received, and the number of chewing times A technique for recording and displaying as is also known.

  On the other hand, in karaoke, a technique for reflecting the appropriateness of the mouth shape of the singer in the scoring result is known. For example, as in Patent Document 3, shape data representing the ideal aspect ratio of the mouth is obtained for each vowel. Is stored as ideal shape data, and the syllables to be pronounced by the singer identified based on the lyric data in the music data being played are shape data corresponding to the vowels constituting the syllable. Extracting from the data, calculating the shape adaptability of the aspect ratio of the singer's mouth measured from the captured image of the camera with respect to the aspect ratio of the extracted shape data, and generating image score data based on the shape adaptability Technology is known.

Japanese Patent Application Laid-Open No. 07-213510 Japanese Patent Application Laid-Open No. 07-171136 JP 2005-242230 A

  By the way, singing karaoke music according to the lyrics moves the chin like chewing, so the singer can wear a device that can detect the number of mastications described in Patent Document 1 and Patent Document 2, so It is possible to detect the movement of the jaw at the time, but it is a technology that only counts the number of times the jaw has moved, and only the number of times the jaw has actually moved is displayed as a result. There is a problem that it is not possible to compare ideal jaw movements when singing according to the lyrics, and it is not possible to provide singing support that promotes good utterance.

  In addition, the technique described in Patent Document 3 is merely a technique for scoring a song, and it is determined whether or not a smooth jaw movement can be performed along the lyrics when singing only by specifying the mouth shape at the time of singing. There is a problem that it cannot be used as a singing support that promotes good utterance.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide a karaoke apparatus that supports singing that encourages utterance that allows the user to be aware of the jaw movement during singing.

  In order to solve the above-mentioned problems, the invention of claim 1 is a karaoke apparatus capable of communicating with a detecting device for detecting movement of a singer's jaw by singing and capable of receiving the detection signal, and playing a musical piece. In accordance with the syllable of the lyric character included in the lyrics data of the song to be sung, and the detection operation identification means for identifying the opening / closing operation state according to the movement of the singer's jaw based on the signal received from the detection device The reference information processing means for specifying the reference data for estimating the opening / closing operation state according to the movement of the singer's jaw based on a predetermined number of syllable tables in which the syllable and the jaw opening / closing operation state are set in advance is sung Singing timing data is associated with each lyric character included in the lyric data of the music, and the syllable of the syllable in which the period to sing based on the singing timing data has arrived. And reference data specified in Arensu information processing means compares the opening and closing operation state of the identified jaws by the detection operation identification means, a configuration having a comparison evaluation processing means for displaying the evaluation of the comparison results.

According to the second to fourth aspects of the present invention, “the predetermined number of syllable tables is a single syllable table associated with the opening / closing operation state of the jaw when uttering one syllable, and the first syllable is uttered for two syllables. A syllable table in which the jaw opening / closing operation state at the time of uttering the second syllable from the state is associated with the pair in the order of the two syllables, and the reference information processing means starts from the first character of the lyrics data Syllable specifying means for sequentially specifying syllables, and the lyrics data is divided into a plurality of phrase sections, and when the syllable specified by the syllable specifying means is the head of a phrase section, the jaw opening / closing operation state corresponding to the syllable Is determined from the single syllable table, and in the case of a syllable other than the head of the phrase section, the syllable preceding the syllable and the jaw opening / closing operation state corresponding to the pair in the order of the syllable are described in the double syllable table. And a reference identifying means for al identified, and the having "configuration,
“Singing timing data associated with each lyric character included in the lyric data of the music is color change timing data for each lyric character included in the lyric data, and the reference specifying means includes the phrase section, The lyric character strings that change the color of the lyrics continuously are classified as a batch, and the opening and closing operation state of each syllable jaw is specified for each category.
“Singing timing data associated with each lyric character included in the lyric data of the music is data in which a lyric character is associated with each note of the musical score data of the melody part included in the performance data of the music, The reference specifying means has a configuration in which the phrase section is divided into lyrics character strings in which the pronunciation of notes is continuously performed, and the opening / closing operation state of each syllable jaw is specified for each division.

  According to the first aspect of the present invention, the opening / closing operation state corresponding to the movement of the singer's jaw is identified from the signal received from the detection device during the performance of the music, and the lyrics characters included in the lyrics data of the music to be sung Lyric data of music by specifying reference data for estimating the opening / closing operation state according to the movement of the singer's jaw according to the syllable based on a predetermined number of syllable tables in which the syllable and the opening / closing operation state of the jaw are set in advance. The reference data of the syllable that has reached the period to be sung based on the singing timing data associated with each lyric character included in the lyric character is compared with the opening / closing operation state of the jaw identified by the detection operation identifying means, and the comparison result It is ideal based on the movement of the jaw when the singer sings and the lyrics of the song, in accordance with the lyrics displayed on the screen when playing the karaoke song. Since the movement of the chin is compared for each syllable, it is possible for the singer to check for each syllable whether the movement of the chin is smooth or not. It can support singing that promotes good utterance.

  According to the invention of claim 2, the single syllable table in which the opening / closing operation state of the jaw when speaking one syllable is associated, the second syllable is uttered from the state where the first syllable is uttered for two syllables. It has a double syllable table that associates the opening and closing movement state of the jaw with the pair in the order of the two syllables, specifies the syllables sequentially from the first character of the lyric data, and divides the lyric data into a plurality of phrase sections When the specified syllable is the beginning of a phrase section, the jaw opening / closing operation state corresponding to the syllable is specified from the single syllable table, and when it is a syllable other than the beginning of the phrase section, the syllable before the syllable and the corresponding syllable By configuring the jaw opening / closing operation state corresponding to the syllable order from the double syllable table, the jaw opening / closing operation state at the time of song singing is estimated and specified from the lyric data by reference data. From, there is no need to create a pre-data to the karaoke song reaches several million songs, those that can be realized with a simple configuration.

  According to the third and fourth aspects of the invention, the singing timing data associated with each lyric character included in the lyric data of the music is the color change timing data for each lyric character included in the lyric data or the performance data of the music. This is data in which lyrics characters are associated with each note of the musical score data of the melody part included in the lyric part. By categorizing the lyrics character string as a whole and specifying the opening / closing operation state of each syllable jaw for each division, the reference data of the opening / closing operation state of the jaw when singing a song by lyric data or performance data The estimation and identification can be made with higher accuracy.

It is a block block diagram of the karaoke apparatus which concerns on this invention. It is explanatory drawing of the single syllable table of FIG. 1, and a double syllable table. It is explanatory drawing of the lyric data of a reference specific object. It is a block block diagram of the detection apparatus of FIG. It is explanatory drawing of the jaw movement specific process based on the lyric data in the reference specific means of FIG. It is explanatory drawing of the jaw movement specific process of the individual syllable of FIG. It is explanatory drawing of the comparative evaluation in the comparative evaluation means of FIG. It is explanatory drawing of phrase area specification based on the other lyric data in the reference specific | specification means of FIG. 1, and reference specific.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram of the karaoke apparatus according to the present invention, FIG. 2 is an explanatory diagram of the single syllable table and the double syllable table of FIG. 1, and FIG. FIG. 4 shows a block diagram of the detection apparatus of FIG.

  In FIG. 1, a karaoke apparatus 11 is connected to a karaoke main body 12 serving as a main apparatus by wired or wireless connection, and a display unit 13, a mixing amplifier 14, a microphone 15, a speaker 16, and a remote input / output device 17 are connected. The karaoke main body 12 is configured to be communicable with a predetermined number of detection devices 18 (18A to 18N) (the detection device 18 will be described with reference to FIG. 4).

  The display unit 13 displays a normal music selection display, a background image during karaoke performance, and the like. For example, a liquid crystal display (LCD), a plasma display (PDP), and other various displays can be employed. The mixing amplifier 14 mixes the audio signal from the microphone 15 with the music performance signal sent from the karaoke main body 12, amplifies it, and outputs it from the speaker 16.

  The remote input / output device 17 transmits / receives data to / from the karaoke main body 12 using a wired system or a wireless system (IR system, Bluetooth (registered trademark) mechanism piconet connection system, etc.) by a terminal transmission / reception unit (not shown). Therefore, at least a login / logout processing means 17A, a music selection music registration means 17B as music search means, a detection device selection means 17C, and a remote control display unit 17D are provided as appropriate.

  The login / logout processing means 17A obtains a user name and user ID from the user in response to a login operation by the user, stores the user name and user ID in the RAM 23 as user information 42, and performs a process of permitting login. This is a program that performs use termination processing in response to a logout operation. The remote input / output device 17 can search and reserve a song that the user wants to sing using a GUI function described later, but the user preferably logs in to the karaoke device 11 first.

  The music selection music registration means 17B is a program for searching the music desired by the user based on a table including data such as music ID, music title, artist name, etc. and transmitting the music ID of the music selected to the karaoke main body 12. It is. The transmitted music ID is registered in the RAM 23 reservation queue 41 in the karaoke main body 12.

  The detection device selection means 17C is a program that associates the detection device 18 with the logged-in user. Specifically, a list of the detection devices 18A to 18N that have already been paired with the karaoke device 11 is displayed on the remote control display unit 17D, and the detection device 18 to be used is selected from the list. The user is associated with the detection device 18 and the information is stored in the RAM 23 as one of the user information 42. And if a user performs reservation operation after selecting himself as a singer at the time of a music reservation, the karaoke apparatus 11 will detect the detection signal from the detection apparatus 18 matched with the said user at the time of the performance of the said music. Will only receive.

  The remote control display unit 17D has a liquid crystal display (LCD) and a touch sensor stacked to be used for input / output, and can input data such as music selection by the touch sensor corresponding to displayed icons and the like. It has a GUI user interface function.

  The karaoke main body 12 includes a bus 20, a central control unit 21, a ROM 22, a RAM 23, a video display control unit 24, a music performance control unit 25, a sound source (synthesizer) 25A, a scoring unit 26, an A / D conversion unit 26A, and a transmission / reception unit 27. , A storage unit 28, a detection signal receiving unit 31, a detection operation identification unit 32, a syllable identification unit 33, a reference identification unit 34, and a comparative evaluation processing unit 35. The syllable specifying means 33 and the reference specifying means 34 constitute reference information processing means.

  In the RAM 23, storage areas for a reservation queue 41, user information 42, target lyrics data 43, reference data 44, and comparison result data 45 are formed. The storage unit 28 also stores a music database (music DB) 51, a video database (video DB) 52, a reference database (reference DB) 53, a single syllable table 54, and a double syllable table 55. In addition, about each said structure, even if it is an element part which is not directly related to the summary of this invention, in order to show that most can be applied also in the conventional karaoke system, the whole component is demonstrated.

  The central control unit 21 is a physical CPU that performs overall processing control of the system, and performs algorithm processing based on a program stored in the ROM 22. The RAM 23 has a storage area for a reservation queue 41, user information 42, target lyrics data 43, reference data 44, and comparison result data 45, and a work area for developing and executing the various programs. It is a concept that includes a case where it is configured by, for example, a semiconductor memory and is virtually built on a hard disk.

  The target lyrics data 43 stored in the RAM 23 is the lyrics data read from the music DB 51 based on the music ID registered in the reservation queue 41 for music selection reservation. As shown in FIG. 3, the lyric data of this music is composed of a page in which all lyrics characters are divided into appropriate measures, a line delimited by one horizontal character string displayed on the display unit 13 and the like, and each character. Character number and character code, lyric text or ruby type code, display layout indicating the display position of the display unit 13 or the like, and color change timing for color change processing on the performance time series for each character. The color change timing includes a color change start time and an end time. For example, the color change timing represents an elapsed time from the start of performance in ms units.

  That is, this lyric data is divided into pages and lines, the page is the maximum display unit for one screen of the display unit 13, and the line is a character string for one horizontal line when displayed on the display unit 13 or the like. This page or line is divided into a plurality of phrase sections in the reference specifying means 34 described later (see FIG. 5), and an opening / closing operation state specifying process corresponding to the jaw movement described later is performed. The reading process is reserved and registered in the reservation queue 41 and is performed by the central control unit 21 at the start of performance. However, a program specialized for this process may be provided separately.

  The comparison result data 45 stored in the RAM 23 is a comparison result by the comparative evaluation processing means 35 and is stored in real time for each syllable, and becomes data of all comparison results of the entire singing when the singing is completed.

  The video display control means 24 is a program or an electronic circuit that outputs the background video extracted from the video DB 52 and the lyrics data of the music extracted from the music DB 51 to the display unit 13 during performance. Note that the evaluation in the comparative evaluation processing means 35 may be displayed. The music performance control unit 25 includes, for example, a sequence program, and drives a sound source (synthesizer) 25A according to the note data extracted from the music DB 51 with the music ID. The output of the sound source 25A is sent to the mixing amplifier 14 as a performance signal. Is output.

  The scoring means 26 is for scoring by detecting the pitch of the singing voice by the user and comparing it with the pitches of the reference data of the main melody for each music, which will be described later. It is a program that performs scoring processing based on reference data for singing a voice signal digitally converted by 26A. Specifically, for example, a technique described in Japanese Patent No. 4229915 can be used.

  The transmission / reception unit 27 is used to exchange data with the remote input / output device 17 using a wired system or a wireless system (such as an IR system or a Bluetooth (registered trademark) mechanism piconet connection system). Electronic circuit and program.

  The music DB 51 stored in the storage unit 28 stores note data, lyrics data, and the like for each music. Regarding performance, specifically, performance data of a karaoke piece of music in a predetermined data format having a music table in which a music ID, a music title, and an artist ID (artist name) are associated and managed by the music ID for each piece of music (for example, This is a database storing music IDs as file names for music data (files) composed of MIDI (registered trademark) format melody part musical score data). The video DB 52 stored in the storage unit 28 is a database in which music IDs are stored as file names for background video data corresponding to each music.

  Reference DB53 memorize | stored in the said memory | storage part 28 is scoring reference data of the main melody for every music, Comprising: The karaoke music memorize | stored in the said music DB51 and the song by the singer according to the song section of the said karaoke music are carried out. It is a database that stores and stores scoring reference data used as evaluation criteria for evaluation and analysis.

  The single syllable table 54 stored in the storage unit 28 utters the target syllable in the case where the target syllable in the lyrics data is the head of a phrase section that is processed by the reference specifying unit 34 described later. At this time, it is a table that specifies the opening / closing operation state of the jaw that moves in the direction of opening or closing the jaw, and as shown in FIG. 2 (A), one of “vowel” or “consonant + vowel” that is the target syllable. Jaw movement (opening / closing operation state) is preset for one syllable. The syllable at the beginning of the phrase section is set to move when the syllable at the beginning of the phrase section is uttered from a state where the chin is closed. 2A and 2B, “↑” represents a movement for opening the jaw, “↓” represents a movement for closing the jaw, and “−” represents a state where there is no movement of the jaw.

  The double syllable table 55 stored in the storage unit 28 stores the second syllable between the syllable before the target syllable and the target syllable when the target syllable in the lyrics data is a syllable other than the head of the phrase section. 2 is a table that specifies movements of the jaw opening direction or closing direction when uttering two syllables in order. As shown in FIG. 2B, “vowel” or “ “Consonant + vowel”, the target syllables “vowel” and “consonant + vowel”, and special vocal syllables (for example, “vowel a” and “consonant + vowel a”, “vowel e” and “consonant + vowel e”) , And “vowel o” and “consonant + vowel o” are “ha (ha)”, “he (he)”, “ho (ho)”, “n (n)”). The opening / closing operation state of the jaw is set in advance for the group according to the order. Note that the syllables other than the head of the phrase section set the jaw movement when uttering the second syllable from the state of uttering the first syllable for two consecutive syllables.

  The detection signal receiving unit 31 is a program or an electronic circuit that receives a detection signal transmitted from the detection device 18 described later. The detection operation identifying means 32 is a program or an electronic circuit for identifying the open / closed state of the jaw from the signal received from the detection device 18 during the performance of the music. For example, a voltage level signal (which may be a data signal specialized for the level value) is received from the detection device 18 as the detection signal, the level value is specified based on the signal, and the jaw is set according to the preset level value. Is identified based on the correspondence between open, closed, and stationary open / closed states.

  The reference information processing means composed of the syllable specifying means 33 and the reference specifying means 34 uses reference data for estimating the opening / closing operation state of the singer's jaw movement according to the syllable of the lyrics data of the song to be sung. 3 is calculated based on the preset single syllable table 54 and the double syllable table 55. The syllable specifying means 33 stores the target lyrics data 43 shown in FIG. Is a program for specifying syllables sequentially from the first character of the lyrics data (described with reference to FIG. 5).

  The reference specifying unit 34 divides the target lyrics data 43 into a plurality of phrase sections (the division will be described with reference to FIGS. 5 and 8), and the syllable specified by the syllable specifying unit 33 is the head of the singing section. The jaw opening / closing operation state corresponding to the syllable is specified from the single syllable table 54, and in the case of a syllable other than the head of the singing section, the jaw opening / closing operation corresponding to the pair in the order of the syllable before the syllable and the syllable This is a program that specifies the state from the above-mentioned double syllable table 55 and stores it as reference data 44 in the RAM 23 (described in FIGS. 5 and 6).

  The comparative evaluation processing means 35 opens and closes the jaws specified by the reference information processing means (the syllable specifying means 33 and the reference specifying means 34) for the syllable in which the period to be sung has arrived based on the singing timing data described later. The reference data 44 stored in the RAM 23 as the state is compared with the data of the jaw opening / closing operation state identified by the detection operation identifying means 32, and the comparison result is evaluated and the remote control display unit 17D of the remote input / output device 17 is evaluated. Alternatively, the program is displayed on the display unit 13 (may be combined with the remote control display unit 17D) and the comparison result is stored in the RAM 23 as the comparison result data 45. The comparison process and the evaluation process will be described with reference to FIG.

  Here, the detection device 18 shown in FIG. 1 is a wearable terminal, which is always provided in a karaoke room or the like, and is selectively held by the user. The receiver 31) is paired by short-range wireless communication. As shown in FIG. 4, the detection apparatus 18 includes a detection processing unit 62 and a detection transmission unit 63 in the apparatus main body 61, and a detection sensor 64 extended and connected to the outside. The detection processing means 62 is provided with a program that holds the device ID and manages the entire device. When the user logs in, the detection processing unit 62 selects and specifies the detection device 18 (device ID) and uses it in the RAM 23. Stored as part of the person information 42.

  The detection transmission means 63 transmits the detection signal detected by the detection sensor 64 to the detection signal reception unit 31 of the paired karaoke apparatus 11 as, for example, a voltage level signal or a data signal specialized for the level value. A program or an electronic circuit. The detection sensor 64 includes a detection element such as a pressure sensor that detects movement of the jaw by singing, and uses a sensor that can be inserted into the ear canal of the user or a sensor that can be worn around the ear canal. It is something that can be done.

  Therefore, FIG. 5 shows an explanatory diagram of the jaw movement specifying process based on the lyrics data in the reference specifying means of FIG. 1, FIG. 6 shows an explanatory diagram of the individual syllable jaw movement specifying process of FIG. 5, and FIG. The explanatory view of comparative evaluation in one comparative evaluation means is shown. Here, it is assumed that the target lyrics data 43 is stored in the RAM 23 shown in FIG. 3 when a music piece is reserved in the reservation queue 41.

  When the music selection reservation is made, the syllable specifying means 33 performs the lyrics as shown in FIG. 5 for the target lyrics data 43 shown in FIG. 3 read from the RAM 23 based on the music ID of the music. In order from the first character of the data, the character whose character code is hiragana is specified as a syllable (kanji is not specified).

  The reference specifying means 34 divides the target lyrics data 43 into a plurality of phrase sections as shown in FIG. For example, in the target lyric data 43 of FIG. 3, lyric character strings in which the lyric color is continuously changed are set as sections that are grouped together, and the color change associated with each lyric is changed. Based on the timing data, if the start timing of the target syllable and the end timing of the syllable before the target syllable are continuous, then the two syllables are continuously color-changed, Include in one phrase section.

  In the specific example of FIG. 3, the end timing of the character number 1 “A” in the lyrics data and the start timing of the subsequent character number 2 “TA” are both continuous at 9750 ms, which is the elapsed time from the start of the performance. Thereafter, the continuous state continues from the character number 7 “mo” to the character number 8 “no”, and this is included in one phrase section.

  On the other hand, when the start timing of the target syllable and the end timing of the syllable before the target syllable are not consecutive, the two syllables are not continuously changed in color, and this is the phrase Since it is a delimiter, the target syllable is the beginning of the phrase. In the specific example of FIG. 3, the end timing of “no” of the character number 8 in the lyrics data is 12700 ms, which is the elapsed time from the start of the performance, and the start timing of the subsequent “u” of the character number 10 is from the start of the performance. The elapsed time is 13000 ms, and there is a non-continuous interval of 300 ms between them, which is a phrase delimiter.

  Subsequently, the reference specifying means 34 specifies the opening / closing operation state of the corresponding jaw with respect to the syllable specified by the syllable specifying means 33 with reference to the single syllable table 54 for the first syllable of the phrase section, For the syllables other than the head of the phrase section, the double syllable table 55 is referred to and the jaw opening / closing operation state corresponding to the pair in the order of the syllable immediately before the target syllable and the target syllable is specified. Is.

  Specifically, in the example of FIG. 6A, since the syllable of “A” [(consonant +) a] of character number 1 in FIG. 5 along the performance is the head of the phrase section, it is a single syllable. The jaw movement “↑” corresponding to the target syllable [(consonant +) a] is specified with reference to the table 54. In the example of FIG. 6B, the syllable of “ta” [(consonant +) a] of the character number 2 in FIG. 5 along the performance is in the middle of the phrase section, so the immediately preceding syllable is “A”. ”[(Consonant +) a], referring to the double syllable table 55, the jaw movement“ ↓ corresponding to the target syllable [(consonant +) a] → [(consonant +) a] ” ↑ ”is specified.

  In the example of FIG. 6C, since the syllable of “ku” [(consonant +) u] of the character number 6 of FIG. 5 along the performance is in the middle of the phrase section, the immediately preceding syllable is “O” [ (Consonant +) o] is specified, and the jaw movement “↓” corresponding to the target syllable [(consonant +) o] → [(consonant +) u] is determined by referring to the double syllable table 55. Identify. Furthermore, in the example of FIG. 6D, the syllable of “M” [(consonant +) o] of the character number 7 of FIG. 5 along the performance is in the middle of the phrase section, so the immediately preceding syllable is “ku”. ”[(Consonant +) u]”, referring to the double syllable table 55, the jaw movement “↑ corresponding to the target syllable [(consonant +) u] → [(consonant +) o]” Is specified.

  Then, in the reference specifying means 34, the reference data specifying the jaw opening / closing operation state as described above is specified for each syllable specified by the lyrics data, and the reference data specified by the syllable of all the lyrics data is stored in the RAM 23. Is stored as reference data 44.

  Therefore, the comparative evaluation processing means 35 reads the reference data 44 from the RAM 23 when the karaoke music reserved for music selection is played, the performance of the karaoke music reserved for music selection is started, and the period to be sung based on the singing timing data. By comparing the reference data of the syllables that have arrived with the detection motion (open / close operation state of the jaw) received and identified from the detection device worn by the singer during the period of singing of the syllable, Give an evaluation of the comparison results.

  Specifically, as shown in FIG. 7, when the detection operation received during the period of singing the syllable “A” on the performance time axis is “↑ (open chin)”, the syllable Since the reference data of jaw movement corresponding to “A” is “↑”, it means that an ideal jaw movement has been made for the syllable, for example, the comparison result is “○”, and the following syllable “ When the detection signal received during the singing period is “↓ (jaw closed)”, the jaw movement reference data corresponding to the syllable “ta” is “↓ ↑”. For syllables, the ideal jaw movement has not been achieved, and for example, the comparison result is “x”.

  The comparative evaluation processing unit 35 displays the evaluation of the comparison result on one or both of the remote control display unit 17D and the display unit 13 of the remote input / output device 17. For example, when the evaluation content is displayed in real time during the singing, the number of times “○” and the number of times “×”, which are comparison results at the time when the period for singing an arbitrary syllable is over, It is displayed in real time at a predetermined location on the screen of the display unit 13 or the remote control display unit 17D. In this way, the user is conscious of increasing the number of times “◯” from “X”, that is, singing while considering the movement of the jaw.

  As a process of evaluation display from another comparative evaluation, the comparative evaluation processing means 35 counts the number of times each jaw is opened and closed based on the movement of the jaw by the detection operation identified from the reference data 44 and the received detection signal. Specifically, the jaw movement during the period to sing the syllable “a” is “↑”, and this is once, and the jaw movement during the period to sing the subsequent syllable “ta” is “↓ ↑”. When the performance progresses to “no”, the number of jaw movements in the reference data 44 is nine. Each time the syllable singing period ends, the count value based on the reference data 44 is compared with the count value of the detection operation, and evaluation is performed according to the difference between the comparison results.

  That is, based on the reference data 44 and the open / close operation state corresponding to the jaw movement specified from the detection operation, the number of open / close times of each jaw is counted from the specified “↑”, “↓”, “↓ ↑”. When the period for singing an arbitrary syllable is over, the count value of the detection operation of the detection signal is less than a predetermined value relative to the count value based on the reference data 44, meaning that the jaw is consciously moved. “↑” is evaluated, and if the count value of the detection operation of the detection signal is more than a predetermined value, the evaluation value is “↓” in the sense that the jaw is moved too much, and the count value of the detection operation of the detection signal is When the difference from the number of times to move is within a predetermined value, “GOOD” in the sense that the jaw is moving smoothly is displayed in real time as an evaluation. Thus, it is possible to support the singing of how to continue the “GOOD” display, that is, singing while taking care of the jaw movement.

  On the other hand, when the evaluation content is displayed after singing, the comparison evaluation processing means 35 reads the comparison result data 45 for each singing syllable from the RAM 23 and displays it after the singing. For example, if the comparative evaluation processing means 35 transfers the comparison result data 45 in association with the music ID and the user ID to the remote input / output device 17 after singing, the singer and the comparison result data 45 are associated with each other. The singer displays the syllable (hiragana) of the sung song on the remote control display unit 17D, and the comparison result of each syllable in the comparison result data 45 is evaluated as “○” in blue, “×” Displays characters in red. Thereby, the singer can confirm the syllables in which the jaw moves smoothly and the syllables in which the jaw does not move.

  Next, FIG. 8 shows an explanatory diagram of phrase section specification and reference specification based on other lyrics data in the reference specifying means of FIG. Here, the phrase section specifying process and the reference specifying process are shown. Further, as shown in FIG. 8A, the RAM 23 has a storage area for the target performance data 71, and the performance data read from the music DB 51 based on the music ID reserved for music selection in the reservation queue 41. Is memorized.

  Therefore, in FIG. 8B, the reference specifying means 34 first converts each musical note data of the singing melody part included in the musical performance data from the target lyrics data 43 and the target musical performance data 71 stored in the RAM 23. On the other hand, lyric character / melody correspondence data (corresponding data of note / rest, pitch, and lyrics in FIG. 8B) is created in association with lyric character data to be sung. Specifically, the syllable corresponding to the note is specified from the degree of coincidence between the pronunciation timing of each note in the musical score data of the singing melody part and the color change timing of each syllable in the target lyrics data 43. At this time, the syllable color change is set to be started before the start of the pronunciation of each note so that the singer can easily sing, so this timing difference should be taken into consideration. Further, even when one syllable is pronounced with a plurality of notes, it is possible to associate them according to the degree of coincidence between the color change time of the syllable and the pronunciation duration of the two notes.

  Subsequently, the phrase section in the case where the movement of the jaw is specified based on the lyric character / melody correspondence data is classified as a batch of lyric character strings in which the melody part's notes are continuously pronounced. In other words, while the notes are continuous, the syllables associated with the two notes are continuously pronounced, and thus are included in one phrase section. For example, notes that correspond to the syllable “Atamagomono ni dashi” shown in FIG. 8B are included in one phrase section.

  In addition, since the two notes before and after the rest are not continuously pronounced by the rest, the rest is a phrase delimiter, and the syllable associated with the next note after the rest is the head of the phrase. For example, since there is a quarter rest next to the half note corresponding to “shi” in the syllable “˜ue ni dashi” shown in FIG. 8 (B), the syllable “shikata” corresponding to the next dotted quarter note. “” Of “no” is the head of the phrase.

  Then, as shown in FIG. 8B, the jaw movements (“↑”, “↓”, “↓ ↑”) identified with reference to the single syllable table 54 and the double syllable table 55 are set as the target syllables. By associating, the reference data 44 for each syllable based on the lyrics data and the performance data is specified. The reference data 44 specified and stored in this way is the syllable in which the period to be sung has arrived in the processing in the comparative evaluation processing means 35 based on the pronunciation timing of each note of the musical score data of the singing melody part. The reference data is specified.

  This sort of phrasal data and performance data can be used to estimate and specify the jaw opening / closing operation state during song singing with higher accuracy, and karaoke of tens of thousands of songs. It is not necessary to create data in advance for music, and can be realized with a simple configuration.

  As described above, according to the lyrics displayed on the screen during the performance of karaoke music, the movement of the jaw when the singer sings and the ideal movement of the jaw based on the lyrics of the music sings Since it is compared for each syllable, it is possible for the singer to check for smooth chin movements at the time of singing for each syllable. It can support singing.

  Moreover, since the jaw opening / closing operation (reference data) that will open and close during song singing is estimated and specified from the lyrics data and performance data, data is created in advance for tens of thousands of karaoke songs. This is not necessary and can be realized with a simple configuration.

  The karaoke apparatus of the present invention can be used in industries such as manufacture, use, and sales of a karaoke terminal having a basic karaoke function and a karaoke server.

DESCRIPTION OF SYMBOLS 11 Karaoke apparatus 12 Karaoke main body 17 Remote input / output apparatus 18 Detection apparatus 28 Memory | storage part 31 Detection signal receiving part 32 Detection operation | movement identification means 33 Syllable identification means 34 Reference identification means 35 Comparison evaluation processing means 43 Target lyrics data 44 Reference data 45 Comparison result Data 53 Reference database 54 Single syllable table 55 Dual syllable table 63 Detection transmission means 64 Detection sensor

Claims (4)

A karaoke device that can communicate with a detection device that detects the movement of a singer's jaw by singing and can receive the detection signal,
A detection operation identifying means for identifying an opening / closing operation state according to the movement of the singer's jaw based on a signal received from the detection device during the performance of the music;
Reference data for inferring the opening / closing operation state according to the movement of the singer's jaw according to the syllable of the lyric character included in the lyrics data of the song to be sung, a predetermined syllable and the opening / closing operation state of the jaw Reference information processing means for identifying based on a number of syllable tables;
Singing timing data is associated with each lyric character included in the lyric data of the song to be sung, and the reference specified by the reference information processing means of the syllable in which the period to be sung has arrived based on the singing timing data Comparison evaluation processing means for comparing the data and the open / close operation state of the jaw identified by the detection action identification means, and displaying the evaluation of the comparison result,
A karaoke apparatus comprising: The karaoke apparatus according to claim 1,
The predetermined number of syllable tables is a single syllable table that correlates the opening / closing operation state of the jaw when uttering one syllable, and when uttering the second syllable from the state that uttered the first syllable for two syllables Is a double syllable table in which the opening / closing operation state of the chin is associated with a pair in the order of the two syllables,
The reference information processing means includes
Syllable specifying means for sequentially specifying syllables from the first character of the lyrics data;
The lyrics data is divided into a plurality of phrase sections, and when the syllable specified by the syllable specifying means is the head of the phrase section, the jaw opening / closing operation state corresponding to the syllable is specified from the single syllable table, and the phrase A reference specifying means for specifying, from the double syllable table, a jaw opening / closing operation state corresponding to a pair in the order of the syllable preceding the syllable and the syllable in the case of a syllable other than the head of the section;
A karaoke apparatus comprising: A karaoke apparatus according to claim 2,
Singing timing data associated with each lyric character included in the lyric data of the music is color change timing data for each lyric character included in the lyric data,
The karaoke characterized in that the reference specifying means classifies the phrase section as a batch of lyrics character strings in which the color of the lyrics is continuously changed, and specifies the opening / closing operation state of each syllable jaw for each section. apparatus. A karaoke apparatus according to claim 2,
Singing timing data associated with each lyric character included in the lyric data of the music is data in which a lyric character is associated with each note of the musical score data of the melody part included in the performance data of the music,
The karaoke apparatus characterized in that the reference specifying means divides the phrase section as a batch of lyrics character strings in which the pronunciation of notes is continuously performed, and specifies the opening / closing operation state of each syllable jaw for each section .

Images