JP5803172B2 - Evaluation device - Google Patents

Description

  The present invention relates to a technique for displaying a character when evaluating a sung voice.

  Conventionally, there is a technique for displaying a character when assisting a singer who sings in karaoke or the like to sing well. In Patent Document 1, when the above-mentioned assistance is performed by outputting a synthesized singing voice (so-called guide vocal), a character having a mouth shape or facial expression that matches the phonological tone, pitch, and pronunciation timing of the singing voice is disclosed. The technology to display is described.

JP 2001-42879 A

In the technique described in Patent Document 1, while various expressions are displayed in one piece of music, patterns of expressions that are displayed for the same piece of music and the timing for displaying them are determined. That is, no matter how the singer sings, there is no reaction to the contents, so it is difficult to give the singer a sense of realism as if singing with the displayed character. For the same reason, the singer does not know whether he is singing in the same way as the guide vocal, that is, the standard singing voice to be sung. It is difficult to judge whether to continue singing or to use the above guide vocal more.
The present invention has been made in view of such circumstances, and one of its purposes is to display a character that reacts to the extent that the voice sung by the singer is in line with the standard to be sung. It is.

In order to solve the above-described problem, the present invention provides a storage unit that stores position display data indicating a position where a song should be sung when reproduced, and reference data indicating a reference for evaluating the singing voice, and the storage unit Read out the position display data from the reproduction means, and the singing voice represented by the audio signal supplied from the sound collection means when the reproduction means is reproducing the position display data, and read out from the storage means An evaluation unit that evaluates the singing voice by comparing with a reference indicated by the reference data, and a first image data indicating a character image is output while the reproduction unit reproduces the position display data, and When the singing voice is evaluated by the evaluation means, an image representing the facial expression of the character with a facial expression corresponding to the result of evaluating the singing voice is shown. An output means for outputting the second image data, and outputs data indicating the image obtained by changing the facial expression as the second image data if a particular singing technique from singing voice which the audio signal represented is detected And an output means .

Moreover, in another preferable aspect, the evaluation means compares the first singing voice represented by the audio signal with a reference indicated by the reference data read from the storage means to obtain the first singing voice. If the evaluation score of the first singing voice is higher than the evaluation score of the second singing voice compared with the reference in the past, the evaluation score of the first singing voice is added. Features.

Further, the present invention provides a storage means for storing position display data indicating a position where a song should be sung when reproduced, and reference data indicating a reference for evaluating the singing voice; and the position display data from the storage means. Reproducing means for reading and reproducing; first singing voice represented by an audio signal supplied from the sound collecting means when the reproducing means reproduces the position display data; and the reference data read from the storage means Is an evaluation means for comparing the first singing voice with the evaluation point of the second singing voice compared with the reference in the past. If the evaluation score is high, an evaluation means for adding to the evaluation score of the first singing voice, and a first image indicating a character image while the reproduction means reproduces the position display data. A second image that outputs image data and shows the facial expression of the character as a facial expression corresponding to a result of the evaluation of the first singing voice when the evaluation means evaluates the first singing voice; And an output means for outputting image data .

In another preferable aspect, the output means outputs, as the second image data, data indicating an image in which the facial expression is changed when a specific singing technique is detected from the singing voice represented by the audio signal. characterized in that it.

  According to the present invention, it is possible to display a character that reacts to the extent that the voice sung by the singer is along the reference to be sung.

It is a block diagram which shows the whole structure of the karaoke apparatus which concerns on embodiment. It is a block diagram which shows the functional structure of the control part of a karaoke apparatus. It is a figure which shows an example of the image displayed on a display. It is a figure which shows an example of the image displayed on a display. It is a figure which shows an example of the image displayed on a display. It is a figure which shows the structure of the karaoke apparatus which concerns on a modification. It is a figure for demonstrating a mode that the direction of a display is adjusted.

[Embodiment]
Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing the overall configuration of the karaoke apparatus 1. The karaoke apparatus 1 is installed in, for example, a karaoke shop and provides a user with a so-called karaoke function and evaluates a voice (singing voice) sung by the user (singer), according to the present invention. It corresponds to an example of “evaluation device”. The karaoke apparatus 1 includes a control unit 10, an operation unit 21, an acoustic processing unit 22, a sound collection unit 23, a sound emission unit 24, a stereoscopic image display unit 25, and a storage unit 30.
The control unit 10 includes a CPU (Central Processing Unit) and a memory. The CPU controls each unit of the karaoke apparatus 1 by executing a program stored in the memory. The memory includes a ROM (Read Only Memory) and a RAM (Random Access Memory), and stores programs and data used by the CPU.
The operation unit 21 includes a plurality of operation buttons, and supplies operation data indicating the contents operated by the user to the control unit 10.
The sound collection unit 23 is a sound collection unit such as a microphone, and receives a singing voice emitted by a singer and outputs an audio signal indicating the singing voice to the acoustic processing unit 22.
The sound emitting unit 24 is a sound emitting unit such as a speaker, and emits an audio signal output from the sound processing unit 22.
The acoustic processing unit 22 includes a signal processing circuit such as a DSP (Digital Signal Processor), a sound source that generates an audio signal from a MIDI (Musical Instrument Digital Interface) format signal, and the like. The sound processing unit 22 performs A / D conversion on the audio signal input from the sound collection unit 23 and outputs the audio signal to the control unit 10. The sound processing unit 22 receives MIDI format data from the control unit 10 and generates an audio signal based on the data. The sound processing unit 22 performs signal processing such as effect processing and amplification processing on the audio signal thus generated, the audio signal output from the control unit 10, the audio signal input from the sound collection unit 23, and the like. The sound is output to the sound emitting unit 24.
The stereoscopic image display unit 25 includes a display corresponding to the stereoscopic image, and displays a stereoscopic image corresponding to image data indicating the stereoscopic image supplied from the control unit 10.

The storage unit 30 includes a hard disk, and stores a music database 310 and a character database 320.
In the music database 310, data related to music played in karaoke is recorded, and in detail, accompaniment data 311, reference data 312 and lyric data 313 are recorded. The accompaniment data 311 is data indicating the content of the accompaniment of the music and is described in, for example, the MIDI format. The reference data 312 is data indicating a standard to be sung, and specifically indicates the pitch of each component sound to be sung. These pitches, that is, the pitches of the constituent sounds, are associated with the period during which each constituent sound is to be sung in the accompaniment indicated by the accompaniment data 311, and the singing voice input to the sound collection unit 23 during this period Used to evaluate. Hereinafter, this period is referred to as an “evaluation period”. The start and end of the evaluation period are represented by the time elapsed after the accompaniment data 311 is started. In the reference data 312, for example, the pitch and length of each constituent sound (corresponding to the evaluation period) are described in the MIDI format. The lyrics data 313 includes data indicating the contents of the lyrics of the music and data indicating the timing for changing the color of the lyrics telop displayed on the stereoscopic image display unit 25.

  In the character database 320, data relating to an image displayed on the stereoscopic image display unit 25 during singing is recorded, and in detail, character image data 321 is recorded. The character image data 321 is data (image data) indicating images of various expressions of various characters. The character here is, for example, a person represented by a picture, an animal, or an anthropomorphic object (such as a robot), and also includes a character (so-called avatar) used on an Internet community site. In addition, a photograph of a real person, an animal, or a thing may be used for this picture. Here, the character's facial expression is a character's emotion or information that you want to convey, such as a facial expression or gesture, such as a smiling expression, an angry expression, a sadly walking action, For example, a jumping action, an action of waving at the tempo, or showing a certain timing.

Next, functional blocks constructed when the control unit 10 executes a functional program in such a hardware configuration will be described with reference to FIG.
FIG. 2 is a block diagram illustrating a functional configuration of the control unit 10. The control unit 10 includes a reproduction unit 111, an evaluation unit 112, a specification unit 113, an image generation unit 114, and a display control unit 115.
When the operation data for instructing the reproduction of the music is supplied from the operation unit 21, the reproduction unit 111 reproduces the accompaniment data 311 and the lyrics data 313 of the music shown in FIG. Specifically, the playback unit 111 reads the accompaniment data 311 and the lyrics data 313 of the music from the music database 310. Then, the playback unit 111 supplies the accompaniment data 311 to the sound processing unit 22 to emit the accompaniment sound, and supplies the lyrics data 313 to the display control unit 115 to display the lyrics. At this time, the reproduction unit 111 supplies these data so that the accompaniment sound emission and the lyrics display are started at the same timing. While reproducing the accompaniment data 311, the reproducing unit 111 evaluates data indicating the time (elapsed time) that has elapsed since the reproduction of the accompaniment data 311 is started every several msec, and the image generating unit 114. And the display control unit 115.

  The evaluation unit 112 is a means for evaluating the singing voice of the singer by comparing with the reference indicated by the reference data 312, and operates in detail as follows. The evaluation unit 112 collects the sound shown in FIG. 1 via the sound processing unit 22 in a state where the data indicating the elapsed time is supplied from the reproduction unit 111 (that is, the accompaniment data 311 is reproduced). The audio signal is supplied from the unit 23 to operate. First, the evaluation unit 112 reads the reference data 312 of the music indicated by the operation data supplied from the operation unit 21 from the music database 310. Next, the evaluation unit 112 determines the pitch of the singing voice represented by the audio signal supplied at the time when the data indicating the elapsed time is supplied from the reproduction unit 111, and the reference (pitch of the constituent sound) indicated by the reference data 312. ) And the one associated with the evaluation period including this elapsed time, and the difference value is calculated, for example, as a cent value. The evaluation unit 112 calculates the difference value every time data indicating elapsed time is supplied from the reproduction unit 111 (every several msec). Then, the evaluation unit 112 calculates the sum of the difference values calculated during the evaluation period as an evaluation value, and subtracts it from the score that is the basis of the evaluation score when the value is larger than the first threshold (deduction point) To do). The evaluation unit 112 calculates an evaluation value until the reproduction of the accompaniment data 311 is completed, and calculates the score of the result as a final evaluation point. As described above, the evaluation score is larger as the value of the evaluation value is larger, that is, as the pitch is separated from the reference, and as the evaluation period is deducted, that is, as the number of times the pitch is separated from the reference is larger, Lower. Note that the first threshold is a value indicating the degree to which the pitch difference between the singing voice and the constituent voices is allowed. For example, if the singing is evaluated strictly, the first threshold is reduced, and if the singing is evaluated gently, it is increased. good.

  The evaluation unit 112 evaluates a singing voice having an evaluation value equal to or lower than the first threshold as “good”, evaluates a singing voice higher than the first threshold and lower than the second threshold as “normal”, and is higher than the second threshold. Evaluate singing voice as “bad”. The second threshold value is a value for determining a place where the deduction is large and the evaluation is bad, that is, a place where the singer is not good. Similarly to the first threshold value, the second threshold value may be reduced when singing is evaluated strictly, and may be increased when evaluated gently. The evaluation unit 112 supplies the image generation unit 114 with data indicating these evaluation results (“good”, “normal”, “bad”) every time the evaluation period ends, and the evaluation value for the last criterion An evaluation score calculated by including up to this evaluation value is calculated. In addition, when the evaluation result is “bad”, the evaluation unit 112 supplies the data to the specifying unit 113 together with data indicating the criteria compared in this evaluation.

The identifying unit 113 identifies and identifies a period associated with a reference that has the same pitch as the reference indicated by the data supplied by the evaluation unit 112 among the periods associated with the reference indicated by the reference data 312. Data indicating the determined period (for example, data indicating the start time and end time of the evaluation period) is supplied to the image generation unit 114.
When the operation data indicating the music is supplied from the operation unit 21, the image generation unit 114 reads the character image data corresponding to the accompaniment data of the music from the character database 320. The image generation unit 114 generates image data obtained by editing or synthesizing the read image and supplies the image data to the display control unit 115. The read image data is, for example, image data indicating an image of a character that becomes a facial expression that expresses excitement in rust and an expression that evokes a vibrato singing method when it is desired to vibrato. In these images, the expression changes during the reproduction of the accompaniment data, but the manner of the change is constant. If the accompaniment data is the same, the expression changes in the same way every time. The image data indicating the character image corresponds to an example of “first image data” according to the present invention. The image generation unit 114 operates as described above while data is not supplied from the evaluation unit 112 and the specifying unit 113.

  On the other hand, when data indicating an evaluation result is supplied from the evaluation unit 112, the image generation unit 114 reads out image data of an image representing a facial expression character corresponding to the evaluation result from the character database 320. Further, when the data indicating the evaluation period is supplied from the specifying unit 113, the image generation unit 114 reads the character image data 321 shown in FIG. 1 representing the character of the facial expression corresponding to the evaluation period from the character database 320. In each case, the image generation unit 114 generates image data obtained by editing or synthesizing these character images and supplies the image data to the display control unit 115. In any case, the image data generated in these cases indicates an image of a character with a facial expression corresponding to the result of evaluation of the singing by the evaluation unit 112, and “second image data” according to the present invention. It corresponds to an example. In these cases, the image generation unit 114 supplies the generated image data to the display control unit 115 instead of the character image data corresponding to the accompaniment data described above. The display control unit 115 controls the operation of the stereoscopic image display unit 25, and the image represented by the image data supplied from the image generation unit 114 is described above according to the elapsed time indicated by the data supplied from the reproduction unit 111. Display on the display.

The images displayed by the operations of the above units will be described in detail with reference to FIGS. Each of these drawings is a diagram illustrating an example of an image displayed on the display in the karaoke apparatus 1. In FIGS. 3 and 4, images displayed on the display when time elapses are shown in the order of (a), (b), and (c).
FIG. 3 is a diagram illustrating an example of an image in which a character corresponding to a result of evaluating a song is displayed. In these images, lyrics A, reference image B, singing result line C (C1, C2, C3), and character D (D1, D2, D3) are displayed. As the lyrics A, the lyrics indicated by the lyrics data 313 shown in FIG. 1 are displayed.
In the lyrics A, the portion corresponding to the portion where the accompaniment data 311 shown in FIG. 1 is reproduced is changed from a white character to a black character, so that the position where the singer should sing now can be understood. It is displayed.
The reference image B is an image indicating the reference of the singing voice described above, and in detail, is a bar-shaped image representing the pitch of each constituent sound and its evaluation period. The reference image B is displayed so as to be superimposed on the staff, and the time is defined in the direction indicated by the arrow R1 along each line. The reference image B represents the pitch according to where it is displayed on the staff and represents the evaluation time according to the length in the direction along the arrow R1. The reference image B is displayed based on the pitch indicated by the reference data 312 shown in FIG. 1 and the evaluation period associated therewith.

The singing result line C is a line showing the pitch trajectory of the singing voice superimposed on the staff score displayed together with the reference image B, and is indicated by a two-dot chain line in the following figures. Specifically, the singing result line C is a point that the pitch of the voice represented by the audio signal output from the sound collection unit 23 shown in FIG. It is the line which connected in order.
The character D is an image displayed according to the result of evaluating the voice of the song. In FIG. 3A, the character D1 having a facial expression expressing the feeling of “normal” is displayed as a result of evaluating the portion indicated by the evaluation period X1 in the singing result line C1 as “normal”. In FIG. 3B, as a result of evaluating the portion indicated in the evaluation period X2 of the singing result line C2 as “bad”, a character D2 having a facial expression expressing an emotion of “bad” is displayed. In FIG. 3C, as a result of evaluating the portion indicated in the evaluation period X3 in the singing result line C3 as “good”, a character D3 having a facial expression expressing the feeling of “good” is displayed. The images of the character D are displayed when each evaluation period elapses and data indicating the evaluation result is supplied from the evaluation unit 112 shown in FIG. 1 to the image generation unit 114.

  FIG. 4 is a diagram illustrating an example of an image in which a character with a facial expression corresponding to the evaluation period specified by the specifying unit 113 is displayed. FIG. 4 will be described with a focus on differences from FIG. In FIG. 4, the character E (E1, E2, E3) has an expression to convey the above-mentioned “I am not good”. In this example, the reference evaluation period Y1 starting from time t3 is the reference evaluation period that is common to the reference (pitch is G) in the evaluation period X2 evaluated as “bad” in FIG. 3B. 4 (a) and 4 (b) show images displayed at a time point before time T1 or a time point before time T2 before time t3, respectively, and FIG. 4 (c) is displayed at time t3. The image is shown.

  The character E1 shown in FIG. 4A is an image that shows that it is close to “I am not good” and how to sing at that time. More specifically, the character E1 indicates that “I am not good” is approaching by raising the left arm E1L. Further, the character E1 indicates that it is better to raise the right arm E1R so that the voice is raised slightly when the voice is emitted. The character E2 shown in FIG. 4B indicates that the “I am not good at hand” is approaching by lowering the left arm E2L little by little in the direction of the arrow R2. The character E3 shown in FIG. 4 (c) indicates that he / she has reached “I'm not good” by lowering the left arm E3L all at once in the direction of the arrow R3. As described above, these characters E are displayed “before the time t3, which is the timing when the evaluation period Y starts”, indicating that this timing is approaching due to the change in facial expression. . Thereby, it becomes easy for a singer to utter a sound in accordance with the start of the evaluation period Y, and the karaoke apparatus 1 can assist the singer to sing well.

FIG. 5 is a diagram illustrating examples of images representing various facial expressions of a character. FIG. 5A is an image for telling the singer how to correct the pitch (correction method). Character G (G1, G2) tells whether to raise or lower the pitch by raising the right arm, and when the character G1 with the right arm raised is displayed, it should indicate that the pitch should be raised. When the character G2 with the lower right arm is displayed, it is informed that “pitch should be lowered”. Note that the number of fingers raised by the hand may represent the degree of pitch that should be raised (or lowered). For example, one finger increases the semitone, and two fingers increases (or decreases) the pitch. You may show that. Further, the degree of pitch may be expressed by the angle at which the right arm is raised.
FIG. 5B is an image for telling the singer how to correct the volume. Character H (H1, H2, H3) tells you what to do with the volume by opening your mouth. In this example, when the character H1 with a small mouth is displayed, it is notified that “the volume should be reduced”, and when the character H2 with a wide mouth is displayed, “the volume is increased”. When the character H3 with a closed mouth is displayed, it is informed that “You should not speak”.

  FIG.5 (c) is an image for conveying the volume correction method to a singer with a three-dimensional image. The character J (J1, J2, J3) is an image for indicating how much the volume should be increased in the way the image is projected. For convenience of explanation, the horizontal shifts K1, K2, and K3 (K1 <K2 <K3) from the non-stereoscopic image are virtually shown to the extent that each character J appears to jump out. It was supposed to be. In this example, when the character J1 with the smallest degree of appearing popping out (K1) is displayed, it is notified that “the volume should be slightly increased”, and the character J3 with the largest degree of appearing popping out (K3) When it is displayed, it tells you that “the volume should be very loud”, and when character J2 is displayed with a degree of popping out (K2) between them, it should be “increase the volume” I tell you.

  In the karaoke apparatus 1, the singer can hear the voice that he / she is not good at by telling the timing to execute the various contents conveyed by the facial expression shown in FIG. 5 by the facial expression of the left arm shown in FIG. 4. You can know when and how to emit. For this reason, even if the singer is not good at voice, since the singer makes a preparation for improving it in advance, the possibility of improving the evaluation is increased. Further, the karaoke apparatus 1 displays an image representing a character while reproducing the accompaniment data 311 and changes the display of the character according to the result of evaluating the singing. Thereby, the karaoke apparatus 1 can display the character which reacts to the extent to which the singing voice is along with the reference | standard which should be sung, and a singer is sung with this character. Can be given.

  Moreover, as shown in FIG.5 (c), the karaoke apparatus 1 can display the image of a character in three dimensions, and can tell a singer how to sing by the extent to which the image jumps out. The display included in the stereoscopic image display unit 25 illustrated in FIG. 1 is an image that can be viewed stereoscopically when viewed from a predetermined area with respect to a surface (display surface) for displaying an image. For example, a display that a singer sees from the stage of a karaoke room is often invisible or difficult to see by anyone other than the singer. If the character described above is displayed on this display and the way of singing is transmitted to such an extent that it appears to pop out, it can be made difficult for a non-singer to understand that the character is telling how to sing. Thereby, the singer can improve how to sing so that the singing is highly evaluated by having the character convey the way of singing in a way that is difficult for others to notice.

[Modification]
The above-described embodiment is merely an example of implementation of the present invention, and various applications and modifications are possible as follows, and can be combined as necessary.

(Modification 1)
The karaoke apparatus according to the present invention may display an image so that a singer can see a character stereoscopically. For example, when the karaoke device has a plurality of displays on the display unit, it is possible to detect the position of the singer and display the character only on the display that can be seen from that position, or the character only on the display May be displayed three-dimensionally. Further, the karaoke apparatus may display the character on a display whose orientation is adjusted so that the display can be viewed from the detected position by rotating the display with a motor or the like.

  FIG. 6 is a diagram illustrating a configuration of the karaoke apparatus 1a according to the present modification. The sound collection unit 23 includes two microphones 231 (231a and 231b), and the sound emission unit 24 includes two speakers 241 (241a and 241b). The stereoscopic image display unit 25 includes two displays 251 (251a, 251b), and each display is provided with a motor 253 (253a, 253b). The motor 253 rotates each display. Each display can be rotated by each motor to change the direction in which an image is displayed. The storage unit 30a stores display data 330a. The display data 330a is data indicating the position where each display is installed and the direction in which these images are displayed. These positions and orientations are input by operating the operation unit 21 shown in FIG. 1 when each display is installed, and are stored as display data 330a.

The control unit 10a includes a position detection unit 117a, a direction calculation unit 118a, and a motor control unit 119a. The position detection unit 117a is a means for detecting a position where the singer is singing (singing position). Specifically, the position detection unit 117 a outputs data indicating sound (measurement sound) for measuring the position of the microphone 231 to the two speakers 241 via the acoustic processing unit 22 and measures from these speakers 241. Sound is emitted. At this time, the position detection unit 117a acquires the time when the speaker 241 emits the measurement sound. Subsequently, the position detection unit 117a calculates the time at which the measurement sound is collected from the audio signal representing the sound collected by each microphone 231. The position detection unit 117a calculates the distance from the speaker that emitted the measurement sound to the microphone that collected the sound from the time from when the measurement sound is emitted until the sound is collected, and this is calculated for the two speakers 241. By calculating, the position of the microphone 231 is measured. The position detection unit 117a detects the position of the microphone from which the sound evaluated by the evaluation unit 112 is collected among the measured positions of each microphone as the singing position. The position detection unit 117a supplies data indicating the detected singing position to the direction calculation unit 118a. The microphone 231, the speaker 241, and the position detection unit 117a cooperate to function as a detection unit that detects the singing position.
The direction calculation unit 118a determines the display on which the character is to be displayed from the position indicated by the supplied data and the position and orientation indicated by the display data 330a read from the storage unit 30a, and determines the direction in which the display should display the image. calculate. Details of the operation of the direction calculation unit 118a will be described with reference to FIG. The direction calculation unit 118a supplies calculation result data indicating the calculated direction and a display to display an image in the direction to the image generation unit 114a, the display control unit 115a, and the motor control unit 119a.

The image generation unit 114a generates image data including the image representing the character and the image data not including the image, and supplies the image data to the display control unit 115a.
The display control unit 115a displays the image indicated by the image data including the image representing the character among the image data supplied from the image generation unit 114a on the display indicated by the calculation result data supplied from the direction calculation unit 118a. The image indicated by the image data not including this image is displayed on a display other than this display.
The motor control unit 119a controls the operation of the motor 253 provided in the display indicated by the calculation result data supplied from the direction calculation unit 118a, and adjusts the display to face the direction indicated by the calculation result data. The display control unit 115a, the direction calculation unit 118a, and the motor control unit 119a cooperate to function as a “display control unit” according to the present invention.

FIG. 7 is a diagram for explaining how the direction of the display is adjusted. FIG. 7 schematically shows each display and users M1, M2, and M3 including a singer viewed from above in the vertical direction. The displays 251a and 251b display images on the display surfaces 252a and 252b, respectively. In FIG. 7, the boundaries between the first region in which images displayed on the respective display surfaces are viewed stereoscopically and the other second regions are indicated by dotted lines Xa and Xb, respectively. That is, the first region is a region sandwiched between two dotted lines Xa, and is also a region sandwiched between two dotted lines Xb.
In Fig.7 (a), the user M1 is a singer and has input the audio | voice from the microphone 231a. In this case, the position detector 117a shown in FIG. 6 detects the position of the microphone 231a used by the singer as the singing position. Then, since the detected singing position is included in the region indicated by the dotted line Xa, the direction calculation unit 118a illustrated in FIG. 6 determines that the display on which the character is to be displayed is the display 251a, and the display displays an image. As the direction to be calculated, the direction that is currently facing is calculated. Thereby, the display control unit 115a illustrated in FIG. 6 displays an image including a character on the display 251a, and displays an image including no character on the display 251b. In addition, the motor control unit 119a illustrated in FIG. 6 does not rotate the motor because the calculated direction is the current direction.

  FIG. 7B shows a state where the singer is changed to the user M3 from the state of FIG. 7A and the user M3 is inputting sound from the microphone 231b. In the state shown in FIG. 7A, there is no display capable of stereoscopically viewing an image from the position of the microphone 231b. In this case, the direction calculation unit 118a calculates the direction of each display when the detected singing position is directed to be included in the region indicated by the dotted line Xa or the dotted line Xb. Then, the direction calculation unit 118a selects a display (display 251b in the example of FIG. 7B) having the smallest angle of rotation for directing the calculated direction from among the displays capable of facing the calculated direction. Judge as the display to be displayed. Subsequently, the direction calculation unit 118a calculates a direction in which the display 251b should display an image. Then, the display control unit 115a displays an image including a character on the display 251b, and the motor control unit 119a changes the display 251b from the state indicated by the two-dot chain line in FIG. 7B to the state indicated by the solid line. Rotate until As a result, the singer can see the character displayed on the display in a three-dimensional manner without changing the orientation of the display or changing his position. This makes it difficult for others to understand the character's pop-up, regardless of where the singer is singing, and the singer sings so that the singing is highly appreciated without being known to others. Can be improved.

(Modification 2)
The karaoke apparatus according to the present invention may shoot a singer's image and synthesize the captured image with a character image for display. In this case, the karaoke apparatus includes a photographing unit that photographs a singer's image, and the image generation unit generates image data obtained by combining the image captured by the photographing unit and the various images described above. What is necessary is just to output and display on the display which a display part has. Thereby, the karaoke apparatus can give the singer an atmosphere that makes a duet with the character, and can enhance the sense of reality compared to the case where the synthesized image data is not displayed.

(Modification 3)
The karaoke apparatus according to the present invention may convey emotions and the like with various facial expressions in addition to the facial expressions of the characters shown in the above-described embodiments. For example, the character may be watched and the timing may be transmitted, or when the character is sung well (when the evaluation is high), the character may jump up and the singer can be excited. Regardless of the reference, the facial expression may be changed if, for example, vibrato is detected. Further, the content transmitted to the character's facial expression may be associated in any way as long as the singer can recognize the content transmitted in association. For example, a table in which facial expressions and contents indicated by the facial expressions are associated may be provided to the user in advance. Thereby, even if it is a case where how to receive a facial expression changes with people, the emotion or content which should be conveyed can be conveyed with respect to a singer. Further, this correspondence may be set by the user by operating the operation unit 21 or the like. In addition to the character's facial expression, emotions etc. may be conveyed by the background, for example, as the evaluation changes from high to low, the background color changes from blue to red, and mistakes increase. May be warned. Moreover, a character may be displayed and an audio | voice may be uttered. When uttering voice, the character may be displayed with the mouth shape changed according to the sound to be generated.

(Modification 4)
The karaoke apparatus according to the present invention does not have to represent each character's facial expression with different images. For example, by combining several parts that form the facial expression, dozens of different facial expressions can be used. May be used, or a facial expression may be generated using a parameter. As a result, the capacity of the character image data can be reduced as compared with the case of preparing images for the number of facial expressions.

(Modification 5)
In the present invention, this may be evaluated using various characteristics of the singing voice as a reference. For example, features such as beat length or volume may be used as the reference, and singing techniques such as “vibrato” or “fist” may be used as features.

(Modification 6)
The control part which concerns on this invention may evaluate a song by not only the method demonstrated in embodiment mentioned above but various methods. For example, when the evaluation value calculated by the control unit is less than or equal to the first threshold value, two points are added, and when the evaluation value is less than or equal to the second threshold value, one point is added, and the total is used as an evaluation point. May be. Further, the evaluation period may be a period different from the period in which the constituent sounds are to be sung, for example, a certain time such as 1 second or 2 seconds may be set as the evaluation period, and the length of the evaluation period is operated by the user. The setting may be made by operating the unit 21 or the like.

(Modification 7)
In the embodiment described above, the karaoke apparatus according to the present invention displays a facial expression character according to the evaluation result by the evaluation unit 112, and also displays a facial expression character for transmitting the evaluation period specified by the specifying unit 113. Although different images are displayed, these may be displayed simultaneously or only one of them may be displayed. If the former is displayed, it is easy to give a sense of reality to the singer, and if the latter is displayed, it is possible to assist the singer in singing well.

(Modification 8)
In the present invention, during the evaluation period specified by the specifying unit 113, if the user can sing better than when singing in the past, the evaluation point may be increased. Specifically, in the evaluation unit 112, the evaluation period is specified by the specifying unit 113, and the evaluation value is smaller than the evaluation value with respect to the previous standard among the standards having the same pitch. In this case, one point is added to the evaluation score. At this time, points may be added when the evaluation value becomes smaller than the average value of a plurality of past evaluation values, or the points to be deducted may be reduced instead of adding points.

(Modification 9)
The karaoke apparatus according to the present invention includes means for displaying a stereoscopic image (stereoscopic image display unit) in the above-described embodiment, but may include means for displaying a two-dimensional image. In this case, the image generation unit may generate an image that appears three-dimensionally as shown in FIG. 5C as a two-dimensional image using a perspective method. Further, instead of displaying an image that appears to pop out, an image of a character expressed in a plane as shown in FIGS. 4, 5A, and 5B may be displayed. Even in this case, the karaoke apparatus can display a character that reacts to the extent that the singing voice is in line with the reference to be sung.

(Modification 10)
The karaoke apparatus according to the present invention includes the display control unit and the stereoscopic image display unit in the above-described embodiment, but these units may be included in an external device. In this case, the image generation unit may output image data to a display control unit provided in the external device via a network, for example.

(Modification 11)
The karaoke apparatus according to the present invention may represent emotions or information to be conveyed depending on the background in addition to the facial expression of the character. In this case, background data, which is image data to be displayed as a character background, may be stored in the storage unit. The background referred to here includes, in addition to the so-called background representing the background behind the scene, the same as the character's facial expression, representing the character's emotion or information to be conveyed in color, shape, or character.

(Modification 12)
In the karaoke apparatus according to the present invention, the time at which the singer should sing or the position at which the singer should sing may be represented not only by the accompaniment sound but also by an image or the like. For example, in the above-described embodiment, as shown in FIG. 3, the boundary where the color changes by changing the color of the lyrics A is represented as the position where the singer should sing now. Both can know where to sing. Further, the present invention is not limited to this, and a mark indicating the currently played position may be displayed on the staff shown in FIG. 3 and moved in the direction of the arrow R1 to notify the position to be sung. The “accompaniment data 311” and “lyric data 313” according to the embodiment described above and the data for displaying the mark according to this modification are all data representing the position at which the music should be sung when reproduced. This corresponds to an example of “position display data” according to the invention.

(Modification 13)
In the above-described embodiment, the image generation unit 114 supplies the generated image data (second image data) to the display control unit 115 instead of the character image data (first image data) corresponding to the accompaniment data. Not only this but these image data may be supplied to the display control part 115 together. For example, the image generation unit 114 represents, as the second image data, data representing an image representing the facial expression of the character indicated by the first image data with the above-described evaluation result or the facial expression corresponding to the evaluation period specified by the specifying unit 113. The display control unit 115 may display the image indicated by the second image data on the image indicated by the first image data so as to overlap (overlay) the image. Even in this case, the karaoke apparatus 1 can display a character that reacts to the extent that the singing voice is along the reference to be sung.

(Modification 14)
The present invention can also be specified as a program for causing a computer to function as the evaluation apparatus according to the present invention. Such a program may be provided in a form recorded on a recording medium such as an optical disk, or may be provided in a form such that the program is downloaded to a computer via a network such as the Internet, and the program can be installed and used. Is possible.

DESCRIPTION OF SYMBOLS 1 ... Karaoke apparatus 10, 10a ... Control part, 21 ... Operation part, 22 ... Sound processing part, 23 ... Sound collection part, 24 ... Sound emission part, 25 ... Three-dimensional image display part, 30 ... Memory | storage part, 111 ... Playback , 112 ... evaluation unit, 113 ... identification unit, 114 ... image generation unit, 115, 115a ... display control unit, 117a ... position detection unit, 118a ... direction calculation unit, 119a ... motor control unit, 231 ... microphone, 251 ... Display, 252 ... display surface, 253 ... motor, 310 ... music database, 311 ... accompaniment data, 312 ... reference data, 313 ... lyrics data, 320 ... character database, 321 ... character image data, 330 ... display data

Claims (4)

Storage means for storing position display data representing a position at which the song should be sung when played and reference data indicating a reference for evaluating the singing voice;
Reproduction means for reading out and reproducing the position display data from the storage means;
The singing voice is compared by comparing the singing voice represented by the audio signal supplied from the sound collecting means when the reproducing means is reproducing the position display data with the reference indicated by the reference data read from the storage means. An evaluation means for evaluating
While the reproduction means reproduces the position display data, the first image data indicating the character image is output, and when the singing voice is evaluated by the evaluation means, the expression of the character is represented by the singing voice. Output means for outputting second image data indicating an image represented by a facial expression according to the evaluated result, and changing the facial expression when a specific singing technique is detected from the singing voice represented by the audio signal And an output means for outputting the data indicating the processed image as the second image data . The evaluation means compares the first singing voice represented by the audio signal with a reference indicated by the reference data read from the storage means, and evaluates the first singing voice by an evaluation point, and the reference If the evaluation score of the first singing voice is higher than the evaluation score of the second singing voice compared to the past, the evaluation score of the first singing voice is added.
The evaluation apparatus according to claim 1, wherein: Storage means for storing position display data representing a position at which the song should be sung when played and reference data indicating a reference for evaluating the singing voice;
Reproduction means for reading out and reproducing the position display data from the storage means;
The first singing voice represented by the audio signal supplied from the sound collecting means when the reproducing means is reproducing the position display data is compared with the reference indicated by the reference data read from the storage means. An evaluation means for evaluating the first singing voice based on an evaluation point, and if the evaluation point of the first singing voice is higher than the evaluation point of the second singing voice compared with the reference in the past, the first An evaluation means for adding to the evaluation score of one singing voice;
While the reproduction means reproduces the position display data, the first image data indicating the character image is output, and when the first singing voice is evaluated by the evaluation means, the expression of the character is Output means for outputting second image data indicating an image represented by an expression according to a result of evaluation of the first singing voice;
An evaluation apparatus comprising: When the specific singing technique is detected from the singing voice represented by the audio signal, the output means outputs data indicating an image in which the facial expression is changed as the second image data.
The evaluation apparatus according to claim 3.

Images