JPH11212582A - Karaoke device provided with choreography scoring function - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase the entertainingness of KARAOKE by comparing dancer's posturing data generated on the basis of the detected posturing of a user with posturing data wherein the posturing of a model dancer and scoring the dance performance of a user. SOLUTION: Optical motion capture technologies are employed so as to generate dancer posturing data in a KARAOKE hall. A motion detection part 40 samples video signals from two CCD cameras 42 at intervals of playing elapsed time and processes their images to detect the coordinate position of each mirror 41. The coordinate positions are converted to the same coordinate system with posturing data to generate dancer posturing data, which are outputted sequentially according to the posturing of the dancer. A scoring part 50 compares the dancer posturing data with posturing data sequentially in playing time series to obtain differences between the vectors of respective human body positions of the two data sequentially in playing time-series order, specific points based upon the differences are added, and the final totalization result is converted to scores based upon 100 as the top scores, thereby outputting score information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a karaoke apparatus, and more particularly, to a karaoke apparatus having a function of scoring a performance in which a user dances according to choreography.

[0002]

2. Description of the Related Art In a karaoke apparatus which processes karaoke data of a designated karaoke song and outputs the accompaniment music and the lyrics image of the song in synchronization with a speaker and a display, the singing situation of the singer is scored, and the scoring result is given. There is a singing scoring function for displaying a score or the like on a display. This singing scoring function analyzes the singing state by comparing the pitch extracted from the corresponding karaoke data with the user's pitch input to the microphone when singing the karaoke song requested by the user,
The score corresponding to the singing ability is given by summing up the analysis results sequentially output during the performance processing.

[0003]

One of the ways to enjoy karaoke is to sing like the original singer of a karaoke song. In that case, singing along with the choreography can make the former singer's image more appealing to accompanying persons than simulating only the singing state, and the atmosphere of the place can be greatly enhanced.

[0004] However, the conventional scoring function is inadequate for judging the superiority of only the singing ability, and is not sufficient for judging how similar it is to the original singer. Also, for those who have poor singing skills or who want to enjoy karaoke by dancing to the choreography, if they can "match" with the performance of the dance, they can enjoy karaoke more.

Accordingly, an object of the present invention is to provide a karaoke apparatus having a function of scoring a user's choreography.

[0006]

The karaoke apparatus of the present invention has the following requirements (1) to (3). (1) Choreography data dedicated to each song individually associated with each karaoke song is prepared in an appropriate storage unit.
This choreography data describes the actions of a model player who dances to the original song of the karaoke song, corresponding to the performance time series of the karaoke song. (2) A motion detecting means is provided for detecting the performance of the karaoke user dancing to the karaoke music and sequentially outputting the detection result as dancer performance data. (3) When performing a karaoke music piece, the choreography data and the dancer's work data are sequentially compared, and the results of the comparison are totaled to give a score to the user's dance performance and output the score.

The motion detecting means detects the output signal from a sensor attached to the human body part of the karaoke user to specify the coordinate position data of the human body part and based on the coordinate position data, User-created data may be created and sequentially output.

Alternatively, the motion detection means receives an appropriate beacon signal output from a marker attached to a human body part of the karaoke user, analyzes the received signal strength and the received direction, or captures the signal by a video camera. The coordinate position of the human body part may be specified by sequentially performing image processing on an image.

[0009] The present invention may further include means for generating a choreography animation image by computer graphics by processing the choreography data, and means for outputting the choreography animation image to a display.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS === Basic Configuration and Operation === FIG. 1 is a configuration diagram of a karaoke apparatus according to an embodiment of the present invention. The central control unit 11 has a CPU, RAM, ROM inside.
The central control unit 11 controls the karaoke apparatus 1 by performing appropriate information communication and control on the peripheral components via the data bus 60 and the control bus 70. In addition,
In this embodiment, the karaoke apparatus 1 is a performance terminal in a communication karaoke system, and a karaoke host apparatus 110.
Karaoke data distributed through the telephone line 100 via the modem 25,
Accumulate in 2.

The central control unit 11 transmits music selection information input from the operation panel 17 or the remote control transmitter 20 to the operation control unit 1.
When the karaoke data is received via the hard disk drive 12, the karaoke data is retrieved from the hard disk device 12 and processed. Accompaniment music generation data such as MIDI data in the karaoke data is sequentially transferred to the synthesizer 13, and accompaniment music of the karaoke music is generated. The audio signal of the accompaniment music is mixed by the mixing amplifier 14 with the voice signal of the singer from the microphone 15 and output to the speaker 16.

The lyric rendering data in the karaoke data is transferred to the video RAM 21, and the lyric images synchronized with the accompaniment music are sequentially developed into bitmaps. The lyric image is superimposed on the background image from the video CD changer 23 by the image control unit 22 and is displayed on the display 24.
Is output to

In addition to the basic functions described above, the karaoke apparatus 1 compares a performance of a model performer dancing to a karaoke song with a performance of a user, and scores a choreography score of how similar the choreography is. It has. Hereinafter, the choreography scoring function will be described.

=== Choreographic Scoring Function === In order to achieve the choreographic scoring function, the karaoke apparatus 1 includes a motion detecting section 40 and sensors 41 and 2 attached thereto.
A CCD camera 42 and a scoring unit 50 are provided. Further, when scoring the performance of a user (dancer) who dances to the karaoke music, choreography data dedicated to each karaoke music as a reference is stored in the hard disk device 12. The outline of the choreographic scoring function will be described below.

Motion Capture Technology In order to compare the actions of a model performer and a dancer, it is necessary to convert the actions into data. Motion capture technology is used to convert the actions into data. In this technique, a sensor is attached to a human body part such as a human head or a joint, and a signal output by the sensor according to the position or movement is analyzed to detect a three-dimensional coordinate position of each sensor. Then, the operation is converted into data by associating the position of each sensor with each human body part according to the human body structure. There are various types of motion capture technology using a sensor, such as an optical type and a magnetic type, depending on the type of the sensor.

There is also a method in which a marker for transmitting a beacon signal such as light or radio wave is used instead of the sensor, and a beacon signal from each marker is received, and the position of each marker is specified based on its intensity and direction. Of course, it is also possible to detect a coordinate change of each human body part by sequentially performing image processing on a video captured by a video camera or the like.

Choreography Data The choreography data describes the actions performed when the model performer dances to the karaoke music, and the coordinate positions of each human body part of the model performer correspond to the performance time series of the karaoke music. Things. FIG. 2 shows a schematic structure of choreography data. Elapsed times (t0, t1,...) Are set at predetermined time intervals from the start of the performance of the karaoke song, and the three-dimensional coordinate positions of each human body part such as a joint or a head are associated with each elapsed time. I have. In this embodiment, the head, both elbows, both hands, waist, both knees, and both feet are set as the human body parts.

Creation of Dancer Performance Data In order to score a dancer's performance based on the choreography data, the motion detection unit 40 determines the performance of the dancer by performing the performance elapsed time (t0, t1,
...) And output as dancer work data in an appropriate format.

In order to actually create dancer performance data at a karaoke hall, this embodiment employs an optical motion capture technology. This method uses a spherical mirror that selectively reflects special illumination light as a sensor,
The reflected light from the mirrors is captured by a plurality of video cameras, and the position of each mirror is detected by triangulation. The motion detection unit 40 samples the video signals from the two CCD cameras 42 for each performance elapsed time and performs image processing.
The coordinate position of each mirror 41 is detected. Then, the coordinate position is converted into the same coordinate system as the choreography data to create dancer work data. The created dancer performance data is sequentially output almost in real time according to the performance of the dancer.

Output of scoring information The scoring unit 50 sequentially compares the dancer's work data and the choreography data sequentially output by the motion detection unit 40 in the order of performance time series. Specifically, in each data, the change (vector) of the coordinate position between a certain performance time point (tn) and the next time point (tn + 1) for each human body part
Is calculated, and whether each body part is moving in a predetermined direction,
Alternatively, the performance of the model performer and the performance of the dancer are compared by comparing whether or not they have been stopped for a predetermined time.

Then, the difference between the vectors for each human body part in the two data is sequentially obtained in the order of performance time, and a predetermined point is added in accordance with the difference. When the end of the performance is recognized, the scoring information is output as a score obtained by converting the final total result into a score of 100 points. Next, the user interface environment in the scoring function and the operation in the karaoke apparatus 1 will be described.

Use Environment and Operation of the Choreographic Scoring Function A user inputs a karaoke song request using the remote control transmitter 20 or the like. At this time, a user input to use the choreographic scoring function at the time of performance processing of the music is also performed. Upon receiving the series of user inputs, the central control unit 11 reserves the performance of the karaoke music as a choreographic music to be scored.

When the order of performance of the corresponding karaoke music comes, the display prompting the user to mount the sensor (mirror) 41 is output on the display 24, and the performance processing is suspended until a user input to start scoring is received. And Further, the corresponding choreography data is extracted from the hard disk device 12 and transferred to the scoring unit 50. The choreography data is stored in an appropriate storage unit such as a RAM inside the scoring unit 50.

The dancer attaches a predetermined sensor 41 to a predetermined human body part, stands at a predetermined position in the karaoke hall, and inputs a command to cancel the pause of the performance processing by the remote control transmitter 20 or the like. And the dancer dances to the performance.

The central control unit 11 operates the motion detection unit 40 and the scoring unit 50 in conjunction with the performance processing of the corresponding karaoke music. During the performance, the motion detection unit 40 sequentially outputs dancer performance data, and the scoring unit 50 sequentially compares the dancer performance data with the choreography data. Scoring part 5
0 returns the scoring information when the end of the performance is received from the central control unit 11. The central control unit 11 performs processing and control for outputting the acquired scoring information to the display as a score display.

=== Others === In the above embodiment, the number of human body parts set in the choreography data is the same as the number of sensors attached to the dancer, but a large number of human body parts are set in the choreography data. The difficulty level may be freely set by appropriately increasing or decreasing the number of sensors to be mounted (the higher the number of sensors, the higher the score unless the dancer is more faithful to the choreography data).

Although the performance of the model performer and the performance of the dancer are scored by comparing the choreography data and the vector data of each human body part in the dancer performance data, the model performer and the dancer or each dancer are scored. A measurement error may occur due to a difference in body type, a difference in the choreography data creation and the choreography scoring, or a difference in the positional relationship of the measuring devices at each karaoke venue, and the scoring results may be inaccurate. In such a case, for example, it is also possible to make appropriate corrections, such as normalizing the maximum change amount in the choreography data and the dancer's work data for the human body part to be the same, and scoring by relative values.

It is also possible to display an animation by computer graphics using the choreographic data. This saves the dancer from having to remember the choreography in advance.

It is easy to add a conventional singing scoring function in addition to the choreographic scoring function. If these two scoring functions are used during the performance of karaoke music, various applications such as displaying the total score of singing and choreography are displayed. Can be considered.

[0030]

According to the present invention, choreography data describing the performance of a model performer dancing in accordance with the choreography dedicated to each karaoke song is compared with dancer performance data created based on the detected performance of the user. Score points for dance performance. This increases the entertainment of karaoke for those who have poor singing skills or who enjoy karaoke by dancing to the choreography.

Further, the choreography data is the time change of the coordinate position in each human body part of the model performer, and the dancer's work data is created by the motion capture technology, so that the model performer's and the dancer's work are compared with each other. The comparison can be made by changing the coordinates. There are various methods of motion capture, such as sensors, markers, image processing, and the like, and an optimal configuration can be adopted according to the size and shape of the karaoke hall.

Furthermore, if an animation choreography image by computer graphics is displayed on the display based on the choreography data, it is not necessary to remember the choreography in advance.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a karaoke apparatus according to an embodiment of the present invention.

FIG. 2 is a schematic configuration diagram of choreography data processed by the karaoke apparatus.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Karaoke apparatus 11 Central control part 12 Hard disk drive 40 Motion detection part 41 Sensor 50 Scoring part

Claims (5)

[Claims] 1. A karaoke apparatus having a choreographic scoring function, which has the following requirements (1) to (3). (1) Choreography data dedicated to each song individually associated with each karaoke song is prepared in an appropriate storage unit.
This choreography data describes the actions of a model player who dances to the original song of the karaoke song, corresponding to the performance time series of the karaoke song. (2) A motion detecting means is provided for detecting the performance of the karaoke user dancing to the karaoke music and sequentially outputting the detection result as dancer performance data. (3) When performing a karaoke music piece, the choreography data and the dancer's work data are sequentially compared, and the results of the comparison are totaled to give a score to the user's dance performance and output the score. 2. The karaoke user according to claim 1, wherein the motion detecting means detects an output signal from a sensor attached to a human body part of the karaoke user to specify coordinate position data of the human body part, A karaoke apparatus, wherein the dancer's work data is created based on position data and sequentially output. 3. The motion detection means according to claim 1, wherein the motion detection means receives an appropriate beacon signal output by a marker attached to a human body part of the karaoke user, and analyzes the reception intensity, the reception direction, and the like. A karaoke apparatus comprising: identifying a coordinate position of the human body part; creating the dancer's work data based on the coordinate position data; and sequentially outputting the data. 4. The method according to claim 1, wherein the motion detection unit specifies a coordinate position of the human body part by sequentially performing image processing on a video captured by a video camera,
A karaoke apparatus wherein the dancer's work data is created based on the coordinate position data and sequentially output. 5. A means according to claim 1, wherein said choreography data is processed to generate a choreography animation image by computer graphics, and means for outputting said choreography animation image to a display. A karaoke apparatus comprising: