WO2009030891A2

WO2009030891A2 - Karaoke system

Info

Publication number: WO2009030891A2
Application number: PCT/GB2008/002968
Authority: WO
Inventors: Ali Kord
Original assignee: Digital Intertive Booths Systems Ltd
Priority date: 2007-09-07
Filing date: 2008-09-03
Publication date: 2009-03-12
Also published as: WO2009030891A3; GB0717379D0

Abstract

The present application relates to a karaoke system (1 ). The system comprises a motion tracking system (3) for tracking the movements of an individual (5) performing karaoke. A character animation unit (11 ) is provided for animating a virtual puppet (7) based on an output of the motion tracking system (3). The karaoke system (1 ) may also be suitable for animating virtual puppets (7) for a plurality of individuals (5).

Description

KARAOKE SYSTEM

FIELD QF THE INVENTION

The present invention relates to a karaoke system and a method for performing karaoke.

BACKGROUND OF THE INVENTION

Karaoke is a popular form of entertainment in clubs, bars and in private karaoke rooms. Karaoke machines typically have one or more handheld microphones to allow an individual or a group of individuals to sing along to a backing track. The vocals are amplified and played over one or more loudspeaker(s) along with the backing track.

The song lyrics are usually displayed on a screen while the backing track plays to prompt the individual(s) performing. For example, a music video with the song lyrics superimposed may be displayed while the backing track plays.

It is an objective of the present invention, at least in preferred embodiments, to provide improvements over conventional karaoke machines.

SUMMARY OF THE INVENTION

Viewed from a first aspect, the present invention relates to a karaoke system comprising a motion tracking system for tracking the movements of an individual performing karaoke and a character animation unit for animating a virtual puppet based on the output of the motion tracking system. In use, the movements of the individual while they perform may be mapped on to the virtual puppet. The animation of the virtual puppet is preferably performed substantially in real-time so that the movements of the puppet are at least substantially synchronised with the individual's movements. The virtual puppet may be displayed on a screen alongside the individual performing karaoke. At least in preferred embodiments, the virtual puppet karaoke system may enhance the karaoke experience and may provide additional entertainment. The karaoke system according to the present invention may be referred to as a virtual puppet karaoke system or a 3D karaoke system.

The karaoke system preferably comprises a database containing data relating to a range of different virtual puppets. The data may, for example, include graphical data relating to the appearance of the virtual puppet and/or animation data relating to how the virtual puppet is animated. Preferably, a virtual puppet may be selected from the database thereby enabling an individual to select the virtual puppet to be displayed. The virtual puppets may, for example, be caricatures of pop stars.

The karaoke system preferably comprises a prompter or autocue for displaying song lyrics. The lyrics are preferably displayed on a screen. The lyrics may be read from the storage medium on which the backing track is stored. The storage medium may be a compact disc (CD), a digital versatile disc (DVD) or other digital medium.

The karaoke system preferably comprises a data storage device for storing data relating to the performance. The data storage device is preferably adapted to store video data generated by the character animation unit. The data storage device may also store audio data, for example comprising an individual's vocals and/or the backing track. The data storage device may comprise solid-state memory, magnetic storage, optical storage or any other appropriate storage medium. The stored data may subsequently be transmitted over a local area network (LAN), wide area network (WAN) or the internet. Thus, the data may be sent to the individual, for example as an email attachment.

Preferably, the karaoke system comprises a data recording device. The recording device may record data generated by the motion tracking system and/or the character animation unit. Preferably, the recording device records video data relating to the animation of the virtual puppet. The recording device may also record audio data, for example comprising an individual's vocals and/or the backing track. The recording device preferably records the data onto a transferable medium, such as a CD or a DVD. The recording device may record the data stored in the data storage device.

The data is preferably recorded in a format suitable to enable the animation of the virtual puppet to be viewed again in the future. The data may be raw data, for example derived from the motion tracking system and/or character animation unit, and require a viewing application to re-create the animation. Preferably, however, the data is recorded in a video format to allow the virtual puppet animation to be re-played. Preferably, the recorded data may be re-played on one or more of the following devices: a personal computer (PC), personal digital assistant (PDA), cellular telephone, portable media player or home media player (such as a video disc, DVD, HD-DVD or Blu-Ray disc player). The recording device may also record audio data, for example comprising the individual's vocal track and/or the backing music. The data may, for example, be stored as an mpeg file.

In use, the tracking system may track at least some of the individual's movements and transfer the tracked movements onto the virtual puppet. In use, the virtual puppet may perform movements corresponding to those performed by the individual. The tracking system preferably tracks the movement of at least one of the individual's limbs ^" and/or the individual's head.

A camera for tracking the individual's movements or expressions could be employed, but it will be appreciated that this may increase the complexity of the system. The motion tracking system preferably comprises at least one sensor. Preferably, a plurality of sensors is provided. In use, the at least one sensor may be provided on the person of the individual. In use, at least one sensor is preferably located on each arm of the individual to be mapped onto the virtual puppet. At least one sensor may also be provided on each leg of the individual. In use, asensor is preferably provided at an individual's joints or on each side of the individual's joints. The at least one sensor may be an inertial sensor, such as an accelerometer or inertial gyroscope. Alternatively, the karaoke system may comprise at least one electromechanical, magnetic, pressure (piezo) pads or optical sensor.

The position of the or each sensor may be determined relative to a reference point. The reference point may be fixed. Preferably, however, the reference point is movable and may be tracked dynamically. The motion tracking system preferably comprises a plurality of receivers for receiving a signal transmitted from a transmitter. In use, the transmitter may be provided on the person of the individual. The tracking system may monitor the position of the transmitter to provide a reference point. This reference point may be used to determine the relative positions of other sensors. The transmitter may be an optical transmitter, a sonic transmitter or a radio transmitter.

Alternatively, the motion tracking system may comprise a plurality of transmitters. A receiver is preferably provided on the person of the individual for receiving signals transmitted from said transmitters. The receiver may determine its relative position to provide a reference point. For example, sound at different frequencies may be emitted from the transmitters simultaneously (or at predetermined times or intervals) and the relative time at which the different frequencies detected by the receiver used to determine the relative position of the receiver. The transmitters may be optical transmitters, sonic transmitters or radio transmitters. The karaoke system preferably comprises a lip synchronisation unit for animating the lips and/or mouth of the virtual puppet. The virtual puppet may then be animated to mimic the movements of the individual's mouth while they sing. The lip synchronisation unit preferably forms part of the character animation unit.

The karaoke system may comprise a microphone. One or more inertial sensors may be provided in the microphone. The microphone may be handheld or mounted on a headpiece. The karaoke system may comprise a display device for displaying the virtual puppet. The display device may, for example, comprise a monitor or a projector for projecting the virtual puppet onto a screen.

The karaoke system may be provided with processing means for removing the vocal track from a song.

The karaoke system may comprise processing means for altering the sound of an individual's voice. For example, the pitch or tone of the voice may be changed for comic effect.

The character animation unit and/or the lip synchronisation unit may be provided in software or hardware. Preferably, a processor is programmed to perform the character animation and/or lip synchronisation.

Viewed from a further aspect, the present invention relates to a karaoke system comprising a microphone and a character animation unit for animating a virtual puppet having a mouth, wherein, in use, the character animation unit animates the mouth of the virtual puppet in response to an output of the microphone. Thus, at least in preferred embodiments, the karaoke system may enable a virtual puppet to be displayed with lip synchronisation substantially matching that of the individual performing karaoke.

The karaoke system preferably comprises a display device for displaying the virtual puppet. Preferably, the karaoke system comprises a data storage device. The data storage device is preferably adapted to store video data generated by the character animation unit. The data storage device may be adapted also to store audio data such as an individual's vocals and/or a backing track.

A data recording device is preferably provided. Preferably, the recording device is suitable for recording data generated by the character animation unit. The recording device may record audio and/or video data. The data is preferably recorded on a CD, DVD, HD-DVD or the like.

The karaoke systems described herein may be adapted for simultaneous use by a group of individuals. Preferably, the motion tracking system can distinguish between different individuals in the group and map their respective movements onto different virtual puppets to be displayed.

Furthermore, the karaoke systems described herein may be provided with means for accepting payment. The payment means may be suitable for accepting payment from a credit or debit card, or for accepting a cash payment. If payment means is provided, the karaoke machine preferably only operates once payment has been made. The means for accepting payment is preferably automatic. Viewed from a yet further aspect, the present invention relates to a method of performing karaoke, the method comprising the steps of:

(a) tracking the movements of an individual performing karaoke;

(b) mapping the tracked movements of the individual onto a virtual puppet; and (c) displaying the virtual puppet in real-time.

Preferably, audio and/or video data is recorded on a storage medium.

The term virtual puppet used herein is intended to mean a computer generated character and the term virtual actor could be used in its place. The movements of the virtual puppet are typically modelled, mapped or related to those of an operator. It will be appreciated that the virtual puppet may be a cartoon character, mascot, avatar, virtual actor, caricature, or other computer generated character.

BRIEF DESCRIPTION OF DRAWINGS

A preferred embodiment of the present invention will now be described, by way of example only, with reference to the accompany Figures, in which:

Figure 1 shows a schematic view of a karaoke system in accordance with a preferred embodiment of the present invention; and

Figures 2 and 3 illustrate an individual using the karaoke system according to the present invention.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

A karaoke system 1 for displaying a virtual puppet in accordance with the present invention is shown schematically in Figure 1.

A motion tracking system 3 is provided for tracking the movements of the individual 5 while they perform. The motion tracking system 3 comprises a plurality of sensors provided in a harness 9 to be worn by the individual, as shown in Figures 2 and 3. The sensors may be inertial, electromechanical, magnetic, pressure (piezo) or optical. In the present embodiment, the harness is provided with separate sensors for the operator's head and torso, and for each hand, forearm and upper arm (eight inertial sensors in total). Thus, the motion tracking system 3 tracks the movements of the upper part of the operator's body. A suitable motion tracking system 3 is the Gypsy Gyro™ system supplied by Animazoo UK Ltd., of Quayside Offices, Basin Road South, Brighton BN41 1WF (www.animazoo.comV A reference point for the individual 5 wearing the harness 9 is tracked dynamically, for example using wireless or mechanical tracking. A transmitter is provided on the harness 9 for transmitting a signal while the karaoke system is in use. A series of receivers (not shown) are provided for receiving the signal and determining the relative position of the transmitter. Thus, a reference point for the other sensors provided in the harness 9 may be determined. The transmitter may be a radio, sonic, ultrasonic or optical transmitter.

The data from the motion tracking system 3 is sent to a computer 11 , such as a graphics personal computer, which serves as a character animation unit. The data from the harness 9 may be sent to the computer 11 via an umbilical cable or over a wireless connection. The computer 11 is programmed to generate the virtual puppet 7 and uses motion tracking data from the motion tracking system 3 to animate the virtual puppet 7. If, for example, the individual 5 performs an action, such as raising an arm, the movement will be tracked by the motion tracking system 3 and the computer 11 will map the tracked movement onto the virtual puppet 7 so that it performs the same action. Thus, the movements of the individual 5 and the virtual puppet 7 may be substantially synchronised. The computer 11 is provided with a keyboard 13 to enable an operator (not shown) to operate the karaoke system 1. The operator may use the keyboard 13 to select a requested track or song. Furthermore, the keyboard 13 may be used to select a virtual puppet from several character models stored in a database. Equally, the operator may use the keyboard 13 to change the appearance of the virtual puppet 7 to represent different moods or facial expressions while the individual 5 performs. The keyboard 13 could be replaced or supplemented with another input device, such as a touch-screen device. The computer 11 displays the virtual puppet 7 on a main screen 15. A first coupler

17 is provided to enable the virtual puppet 7 also to be displayed on a monitor 19 to be viewed by the individual 5 and/or the operator.

The karaoke system 1 is provided with a microphone 21 connected to the computer 11 via a second coupler 23. The individual 5 uses the microphone 21 and their vocals are output through a loud speaker 25. The computer 11 is programmed to provide lip synchronisation for the virtual puppet 7 responsive to sounds picked up by the microphone 21. The lip synchronisation is performed substantially in real-time. Thus, the virtual puppet 7 is animated such that its mouth and/or lips are substantially synchronised with those of the individual 5. A sound system comprising a media player 27, an amplifier 29 and stereo speakers 31 is provided to enable a backing track to be played. The media player 27 may, for example, be a DVD player. The lyrics for the song are displayed on the monitor 19 to provide the individual 5 with prompts while they perform.

The karaoke system 1 is provided with a data recorder 33 for recording audio and video data. The video data typically relates to the animation of the virtual puppet 7 and the audio data is the vocals of the individual 5 recorded during their performance. The data recorder 33 preferably comprises a writer for writing the audio and video data onto a data storage medium, such as a CD or DVD, that can be taken away by the individual as a permanent record of their performance. Alternatively, or in addition, the karaoke system 1 may be connected to a network, such as a LAN, WAN or the internet, to allow the recorded audio and video data to be transmitted electronically. For example, a performance could be emailed to the individual who performed.

The operation of the karaoke system 1 will now be described. The individual 5 puts on the harness 9 and performs karaoke in the normal manner by singing along to a backing track played over the stereo loudspeakers 31. The individual uses the microphone 21 and their vocals are amplified and output through the loudspeaker 25. Throughout the performance the song lyrics are displayed on the monitor 19 to provide ah autocue for the individual 5.

While the individual performs, their movements are tracked by the motion tracking system 3. The movement data is transmitted to the computer 11 and used to animate the virtual puppet 7 displayed on the main screen 15. Since the tracked movements of the individual 5 are mapped directly onto the virtual puppet 7, the movements of the virtual puppet 7 are substantially synchronised with those of the individual 5. Furthermore, the vocals of the individual 5 are lip-synched onto the virtual puppet 7 so that it appears to sing the vocals output through the loud speaker 25. The operator and/or the individual may optionally refine the actions and appearance of the virtual puppet 7 during the performance. The appearance of the virtual puppet 7 may be changed to display different predetermined characteristics, such as happiness, sadness or anger. These further refinements may be controlled by the keyboard 13 or a separate controller (not shown) operable by the individual or an operator.

The data storage device 33 stores data of the individual's performance for future playback. The stored data comprises video data relating to the animation of the virtual puppet 7; and audio data relating to the individual's vocals and the backing track. The data storage device 33 preferably comprises a data writer for writing the audio and video data to a transferable medium, such as a CD or DVD. The animation of the virtual puppet 7 is preferably recorded in a format suitable to enable it to be viewed on a domestic media player, such as a DVD player or personal computer. ,

A separate camera may also be provided to film the individual's performance. The filmed performance may be recorded along with the other audio and video data for future reference. Preferably, the filmed performance and the corresponding animation can be played back together to provide a comparison.

The karaoke system 1 has been described as having a single screen 15 on which the virtual puppet 7 is displayed. The karaoke system 1 may be connected to one or more additional screens or displays for displaying the virtual puppet 7. The additional screen(s) or display(s) may be proximal or distal to the karaoke system 1.

It is not essential that the karaoke system is operated by a dedicated operator. For example, the individual who performs could operate the system. Preferably, the karaoke system is at least partially automated. The song to be performed and/or the virtual puppet to be displayed could, for example, be selected from on-screen menus. Equally, payment could be made to use the karaoke system via an on-screen payment system or a dedicated payment device.

It will be appreciated that various changes and modifications may be made to the karaoke system described herein without departing from the spirit and scope of the present invention. For example, the karaoke system may be provided with several motion tracking systems and optionally also microphones to allow a group of individuals to perform simultaneously.

Claims

CLAIMS:

1. A karaoke system comprising a motion tracking system for tracking the movements of an individual performing karaoke and a character animation unit for animating a virtual puppet based on an output of the motion tracking system.

2. A karaoke system as claimed in claim 1 further comprising a prompter or autocue for displaying song lyrics.

3. A karaoke system as claimed in claim 1 or claim 2 further comprising a data storage device for storing data generated by the character animation unit.

4. A karaoke system as claimed in claim 3, wherein the data storage device is adapted to store video data generated by the character animation unit.

5. A karaoke system as claimed in claim 3 or claim 4, wherein the data storage device is adapted to store audio data.

6. A karaoke system as claimed in any one of the preceding claims further comprising a recording device for recording data generated by the character animation unit on a storage medium.

7. A karaoke system as claimed in any one of the preceding claims further comprising at least one sensor.

8. A karaoke system as claimed in claim 7, wherein, in use, said at least one sensor is provided on the person of the individual.

9. A karaoke system as claimed in claim 7 or claim 8, wherein said at least one sensor is inertial, electromechanical, magnetic, pressure, piezo or optical.

10. A karaoke system as claimed in any one of the preceding claims, wherein the motion tracking system comprises a plurality of receivers for receiving a signal transmitted from a transmitter.

11. A karaoke system as claimed in claim 10, wherein, in use, at least one transmitter is provided on the person of the individual.

12. A karaoke system as claimed in claim 11 , wherein said at least one transmitter is an optical transmitter, a sonic transmitter or a radio transmitter.

13. A karaoke system as claimed in any one of the preceding claims, wherein the character animation unit provides lip synchronisation for the virtual puppet.

14. A karaoke system as claimed in any one of the preceding claims further comprising a microphone.

15. A karaoke system as claimed in claim 14, wherein one or more inertial sensors are provided in the microphone.

16. A karaoke system as claimed in any one of the preceding claims further comprising a display device for displaying the virtual puppet.

17. A karaoke system as claimed in claim 16, wherein the display device comprises a monitor or a projector.

18. A karaoke system as claimed in any one of the preceding claims comprising processing means for altering the sound of the individual's voice.

19. A karaoke system as claimed in any one of the preceding claims, wherein the motion tracking system is suitable for tracking the movements of a plurality of individuals performing karaoke together and the character animation unit is suitable for animating respective virtual puppets for each of the individuals.

20. A karaoke system comprising a microphone and a character animation unit for animating a virtual puppet having a mouth, wherein, in use, the character animation unit animates the mouth of the virtual puppet in response to an output of the microphone.

21. A karaoke system as claimed in claim 20 further comprising a display device for displaying the virtual puppet.

22 A karaoke system as claimed in claim 20 or claim 21 further comprising a data storage device for storing data generated by the character animation unit.

23. A karaoke system as claimed in claim 22, wherein the data storage device is adapted to store video data generated by the character animation unit.

24. A karaoke system as claimed in claim 22 or claim 23, wherein the data storage device is adapted to store audio data.

25. A karaoke system as claimed in any one of claims 20 to 24 further comprising a recording device for recording data generated by the character animation unit on a storage medium.

26. A karaoke system as claimed in any one of the preceding claims further comprising means for accepting payment.

27. A method of performing karaoke, the method comprising the steps of:

(a) tracking the movements of an individual performing karaoke;

28. A method as claimed in claim 27, wherein audio and video data is recorded on a storage medium.

29. A karaoke system substantially as herein described with reference to the accompanying Figures.

30. A method of performing karaoke substantially as herein described with reference to the accompanying Figures.