WO2006018299A1

WO2006018299A1 - User input device including user input evaluation and method

Info

Publication number: WO2006018299A1
Application number: PCT/EP2005/008928
Authority: WO
Inventors: Claas Derboven; Andreas Merz; Markus Cremer; Michael Saupe
Original assignee: Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Current assignee: Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Priority date: 2004-08-19
Filing date: 2005-08-17
Publication date: 2006-02-23
Anticipated expiration: 2007-02-19
Also published as: DE102004040281A1

Abstract

In order to enable a more flexible application, a user input device comprising a user input evaluation is described including provisioning means (16) for providing an audio signal, reproduction means (20) for reproducing the audio signal; input means (12) for detecting user inputs; melody examination means (22) for subjecting the audio signal to a melody recognition in order to obtain rhythm information; control means (24) for, accompanying the reproduction, controlling a visual display (14) depending on the rhythm information in order to visually display a prescribed manner how user inputs should be performed at the input means (12), and comparison means (26) for comparing the user inputs to the prescribed manner in order to obtain a user input evaluation.

Description

User Input Device including User Input Evaluation and

Method

Description

The present invention relates to a user input device with user input evaluation. Such user input devices are for example employed as game devices in entertainment industry.

In the field of entertainment industry and there in particular in the field of entertainment software, for several years video games have experienced a high popularity which are based on the rhythm-based triggering of certain sensors built into the input devices. In particular, a plurality of games exist that realize this request via so called dance mats including a plurality of pressure sensors. Here, a dance mat lying on the floor is arranged in front of a screen. According to the beat of a simultaneously played music, symbols are displayed on the monitor indicating to the player standing in the middle of the dance mat which sensor should be activated. In the dance mat, film-integrated switches are integrated as sensors which check whether the user performed the prescribed "dancing steps" "according to beat", i.e. at the requested points of time. There is a plurality of other games operating with similar principles and/or different hardware. Thus, in a different game the sensors are for example integrated in drums.

So that the patterns displayed on the screen go with the simultaneously played music, the patterns are created manually and individually adapted to the pieces of music by musically skilled "pattern creators", wherein the same may here integrate any musical characteristics of pieces of music into the patterns. All games already on the market have in common that the user may not extend the game by his own music. He is always dependent on the pieces of music contained on the accompanying games CD or DVD or which are available by download service for example from the game provider. The hitherto available variants of those games are limited with regard to the selection of music which clearly represents a great disadvantage. The acceptance among the players for games of the above-mentioned type would obviously be higher if the music selection were not limited.

It is therefore the object of the present invention to provide a user input device with user input evaluation and a method for user input comprising user evaluation facilitating a more flexible application.

This object is achieved by a user input device according to claim 1 and a method according to claim 11.

The central idea of the present invention is that a more flexible applicability of a user input device, comprising user input evaluation controlling a visual display accompanying a music playback to visually display a prescribed manner how a user should perform user inputs at an input means, may be achieved by obtaining rhythm information from the audio signal via a rhythmical examination of the audio signal, which is then used in order to control the visual display, i.e. in a way that fits the audio signal. This proceeding in particular enables the input of any audio signal into the user input device. In other words, the system is not restricted to a predetermined set of audio signals.

According to a special embodiment of the present invention the rhythmic examination includes at least or exclusively a tempo analysis of the audio signal, such that as at least one part of the rhythm information tempo information is obtained that defines an associated tempo for each predetermined point in time of the audio file, like e.g. a tempo value between all quarter note pairs of the audio signal. With a speed depending on this tempo information then a sequence of visual patterns is reproduced which was for example generated in advance.

According to a further aspect of the present invention, the sequence of visual or graphical patterns, respectively, is randomly generated. The generation of the sequence of patterns may, however, also be performed depending on melody or song setup information, respectively, that may be extracted from the audio signal. The melody information may for example include a melody or bass line of the audio signal.

According to a special embodiment of the present invention, the user input device includes a dance mat as an input means, a monitor as output means, a loudspeaker as a reproduction means and a CD, DVD, or MP3 player as a provisioning means, wherein the user input device functions as a game device and performs a comparison between the user inputs and the prescribed manner defined by the visual display in order to obtain a user input evaluation.

In the following, preferred embodiments of the present invention are explained in more detail with reference to the accompanying drawings, in which:

Fig. 1 shows a diagram of a user input device according to the present invention;

Fig. 2 shows an exemplary snapshot of the content displayed on the screen of the device of Fig. 1;

Fig. 3 shows a flowchart for illustrating the basic course of the game in the device according to Fig. 1; Fig. 4 shows a flowchart for illustrating an actual game course in the device of Fig. 1; and

Fig. 5 shows a flowchart for illustrating the basic game course in the device of Fig. 1 according to an alternative embodiment.

It is to be noted that the following embodiments described with reference to the figures relate to a special game device in which the present invention may be implemented. The present invention is, however, not limited to this game device comprising a dance mat. In addition to that, input devices according to the present invention may not only be used in the entertainment sector. It would further be possible to use user input devices comprising user input evaluations also for medical purposes, like e.g. for the examination of the powers of concentration, the motor powers or the reactions of a patient, or for a therapeutical treatment of patients with regard to these abilities.

Fig. 1 shows a game device 10 in which the present invention is implemented. The game device 10 includes a dance mat 12 as a user input device, a monitor 14 as a visual display, an audio provisioning means 16 for audio provisioning and a computer 18 for evaluation and control purposes which is connected to the dance mat 12, the monitor 14 and also the provisioning means 16. Additionally, the monitor 14 includes presently as an example integrated speakers 20 as a reproduction means.

On the computer 18 certain processes are running which are explained in more detail in the following. In particular, the computer 18 is programmed or able, respectively, to perform a rhythm recognition process 22, a pattern recognition process 24 and a comparison/evaluation process 26. In other words, on the computer 18 rhythm recognition means 22 for a rhythmic examination of the audio signal, a pattern generator 24 for a pattern generation depending on the result of the rhythm examination by means 22 and for a corresponding control of the monitor 14 and comparison/- evaluation means 26 for a final input evaluation are implemented.

While the computer 18 may be located at any location, the dance mat 12 is arranged on a floor in front of the monitor 14 so that a player standing or playing, respectively, on the dance mat 12, may easily see the monitor 14.

The provisioning means 16 may function as a mere memory in which a predetermined plurality of audio signals or audio files, respectively, representing respective pieces of music, is stored. In this case, the provisioning means just like the computer 18 may be arranged at any location or even be integrated in the computer. It may, however, further be the case that the provisioning means 16 is able to read audio files from portable data carriers brought along by the player, like e.g. CDs, DVDs or memory cards, or comprises an interface which enables a player to have an audio file read in by the provisioning means, like e.g. an infrared interface. By this, a user is enabled to integrate audio files brought along by himself into the game course explained in the following. In this case it is of course advantageous if the provisioning means 16 is arranged close to the dance mat 12 and the monitor 14.

Fig. 2 shows an example of a snapshot of a content displayed on the screen 28 during the actual course of the game. As will be explained in more detail in the following, on the screen 28 during the course of the game, matching the beat of the music, symbols are faded in informing the player in which way he should move on the dance mat 12. According to a preferred embodiment, the symbols are arrows 30 either pointing to the left, the bottom, the top or the right. Certain fields of the dance mat 12 correspond to the four different possibilities or the possible arrow - -

directions, respectively, for the arrows 30 on the screen 28. In the preferred embodiment, the dance mat 12 is separated into nine fields or nine partial areas 32, respectively, visible for the player, which together form a 3x3 field. On the middle field on which in Fig. 1 as an example footprints 34 are shown, the player is mainly standing during the game or he is acting starting from this field, respectively. On the field in front of him an arrow 36 is shown pointing to the front or in the direction of the monitor 14, respectively, on the right middle field next to him an arrow 38 is illustrated pointing to the right with respect to the direction of view to the front to the monitor 14, on the middle rear field an arrow 40 is shown pointing to the back and on the left middle field an arrow 42 is shown pointing to the left. The fields with the arrows 36 - 42 corresponds to the arrows 30 on the monitor 28. In the area of these fields with the arrows 36 to 42 the film-integrated switches (not shown) are arranged in the mat 12 in order to enable the computer 18 to determine for example by query whether the player is currently stepping onto the fields with the arrows 36 - 42. In particular, this way the computer is enabled to determine whether, and if yes, with which temporal accuracy, the player hits the fields with the arrows 36 - 42 in the way indicated by the arrows 30 on the screen 28. At that, the arrows 30 scroll from top to bottom on the screen 28. In the moment in which an arrow 30 reaches an associated arrow template 32 at the top margin of the screen 28, the player has to step onto the corresponding arrow 36 - 42 of the dance mat to obtain points. Thus, the player already sees a few seconds before the actual action which arrows 30 come up next or become relevant next, respectively.

As above basically only the setup of the game device of Fig. 1 was described, in the following with common reference to Figs. 1 - 3 the approximate course of the game of the game device 10 is described. The course of the game is controlled by a game program running on the computer 18 and including the functionalities 22, 24 and 26.

The game starts with the provisioning of an audio file in step 50. The audio file provided in step 50 is provided to be reproduced during the actual game via the loudspeakers 20. The audio file 50 may be provided in any way, like e.g. by a random selection from a predetermined fixed set of audio files or by the player selecting from a predetermined fixed set of audio files. In order to be able to do the selection, the player may for example use an input device provided in the provisioning means 16, like e.g. a keyboard. It is possible, however, that free fields in the dance mat 12, i.e. those without arrows, are used as function input fields. In Fig. 1 as an example the front two corner fields right and left at the front are provided with signs, i.e. a cross and a circle, and are further equipped with film-integrated switches. By use of those two fields, the player could for example navigate through a menu before the actual start of the game, which is displayed on the monitor 14, in order to make a selection from available audio files. In the case in which the provisioning means 16 enables inserting portable data carriers, like e.g. CDs, DVDs or memory cards, the player may anyway supplement audio files provided by the game device by own audio files or a single own audio file, so that same are additionally available in order to be selected. Of course it is further possible that the game device 10 itself comprises no pre-stored audio files and that the player is thus dependent on his own audio files.

After the user has selected his audio file this way at the start of the game program, the computer 18 or the rhythm recognition means 22, respectively, within the scope of the game program, subjects this audio file to a tempo analysis or tempo recognition 52, respectively. The tempo recognition is for example performed based on a method of Jan Rohden described in the patent DE 10123281 Cl with the title "Vorrichtung zum Analysieren eines Audiosignals hinsichtlich von Rhythmusinformationen des Audiosignals unter Verwendung einer Autokorrelationsfunktion" (device for the analysis of an audio signal with regard to the rhythm information in the audio signal using an auto¬ correlation function) . Preferably, the tempo recognition 52 leads to the generation of a list containing the tempi or tempus values, respectively, of the song presented by the audio file, between all quarter note pairs or for all time intervals between such pairs, respectively. This way, also possible tempo changes within the song may be considered.

Within the scope of the running game program, the computer 18 or the pattern generator 24, respectively, then performs a pattern generation 54. In this pattern generation 54 patterns in the sense of symbol or arrow sequences, respectively, are generated, according to which the player should dance. According to a preferred embodiment of the present invention, the complete sequences for a song for an audio file, respectively, are generated before the beginning of the actual game happening.

In the present embodiment, the patterns may for example be combined from many individual patterns, which in turn respectively consist of four or eight or another number of rows. In one row one or two arrows are located. In other words, the pattern generation 54 may consist in generating a sequence of individual patterns and thus a sequence of rows by a selection from a predetermined fixed set of possible individual patterns, respectively having a predetermined number of rows to one or two arrows 30 each. In the case of two arrows in one row, the same of course direct into different directions, as according to the example of Fig. 2 arrows 30 with a different arrow direction are indicated in different columns or scrolled along the column direction, respectively. If two arrows are located in one row, this means that the player has to step onto two fields 32 of the dance mat 12 simultaneously. This - -

case for example occurs in Fig. 2 in the row directly below the writing "55 combo".

The pattern generation or the selection below the individual patterns, respectively, may be controlled by random. The pattern generation or pattern selection may also be made dependent on the setup of the song and melodical aspects. Thus, for example by means of a filter bank in the computer 18 melody or bass lines might be examined and the pattern may be adapted to the same. The rhythm of the song underlying the audio file could also be used for pattern generation.

After the tempo recognition 52 was performed by the rhythm recognition 22 and the pattern generation 54 was performed by the pattern generator 24, in a step 56 the actual game starts whose actual game course is explained in more detail with reference to Fig. 4.

At the end of the game, which is for example determined by reaching the end of the audio file, an evaluation 58 or a conclusion in the form of a statistic about the individual evaluations of the dancing steps of the player or user, respectively, takes place. Apart from that, for example an overall ranking is determined, i.e. a listing of the best games at the game device 10.

Fig. 4 shows in more detail the actual game course 56. As already mentioned, the actual game course 56 begins after the pattern generation 54 and the tempo recognition 52. The actual game course 56 begins in particular with an initialization in a step 100. In the initialization 100 for example the screen 28 is prepared, i.e. for example the arrow templates 32 are drawn, etc. The initialization 100 may further be provided to activate the sensors in the dance mat 12 or to guarantee a synchronization of the following process steps. After the computer 18 has performed all necessary initialization measures 100, it controls the loudspeakers 20 to run the audio file or music, respectively. It is further possible that the computer 18 merely triggers playing 102 the music, like e.g. by a command or a trigger signal, respectively, to the provisioning means 16 to directly pass on the audio signal to the loudspeakers 20 - without a detour via the computer 18.

During playing 102 the music the computer 18 performs individual processes, i.e. a query 104 of the sensors 36 - 42 of the dance mat 12, a graphical pattern representation 106 or a control of the monitor 14, respectively, a comparison 108 of the patterns to the query result or the sensor signals from the sensor mat 12, respectively, and an evaluation 110. The reproduction of music 102, the graphical representation 106 of the arrows 30, the query 104 of the dance mat 12 or the pattern and the examination 108 whether the input corresponds to the arrows, run synchronously. In particular, presently as an example the processes 104 and 106 are performed in parallel by the computer 18 in order to compare, in a subsequent process 108, the results of those processes, i.e. the actual dancing performance and the set prescribed manner, and to subsequently subject the comparison result to an evaluation in the evaluation process 110. After the process 110 the processes 104 and 106 are performed again, whereby the processes 104 to 110 are passed through cyclically, while the music is played 102.

Within the scope of the query process 104 the computer 18 receives for example at certain points of time with short time intervals between the same information about whether the player is actually stepping onto a certain one of the fields 32 with the arrows 36 - 42 or not, in order to obtain a sequence of binary values for each of those fields. Within the scope of the graphic pattern representation 106 the computer 18 controls the monitor 14 in order to indicate the sequence of patterns or the rows with arrows 30 and at that scroll the same to the top row by row on the screen 28. The speed of scrolling here depends on the tempo of the piece of music which is simultaneously played in the process 102, i.e. in particular depending on the tempo at the section of the piece of music which is being played.

As it was already mentioned above, a point of time at which an arrow 30 on the screen 28 reaches its associated arrow template 32 at the top margin of the screen 28 defines the set point of time at which the player should enter the corresponding field containing the arrow 36, 38, 40 or 42, respectively. In the process 108 the computer 18 compares these set points of time to the points of time at which the player actually stepped onto the corresponding field, i.e. for example the initial point of time of "stepping", which is for example noticed by a sequence of binary ones in the query signal of the corresponding film-integrated switch. In particular it may be provided that the passing of the templates 32 by the arrows 30 not only determines initial points of time at which a corresponding field should be actuated or entered by a user, respectively, but also final points of time, at which the activation or entering, respectively, of this field should be finished. In other words, according to this alternative, with each arrow 30 not only a set point of time would be determined, but even a time interval set by two set points of time, i.e. an initial point of time and a final point of time, in which the field should continuously be activated or entered, respectively.

In the process 110 the computer 118 performs an evaluation for each row which describes whether, and if yes, how accurate in time the user hits the right field 32 on the input device 12, wherein, as already mentioned above, the evaluation occurs on the basis of the time difference of a point of time determined by passing the arrows across the corresponding template 32 with regard to the actual activation or entry point of time, respectively, on the basis of the examination whether the actual actuation or entry time, respectively, occurs in a time interval determined by the passing of a respective template 32 by the respective arrow 30, on the basis of an intersection of the actual time interval of an actuation or an entering of a field, respectively, with a set time interval determined by passing a template 32 by the associated arrow, or on the basis of similar comparisons of the detected actuations with the passings of the templates 32 by the arrows 30. The evaluations are for example classified into steps similar to school grades, like e.g. very good, satisfactory, etc., or 1, 2, 3, etc., respectively, classified and counted per step or row. As already mentioned, the player then receives a conclusion 58 in the form of a statistic about the individual evaluations at the end of the game.

In contrast to the games described in the specification introduction, that do not offer the possibility to introduce own music into the game or to play it to the same, the game device 10 makes it possible to design the game more freely with regard to the selection of music by the player being able to freely select his own music.

With reference to the above-mentioned description, it is noted, that the game device 10 is not restricted to the indicated setup. Instead of a dance mat as an input means, also a drum or a plurality of drums or a keyboard may be used. Depending on the case, also other sensors apart from film-integrated switches are used.

In the preceding embodiment, the rhythm recognition means 22 only performed a tempo analysis. It is further possible, however, that the same extracts further rhythm information from the audio signal, like e.g. beats or beat limits, respectively, and the like. Those further information may then, if applicable, also be integrated into the speed control. Conversely, the rhythm recognition means might also extract only points of time of the beginning of the notes of the melody or main melody, respectively, underlying the audio signal from the audio signal from which inherently of course also tempo information results. In this case, the rhythm information only contains the points of time of the beginning of the note. These then represented the set points of time for actuating the dance mat. The arrow rows or patterns, respectively, would, however, here be scrolled with such a variable speed, that the passes of the templates 32 by the arrows 30 are synchronized with the points of time of the beginning of the notes. The rhythm recognition means could arrange the points of time of the beginning of the notes from a transient audio signal or an already transcribed audio signal - namely by an automatic transcription. Additionally, it could associate the individual points of time of the beginning of the notes to individual fields of the mat, for example depending on the associated note pitch or the like. A further embodiment using a melody recognition is described in the following with reference to Fig. 5.

Further, the above-described game device may also be improved in so far that the created patterns not only match the music rhythmically but also depend on the setup of the song and on melodical aspects. Thus, for example using a filter bank, for example realized in the computer 18, melody or bass lines may be examined and the patterns may be adapted to the same, or the above-mentioned selection among the patterns may be performed depending on those lines, respectively. Also an extraction of the rhythms and not only of the tempo would be possible, wherein the rhythm information may also be used for pattern generation or selection, respectively. - IA -

The above-mentioned game program running on the computer 18 may thus be referred to as a rhythm-based entertainment software integrating a special periphery, i.e. a special periphery in the form of the dance mat.

With reference to Fig. 3, it is explicitly noted, that it is neither required that the tempo recognition 52 and the pattern generation 54 are performed temporally separated from the actual course of the game 56 before the actual course of the game 56, nor that the same have to be performed one after the other. It would rather be further possible to let this processes run simultaneously to the actual course of the game.

Above it was already noted, that as an input means also something different from a dance mat may be used. In a similar way it is of course also possible to use a different display means than the monitor 14 for a visual display. So, for example, only lamps might be used that may be switched on and off, like e.g. LEDs, in order to indicate in which way and when a player should perform a certain input or actuation, respectively. In the case of the dance mat of Fig. 1 those LEDs might for example be arranged at the corners of a rhombus, in order to intuitively correspond to the fields 36 - 42. Accordingly, the prescribed manner of the user input may also be illustrated different to a sequence of arrow rows.

One possible generalization is for example, that within the scope of the pattern generation depending on the rhythm information for each individual partial user input device - in Fig. 1 also for each film-integrated switch - a signal, like e.g. a digital or binary signal, is generated comprising a sequence of signal events, like e.g. signal pulses. These individual signals are visually displayed on different visual output devices, like e.g. by the lighting up of an LED when a signal event occurs. The point in time of the visual display of a signal event then determines a set point in time in a predetermined way in which the player is to activate the individual partial user input device corresponding to this visual display. The set point in time is for example a predetermined time period after the beginning of the display of the signal event, i.e. for example one second after the beginning of the lighting up of the corresponding LED, or at the end of the visual display of the signal event, i.e. for example when reaching the arrow template 32 by the corresponding arrow. The set points in time are then compared to the actual actuation points of time of the respectively corresponding partial user input devices. A sequence of patterns generated by the pattern generator would in this case correspond to the visual impression of the visual displays for all partial user input devices, i.e. e.g. of all LEDs at subsequent points in time. The creation of the signals could in this case similarly be performed in two stages, as described above, i.e. for example by a random generation of the signal and a subsequent "reproduction" or visual display of the signals with a speed depending on the rhythm information. It would also be possible, however, to directly integrate the rhythm information into the process of creating the signals associated with the individual partial user input devices, for example by associating partial bands of the audio signal to different partial user input devices and rhythmically examining the same separate from each other in order to set the signal events of the individual signals in the individual signals separate from each other depending on the rhythm information received for the respective partial band.

It would further be possible to use another reproduction means instead of the integrated loudspeakers 20, like e.g. a headphone.

With reference to the audio file provided by the provisioning means (16) or the audio signal provided by the same, respectively, it is again noted, that it is not important in which format the same is present. It may be present in a compressed or an uncompressed form. In every case the rhythm recognition (22) should preferably be implemented such that it is not necessary that the provided audio signal is present in a note-type or a transcribed representation, respectively, like e.g. in an MIDI format. The reason for this is that a user rarely has a meta- representation of such a kind of his favorite piece of music. Rather, the rhythm information extraction should preferably be possible directly from PCM encoded or generally from digitally available audio signal data or from compressed data, like e.g. MP3 data, in which the digital audio samples are for example transformed into a time/frequency range or separated into partial bands, respectively.

It was already mentioned above and shortly outlined that or how, respectively, for a pattern generation also a melody recognition may be used. Fig. 5 shows a further embodiment of the present invention using such a melody recognition. After the embodiment of Fig. 5 the course of the game starts in a step 150 with the provisioning of the audio file which is reproduced during the actual game via the loudspeakers 20. The audio file 150 is thereupon subjected to a melody recognition in a step 152. The melody recognition subjects the audio file provided in step 150 to an auto transcription to extract a preferably monophonic melody in a note representation from the audio file in which the melody underlying the piece of music of the audio file is written in a format from which the note sequence of the melody may be seen and in which for each of note of the melody for example the point in time of the beginning of the note, the note pitch and the note length are provided. Apart from that, the audio file is subjected to a harmony recognition 153 which is provided to detect chords in the audio file or the piece of music represented by the same, respectively. The harmony recognition 153 may perform a key recognition for this purpose, in order to limit the number of possible chords to be detected to a certain subset associated with the recognized key. The result of the harmony recognition 153 is chord information indicating the chords occurring in the piece of music represented by the audio file together with their occurrence times. As it is indicated by a dashed arrow between the blocks 152 and 153, the harmony recognition may take place for example using the note sequence obtained by the melody recognition 152. For example, in a final step the note sequence of the melody recognition 152 may be used to examine the sequence of chords and the occurrence times again with regard to their coherence or the matching with the melody represented by the note sequence, respectively, within the scope of the harmony recognition 153, in order to perform a modification or adaptation, respectively, in discrepancies of the recognized chords or the chord recognition information, respectively, as they are finally output by the harmony recognition 153. Alternatively, also the result of the harmony recognition may be used to improve or correct the melody recognition. The note sequence information and the chord information of steps 152 and 153 are subsequently used in a step 154 for pattern generation. Here, for example the pattern generation 154 may primarily detect repeating or recurring chords or chord sequences in the piece of music of the audio file on the basis of the chord information of the harmony recognition 153. Subsequently, within the pattern generation 154 a sequence of arrow rows is generated, as described above, which is then reproduced when playing, together with the reproduction of the audio file, wherein according to the embodiment of Fig. 5 for example the tempo in which the arrow sequence is scrolled is always the same, the arrow lengths, however, determining the time period or the time intervals of passing the arrows across the templates, respectively, are set by the pattern generation 154 depending on the note length from the note sequence of the melody recognition 152, and wherein in the pattern generation 154 also the sequence of arrow rows is determined such that the scroll points in time or scroll time intervals, respectively, determined by the same in reproduction, are the same for the time sections of the same chords or the same chord sequences, respectively, detected at the beginning of step 154. Further, the step of pattern generation 154 is performed so that the resulting set points in time or set intervals, respectively, determined by the passing of the templates 32 by the arrows 30, are synchronized with the points in time of the beginning of the notes of the note sequence. By the integration of the result of the search for like locations with like chords or like chord sequences, in the automatic pattern generation in step 154 the setup of the song is overall taken into account more. The pattern generation may otherwise take place in the above-described way, like e.g. comprise a random combination of the arrows in the rows. In the pattern generation, the note pitches may further be used for the generation of arrow rows or for the selection of the fields 32 to be actuated, respectively. Subsequently, in steps 156 and 158 the actual game takes place with a subsequent evaluation, wherein those steps are performed in the same way, like the above-described steps 156 and 158 of Fig. 3, apart from the fact that according to the present embodiment the tempo of scrolling the arrow pattern is exemplarily constant and only the arrow lengths are varied.

It is explicitly noted again, that depending on the circumstances, the inventive scheme for generating polyphonic melodies may be implemented in software. The implementation may take place on a digital storage medium, in particular a floppy disc or a CD with electronically readable control signals which may cooperate with a programmable computer system so that the corresponding method is performed. In general, the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for performing the inventive method when the computer program product runs on a computer and/or a corresponding digital or analog building block. In other words, the invention may thus be realized as a computer program having a program code for performing the method when the computer program runs on a computer.

Claims

1. A user input device having a user input evaluation, comprising

provisioning means (16) for providing an audio signal;

reproduction means (20) for reproducing the audio signal;

input means (12) for detecting user inputs;

rhythm examination means (22) for subjecting the audio signal to a rhythmical examination in order to obtain rhythm information;

means for extracting melody or song setup information, respectively, or a melody or bass line from the audio signal;

control means (24) for, in addition to the reproduction, controlling a visual display (14) depending on the rhythm information in order to visually display a prescribed manner how the user inputs are to be performed at input means (12) , wherein control means (24) is implemented to generate a sequence of visual patterns (30) depending on the melody or song setup information, respectively, or the melody or bass line, and to control the visual display (14) such that the same reproduces this sequence of visual patterns (30) in a speed depending on the rhythm information; and

comparison means (26) for comparing the user input to the prescribed manner, in order to obtain a user input evaluation.

2. The user input device according to claim 1, wherein the rhythm examination means (22) is implemented to subject the audio signal to a tempo analysis such that at least as one part of the rhythm information tempo information is obtained defining for each point in time of the audio signal an associated tempo.

3. The user input device according to claim 1 or 2, wherein control means (24) is implemented to perform the generation of the sequence of visual patterns (30) completely before reproducing the same.

4. The user input device according to one of the preceding claims, wherein control means (24), input means (12), reproduction means (20) the comparison means (26) are implemented in order to operate synchronously.

5. The user input device according to one of the preceding claims, wherein input means (12) includes a dance mat (12) , a keyboard or a drum.

6. The user input device according to one of the preceding claims, wherein the user input device includes a monitor (14) as a visual display, wherein control means (24) is implemented to perform a scrolling of the screen of the monitor (14) with a speed depending on the rhythm information.

7. A method for a user input having a user input evaluation, comprising the following steps:

providing (50) an audio signal;

reproducing (102) the audio signal;

detecting (104) user inputs; subjecting (52) the audio signal to a rhythmical examination in order to obtain rhythm information;

extracting melody or song setup information, respectively, or a melody or bass line from the audio signal;

accompanying the reproduction, controlling (106) a visual display depending on the rhythm information in order to visually display a prescribed manner, in which user inputs should be performed, wherein the step of controlling further comprises the following substeps:

generating a sequence of visual patterns (30) depending on the melody or song setup information, respectively, or the melody or bass line; and

reproducing the sequence of visual patterns (30) with a speed depending on the rhythm information; and

comparing (108) the user inputs to the prescribed manner in order to obtain a user input evaluation.

8. A computer program having a program code for performing the method of claim 7, when the computer program runs on a computer.