DE102004040281A1 - User input device with user input rating and method - Google Patents

Abstract

In order to enable a more flexible application, a user input device with user input evaluation is described, which comprises a provision device for providing an audio signal; a reproducing device for reproducing the audio signal; an input device for detecting user inputs; a rhythm examiner for examining the audio signal of a rhythmic examination to obtain rhythm information; driving means for accompanying the reproduction, driving a visual display depending on the rhythm information to visually indicate a default manner in which user inputs are to be made to the input means, and comparing means for comparing the user input with the default way to obtain a user input score.

Description

The The present invention relates to a user input device with user input rating. Such user input devices will be for example as play equipment used in the entertainment industry.

in the Area of the entertainment industry and there especially in the area The entertainment software has been enjoying video games for several years a big one Popularity based on the rhythm-based release of certain sensors, the in input devices are built based. In particular, a variety of Play, this question about so-called. Dance mats, which includes a variety of pressure sensors realize. There is one on the floor lying dance mat arranged in front of a screen. Matching Clock of a simultaneously played music will be symbols on the Monitor, which is the one in the middle of the dance mat Tell players which sensor to activate. In the dance mat Foil switches are incorporated as sensors which check whether the user is able to follow the predetermined "dance steps" "in time", i.e. at the required times, carried out Has. There are a lot of other games that use similar principles and / or other hardware. So are in another game the sensors housed for example in drums.

In order to the patterns that are displayed on the screen at the same time played music, the patterns or patterns of musical preformed "pattern creators" manually and individually to the music pieces adapted, while the same all musical properties the music pieces into the patterns can let.

All Games on the market so far have the commonality of that the user does not extend the games to his own music can. He always depends on the music on the accompanying game CD or -DVD included or download service, for example from Game drivers available are. The previously available Variants of these games are therefore in relation to the music selection limited, which is quite a disadvantage. The Acceptance among the players for Games of the kind mentioned above would be certainly bigger, though the music selection would not be restricted.

The It is therefore an object of the present invention to provide a user input device with user input rating and a method for user input with user assessment to create a more flexible application enable.

These The object is achieved by a user input device according to claim 1 and a method according to claim 10 solved.

Of the The core idea of the present invention is that a more flexible applicability of a user input device with user input rating, accompanying a music playback to a visual display, to visually indicate a default way in which a user inputs user information to make an input device, thereby achieved can that over one rhythmic examination of the audio signal rhythm information are obtained from the audio signal, which are then used to control the visual display, in a way which matches the audio signal. This procedure makes it possible in particular, any audio signal into the user input device to be able to enter. In other words, the system is not on a predetermined set of audio signals limited.

According to one special embodiment The present invention includes rhythmic examination at least or exclusively one Tempo analysis of the audio signal, such that as at least a part the rhythm information is obtained tempo information that for every predetermined one At the time of the audio file define an associated tempo, such as e.g. a tempo value between all ¼-note pairs of the audio signal. With a Speed, which depends on these tempo information, will then a sequence of visual patterns played, for example has been generated in advance.

According to one Another aspect of the present invention is the consequence of visual or graphic patterns at random generated. However, the generation of the sequence of patterns can also be dependent on Melody or song structure information is carried out from the audio signal can be extracted. The melody information may be, for example comprise a melody or bass line of the audio signal.

According to a specific embodiment of the present invention, the user input device comprises a dance mat as input device, a monitor as output device, a loudspeaker as reproduction device and a CD, DVD or MP3 player as providing device, the user input device functioning as a game device and a comparison between hires the user input and the default manner defined by the visual display to obtain a user input rating.

preferred embodiments The present invention will be described below with reference to FIGS enclosed drawings closer explained. Show it:

1 a schematic representation of a user input device according to the present invention;

2 an exemplary snapshot of the content displayed on the screen of the device 1 is shown;

3 a flowchart for illustrating the basic game play in the device according to 1 ; and

4 a flowchart for illustrating the actual game play in the device of 1 ,

It It should be noted that the following, referring to the figures described embodiments on a special game device in which the present invention is implementable. However, the present invention is not applicable to this dance mat game machine limited. In addition, you can Input devices according to the present invention Invention are not used only in the entertainment sector. Conceivable would it be further, user input devices with user input rating also for medical purposes, such as for examination the ability to concentrate, the motor ability or the reaction skills a patient, or for the therapeutic treatment of patients in terms of of these skills.

1 shows a game machine 10 in which the present invention is implemented. The game device 10 comprises as a user input device a dance mat 12 , as a visual display a screen 14 For audio provision, an audio delivery device 16 and for evaluation and control purposes, a computer 18 that with both the dance mat 12 , the monitor 14 as well as with the provisioning device 16 connected is. In addition, the monitor includes 14 in the present example, integrated speakers 20 as a playback device.

On the computer 18 certain processes take place, which are explained in more detail below. In particular, the computer 18 programmed or capable of a rhythm recognition process 22 , a pattern generation process 24 and a comparison / evaluation process 26 perform. In other words, on the computer 18 a rhythm recognition device 22 for the rhythmic examination of the audio signal, a pattern generator 24 for pattern generation depending on the result of the rhythm examination by the device 22 and for the corresponding control of the monitor 14 and a comparison / evaluation device 26 implemented for the final input rating.

While the computer 18 can be located at any place is the dance mat 12 on a floor in front of the monitor 14 arranged so that a player on the dance mat 12 stands or plays, the monitor 28 can see comfortably.

The provisioning device 16 may act as a pure memory in which a predetermined plurality of audio signals or audio files representing respective pieces of music are stored. In this case, the provisioning device may be the same as the computer 18 be located at any location or even integrated in the computer. However, it may be further that the provisioning device 16 is capable of reading audio files from portable media brought by the player, such as CDs, DVDs or memory cards, or having an interface that allows a player to read in an audio file from the providing device, such as an infrared interface. As a result, it is possible for a user to embed self-brought audio files into the game sequence set out below. In this case, it is of course advantageous if the provisioning device 16 near dance mat 12 and monitor 14 is arranged.

2 shows an example of a snapshot of a content on the screen 28 during the actual gameplay. As will be explained in more detail below, will be on the screen 28 During the course of the game, according to the beat of the music, symbols are displayed informing the player how he is sitting on the dance mat 12 to move. According to a present exemplary embodiment, the symbols are arrows 30 pointing to either left, down, up, or right. The four different possibilities or possible arrow directions for the arrows 30 on the screen 28 correspond to certain fields of the dance mat 12 , In the present embodiment, the dance mat is 12 visible to the player in nine fields or nine sections 32 divided, which together give a 3 × 3 field. In the middle field, on the in 1 exemplary footprints 34 are shown, the player stands during the game primarily or from this field he acts. On the field in front of him is an arrow 36 pictured, the forward or toward the monitor 14 shows, on the right middle field next to him is an arrow 38 shown to the right relative to the viewing direction forward to the monitor 14 shows, on the middle back field is an arrow 40 shown pointing to the rear, and on the left middle panel is an arrow 42 shown pointing to the left. These are the fields with the arrows 36 - 42 that the arrows 30 on the screen 28 correspond. In the area of these fields with the arrows 36 - 42 are foil switches (not shown) in the mat 12 arranged it to the computer 18 to allow, for example, by questioning whether the player is watching the boxes with the arrows 36 - 42 occurs. In particular, this allows the computer to determine if and if so, with what timing accuracy, the player uses the arrows 36 - 42 in the way meets, as by the arrows 30 on the screen 28 are displayed. The arrows scroll 30 on the screen 28 from the bottom up. At the moment, by an arrow 30 an associated arrow template 32 at the top of the screen 28 reached, the player must, in order to gain points, on the appropriate arrow 36 - 42 kick the dance mat. So the player sees a few seconds before the actual action, which arrows 30 come next or become relevant.

After the above essentially only the structure of the game device of 1 is described below with common reference to the 1 - 3 the rough course of the game of the game device 10 described. The course of the game is controlled by a game program running on the computer 18 expires and the functionalities 22 . 24 and 26 includes.

The game begins by providing an audio file in step 50 , The in step 50 provided audio file is provided during the actual game on the speakers 20 to be reproduced. The audio file 50 may be provided in any manner, such as by random selection from a predetermined fixed set of audio files or by the player selecting among a predetermined fixed set of audio files. For example, to make the selection, the player may choose one in the provisioning facility 16 provided input device, such as use a keyboard. Furthermore, it is possible that in the dance mat 12 Free fields, ie those without arrows, are used as function input fields. In 1 By way of example, the front two corner panels on the right and left front are provided with symbols, namely a cross and a circle, and are further equipped with membrane switches. By means of these two fields, for example, the player could navigate through a menu prior to the actual start of the game on the monitor 14 is displayed to select from available audio files. In the case that the provisioning device 16 allows the insertion of portable media, such as CDs, DVDs or memory cards, the player can supplement the already provided by the game device audio files to their own audio files or a separate audio file, so that they are additionally available to be selected. Of course, it is also possible that the game device 10 does not have pre-stored audio files and thus the player relies on his own audio files.

After the user has selected his audio file at the start of the game program, the computer subjects 18 or the rhythm recognition device 22 as part of the game program this audio file of a tempo analysis or tempo detection 52 , The tempo detection is performed, for example, based on a method of Jan Rohden described in the patent DE 10123281 C1 entitled "Apparatus for analyzing an audio signal with respect to rhythm information of the audio signal using an autocorrelation function". Preferably, the tempo detection leads 52 to generate a list containing temps of the song presented by the audio file between all 1/4 note pairs or for all time intervals between such pairs. In this way, also possible tempo changes within the song can be considered.

The computer runs as part of the running game program 18 or the pattern generator 24 then a pattern or pattern generation 54 by. In this pattern generation 54 Patterns are generated in the sense of symbol or arrow sequences to which the player should dance. According to a preferred embodiment of the present invention, the complete sequences for a song or for an audio file are generated before the start of the actual game action.

For example, in the present embodiment, the patterns may be composed of many individual patterns each consisting of four or eight or a different number of lines. One line contains one or two arrows. In other words, the Pattergenerierung 54 For example, by selecting from a predetermined fixed set of possible Einzelpatterns each having a predetermined number of lines to one or two arrows 30 create a sequence of single patterns and thus a sequence of lines. In the case of two arrows in a row, these naturally indicate in different directions, since after the example of 2 arrows 30 be displayed with a different direction of the arrow in different columns or scrolled along the column direction. If there are two arrows in a row, this means that the player has two spaces 32 the dance mat 12 must enter at the same time. This case occurs in 2 For example, in the line that is just below the "55 compo" lettering.

Patter generation or the selection under the individual patterns can be random. The Pattergenerierung or pattern selection can also be made dependent on the song structure and melodic aspects. For example, using a filter bank in the computer 18 Check melody or bass lines and adapt the patterns to them. Also, the rhythm of the audio file underlying songs could be used for Pattergenerierung.

After the tempo detection 52 through the rhythm recognition 22 and pattern generation 54 through the pattern generator 24 have been carried out, starts in one step 56 the actual game, its actual gameplay referring to 4 will be discussed in more detail.

At the end of the game, which is determined for example by reaching the end of the audio file, finds an evaluation 58 or a summary in the form of a statistic about the individual assessments of the dance steps of the player or user instead. In addition, for example, an overall ranking is determined, ie a listing of the best games on the gaming machine 10 ,

4 shows the actual gameplay in more detail 56 , As already mentioned, the actual game begins 56 after pattern generation 54 and the tempo detection 52 , The actual gameplay 56 especially begins with an initialization in one step 100 , At initialization 100 For example, the screen becomes 28 prepared, ie, for example, the arrow stencils 32 drawn, etc. The initialization 100 may also be provided to the sensors in the dance mat 12 or to ensure synchronization of the following process steps.

After the computer 18 all necessary initialization measures 100 he controls the speakers 20 to play the audio file or music. It may also be that the computer 18 the playing 102 the music only triggers, such as by a command or a trigger signal to the provision device 16 to directly send the audio signal to the speakers 20 to forward - without detour via the computer 18 ,

While playing 102 the music is led by the computer 18 single processes, namely a query 104 the sensors 36 - 42 the dance mat 12 , a graphic pattern representation 106 or a control of the monitor 14 , a comparison 108 the pattern with the query result or the sensor signals from the sensor mat 12 and a rating 110 , The music playback 102 , the graphic representation 106 the arrows 30 , the query 104 the dance mat 12 or the stalemate and the review 108 whether the input matches the arrows, run synchronously. In particular, the present case will be exemplified by the computer 18 the processes 106 and 106 run parallel to in a subsequent process 108 compare the results of these processes, namely the actual dance behavior and the target specification, with each other, and then the comparison result of an evaluation in the evaluation process 110 to undergo. After the process 110 become the processes 104 and 106 re-executed, eliminating the processes 104 - 110 be cycled while the music is playing 102 becomes.

As part of the query process 104 the computer gets 18 for example, at certain times with short time intervals between the same information as to whether the player is currently on a particular one of the fields 32 with the arrows 36 - 42 or not, to obtain a sequence of binary values for each of these fields.

As part of the graphical pattern representation 106 the computer controls 18 the monitor 14 to the sequence of patterns or lines of lines 30 display the same line by line on the screen 28 scroll up. The speed of scrolling depends on the tempo of the piece of music, which is at the same time in the process 102 is played, in particular depending on the tempo at the point of the piece of music that is currently playing.

As mentioned earlier, a point in time at which an arrow defines 30 on the screen 28 his associated arrow template 32 at the top of the screen 28 reached, the target time at which the player the corresponding field with the arrow 36 . 38 . 40 respectively. 42 to enter. The computer compares with these target times 18 in the process 108 the times at which the player actually stepped on the corresponding field, ie, for example, the start time of a "kicking", which is noticeable for example by a sequence of binary ones in the interrogation signal of the corresponding slide switch.

In the process 110 takes the computer 118 For each line before a review, which describes whether and if so, how exactly in time, the user the right one field 32 on the input device 12 has met. For example, scores are graded in grades similar to grades, such as very good, satisfying, etc., or 1, 2, 3, etc., and counted per level or row. As already mentioned, the player then receives a summary at the end of the game 58 in the form of statistics on the individual ratings.

In contrast to the games described in the introduction which do not offer the possibility of incorporating or playing their own music in the game, the gaming device allows 10 to make the game more free in terms of music selection, allowing the player to freely choose their own music.

With reference to the foregoing description, it should be noted that the gaming device 10 is not limited to the construction shown. Instead of a dance mat as an input device could also serve a drum or a plurality of drums or a keyboard. Depending on that, other sensors are used as foil switches.

In the foregoing embodiment, the rhythm detecting means performed 22 just a tempo analysis through. However, it is also possible that it extracts further rhythm information from the audio signal, such as clocks and the like. This further information could then also be included in the speed control.

Furthermore, the game device described above could also be improved in that the patterns created not only rhythmically match the music, but also depend on the song structure and melodic aspects. For example, using a filter bank, for example, in the computer 18 is realized, melody or bass lines examined and the patterns are adapted to this or the above-mentioned selection among the patterns are made depending on these lines. It would also be possible to extract the rhythms and not just the tempo, whereby the rhythm information could also be used for pattern generation or selection.

The previous mentioned game program, that on the computer 18 can thus be referred to as a rhythm-based entertainment software involving special peripherals, namely special peripherals in the form of the dance mat.

Referring to 3 It is still explicitly stated that it is not required that the tempo detection 52 and pattern generation 54 temporally separate from the actual gameplay 56 before the actual game play 56 be carried out, nor that they are carried out one after another. Rather, it would also be possible to run these processes simultaneously to the actual gameplay.

In the foregoing it has already been pointed out that something else than a dance mat could be used as input device. Similarly, it is of course also possible to use a display device other than the monitor for the visual display 14 , For example, only on / off lights, such as LEDs, could be used to indicate how and when a player should make a particular input or operation. In the case of the dance mat of 1 For example, these LEDs could be located at the corners of a rhombus to intuitively control the fields 36 - 42 correspond to. Accordingly, the default manner of user input could also be represented other than by a series of arrow lines.

A possible generalization consists, for example, in the context of the pattern generation depending on the rhythm information for each individual sub-user input device - in 1 Thus, for each foil switch - a signal, such as a digital or binary signal, is generated which has a sequence of signal events, such as signal pulses. These individual signals are visually displayed on different visual output devices, such as by the illumination of an LED upon the occurrence of a signal event. The timing of visual indication of a signal event then predetermines a target time at which the player is to activate the single sub-user input device corresponding to that visual display. The target time is, for example, a certain period of time after the beginning of the display of the signal event, so for example one second after the beginning of the lighting of the corresponding LED, or at the end of the visual display of the signal event, so for example when reaching the arrow template 32 through the corresponding arrow. The target times are then compared with the actual operating times of the respective sub-user input devices. In this case, a sequence of patterns produced by the pattern generator would correspond to the visual impression of the visual displays for all sub-user input devices, eg all LEDs, at successive points in time. The formation of the signals in this case could be carried out in a similar two-stage manner as described above, for example by randomly forming the signals, and then "playing" or visually displaying the signals at a speed that depends on the rhythm information. Likewise, it would be possible to include the rhythm information equally in the process of forming the signals associated with each sub-user input device, for example by assigning sub-bands of the audio signal to different sub-user input devices and rhythmically inspecting them separately to separate the signal events of the individual signals in the individual signals dependent on the rhythm information obtained for the respective sub-band.

Furthermore, it would be possible to have a display device other than the integrated speakers 20 to use, such as a headphone.

Referring to that of the provisioning device ( 16 ) or the audio signal provided by the same, it is also pointed out that it does not matter in which format the same is the case. It can be in compressed or uncompressed form. In any case, the rhythm recognition ( 22 ) be formed such that it is not necessary that the provided audio signal in a note-like or transscribed representation, such as in a MIDI format, is present. The reason for this is that a user rarely possesses a meta-depiction of such kind from his favorite pieces of music. Rather, the rhythm information acquisition should preferably be possible directly from PCM-coded or generally digitally present audio signal data or from compressed data, such as MP3 data, in which the digital audio samples are transformed into a time / frequency domain or a subbands, for example.

explicit it is again pointed out that depending on the circumstances the scheme of the invention can be implemented in software for generating polyphonic melodies. The implementation may be on a digital storage medium, in particular a floppy disk or a CD with electronically readable control signals done so interact with a programmable computer system can, that the corresponding procedure is carried out. Generally there is the Invention thus also in a computer program product on a machine-readable carrier stored program code for carrying out the method according to the invention, if the computer program product on a computer and / or a corresponding digital or analog module expires. In other words, can the invention thus as a computer program with a program code to carry out the process can be realized when the computer program is up a computer expires.

Claims (11)

User input device with user input rating, with a provisioning device ( 16 ) for providing an audio signal; a playback device ( 20 ) for playing the audio signal; an input device ( 12 ) for detecting user input; a rhythm examination device ( 22 ) to subject the audio signal to a rhythmic examination to obtain rhythm information; a drive device ( 24 ), accompanying to playback, driving a visual display ( 14 ) depending on the rhythm information to visually indicate a default manner to the user input at the input device ( 12 ) to be made; and a comparator ( 26 ) for comparing the user input with the default manner to obtain a user input score. User input device according to claim 1, wherein the rhythm examination device ( 22 ) is adapted to tempo-analyze the audio signal such that tempo information is obtained as at least part of the rhythm information defining an associated tempo for each time point of the audio signal. User input device according to claim 1 or 2, wherein the drive device ( 24 ) is designed to produce a sequence of visual patterns ( 30 ) and the visual display ( 14 ) so as to control this sequence of visual patterns ( 30 ) at a speed that depends on the rhythm information. A user input device according to claim 3, further comprising means for extracting melody information from said audio signal, said drive means ( 24 ) is designed to facilitate the generation of the sequence of visual patterns ( 30 ), depending on the melody or song composition information. User input device according to claim 3, further comprising means ( 18 ) for extracting a melody or bass line from the audio signal to enable generation of the sequence of visual patterns ( 30 ), depending on the melody or bass line. User input device according to one of Claims 3 to 5, in which the drive device ( 24 ) is adapted to the generation of the sequence of visual patterns ( 30 ) completely before playing the same. User input device according to one of the preceding claims, in which the drive device ( 24 ), the input device ( 12 ), the playback device ( 20 ) and the comparator ( 26 ) are designed to operate synchronously. User input device according to one of the preceding claims, in which the input device ( 12 ) a dance mat ( 12 ), a keyboard or a drum. User input device according to one of the preceding claims, in which the user input device displays a monitor ( 14 ) as a visual display, the drive means ( 24 ) is designed to scroll the screen of the monitor ( 14 ) at a speed that depends on the rhythm information. User input method with user input rating, with the following steps: Deploy ( 50 ) an audio signal; Play ( 102 ) of the audio signal; To capture ( 104 ) of user input; Undergo ( 52 ) the rhythmic examination audio signal to obtain rhythm information; Accompanying playback, driving ( 106 ) a visual display dependent on the rhythm information to visually indicate a default manner to which user input is to be made; and comparing ( 108 ) of the user inputs in the default manner to obtain a user input rating. Computer program with a program code to carry out the The method of claim 10 when the computer program is on a computer expires.