WO2011000059A1 - Method of obtaining a user selection - Google Patents
Method of obtaining a user selection Download PDFInfo
- Publication number
- WO2011000059A1 WO2011000059A1 PCT/AU2010/000857 AU2010000857W WO2011000059A1 WO 2011000059 A1 WO2011000059 A1 WO 2011000059A1 AU 2010000857 W AU2010000857 W AU 2010000857W WO 2011000059 A1 WO2011000059 A1 WO 2011000059A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- musical
- user
- selectable
- musical sounds
- sounds
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000004891 communication Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000005236 sound signal Effects 0.000 abstract description 4
- 230000002452 interceptive effect Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000003190 augmentative effect Effects 0.000 description 4
- 230000003292 diminished effect Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 239000011295 pitch Substances 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 241001342895 Chorus Species 0.000 description 1
- 241000282376 Panthera tigris Species 0.000 description 1
- 241000577979 Peromyscus spicilegus Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- HAORKNGNJCEJBX-UHFFFAOYSA-N cyprodinil Chemical compound N=1C(C)=CC(C2CC2)=NC=1NC1=CC=CC=C1 HAORKNGNJCEJBX-UHFFFAOYSA-N 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004297 potassium metabisulphite Substances 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000004557 technical material Substances 0.000 description 1
- 238000013518 transcription Methods 0.000 description 1
- 230000035897 transcription Effects 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/20—Input arrangements for video game devices
- A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
- A63F13/215—Input arrangements for video game devices characterised by their sensors, purposes or types comprising means for detecting acoustic signals, e.g. using a microphone
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
- A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
- A63F13/424—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/80—Special adaptations for executing a specific game genre or game mode
- A63F13/814—Musical performances, e.g. by evaluating the player's ability to follow a notation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/0008—Associated control or indicating means
- G10H1/0016—Means for indicating which keys, frets or strings are to be actuated, e.g. using lights or leds
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/18—Selecting circuits
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H3/00—Instruments in which the tones are generated by electromechanical means
- G10H3/12—Instruments in which the tones are generated by electromechanical means using mechanical resonant generators, e.g. strings or percussive instruments, the tones of which are picked up by electromechanical transducers, the electrical signals being further manipulated or amplified and subsequently converted to sound by a loudspeaker or equivalent instrument
- G10H3/14—Instruments in which the tones are generated by electromechanical means using mechanical resonant generators, e.g. strings or percussive instruments, the tones of which are picked up by electromechanical transducers, the electrical signals being further manipulated or amplified and subsequently converted to sound by a loudspeaker or equivalent instrument using mechanically actuated vibrators with pick-up means
- G10H3/18—Instruments in which the tones are generated by electromechanical means using mechanical resonant generators, e.g. strings or percussive instruments, the tones of which are picked up by electromechanical transducers, the electrical signals being further manipulated or amplified and subsequently converted to sound by a loudspeaker or equivalent instrument using mechanically actuated vibrators with pick-up means using a string, e.g. electric guitar
- G10H3/186—Means for processing the signal picked up from the strings
- G10H3/188—Means for processing the signal picked up from the strings for converting the signal to digital format
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/20—Input arrangements for video game devices
- A63F13/24—Constructional details thereof, e.g. game controllers with detachable joystick handles
- A63F13/245—Constructional details thereof, e.g. game controllers with detachable joystick handles specially adapted to a particular type of game, e.g. steering wheels
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/53—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
- A63F13/533—Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game for prompting the player, e.g. by displaying a game menu
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
- A63F13/63—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor by the player, e.g. authoring using a level editor
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/70—Game security or game management aspects
- A63F13/79—Game security or game management aspects involving player-related data, e.g. identities, accounts, preferences or play histories
- A63F13/798—Game security or game management aspects involving player-related data, e.g. identities, accounts, preferences or play histories for assessing skills or for ranking players, e.g. for generating a hall of fame
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/60—Methods for processing data by generating or executing the game program
- A63F2300/6063—Methods for processing data by generating or executing the game program for sound processing
- A63F2300/6072—Methods for processing data by generating or executing the game program for sound processing of an input signal, e.g. pitch and rhythm extraction, voice recognition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2220/00—Input/output interfacing specifically adapted for electrophonic musical tools or instruments
- G10H2220/135—Musical aspects of games or videogames; Musical instrument-shaped game input interfaces
Definitions
- the present invention relates to methods of obtaining user input.
- a file menu often includes selectable menu items for saving a file, opening a new file, closing a file, printing a file or exiting the text editor and the user may make a selection using a keyboard or a mouse.
- a user interface on a mobile phone or on a lift interface may include an array of buttons. A user may enter a telephone number, or select a floor they wish to go to by pressing one or more buttons on the interface panel.
- a controller in a gaming system may include buttons and a joystick which allow the user to select a menu item or use an item within the game.
- the musical audio signal may be an analogue audio signal such as from a pickup on a guitar, or a microphone external to the instrument, which may then be digitised and processed by a sound card, or a digital signal may be provided such as via a MIDI interface.
- Such systems can often process pre-recorded music files (.wav, .mp3 etc).
- Examples of such software systems include music composition software, in which a user can play a note or a piece of music on the instrument, and the software receives this musical audio input, attempts to recognise the played note or notes, and then records and displays the recognised note or notes on the screen. This facilitates creation and transcription of music to produce a musical score which can then be saved.
- Such systems are typically far from perfect, and the user often has to edit the score to correct notes, add notes and add other musical information to obtain a finished score. This process is generally accomplished using conventional input means, such as a mouse and keyboard, for control of the software.
- a method for obtaining a user selection in a software system from a user with a musical instrument comprising:
- the present invention is suited to applications in software systems which analyse a user's musical input. Because the present invention allows a user to make selections (e.g. menu selections) using a musical instrument, it reduces or may even eliminate the need for other input devices to be used.
- the method may further comprise providing to the user representations of the associated one or more musical sounds. These may be provided by displaying on a display means a musical representation of the associated one or more musical sounds along with the selectable object.
- the one or more musical sounds may include musical notes or chords. They may be permanently or semi-permanently associated with a selectable object, or alternatively one or more musical sounds may be dynamically associated with a selectable object. A combination of these approaches may be used (ie some selectable objects have permanent associations, whilst others have dynamic associations).
- the method may further comprise associating a skill level with the user, and the skill level may be used in choosing the distinct one or more musical sounds to be associated with each of the plurality of selectable objects. Further the skill level may be used in determining if the received musical audio input corresponds to the one or more musical sounds associated with the selectable object.
- a system for obtaining a user selection in a software system from a user with a musical instrument comprising: a processor adapted to perform the method of the first aspect of the present invention.
- a memory in communication with the processor.
- a system for obtaining a user selection in a software system from a user with a musical instrument comprising: means for associating one or more musical sounds with each of a plurality of selectable objects in a software system;
- a computer usable medium encoded with data representing a computer program that can be used to direct a programmable device to perform the method of any one of claims 1 to 11.
- FIGURE 1 depicts the components of a system according to an embodiment of the present invention
- FIGURE 2A represents a display of a menu comprising selectable menu items having a textual representation of a note associated with each selectable menu item according to an embodiment of the present invention
- FIGURE 2B represents the menu in Figure 2A but with textual representations replaced with stave notation representations according to an embodiment of the present invention
- FIGURE 3 A represents a display of a menu with selectable menu items having a textual representation of a chord associated with each selectable menu item according to an embodiment of the present invention
- FIGURE 3 B represents the menu in Figure 3 A showing guitar tablature representations for the same chords in Figure 3 A according to an embodiment of the present invention
- FIGURE 3C represents the menu in Figure 3A showing stave notation representations for the same chords in Figure 3 A according to an embodiment of the present invention
- FIGURE 4 represents a display of a menu with selectable menu items having a stave notation representation of a musical phrase associated with each selectable menu item according to an embodiment of the present invention
- FIGURE 5 represents a display of menu with selectable menu items having a textual representation of a note associated with selectable menu navigation controls for selecting an item in a list according to an embodiment of the present invention
- FIGURE 6 is a flowchart of method for obtaining a user selection in a software system from a user with a musical instrument an embodiment of the present invention.
- FIGURE 7 represents a display of a menu with selectable menu items having a textual representation of a chord associated with each selectable menu item according to an embodiment of the present invention.
- an embodiment of the present invention is described below with reference to an interactive gaming system in which the user plays a guitar.
- the present invention may be utilised for many different instruments, including other string instruments, pianos/keyboards, wind instruments, vocals, etc, where it is possible to obtain musical audio input.
- the present invention may be utilised for a variety of software based systems, including interactive gaming systems, interactive tutoring systems, karaoke systems, and musical composition systems.
- Figure 1 depicts a system 100 according to an embodiment of the present invention.
- the system 100 comprises a microphone 120 for receiving musical audio input from a guitar 110, and an interface device 130, including a processor.
- the system 100 also comprises a video screen 140 as well as a pair of speakers 150.
- the interface device 130 may be a personal computer or a game console. It may a central processing unit (CPU), containing an Input/Output Interface for communication with input and output devices, an Arithmetic and Logic Unit (ALU) and a Control Unit and Program Counter element, and a memory in communication with the CPU.
- CPU central processing unit
- ALU Arithmetic and Logic Unit
- Control Unit and Program Counter element a memory in communication with the CPU.
- the software system running on the interface device 130 could take a wide variety of forms. For example, it could be a musical game, or a tutorial for teaching the user to play the guitar 100. Alternatively, the present invention could be used to allow the user to interact with non- musical software (e.g. the operating system) running on the interface device 130.
- the video screen 140 can display video output from the software system, and the speakers 150 can provide audio output.
- the system 100 may also comprise conventional input devices, such as a mouse or keyboard, or a game console controller.
- Such software systems are interactive, in that they require the user of the software to navigate various menus, or select various selectable items, in order to set parameters or to allow the user to interact in a desired way.
- the user may simulate participation in a musical performance (either simulated or pre-recorded). The user may choose which instrument they wish play, and the song they wish to hear and perform. They may save or restore their game, or make a variety of other selections. In some instances, they may select their skill level - for example, in the system described in this embodiment, the user can select their skill level as being "beginner", “intermediate”, "advanced", "pro” or master” level. This skill level may then be used to assess the quality of the user's performance, with beginner level being the easiest assessment and master being the strictest assessment of the quality of the user's performance.
- Such software based systems use menus with selectable menu items, or icons or other selectable objects displayed on the user interface.
- the user will thus be presented with a plurality of selectable objects in the software system.
- the user can then select a desired object (which corresponds to a setting or method of playing or otherwise interacting with the software), and the software system will receive and process this selection.
- the user of a musical instrument may utilise the musical instrument to make a selection.
- This may be achieved by the software system associating a representation of one or more musical sounds with each of the plurality of selectable objects in a software system.
- Figure 2A shows a display 200 according to an embodiment of the present invention.
- the display includes a menu 210 with three selectable menu items (options 1, 2 and 3), which are labelled "New Game” 220, "Load Game” 230, and "Back” 240.
- Option 1 corresponds to opening a new game file
- option 2 corresponds to loading a pre-existing game file
- option 3 allows the user to go back to a previous menu.
- the software system associates a note with each menu item (selectable object). In the present example the software system has associated the note “E” with option 1 (new game), the note “B” with option 2 (load game), and the note “G” with option 3 (back).
- the menu 210 illustrated in Figure 2A includes a textual representation of the notes, with notes E, B and G represented by the text phrases "Play E String” 222, "Play B String” 232, "Play G String” 242. Other representations of the notes are possible.
- Figure 2B shows another embodiment of the menu in Figure 2A but using stave notation representations of the musical notes E 224, B 234 and G 244.
- the user decides which object they wish to select. For example they may wish to start a new game, in which case they play the E string on their guitar 110.
- the microphone 120 converts the played note into a musical audio input.
- the system 130 receives this musical audio input from the user.
- a processor in the system 130 processes the received musical audio input and compares the received musical audio input signal from the user with an internal representation of the notes E, B and G. In the example, the system identifies that the played note corresponds to the note E, and thus proceeds to select a new game (option 1 ).
- the above method for obtaining a user selection in a software system from a user with a musical instrument is presented as a flowchart 600 in Figure 6.
- the flowchart 600 includes an optional step of associating a skill level associated with the user 610.
- a default value (such as "beginner") may be associated or assigned to the user.
- This information may be stored with the game file, and may also be stored in the memory of the system interface device 130, in another file on a storage means associated with the system interface device 130.
- the processor may receive the information by accessing the appropriate section of memory or accessing the game file or other file storing this information. This may be performed at the start of a gaming session, such as when a game is reloaded, or at a later time in the session as required.
- the method of the invention may be used to allow the user to select their skill level.
- the method comprises the step of associating one or more musical sounds with each of a plurality of selectable objects in a software system. This may be performed as an initialisation step when loading the game and be performed for all possible selectable objects in the software system. It may be a pre-set condition of the game - e.g. the top menu item always corresponds to an A note, the second always corresponds to a B note, etc. In such cases, it may not be necessary to display the musical sound to the user on the display, along with the menu items - this information may be contained within a game manual or other documentation separate to the software system.
- the allocation of musical sounds to selectable objects may be done on a once- off, as needed basis during a gaming session. That is, the first time a selectable objection is to be provided to the user, the software system associates one or more musical sounds with the selectable object, and uses this for the rest of the session.
- the one or more musical sounds is permanently or semi-permanently associated with a selectable object.
- association could be performed dynamically, so that the step of associating one or more musical sounds with the selectable object is performed for each instance of the selectable object.
- selectable objects presented to the user at the same time will need to be associated with distinct combinations of one or more musical sounds to allow distinct selection of the objects. This does not prevent two objects having the same associated one or more musical sounds provided they are not presented to the user at the same time or that they are not in the same musical order (eg playing E-A-G is distinct from playing G-E-A).
- a combination of these approaches may be utilised.
- some selectable objects such as those associated with frequently used tasks, basic navigation, or initial menus (e.g. opening a file, loading or file, going back or exiting) could use permanent or semi-permanent associations. Often these would be single notes or simple chords to facilitate ease of use by all users independent of their skill level, or alternatively to allow game loading and/or rapid menu navigation. Dynamic association could be provided for more items or selectable objects used in more specific areas of the software application, such as a list of songs, selecting skill level etc.
- a skill level may be associated with a user.
- the skill level may be used in choosing the one or more musical sounds to be associated with each of the plurality of selectable objects.
- a selectable object could have a more difficult combination of notes or chords associated it when the user's skill level is "master” as compared to when the user's skill level is "beginner”.
- This also illustrates one example of how associations may be semi-permanent - they may be permanent for a user provided they don't change their skill level. Changing the skill level could then trigger reassociating of musical sounds with selectable objects.
- the step of associating one or more musical sounds with each of a plurality of selectable objects in a software system may be performed well prior to obtaining a user selection, or as part of the dynamic process of obtaining a user selection.
- the system proceeds at step 630 to provide to the user with representations of at least two of the plurality of selectable objects, and (optionally) representations of the associated one or more musical sounds. In many cases this comprises displaying on a display means a musical representation of the associated one or more musical sounds along with the selectable object. This thus provides information to the user on how they may make a selection. The user can then make their choice, and play the appropriate musical sounds indicated by the musical representation.
- the system receives musical audio input from the user.
- this may be an analogue signal from the microphone 120, representing the musical notes played by the user, which is provided to the system interface.
- the system interface may have a sound card, analogue to digital converter, or other appropriate hardware and software which can produce a digital representation of the signal.
- the musical audio input could be a digital signal.
- such hardware may be provided on the instrument and then sent via a wired or wireless link to the system interface device.
- the instrument may provide a MIDI output which can be connected to the system interface device.
- the system interface device may contain circuits and/or hardware for receiving and processing the musical audio input as an analogue signal, without performing any digitisation of the signal.
- the objects to be selected and the associated musical representations will be visually displayed on an electronic display device attached to the computer implementing the method such as an LCD monitor, TV screen, or video projector.
- the system could be adapted for use without a display device, such as for use by vision impaired persons.
- the representations need not be displayed on a display device, but may instead be displayed in a physical form such as printed on large print or in Braille alphabet on a card or other apparatus capable of providing a Braille representation of the musical representation and associated selectable object.
- the system could also provide the selectable items, and associated musical representations, using aural or spoken word representations (this may be done in conjunction with or without a display device).
- a speech synthesiser could be included in the software system or by pre-recording spoken word tracks, which describe the selectable object and the one or more musical sounds which is associated with it.
- This embodiment could be utilised both for systems including an electronic display device, and those without.
- the musical representation of one or more musical sounds can be provided using any of a variety of musical notation schemes, with the choice depending upon the application, skill level of the user or other system design considerations.
- the musical representation may be textual instructions, such as "play an A" or "play an E”. Alternatively common (or uncommon) musical notation schemes may be used.
- the tablature is defined by a 6 x 6 grid.
- the 6 vertical lines corresponding to the 6 strings, with the left most line corresponding to the sixth string or low E and the right most vertical line corresponding to the first string or high E (thinnest string).
- the 6 horizontal lines correspond to the first 6 guitar frets.
- the 5th fret on one string is the same note as the next open string (e.g. the 5th-fret note on the lower E or sixth string is the same note as the open A string or fifth string).
- chords which include three notes, with the first two notes being separated by major third, and the next two notes being separated by a minor third.
- C Major chord is formed from the notes C-E-G.
- Such chords define a major triad of notes.
- chords may be produced such as minor triads (major third lowered by a half step), augmented triads (augmented fifth rather than a perfect fifth), and diminished triads (diminished fifth instead of a perfect fifth).
- Other variants include inverted chords, seventh chords and power chords.
- chord notation symbols in which the first letter indicates the root of the chord (eg "C"), which may then be modified by placing additional symbols after the chord, such as -, "m", “mi” or “min” for minor triad, a “+” or “aug” for an augmented triad, and a degree symbol “°”or “dim” for a diminished triad.
- a combination of musical sounds may be combined to produce a musical phrase.
- the representation scheme will indicate timing of notes, chords, and any gaps between then.
- the term musical phrase is not a precise term, although it typically represents a short piece of music (e.g. several bars). This may correspond to a line in a song, or length of time that the player can play without having to pause to take another breath or rest, but there is no strict definition (e.g. 4 bars long).
- FIG. 2 A shows textual representation of three musical notes, E, B and G
- Figure 2B shows the same notes in stave notation.
- the musical representation 224 corresponds to an "E" note (semibreve)
- musical representation 234 corresponds to a "B” note
- musical representation 244 corresponds to a "G” note.
- Figure 3 A represents display 300 of a menu 310 with selectable menu items ("option 1" 320, "option 2" 330 and “option 3” 340) having a textual representation of a chord associated with each selectable menu item according to an embodiment of the present invention.
- Options 1, 2 and 3 have "C Major Chord” 322, "A Major Chord” 332 and "G Major Chord” 342 associated with them.
- Figure 3B represents the menu in Figure 3A showing guitar chord representations for the same chords in Figure 3A.
- the musical representation 324 corresponds to a "C Major Chord” using tablature representation
- musical representation 334 corresponds to an "A Major Chord” using tablature representation
- musical representation 344 corresponds to a "G Major Chord” using tablature representation.
- Figure 3C represents the menu in Figure 3A showing stave notation representations for the same chords in Figure 3A.
- the musical representation 326 corresponds to a "C Major Chord” using a conventional stave notation representation
- musical representation 336 corresponds to an "A Major Chord” using a conventional stave notation representation
- musical representation 346 corresponds to a "G Major Chord” using a conventional stave notation representation.
- Figure 4 represents display 400 of a menu 410 with selectable menu items ("option 1" 420, “option 2" 430 and “option 3” 440) having a stave notation representation of a musical phrase associated with each selectable menu item according to an embodiment of the present invention.
- Options 1, 2 and 3 have musical phrases for the opening melody of notes for "Twinkle Twinkle Little Star” 422, the textual representation of the chord sequence for the start of the chorus in the song “Eye of the Tiger” by the band Survivor 432, and the opening notes of "Smoke on the Water” by Deep Purple 442.
- Figure 4 illustrates how different musical representations may be combined on the one screen. It should also be noted that selectable objects may allow a user to navigate a list or move around a screen.
- Figure 5 represents a display 500 of menu 510 with selectable menu items Options 1 to 8 indicated by 511 through 518.
- a control panel 530 includes textual representations of a note associated with selectable menu navigation controls for selecting an item in a list according to an embodiment of the present invention.
- the "Up” control 533 has the “E” note associated with it, and a textual representation of "Play E String” 532.
- the "Down” control 535 has the "B” note associated with it, and a textual representation of "Play B String” 534.
- Using the up and down controls moves the selection up and down the list. The current selection is indicated by highlighted box 520.
- Figure 7 depicts yet another example of a display 700 with selectable menu items 710 ("option 1" 711, “option 2" 712, “option 3” 713, “option 4" 714, "option 5" 715, "option 6" 716, “option 7” 717).
- the representations of the musical sounds are fairly basic - each of the selectable menu items 10 as a letter next to it, ranging from A (option 1) to G (option 7).
- a user can select one of the menu items by playing a note corresponding to the relevant letter next to the menu item - e.g. option 1 can be selected by playing an A note.
- the method then comprises the step 650 of selecting one of the plurality of selectable objects if the received musical audio input corresponds to the one or more musical sounds associated with the selectable object.
- the user plays the one or more musical musical sounds associated with the selectable object they wish to select.
- the system interface device 130 then receives the musical audio input and must make a determination of which, if any, of the one or more musical sounds associated with each selectable object correspond to the received musical audio input. This may be performed using software, hardware, or a combination of both. For example, Fast Fourier Transform (FFT), wavelet, or other signal processing techniques could be used to make the determination. Such approaches would generally involve having an appropriate internal representation of the expected musical audio signal.
- FFT Fast Fourier Transform
- the user skill level can be used in determining if the received musical audio input corresponds to the one or more musical sounds associated with the selectable object. That is, the standard of assessment in determining if the played notes correspond to the notes or chords associated with a selectable object may be varied. For example, a low quality or low degree of correlation may be required for low skill levels (beginners) where as the quality of the played notes, including timing, pitch etc, may need to be higher for higher skill levels.
- the method has application in a variety of software based systems. These may be interactive gaming systems, interactive tutoring systems, or even in a karaoke system which could require a user to interact only via singing. Such systems can be made more interactive by varying the musical sounds associated with selectable items. This can increase user participation and interest and can also assist the user to learn notes and improve their musical skills.
- the method could also be used more widely. In a more unusual example one could even go so far as to integrate the system into a lift controller.
- the panel could be a touch screen or an array of buttons, with a note displayed next to each floor.
- a user could then select a floor by singing the note into a microphone associated with the lift controller rather than pressing the button on the panel (eg "doh” or C could be the first floor, “reh” or D could be the second floor).
- Many musical instruments are held using both hands.
- a further advantage of the system is that it allows the user to continue to control the software without the need to put down the instrument so that they provide inputs using traditional means such as a keyboard, mouse or game controller.
- processing may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.
- ASICs application specific integrated circuits
- DSPs digital signal processors
- DSPDs digital signal processing devices
- PLDs programmable logic devices
- FPGAs field programmable gate arrays
- processors controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.
- Software modules also known as computer programs, computer codes, or instructions, may contain a number a number of source code or object code segments or instructions, and may reside in any computer readable medium such as a RAM memory, flash memory, ROM memory, EPROM memory, registers, hard disk, a removable disk, a CD-ROM, a DVD-ROM or any other form of computer readable medium.
- the computer readable medium may be integral to the processor.
- the processor and the computer readable medium may reside in an ASIC or related device.
- the software codes may be stored in a memory unit and executed by a processor.
- the memory unit may be implemented within the processor or external to the processor, in which case it can be communicatively coupled to the processor via various means as is known in the art.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- General Physics & Mathematics (AREA)
- Electrophonic Musical Instruments (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB1200014.7A GB2483046A (en) | 2009-07-03 | 2010-07-05 | Method of obtaining a user selection |
AU2010268695A AU2010268695A1 (en) | 2009-07-03 | 2010-07-05 | Method of obtaining a user selection |
US13/382,000 US20130005470A1 (en) | 2009-07-03 | 2010-07-05 | Method of obtaining a user selection |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2009903118A AU2009903118A0 (en) | 2009-07-03 | Method of obtaining a user selection | |
AU2009903118 | 2009-07-03 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011000059A1 true WO2011000059A1 (en) | 2011-01-06 |
Family
ID=43410384
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/AU2010/000857 WO2011000059A1 (en) | 2009-07-03 | 2010-07-05 | Method of obtaining a user selection |
Country Status (4)
Country | Link |
---|---|
US (1) | US20130005470A1 (en) |
AU (1) | AU2010268695A1 (en) |
GB (1) | GB2483046A (en) |
WO (1) | WO2011000059A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2787501A1 (en) * | 2013-04-05 | 2014-10-08 | Robert Bosch Gmbh | Musical instrument and apparatus for the remote control of an event in the vicinity of a musical instrument |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AR092642A1 (en) * | 2012-09-24 | 2015-04-29 | Hitlab Inc | METHOD AND SYSTEM TO EVALUATE KARAOKE USERS |
US20170017481A1 (en) * | 2014-02-12 | 2017-01-19 | Nokia Technologies Oy | Method and apparatus for updating a firmware of an apparatus |
US9280960B1 (en) * | 2014-12-15 | 2016-03-08 | Amazon Technologies, Inc. | Navigating music using an index including musical symbols |
DE102015107166A1 (en) * | 2015-05-07 | 2016-11-10 | Learnfield GmbH | System with a musical instrument and a computing device |
US20160345184A1 (en) * | 2015-05-20 | 2016-11-24 | International Business Machines Corporation | Signal strength bookmarking for media content |
US20180366096A1 (en) * | 2017-06-15 | 2018-12-20 | Mark Glembin | System for music transcription |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6201178B1 (en) * | 1995-08-28 | 2001-03-13 | Jeff K. Shinsky | On-the-fly note generation and a musical instrument |
US6848996B2 (en) * | 2001-10-15 | 2005-02-01 | Igt | Gaming device with sound recording changes associated with player inputs |
US20080200224A1 (en) * | 2007-02-20 | 2008-08-21 | Gametank Inc. | Instrument Game System and Method |
WO2008157477A2 (en) * | 2007-06-14 | 2008-12-24 | Harmonix Music Systems, Inc. | Systems and methods for simulating a rock band experience |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6175070B1 (en) * | 2000-02-17 | 2001-01-16 | Musicplayground Inc. | System and method for variable music notation |
US6924425B2 (en) * | 2001-04-09 | 2005-08-02 | Namco Holding Corporation | Method and apparatus for storing a multipart audio performance with interactive playback |
US7223913B2 (en) * | 2001-07-18 | 2007-05-29 | Vmusicsystems, Inc. | Method and apparatus for sensing and displaying tablature associated with a stringed musical instrument |
US7806759B2 (en) * | 2004-05-14 | 2010-10-05 | Konami Digital Entertainment, Inc. | In-game interface with performance feedback |
US7521619B2 (en) * | 2006-04-19 | 2009-04-21 | Allegro Multimedia, Inc. | System and method of instructing musical notation for a stringed instrument |
US7842875B2 (en) * | 2007-10-19 | 2010-11-30 | Sony Computer Entertainment America Inc. | Scheme for providing audio effects for a musical instrument and for controlling images with same |
US8017857B2 (en) * | 2008-01-24 | 2011-09-13 | 745 Llc | Methods and apparatus for stringed controllers and/or instruments |
US20090264199A1 (en) * | 2008-04-16 | 2009-10-22 | Macedon Productions, Inc. | Using a Musical Instrument as a Video Game Controller |
US8308569B2 (en) * | 2008-05-12 | 2012-11-13 | Microsoft Corporation | Reward for resurrecting teammate in a multiplayer game |
US7910818B2 (en) * | 2008-12-03 | 2011-03-22 | Disney Enterprises, Inc. | System and method for providing an edutainment interface for musical instruments |
AU2010256342A1 (en) * | 2009-06-01 | 2012-01-19 | Starplayit Pty Ltd | Music display system |
US7893337B2 (en) * | 2009-06-10 | 2011-02-22 | Evan Lenz | System and method for learning music in a computer game |
-
2010
- 2010-07-05 WO PCT/AU2010/000857 patent/WO2011000059A1/en active Application Filing
- 2010-07-05 GB GB1200014.7A patent/GB2483046A/en not_active Withdrawn
- 2010-07-05 AU AU2010268695A patent/AU2010268695A1/en not_active Abandoned
- 2010-07-05 US US13/382,000 patent/US20130005470A1/en not_active Abandoned
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6201178B1 (en) * | 1995-08-28 | 2001-03-13 | Jeff K. Shinsky | On-the-fly note generation and a musical instrument |
US6848996B2 (en) * | 2001-10-15 | 2005-02-01 | Igt | Gaming device with sound recording changes associated with player inputs |
US20080200224A1 (en) * | 2007-02-20 | 2008-08-21 | Gametank Inc. | Instrument Game System and Method |
WO2008157477A2 (en) * | 2007-06-14 | 2008-12-24 | Harmonix Music Systems, Inc. | Systems and methods for simulating a rock band experience |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2787501A1 (en) * | 2013-04-05 | 2014-10-08 | Robert Bosch Gmbh | Musical instrument and apparatus for the remote control of an event in the vicinity of a musical instrument |
Also Published As
Publication number | Publication date |
---|---|
US20130005470A1 (en) | 2013-01-03 |
GB2483046A (en) | 2012-02-22 |
AU2010268695A1 (en) | 2012-02-02 |
GB201200014D0 (en) | 2012-02-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10614786B2 (en) | Musical chord identification, selection and playing method and means for physical and virtual musical instruments | |
US9333418B2 (en) | Music instruction system | |
US9218748B2 (en) | System and method for providing exercise in playing a music instrument | |
US20130005470A1 (en) | Method of obtaining a user selection | |
US6063994A (en) | Simulated string instrument using a keyboard | |
WO2017125006A1 (en) | Rhythm controllable method of electronic musical instrument, and improvement of karaoke thereof | |
US7479595B2 (en) | Method and system for processing music on a computer device | |
US20180268731A1 (en) | Musical Modification Method | |
US10002598B2 (en) | Selective pitch emulator for electrical stringed instruments | |
CN115712473A (en) | Music score display method and device, electronic equipment and storage medium | |
JP2005084069A (en) | Apparatus for practicing code | |
KR101131823B1 (en) | Apparatus and method for studying of chinese language | |
KR102545185B1 (en) | Apparatus for training and testing to improve cognitive auditory function by sensing, method and program of the same | |
KR102564399B1 (en) | Apparatus for improving auditory function that changes training difficulty according to training performance accuracy | |
US11030914B2 (en) | Learning device and method | |
JP6552065B2 (en) | Performance equipment | |
JP2017173640A (en) | Musical performance analyzer and program | |
Wyatt et al. | Ear training for the contemporary musician | |
JP4457199B2 (en) | Code display method | |
JP2007334169A (en) | Keystroke practice device and computer program for keystroke practice | |
Lui | A MUSIC TIMBRE SELF-TRAINING TOOL ON MOBILE DEVICE USING VOLUME NORMALIZED SIMPLIFIED SPECTRAL INFORMATION |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10793447 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010268695 Country of ref document: AU |
|
ENP | Entry into the national phase |
Ref document number: 1200014 Country of ref document: GB Kind code of ref document: A Free format text: PCT FILING DATE = 20100705 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1200014.7 Country of ref document: GB |
|
ENP | Entry into the national phase |
Ref document number: 2010268695 Country of ref document: AU Date of ref document: 20100705 Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13382000 Country of ref document: US |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 15/06/2012) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 10793447 Country of ref document: EP Kind code of ref document: A1 |