JP2005507095A - An interactive game that provides guidance on notation and instrument acquisition - Google Patents

Abstract

A music keyboard (150) is connected to the computer (100). The computer (100) implements a graphical user interface (350) for teaching the user how to play the musical instrument (150). A computer readable music file such as a MIDI file (400) is used to drive the creation of a game object that moves along the path (210i) from the origin to the keys (200i) of the virtual keyboard. In one form, when the user presses a key on the music keyboard (150) within a time window that the game object arrives at the corresponding key (200i) on the virtual keyboard, the user is presented with an audio display, a visual display, and / or Game score is given. In a more structured learning mode, the game can be played using selectable and progressively more difficult tasks that are learned as the user progresses.

Description

【０１７７】
そのポップアップメニュー内から、ユーザは、ただ（上述のような）升目をクリックするだけで、強調表示色、または両手についての運指番号を選択できる。ポップアップメニュー上でどのような選択がされても、それはクリックされたオブジェクトに適用される。強調表示は、オブジェクトのまわりの後光効果の形で行われるべきである。上述のポップアップメニュー上で示される色は、例示の目的のみのためである。加えて、オブジェクトに運指または強調表示が既に割当てられている場合、それらは、上に示すような「ｘ」ボタンをクリックすることによって解除可能である。
【０１７８】
３．注釈付け
ユーザはゲームスクリーン上にメモを付けることができる。これらの注釈は後に．ａｍｍファイルの一部になる。ゲームスクリーン上に一旦付けられた注釈は、必要に応じてファイルに記憶される（保存）。ユーザは注釈を追加および除去できる。注釈は吹き出しの形でスクリーン上に付けられる。選択される吹き出しの形状は多数ある。ゲームスクリーンに注釈を追加するには、ユーザはスクリーン上の空きスペースをクリックする。一旦クリックすると、ポップアップメニューが現われて、吹き出しの形状およびその矢印ポインタの方向を選択することができる。次に、テキストがテキスト編集ボックスを介して入力される。注釈はゲームスクリーンから除去できる。注釈を除去するには、既に既存の注釈をクリックし、次に、それに隣接して現われる「ｘ」ボタンをクリックしなければならない。このゲームはまた、既に入力された注釈の編集も可能にする。注釈を編集するには、削除と同様、ユーザは既に付けられた注釈をクリックし、次に、それに隣接して現われる「編集」ボタンをクリックする。注釈の編集は、その入力と同様に行なわれる。
【０１７９】
４．フリーズメニューオプション
フリーズメニューにおけるオプションの数は、ゲームモードに依存する。異なるゲームモードにおけるフリーズメニューオプションの好ましい利用可能性については、付録Ｂを参照されたい。フリーズメニューオプションを以下に列挙する。なお、これらのオプションのうちのかなりの数は、上述のセクションＣに規定された開始オプションのサブセットである。既に規定されたオプションの規定は、ここでは省略する。
【０１８０】
・運指：オン｜オフ
・オブジェクト：長｜短｜記譜法
・記譜法：臨時記号｜調号
・テーマ選択
・カラー：オン｜オフ
・トラック選択（範囲プレビュー、有効な鍵を含む）
・トラック用の楽器選択
・伴奏トラック選択
・伴奏トラック用の楽器選択
・選択されたトラックの移調
・メトロノーム：オン｜オフ
・テンポ選択
・叩き窓調整
・背景：オン｜オフ｜不透明度％
・小節：オン｜オフ
・オブジェクトガイド：オン｜オフ
・統計（オーバーレイ）：オン｜オフ
・開始鍵選択
・段階的テンポ調整：オン｜オフ
・段階的な叩き窓の閉鎖：オン｜オフ
・ランダムなカラー削除：オン｜オフ
・記譜法への変換：オン｜オフ
・ループソング：オン｜オフ
・歌詞：オン｜オフ
・ＳＦＸ音量および音楽の音量
・曲の自動保存：オン｜オフ
曲の自動保存機能をオンまたはオフにする。このオプションが有効である場合、曲は、変更がされているとして、以前に特定されたファイルに自動的に保存される。
【０１８１】
・曲に名前を付けて保存
ファイル名を入力するよう促す。現在の曲を．ａｍｍファイルに保存する。
【０１８２】
Ｆ．ゲームセットアップ
ゲームセットアップはメインメニューから呼び出し可能である。以下はそのオプションである。
【０１８３】
・メニューオプション：上級｜初心者
このメニューオプションは、ゲームにおけるメニューオプションの２つの異なるセット間で選択することを可能にする。上級セットは、このゲームを最大限に利用することを望み、最大限の柔軟性を必要としているユーザ向けに設計されており、一方、初心者セットは、平均的なユーザ、または子供にとってはあまりに複雑な多数のメニューオプションを排除している。
【０１８４】
・ガンマ補正
このオプションは、スクリーンのガンマ値の調整を可能にする。
【０１８５】
・Ｍｉｄｉイン選択
このメニューオプションは、ゲーム用のＭＩＤＩ入力ポートの選択を可能にする。このゲームは、システム上の可能なＭＩＤＩポートをすべて列挙し、それらを選択用に利用可能にする。
【０１８６】
・Ｍｉｄｉアウト選択
このメニューオプションは、ゲーム用のＭＩＤＩ出力ポートの選択を可能にする。このゲームは、システム上の可能なＭＩＤＩポートをすべて列挙し、それらを選択用に利用可能にする。
【０１８７】
・ゲーム細部レベル：低｜中｜高
ゲーム細部レベルは、ＣＰＵ集約型のゲーム視覚効果の一部を有効および無効にする。細部レベルが低い場合、低価格のコンピュータ上で、視覚効果および全体的な見た目の楽しさを犠牲にしてゲームを実行することが可能になる。
【０１８８】
・音楽およびＳＦＸの音量レベル
スライダの形をしたこれらのメニューオプションは、ゲームについての全体的なサウンドミキサー値を調整する。ＳＦＸ音量オプションは、ゲームで演奏されるすべての．ｗａｖサウンド（サウンドカードのＰＣＭチャンネル）の全体的な音量レベルの調整を可能にし、一方、音楽の音量は、内部のサウンドカードシンセサイザの全体的な音量レベルの調整を可能にする。双方のスライダには０％〜１００％の値が割当てられ得る。
【０１８９】
・鍵盤範囲の設定
これは、コンピュータに接続された鍵盤のサイズを判断するオプションである。ユーザは、ＭＩＤＩ鍵盤上の一番左の鍵と一番右の鍵とを押すよう求められ、ゲームは鍵盤のオクターブサイズを計算する。
【０１９０】
・ベロシティテスト
このオプションは、ユーザのＭＩＤＩ鍵盤をテストして、それがベロシティに対して感度がいいかどうかを判断できるようにする。ユーザは、ＭＩＤＩ鍵盤上のある鍵を２回、１回目はやさしく、２回目は強く叩くよう求められる。受信されたＭＩＤＩベロシティメッセージの値の差に基づいて、プログラムは、ユーザがタッチセンシティブ鍵盤を有しているかどうかを判断する。
【０１９１】
この発明を詳細に説明および図示してきたが、それは例示のみのためであって、限定のためと解釈されるべきでないことは明らかに理解される。この発明の精神および範囲は、添付された特許請求の範囲の用語によってのみ限定される。
【０１９２】
付録Ａ
ＣＤ−ＲＯＭ内容
以下は、この出願への付録として提出されたＣＤ−ＲＯＭの内容の説明である。
【０１９３】
このＣＤ−ＲＯＭは、以下のこのＣＤ−ＲＯＭからのディレクトリのプリントアウトに示すような複数のディレクトリおよびファイルを含む。
【０１９４】
大抵のファイルのファイル拡張説明は、それらの開け方を、当業者に適切に明記している。ファイルは、マイクロソフト・ワード、マイクロソフト・エクセル、またはアドベ・アクロバットリーダを使用して開くことができる。．ｖｓｄファイルは、ビジオを使用して読み込むことができる。
【０１９５】
自明ではないかもしれない他の任意のファイルを開くためのソフトウェアは、http://extsearch.com、または同様のファイル拡張ディレクトリ検索エンジンで見つけることができる。
【０１９６】
【表２】

【図面の簡単な説明】
【０１９７】
【図１】この発明のいくつかの局面を実行するためのハードウェアアーキテクチャのブロック図である。
【図２】仮想鍵盤、ゲームオブジェクトの経路、およびそれらと標準記譜法のト音記号との例示的な関係を示す図である。
【図３】この発明を実行するための例示的なハードウェアおよびソフトウェアモジュールのブロック図である。
【図４】ＭＩＤＩファイルのフォーマットを示す図である。
【図５】ＭＩＤＩファイルのヘッダチャンクのフォーマットを示す図である。
【図６】ＭＩＤＩファイルのトラックチャンクのフォーマットを示す図である。
【図７】ＭＩＤＩファイルのからのＭＥＴＡイベントのフォーマットを示す図である。
【図８】ＭＩＤＩデータのＭＴｒｋチャンクの一部および例示的な意味論の一例を示す図である。
【図９】３つの音の順次演奏のためのＭＩＤＩメッセージを示すデータの例示的な流れの図である。
【図１０】図５のＭＩＤＩデータストリームを解釈してゲームオブジェクトを表示用に生成するための例示的なプロセスのフローチャートである。
【図１１】この発明の一局面に従ったゲームモジュールの例示的な構成を示すブロック図である。
【図１２】この発明のさまざまな局面を実現するために使用可能な例示的なコンピュータのブロック図である。【Technical field】
[0001]
Copyright notice
At least a portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to reproduction by any person of this patent disclosure, while this patent disclosure remains in the Patent Office patent document or record, but in any other respect what copyright Also possess all.
[0002]
Cross-reference to CD-ROM appendix
This application includes (1) an appendix with a CD-ROM that contains specifications for the computer program used to implement the invention disclosed herein. The contents of the CD-ROM are described in more detail in Appendix Document A attached to this document. The contents of the CD-ROM are hereby incorporated by reference in their entirety as if fully set forth in the text of this application.
[0003]
Cross-reference with related applications
This application is a US Provisional Application Serial No. 60 / 347,554, filed October 20, 2001, "Interactive Game Providing Guidance for Reading Music Notation" (Inventor: Hal C. Salter). Salter)) and is related to the provisional application. The contents of the provisional application are hereby incorporated by reference in their entirety.
[0004]
Field of Invention
The present invention relates generally to interactive games, and more specifically, interprets or compiles MIDI (Musical Instrument Digital Interface) or other standard music files into interactive games that teach the user notation while playing the game. Related to the process.
[Background]
[0005]
background
MIDI technology has been a stable source of piano and music sequence programs for editing, printing, playing and composing music for nearly 20 years. Some early, relatively primitive music and piano practice programs and games have resulted from this technique. For example, several types of computer-assisted instruction, including piano instruction, have been used in the home market for several years.
DISCLOSURE OF THE INVENTION
[Problems to be solved by the invention]
[0006]
In the field of music instruction, computers are most often used to provide learners with an interactive display of notation, when a learner presses a key on an accompanying electronic piano, it takes notes. ) Is lit. Or the note lights up and the figure shows which key the learner should press. Other systems have a series of light or fingering instructions that attempt to tell the user which key to play next. Although these programs and systems are designed to allow learners to practice and obtain feedback, they are generally not fully engaged with music beginners, particularly with respect to rhythm. While some typing tutorials have been very successful using such methods, piano and music tutorials are equally successful because of some serious issues specific to that subject. Not.
[0007]
The main problem with these prior art learning systems is that they do not take into account some important differences between learning typing and learning music playing. The first and most obvious difference is that the notes must be played not only in the correct order but also in the rhythm pattern associated with the preceding and following notes. The correct timing of the notes is one of the most difficult aspects of any music teaching method. Until now, only a few applications have been able to deal with this issue satisfactorily and have been isolated from other concepts. The second challenge is that many programs start with notation to teach a user a song. Reading and understanding notation is like reading classical English Beowulf, and its origins are just as old. The staff notation originally represents the hands of a medieval monk and has been adapted to deal with all the musical innovations since.
[0008]
There is also the problem of notation oriented vertically, originally intended to convey the pitch of the speech (and thus the concept of elevation). However, on the piano, the pitch is experienced horizontally (ie, perpendicular to the notation-from left to right instead of low to high). This difference is an easy transition conceptually, but is a difficult transition visually and empirically for learners. This combined difficulty is such that it is estimated that less than 20% of all piano players can read the score fluently.
[0009]
As a result, for beginners, learning the piano has at least four additional challenges compared to learning typing. They must learn piano keys, learn notation, the notation is perpendicular to the piano, and they must strike the right key at the right time. The frustration of these simultaneous tasks abandons many users and makes work more like work than entertainment.
[0010]
Examples of prior patents in the general field of music teaching are outlined below. Each of these references is incorporated by reference in its entirety.
[0011]
U.S. Pat. No. 4,416,182 issued to Wise et al. Discloses a keyboard teaching device for self-study of keyboard instrument learners. This system allows the learner to correlate key positions on the instrument keyboard with notes positions on the scale. A keyboard having a plurality of keys corresponding to musical notes of the scale generates a first set of control signals in the initialization or setup mode, and generates a correlation signal between keys and notes in the instruction or game mode. There is at least one storage element that stores a predetermined combination of logic signals that provides a source for a pseudo-random sequence of one or more notes over a predetermined range in the game mode. The system generates a second set of control signals in response to activation of one or more keys in any game mode. A learner then checks the selection of one or more keys of his or her eyes with a mechanism for generating audio tones and displaying video images according to the first and second sets of control signals. Can do.
[0012]
U.S. Pat. No. 5,183,398 to Monte et al. Discloses a learner teaching apparatus and method that includes learner interactive teaching through a series of lesson frames. This disclosure provides a learner with a keyboard having a plurality of keys corresponding to musical notes of a scale, and generates a key related signal in response to each depressed or released key. A video display and associated audio tone generator allows the learner to respond to visually displayed images and audio tones by selecting one or more keys on the keyboard. The learner is presented with a lesson frame representing teaching activities that require a response on the keyboard by the learner. The learner's keyboard response is compared with the performance criteria and an absolute performance evaluation result is generated. The absolute performance evaluation result is compared with an acceptable achievement level for a particular teaching activity, and the next frame selection signal is generated. Based on the next frame selection signal, the next frame is selected and presented to the learner.
[0013]
U.S. Pat. No. 4,997,374 to Simone discloses a teaching device that includes a changeable two-channel pre-recorded program source and a console unit containing a workbook. The first channel of the program source contains an audio program containing a series of spoken words that are played to be heard by the console unit, and the second channel lights up adjacent to the pre-specified words in the workbook. It contains a series of control signals that act to activate it. The operation of the console unit is linked to the audio program so that light is activated adjacent to the same word as the workbook word is played as heard by the console unit. One embodiment of this apparatus further includes a plurality of depressible user response buttons on the console unit to indicate responses to problems presented in the audio program.
[0014]
U.S. Pat. No. 4,781,099, issued to Koike, presents a question code in sound and the trainee answers by pressing the keys of the notes that make up the code on the keyboard A quiz device is disclosed. This device can generate a plurality of different code data each representing a code, generates question code data selected somewhat randomly from these different code data one at a time, and is specified by the code data of the question Generate the notes that make up the chord. If the answer is correct, the points are added and the next question code is presented.
[0015]
US Pat. No. 5,392,682, granted to McCartney-Hoy, learns how to play a piano with a computerized music keyboard and piano The method is disclosed. This computerized music keyboard includes a piano keyboard connected to a computer. In order to play the selected music on the piano, the computer selects the music to be played on the piano from the music module, the correct keys to be played, the correct order in which those keys should be played, and Programmed to generate a signal indicating the hand and finger to be used to strike each key.
[0016]
U.S. Pat. No. 5,107,743 to Decker has a panel designed to fit snugly on an existing keyboard key and has two or more colors or mounted on the panel. Disclosed is a piano teaching aid in which light having a shape is arranged just above the keys of the piano to be played. To distinguish which hand the user uses to play the piano, the light has more than one color or shape. The panel also includes a finder window that displays alphanumeric codes corresponding to similar codes that appear adjacent to the score to be played. A foot pedal advancement mechanism is used so that the user can use the foot pedal to advance certain actions and adjust the speed at which the light displays the key to be struck. This display can also show the music to be played at a tempo set by the user, which automatically changes from action to action without the use of a pedal. The panel is connected to be stretched at one or more locations so that it can closely fit the various dimensions of the keyboard without disturbing the keys to be played.
[0017]
U.S. Pat. No. 4,331,062 to Rogers discloses an apparatus for visually displaying notes on a note display panel mounted on an electronic piano having a support arm. The electronic piano has a keyboard electrically coupled to an electronic circuit operable to generate audio output in response to one or more key presses. The arm is rotatable in a mount attached to the piano and moves about a first vertical axis. The first movement limiting unit attached to the mount and the arm limits the rotation of the arm around the piano. A panel and a second motion limiting unit attached to the arm limits the rotation of the panel about an axis perpendicular to the arm. One modified structure has an arm fixed to the piano. The panel has a grand staff index associated with a vertically arranged first light mechanism corresponding to chord note information and a second light mechanism corresponding obliquely to ascending note scale information. The keyboard display is placed below the staff index. A third light mechanism associated with the piano key index on the keyboard display, in conjunction with the second light mechanism, provides visual information about the key depressed on the electronic piano. An electrical circuit having an on / off switch electrically couples the electronic piano with the first, second and third lights so that the light can be selectively activated. The electronic circuit includes a switch assembly having a plurality of key switches operable to link the first, second, and third lights with the piano keyboard according to the key of the music being played.
[0018]
U.S. Pat. No. 4,366,741 to Titus is a microprocessor used to control a keyboard and a CRT device that provides video note display simultaneously with one or more key presses. An electronic piano having a connected electronic piano circuit is disclosed. A keyboard display located adjacent to the screen of the CRT device is associated with the light used to indicate the key being played. A manually operated adjustment knob works with the microprocessor to allow screen back erase one note at a time, delete all notes, keep all notes, sharp or flat for each note Indicates the mode and the period of time the key is depressed by stretching the note on the screen. A metronome unit is used with the microprocessor to provide a visual time signature marker on the screen that sequentially traverses the screen. A movable frame connects the CRT device to the piano.
[0019]
U.S. Pat. No. 5,864,868 to Contois discloses a computer system and method for controlling a media playback device. The system provides a user interface for allowing a user to access media works stored in a media database. This interface is also for controlling a media playback device, such as an auto-playing piano or movie playback video device, coupled to a computer for playing the accessed or selected piece of media. In one embodiment, there is a computer interface that allows a user to display only music associated with a selected category, such as jazz or classical music. In another example, the user can instruct the media playback device to automatically play the selected music work associated with the selected music category. In another example, the user can instruct the media playback device to automatically play the selected music piece associated with the selected composer or artist.
[0020]
U.S. Pat. No. 6,204,441, issued to Asahi et al., Uses a personal computer or game device to display music information, in particular a music software program, to record notes, time signatures and tempos. A technique for visual display is disclosed. This disclosure teaches the use of different colors and different brightness to identify certain notations. It has a display screen showing both bass and treble clefs and treble clefs, and keyboard and timing displays.
[0021]
US Pat. No. 6,388,181, issued to Moe, describes a key that is targeted by an animation in which each key to be struck within one beat of time is designated by a colored “sprite”. Figure 2 illustrates a computer graphics animation used in a live video interactive method for a user to play keyboard music while guiding his finger.
[0022]
US Pat. No. 6,066,791 to Renard et al. Teaches the performance of a musical instrument, displays an image on a display device, and preferably uses the musical instrument while paying attention to the image. A system for teaching learners to play notes on a staff is shown.
[0023]
US Pat. No. 5,540,132 issued to Hale shows a technique for teaching children notation. Each note is associated with a clearly identifiable color, which in turn is associated with an object that occurs naturally with that color. They use cartoon characters to clearly strengthen the association in the child's mind.
[0024]
U.S. Pat. No. 6,337,433, granted to Nishimoto et al., Has a plurality of display devices each associated with a performance element, each corresponding to a player's left hand and right hand, respectively. 1 shows an electronic musical instrument having a performance guide function including a pair of display elements, a performance guide method, and a storage medium for storing a program therefor.
[0025]
U.S. Pat. No. 6,284,961, issued to Kimmel, Jr., shows a system of musical notes associated with a certain color of the sound, so that the musical instrument corresponds to the color of the sound being played. A sticker to be attached to the key is used.
[Means for Solving the Problems]
[0026]
Summary of the Invention
The present invention relates to a hand-eye linked game having the purpose of teaching a person how to read notation and how to play a musical instrument such as a piano.
[0027]
The present invention simplifies the notation to a grid, the pitch moves from left (low) to right (high) in parallel with the arrangement of the piano keyboard, and the time dimension moves vertically.
[0028]
In one embodiment, at the beginning of the game, the performer will see a game object representing the musical notes rising from near the bottom of the screen toward the virtual piano keyboard. Their vertical relationship to each other is an indication of relative music time. As these game objects approach the virtual piano keyboard, the corresponding piano keys along the path become visually apparent. The player's purpose is to tap the corresponding key on the music keyboard at the very time when the game object is placed in the predetermined hit window (hit window) facing the virtual keyboard. If the performer strikes the correct key at the right time, audio and image feedback rewards the performer.
[0029]
The main task of the performer in this invention is to hit the corresponding piano keys for as many moving objects as possible at the right time, repeat the task until a certain percentage of accuracy is reached, and advance to the next difficulty level, and so on Learn to play songs unconsciously and understand and read the score at a later level.
[0030]
Objects, features, and advantages of the system of the present invention will become apparent from the following description.
BEST MODE FOR CARRYING OUT THE INVENTION
[0031]
Detailed Description of the Invention
FIG. 1 is a block diagram of a hardware architecture for carrying out some aspects of the present invention. FIG. 1 shows a computer 100 that includes a processing element (CPU) and a display. One typical implementation is a personal computer that is commercially available from a number of sources. Details of the interior of the computer 100 and its use in a network environment are shown in FIG.
[0032]
A piano keyboard 150, preferably having the ability to generate control data in accordance with the MIDI standard, is connected to the computer via a user standard MIDI interface.
[0033]
FIG. 2 is a diagram illustrating an example virtual piano keyboard, game object paths, and their example relationships to standard notation treble clefs. A virtual keyboard is an object that is shown on the screen of a computer display as part of a graphical user interface. The virtual keyboard includes a plurality of individual keys 200i corresponding to piano white keys and a plurality of keys 200j corresponding to piano black keys. For this reason, in this embodiment, the virtual keyboard represents a standard arrangement of keys on a keyboard such as a piano. Similarly associated with each key of the virtual keyboard is a path 210i corresponding to only the white key of the piano keyboard in this example. One of the white keys and the associated path is in this case a notation of an octave note in notation that starts at the center C on the left and goes up to C above the center C. It corresponds to each. The paths shown in bold correspond to the notes seen on the treble clef line in this example.
[0034]
From this figure, it is clear that the pitch increases as the scale rises from left to right as shown in this figure. However, in standard notation, the center C begins on the line below the first line of the standard treble clef shown in FIG. 2, and the C above the center C is between the top of the standard treble clef. Shown in For this reason, the pitch of the notation starts from a lower pitch value and proceeds higher in a treble clef from a lower pitch value to a higher pitch value. However, when viewed from the keyboard, the pitch increases from left to right rather than from bottom to top. As mentioned above, this difference in display creates at least one factor in the difficulty in learning to play the piano.
[0035]
As will be described in more detail below, the game object is introduced along the path 210i at a position facing the virtual keyboard. The game object then advances in an animated manner from near the bottom along the path to the point where the game object encounters the virtual keyboard key 200i. Crossing time is associated with each of the paths, and if three notes are introduced in sequence, they appear to appear at the bottom of the path in turn, and from bottom to top during the crossing time Advance, where they cross each key at different points in time.
[0036]
One of the purposes of this interactive game is that the learner presses a (preferably MIDI) key on the keyboard at substantially the same moment that the game object intersects the virtual key on the graphical user interface. is there. As the user plays the game, the user has to map the visual representation of the notes from the music file that generates the game object to the actual fingering position on the real keyboard interfaced with the virtual keyboard Develop seed ability. The game detects when the user has pressed the correct key at the right moment.
[0037]
FIG. 3 shows a block diagram of exemplary hardware and software modules for carrying out various aspects of the present invention. Although this particular embodiment of the invention will be described with respect to the use of MIDI files and music formatted according to the MIDI standard, other formats for music are well known and used in alternative embodiments of the invention Is possible.
[0038]
In this exemplary embodiment, one or more files 300 formatted in MIDI format are stored, for example, in a library, or downloaded in real time and provided to interpreter 310, where the MIDI data file is sent to the command. Interpreted and converted. The command drives a game object generation module 320 that provides a display of the game object on a graphical user interface on a display screen and provides audio output 370 to a computer speaker. Graphics engine 350 and audio engine 360 conveniently simplify the conversion of game objects into audio and visual components that drive visual display and audio output. The game object generator has access to a library 330 of objects and to a set of game control parameters 340. Each of these modules is described in more detail below.
[0039]
FIG. 4 shows the format of the MIDI file. As described above, the MIDI file format is used to explain the present invention, but the present invention is not limited thereto. An example of another file suitable for carrying out the invention is an MPEG-4 file.
[0040]
Understanding the operation of MIDI files and the MIDI protocol will greatly simplify the understanding of the present invention.
[0041]
MIDI file format
A MIDI file is composed of chunks. Each chunk consists of a 4-byte chunk type and a 4-byte display indicating the length of the bytes contained in the data field.
[0042]
There are two types of chunks: a header chunk (410) (with a chunk type of “MThd”) and a track chunk (420) with a chunk type of “MTrk”.
[0043]
A MIDI file consists of a single header chunk followed by one or more track chunks.
[0044]
FIG. 5 shows the format of the header chunk of the MIDI file. The header chunk contains three fields: chunk type, length, and data. For header chunks, the chunk type is MThd. The length field contains the length of the chunk data portion expressed in bytes. The “format” portion of the data field includes a MIDI file format that can only be format 0, format 1 and format 2. The “tracks” portion of the data field is a binary number indicating the number of track chunks included in the MIDI file. The “division” part of the data field of the header chunk defines the default unit of “delta-time” for the MIDI file. If the most significant bit of the “division” field is 0, the remaining 15 bits indicate the number of “ticks” that should be used to represent and play the music. If the most significant bit is a logic one, there are two components indicated by the remaining 15 bits. Bits 8-14 indicate the number of frames per second (shown as negative numbers) and the least significant 8 bits represent the number of ticks per SMTPE frame.
[0045]
There are three variants of MIDI files. Format 0 contains a single track. Format 1 includes one or more tracks that are all played simultaneously. Format 2 includes one or more independent tracks that can be played or played independently of the others.
[0046]
FIG. 6 shows the format of the track chunk of the MIDI file. It includes a chunk type that is by definition MTrk type and a length field that indicates the length of the data portion of the track chunk. The data portion of the track chunk includes two elements. The first is the “delta_time” part and the “event” part. delta_time is the number of “chicks” from the previous event, expressed as a variable length quantity. There are three types of events defined within the standard. They are “MIDI event”, “SYSEX event”, and “META event”. There is no explicit delimiter between the “delta_time” instance and the “event” instance. This is possible because both fields have a well-defined length.
[0047]
A MIDI event has an arbitrary MIDI channel message. These include channel voice messages and channel mode messages. Messages other than the MIDI channel message to be included in the MIDI file can use the SYSEX event.
[0048]
Most system exclusive messages are fairly simple and are sent as a single packet of bytes starting with F0 and ending with F7. However, there are also system exclusive messages that are used to control device parameters in real time. To deal with different uses, two different types of SYSEX events are defined.
[0049]
META events are used for things such as track names, lyrics and cue points that do not result in MIDI messages being sent, but are still a useful component of MIDI files.
[0050]
A MIDI event has the general form shown in FIG. A MIDI event is preceded by a hexadecimal notation FF, followed by a type field, a length field, and a data field. The type field is a single byte that identifies the type of META event. The length field contains a number of bytes of data following the field. The data field contains 0 bytes or more data.
[0051]
A number of META events have been defined in standards that make it easier to implement the present invention. These include:
[0052]
The “sequence number” is an optional event that should only occur at the start of a track before any delta_time that is not zero.
[0053]
This is usually used to identify each track.
[0054]
Text events are used to annotate tracks with arbitrary text. The copyright display event is available when the copyright display is expressed in ACSI text. It should be the first event in the first track of the MIDI file.
[0055]
The sequence / track name provides the name of the sequence or track in the file. The instrument name provides a description of the instrument used on the track. Lyrics events provide song lyrics. Normally, each syllable has its own lyric event, which occurs when the lyrics are to be sung.
[0056]
Marker events mark important points in the sequence, such as the beginning of a poem.
[0057]
Cue points are used to include cues for events that occur on the stage, such as “raise the curtain” and “leave”.
[0058]
End-of-track events must be used to give the track a well-defined length. This is essential if a track is looped or chained with another track.
[0059]
The set tempo event sets the tempo in microseconds per quarter note. This means a change in the unit length of delta_time ticks. The default tempo is 120 beats / minute. The SMTPE offset event specifies the SMTPE time at which the track should start. The time signature represents a standard time signature for a song, such as 3/4, 6/8, or 2/2. A key event can specify a sharp or flat number and a major or minor flag. Finally, the sequence specific META event allows the manufacturer to incorporate sequencer specific instructions into the MIDI file.
[0060]
FIG. 7 shows the format of a META event from a MIDI file.
[0061]
FIG. 8 is an example of a portion of an MTrk chunk of MIDI data and an exemplary semantics. As shown in FIG. 8, the MTrk chunk starts with an MTrk identifier 800 followed by a length field 810. This is followed by a plurality of ordered pairs 820-1 through 820-j of the delta_time field and the event field, which represent the individual MIDI messages associated with this MTrk chunk.
[0062]
The MIDI protocol consists of messages designed to allow the synthesizer and sequencer to communicate information about which sound to play. A typical MIDI message includes three components. The first component starts with hexadecimal 9 followed by the identification of one of the 16 MIDI channels with a value of 0-F. The second component is a two byte sequence representing the pressed key on the device. In the case of a piano type keyboard, this corresponds to the sound (note) of the pressed keyboard. This 2-byte value ranges from 00-7F. The third component of the MIDI message is a velocity component that identifies the velocity that has been pressed or released. Its value also ranges from 00 to 7F. Thus, if the performer presses the center C key on the keyboard, the keyboard will send a “note on” message containing 90 3C 40. When the performer releases the key, the corresponding “note off” message will contain 80 3C 33. In this case, the key is released more slowly than when it was pressed, as indicated by the velocity message 33 in the release message (compared to the attack velocity 40).
[0063]
Other MIDI messages may include program (instrument) changes, pitch blend messages, control change messages (eg, pedal / foot switch state changes), and timing clock messages.
[0064]
All MIDI messages are unidirectional. There is no confirmation message sent back from the receiver to the transmitter. If the MIDI device does not know what to do with the message, it ignores the message. A MIDI message specific to a MIDI channel is called a channel message. MIDI messages that affect the entire MIDI system or the entire MIDI device are known as system messages. Channel messages and system messages are further divided into several classes. A channel voice message is a message that starts, changes, or stops the sound being played. Channel mode messages affect the entire channel. System real-time messages are used by sequencers to adjust timing and synchronize. They do not contain data bytes. The system common message includes messages such as song position pointer and song select. System exclusive messages are typically used for device-specific extensions to the MIDI protocol.
[0065]
Since MIDI messages are sent and interpreted in real time, it is desirable to reduce the amount of data that must be sent. For normal note-on and note-off messages, it is quite common for some sounds to turn off and on almost simultaneously. In such a case, it is possible to send a single status command such as note-on and subsequent “sound identifier, velocity” pairs without having to repeat the same status byte for each note that is turned on or off. . This reduction in status byte transmission is known as the “running status” in the MIDI message. Only the data bytes to be changed are sent.
[0066]
With the background of MIDI files and protocols, the exemplary operation of the present invention will now be described.
[0067]
FIG. 9 shows an exemplary flow of data describing a MIDI message for the sequential performance of three sounds. In FIG. 9, two MIDI messages are shown for each line. Each begins with a delta_time field. For this reason, the six messages shown in FIG. 9 are preceded by a delta_time increment of 00, 08, 10, 18, 20, and 28.
[0068]
The delta_time increment is followed by a MIDI command sequence. Sequence 92 shows the situation where a sound should be on. Following each 92 message is a message with 82 command fields indicating that the sound should be turned off. Following each command is a sound identifier, which indicates that sound C is turned on and off in the first line. In the second and third lines, the identified sounds are sound E and sound G, respectively. The last component of each message is the velocity component. In the first line, velocity 44 is a measure of how quickly the key is pressed downward (eg, attack velocity). In the next message that the key is being released, velocity 40 indicates that the key is released at velocity 40.
[0069]
FIG. 10 is a flowchart of an exemplary process for interpreting the MIDI data stream of FIG. 5 as part of generating a game object. When a MIDI message representing a note-on object of the form <delta_time><Command><Note><Velocity> is received (1000), a MIDI message representing a note-off property of a similar format optionally follows (1010). If <delta_time> is less than the traversal time, the object is placed on the <Note> path with an optional length property of <delta_time (off)>-<delta_time (on)> ticks (1020). In this form, it can be seen that steps 1000-1020 represent the interpretation of the input MIDI message followed by the creation of the game object. According to steps 1000-1020, messages are continuously received and interpreted. Within the time unit τ of collision with the virtual keyboard, a window is defined (1030).
[0070]
Once the object is thrown onto the note path, the object is moved along the path toward the virtual keyboard every N ticks of the MIDI clock (1040). Video and audio reward presentation (routine) if a MIDI keyboard input is received while the window is open and if the MIDI keyboard input value is equal to the MIDI note value of the object that is about to collide with the virtual keyboard ) Is actuated (1060). In other cases, video and audio sequences representing failure may be activated. This process is repeated iteratively for each tick of the MIDI clock, as indicated by the arrow back to step 1040.
[0071]
Returning to FIG. 3, item 330 represents a library of objects, which allows a game to be modified for different tastes of the corresponding game by replacing various types of objects. Examples of the types of objects used are given below. Also shown in FIG. 3 is a set of parameters 340 that allow the user to customize game behavior or game behavior in an instance.
[0072]
Audio engine 360 is preferably based on the Microsoft® DirectSound 8 application programming interface. Audio engines are generally short via the PCM channel of the sound card. Ability to play WAV files. The sound engine is a constructor, destructor,. Load WAV file, and. This is realized as a class having WAV file playback. The sound engine is primarily used to play menu buttons, clicks, switches, and sound effects associated with gameplay feedback sounds. However, the present invention can be operated with other software engines.
[0073]
The graphics engine 350 operates in conjunction with the object library 330 and the game object generation module 320 to generate graphical objects on the screen that make up each implementation of the game objects used to play the game. The present invention provides three types of song files: MIDI, MIDI karaoke, and. amm can be loaded and played back. The latter is an internal song format that will be further described below. A standard MIDI file has been described above. A MIDI karaoke file is essentially a standard MIDI file in which lyrics are embedded in one of the tracks according to a certain format. This invention is a mainstream file format in the shareware / public domain market. The KAR file format is supported.
[0074]
. The amm file is very similar in structure to a standard type 1 MIDI file. However,. The amm format file also holds annotations, highlighting, and fingering. An annotation is a simple text memo stored on the timeline of a song. These annotations can hold information valuable to the user and can be displayed along with other scrolling game objects during game play. They may be stored as text in an internal file, but on the screen they are shown as balloons with or without arrows pointing to other game objects.
[0075]
. Some notes in the amm file can be highlighted. This is essentially a flag that tells the system that a note should be displayed with an enhanced video effect around it, such as an afterglow effect. Fingering is a little hint to the user as to which finger should be used to play a note on the piano keyboard. Fingerings are assigned to individual notes and hold numbers ranging from 1 to 5. During game play, game objects assigned fingering scroll with numbers placed on them.
[0076]
FIG. 11 is a block diagram showing an exemplary configuration of a game module according to one aspect of the present invention.
[0077]
When the game is first loaded, a splash screen 1100 is displayed. It is centrally located on the desktop and allows the user to click on an icon or link that takes the user to a sponsor company website where the user can download new songs for the game. There may also be a link to HTML documentation for the game.
[0078]
The welcome screen 1110 provides four options. First, it allows selection of the main menu described below. Second, it allows access to the immediate play mode of operation, assuming all pre-set settings of the last song played in the game, described below with respect to item 1140. A third option from the main menu takes the person to a credit screen where game credits are visually apparent. Finally, the main menu 1120 has an option to exit the game and return to the operating system.
[0079]
There are three options from the main menu 1120. Once the quick play option is selected, the user goes directly to the quick play mode described with respect to 1140, where the user is free to load any song, select a number of options, and begin practicing. . There is no need to select or create a profile. In the carrier mode, the second main menu option, the user creates a profile and faces a number of predefined challenges within a number of levels and in a number of different piano methods. These are described in detail below with respect to the carrier mode 1150.
[0080]
A game setup 1130 may also be selected from the main menu. In game setup, a number of game objects can be set, as will be described further below.
[0081]
Immediate play 1140 allows the user to play the game without the hassle of going through an activation option. The previous start option parameter is assumed, including the last song played, while the default song is assumed if the song has not been played since the game was first installed. Immediate play has no menu, it is simply a shortcut to game play.
[0082]
FIG. 12 is a block diagram illustrating details of an exemplary computer that may be used to implement various aspects of the invention. The description of the invention that follows is exemplary. However, it should be clearly understood that the invention may be practiced without the specific details described herein. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention. At least a portion of the present invention is intended to be implemented on or through a network such as the Internet. An example of such a network is also described in FIG.
[0083]
FIG. 12 is a block diagram that illustrates a computer system 100 upon which an embodiment of the invention may be implemented. Computer system 100 includes a bus 102 or other communication mechanism for communicating information, and a processor 104 coupled with bus 102 for processing information. Computer system 100 also includes a main memory 106, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 102 for storing instructions and information to be executed by processor 104. Main memory 106 may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by processor 104. Computer system 100 further includes a read only memory (ROM) 108 or other static storage device coupled to bus 102 for storing instructions and static information for processor 104. A storage device 110, such as a magnetic disk or optical disk, is provided and coupled to the bus 102 for storing information and instructions.
[0084]
Computer system 100 may be coupled via bus 102 to a display 112 such as a cathode ray tube (CRT) for displaying information to a computer user. An input device 114 including, for example, alphanumeric keys and other keys is coupled to the bus 102 for communicating information and command selections to the processor 104. Another type of user input device is a cursor control 116, such as a mouse, trackball, or cursor direction key, that communicates direction information and command selections to the processor 104 and controls cursor movement on the display 112. This input device typically has two degrees of freedom in two axes, a first axis (eg x) and a second axis (eg y), so that the device identifies a location in the plane. be able to.
[0085]
Computer system 100 operates in response to processor 104 executing one or more sequences of one or more instructions contained in main memory 106. Such instructions may be read into main memory 106 from another computer readable medium such as storage device 110. Execution of the sequence of instructions contained in main memory 106 causes processor 104 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of software instructions or in combination with software instructions to implement the present invention. Thus, embodiments of the invention are not limited to any specific combination of hardwired circuitry and software.
[0086]
The term “computer-readable medium” as used herein refers to any medium that participates in providing instructions to processor 104 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and communication media. Non-volatile media includes, for example, optical disks such as storage device 110 or magnetic disks. Volatile media includes dynamic memory, such as main memory 106. The communication medium includes a coaxial cable, a copper wire, and an optical fiber including the wiring that constitutes the bus 102. Communication media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
[0087]
Common forms of computer readable media are, for example, floppy disks, flexible disks, hard disks, magnetic tapes or other magnetic media, CD-ROMs, any other optical media, punch cards, paper tapes, holes Any other physical medium having a pattern, RAM, PROM, EPROM, flash EPROM, any other memory chip or cartridge, carrier wave as described below, or any other computer readable from it Includes media.
[0088]
Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 104 for execution. For example, the instructions may first be held on a remote computer magnetic disk. The remote computer can load the instructions into its dynamic memory and send the instructions using a modem over a telephone line. A modem local to computer system 100 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. The infrared detector can receive the data carried in the infrared signal and appropriate circuitry can place the data on the bus 102. Bus 102 carries the data to main memory 106, from which processor 104 retrieves and executes the instructions. The instructions received by main memory 106 may optionally be stored on storage device 110 either before or after execution by processor 104.
[0089]
Computer system 100 also includes a communication interface 118 coupled to bus 102. Communication interface 118 provides a two-way data communication coupling to network link 120 connected to local network 122. For example, communication interface 118 may be a digital integrated services network (ISDN) card or modem that provides a data communication connection to a corresponding type of telephone line. As another example, communication interface 118 may be a local area network (LAN) card that provides a data communication connection to a compatible LAN. A wireless link may also be implemented. In such an implementation, communication interface 118 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
[0090]
Network link 120 typically provides data communication through one or more networks to other data devices. For example, the network link 120 may provide a connection via the local network 122 to a host computer 124 or to a data device operated by an Internet service provider (ISP) 126. ISP 126 then provides data communication services through a global packet data communication network now commonly referred to as the “Internet” 128. Local network 122 and Internet 128 both use electrical, electromagnetic or optical signals that carry digital data streams. Signals through various networks that carry digital data to and from computer system 100 and signals over network link 120 and through communication interface 118 are exemplary forms of carrier waves that carry information. It is.
[0091]
Computer system 100 sends messages and receives data, including program code, through the network, network link 120 and communication interface 118. In the Internet example, the server 130 may send the requested code for the application program through the Internet 128, ISP 126, local network 122 and communication interface 118. The received code may be executed by processor 104 when it is received and / or stored in storage device 110 or other non-volatile storage for later execution. In this way, computer system 100 may obtain application code in the form of a carrier wave.
[0092]
Each of the play modes will now be described.
[0093]
A. Quick play
Quick Play allows you to play the game without experiencing the career mode (Section 0) challenges. It allows you to load song files and play them freely. Within quick play, performers can choose between training mode and dynamic mode.
[0094]
1. Quick play mode
After selecting quick play, the user can select from two different play modes.
[0095]
・ Training mode
The training mode allows the user to play the game for any song that can be loaded into the game.
[0096]
・ Dynamic mode
The dynamic mode is similar to the training mode, except that there are a number of different dynamics that can be switched on and off. These dynamics present additional challenges. Each dynamic has a number of variables that can be adjusted and will gradually change for the performer as the game progresses. The following is an exemplary list of dynamics.
[0097]
-Gradual tempo adjustment: On | Off
-Gradual window closing: On | Off
・ Random color deletion: ON | OFF
・ Conversion to notation: On | Off
All of the above dynamic game elements are further defined under “Start Options” in Section 0 on page 25.
[0098]
2. Road song
A road song is the part of the game that relates to loading a song stored locally on a computer hard drive. The user can browse for specific songs on the hard drive via file search or a simplified song browser.
[0099]
3. Internet song
The invention features a link that opens a web browser (eg Internet Explorer or Netscape) to a section of the website where more songs can be downloaded from, although songs cannot be loaded directly into the game from the website . Users can download songs manually from the website and then open them from within the game. The link is provided on the splash screen.
[0100]
B. Career mode
Career mode allows the performer to complete the game by experiencing different piano methods and levels and facing different challenges within the level.
[0101]
1. Overview
There is a level (section 0) within the piano method (section 0). A level has a predefined number of issues that are identified and available (section 0). Any task from within the level can be applied to any song from within the level. When a level requirement is met, the level ends. The level requirement may be, for example, that at least 7 out of 10 songs are finished and at least 3 out of 5 assignments are applied to each piece to be played. Once the level requirement is met, the next level is released. By doing more than the minimum requirement, a higher level score can be achieved. Further efforts are rewarded by more points and the release of additional graphic themes.
[0102]
2. Profile
A profile is required to play a game in career mode. The profile tracks the overall game progress in career mode separately for each game user.
[0103]
・ Create profile
One of the first things a user must do before starting to play in career mode is to create a profile. Just as a Windows XP operating system user can do before starting to use the system, the user can select an icon from the pool of available icons and assign his name to it. . The game then asks for input about the user name and in what mode the user wants to start the game, namely beginner, learner or teacher. All of the above information is stored on the computer's hard drive and is stored there along with the overall game progress for the user.
[0104]
Profile loading
If the user returns to the game and has an existing profile that has already been saved, the user can select his profile from a list of available profiles. In this way, the user can continue the game from where he stopped last time.
[0105]
Delete profile
Users can delete existing profiles. A confirmation dialog pops up to ensure that the profile is not accidentally deleted.
[0106]
3. Piano method
The user can select one piano method from the list of available methods. On the piano method selection screen, a list of top performers is displayed along with the score, the number of finished levels, and the total time spent on that method. Each piano method available for play is listed with an icon and some short text description of that method.
[0107]
4). level
The user can select a level from within the piano method. By ending the previous level, the higher level is released. The layout of the level selection screen is the same as that of the piano method selection screen. Each level selection on the list is displayed with a short description and an icon. A top performer list is also displayed for each level highlighted for selection. Once a level is highlighted on the list, statistics of progress achieved on the level are also displayed. Exemplary statistics include the total time spent playing and the number of tasks completed so far within a certain level.
[0108]
Each level has a predefined minimum number of songs and assignments that must be completed to advance to the next level. Each level also has a theme-style reward that can be released once the level is finished.
[0109]
5. Song
A song is the last item in the hierarchy of methods and levels. A number of songs are available for each level. The songs that can be used in the carrier mode are set in advance, and for example, track editing is not permitted. Playing these songs does not have the ability to change the game environment as is possible with the quick play mode described in Section 0, and is performed as is.
[0110]
The songs available within the piano method can be any file format supported by the present invention. Songs available from within the Piano Method cannot be replaced from the operating system. In other words, the user cannot replace these songs with lower difficulty songs in order to get higher scores in the game.
[0111]
6). Task
This game features a number of challenges for its users. The task is intended to support training in various skills. The present invention is characterized by the following types of problems.
[0112]
·tempo
Starting from 50% of the song file tempo, it gradually gets faster. Bonus points are awarded for playing faster than the default song file tempo. The performer must continue to perform within the accuracy parameter, ie 90%. The maximum tolerance is a parameter that can be defined by the creator of the assignment.
[0113]
・ Rhythm accuracy issues
In this task, the slap window is gradually closed to a size predefined by the task creator. The start size is a parameter that can be defined by the user. The performer must stay within the accuracy parameter and continue playing.
[0114]
・ First beat
The challenge is to stay within a predefined accuracy level and hit the first beat. The first beat is defined by the task creator. There are no penalties for hitting other sounds (other than the first beat) in the window.
[0115]
・ Weak beat
Like the first beat task, the weak beat task requires staying within a predetermined accuracy level and hitting the weak beat. There are no penalties for hitting other sounds (other than weak beats) in the slap window.
[0116]
・ Color deletion
At a selected or random time, the color game object representing the note will be black and white. As the song progresses, the amount of game objects that are erased gradually increases.
[0117]
・ Note duration (legato)
The challenge is to keep the piano keys pressed for the correct duration of the notes. The performer must continue to perform, for example, staying within an accuracy parameter of 90%.
[0118]
·rotation
The screen rotates from the vertical position to the horizontal position. The performer must stay within the accuracy parameter and continue playing. As the user learns this task, he learns the relationship between the keyboard and standard notation.
[0119]
・ Visibility
A series of visibility tasks is introduced to guide the performer to memorize the music that the performer is performing. For example, clouds may enter the game field and cover a portion of the screen, making it difficult for the user to see the game object. Visibility issues are further defined during development.
[0120]
・ Loudness (velocity)
Some sections of the song will have different velocity parameters (forte, piano, etc.) that the user needs to stay in. The performer must stay within the accuracy parameter and continue playing. This issue is only available for touch sensitive piano keyboards.
[0121]
7). Creating a custom piano method
Creating a custom piano method involves creating an empty method, naming it, creating a level in it, and adding songs to the level. Custom piano methods do not include rewards, high score lists, or graphical themes. For any song that is part of a custom piano method, all assignments can be played. Custom piano methods should be storable in such a way that they can be copied and shared among users of the present invention. A custom piano method is a folder with files and a simple configuration file. In the custom piano method, all levels are released and can be played. The concept of custom piano methods is to allow teachers to create homework for their students who are users of the present invention.
[0122]
C. Start option
The start option is an option that can be set before the user starts the game. The number of start options available depends on the game mode. Some options are not available in some modes and are grayed out or excluded. However, no matter what mode you start the game with, you need to go through these options to start the game. The only exception to this rule is immediate play, which omits the start option setting and assumes all default values for performance. Preferred availability of start options in different game modes is included in the CD-ROM appendix.
[0123]
Below is a list and a brief description of all start options.
[0124]
Screen orientation: V | H | Rotation
This option specifies the screen orientation. Some users may want to start the game play in the vertical mode (V), while others prefer the horizontal mode (H). It is also possible to start game play in a mode in which the screen begins to rotate stepwise from portrait to landscape.
[0125]
・ Note: On | Off
This switch turns annotations on / off. Annotation. It is included in the amm internal format and is involved in displaying a kind of “sticky note” text memo on the game screen as the annotated game object scrolls.
[0126]
・ Fingering: On | Off
This switch turns fingering information on / off. Like annotations, fingering is. amm internal file format. If fingering is possible, the game object, if so marked, serves as a guide to the user as to which finger to use to correctly point the game object on the piano keyboard 1 to 5 Displays numbers up to.
[0127]
・ Freeze: On | Off
This switch enables or disables the ability to access the freeze menu from within game play, as described below.
[0128]
・ Number of beats per screen
This option allows the user to choose how many beats or bars to display vertically on the screen during gameplay, so that it can be predicted well enough to prepare for the next note in the case of a fast song To do.
[0129]
Object: Long | Short | Notation
This option allows selection between different types of game objects. A long object can be drawn to represent the actual duration of a note, an example being a caterpillar. On the other hand, a short object represents only the starting point of a note, not a duration, ie a soccer ball or a balloon. When notation is selected, the user will see notation instead of game objects during game play.
[0130]
・ Notation: Temporary sign |
This option allows you to choose between different types of notation. It is only available when notation is selected as a game object.
[0131]
・ Theme selection
Theme selection is a set of GUI elements that allow selection of different game themes. The theme includes a background and a different game object. Different backgrounds can be combined with different game objects. However, themes that have not yet been released in career mode (section 0) are not available for selection.
[0132]
・ Color: ON | OFF
This switch turns on / off the color of the game object and the color of the virtual piano keyboard on the screen. Basically, when turned on, all game objects and piano keys are displayed in different colors, and in other cases, these objects are black and white.
[0133]
-Track selection (range preview, including valid keys)
This section allows visual selection of tracks in the loaded song file. Once the song is loaded, a list of available tracks is displayed and the user can select the track he wants to play in the game. Once a track is selected by the user, the program automatically finds a range of notes about it and marks it on the piano keyboard displayed on the screen (ie C2-B3). This range is always calculated for all selected tracks. For example, if two tracks are selected, track 1 and track 2, the range is marked for both as if they were one track. It is important to preview the range. This is because it allows the user to understand whether the selected track range fits the connected MIDI keyboard. In other words, it is practically impossible to play a song spread over 6 octaves on a keyboard having only 4 octaves. In addition, the user can click on individual piano keys and mark them as valid or invalid. By default, all keys in range are marked as valid. During game play, the user is responsible for hitting only valid piano keys.
[0134]
・ Selecting instruments for trucks
The user is given the ability to change the general MIDI instrument for any selected track played in the game. The user can easily select one instrument per track from the drop-down menu of 128 general MIDI level 1 instruments.
[0135]
・ Accompaniment track selection
The user is also given the ability to select an accompaniment track for the song he wants to play for the game. The accompaniment track is played so that it can be heard in the background. These choices are voluntary because some people find accompaniment useful, while others find it distracting.
[0136]
・ Selecting instruments for accompaniment tracks
Similar to selecting an instrument for a track played in the game, the user is given the ability to change the general MIDI instrument for any selected accompaniment track. The user can easily select one instrument per accompaniment track from the drop-down menu of 128 general MIDI level 1 instruments.
[0137]
・ Merging selected trucks
This option merges many selected tracks into one. This operation does not change the contents of the original file, but instead is performed in the computer's memory on the loaded file.
[0138]
・ Delete track
Delete the selected track from memory. The deleted track has not been deleted from the song file.
[0139]
・ Dividing the truck into both hands
This option is useful when only one track is selected. Performing this option splits a single track into two tracks whose music content is configured separately for both hands. That is, the first track should hold the note to be played with the left hand and the second track should be played with the right hand.
[0140]
・ Transposition of the selected track
This option transposes the selected track up or down a semitone.
[0141]
・ Track setup
This section allows the user to set the MIDI output port and channel for any track in the loaded file. For example, the user may wish to listen to accompaniment track # 2 on the 11th channel of a synthesizer located at the 3rd MIDI port.
[0142]
・ Metronome: On | Off
This switch enables or disables the audio metronome that can be heard during game play.
[0143]
・ Practice assignment selection
This section makes it possible to select a task from all the tasks that are built into the game and are part of the career mode, as described above. By making this possible, the user can practice different tasks at will for any song he desires.
[0144]
・ Tempo selection
This option allows the selection of the tempo used during game play. Part of this option is a switch called “MIDI Tempo: On | Off” that enables performance at the tempo specified in the currently loaded song. If this switch is enabled and the current song file features tempo variations, the game speed is automatically adjusted.
[0145]
・ Slash window adjustment
This option allows you to adjust the height of the game window.
[0146]
・ Background: On | Off | Opacity%
An option that enables, disables, or enables adjustment of background opacity. The opacity value is 0% to 100%.
[0147]
・ Measure: On | Off
This switch enables or disables visual measurements that scroll with the game object.
[0148]
・ Object guide: ON | OFF
This switch enables or disables visible path guide lines.
[0149]
・ Statistics (overlay): ON | OFF
This switch turns on / off game statistics, scores, etc. on the screen.
[0150]
・ Keyboard: Automatic | 2 octaves | 4 octaves | 5 octaves
This option allows you to select the size of the on-screen keyboard used for game play. The user can select from a 2-octave version, 4-octave version, or 5-octave version keyboard. Automatic keyboard size selection is also available. The game supports three keyboard sizes: 2 octaves, 4 octaves, and 5 octaves.
[0151]
・ Start key selection
This option makes it possible to select the starting key on the keyboard on the screen. This is very useful for calibrating both the physical MIDI keyboard and the on-screen keyboard if they are of different sizes. Options are C0, C1, C2,..., C11.
[0152]
-Gradual tempo adjustment: On | Off
Enabling this option will gradually increase or decrease the gameplay tempo to the tempo specified in the loaded song file. For example, by starting the game play at a slow tempo, the user can practice the music at a slow but constantly increasing tempo.
[0153]
-Gradual window closing: On | Off
Enabling this option will gradually reduce the height of the slash window during game play. For example, the user can practice music with a slowly but constantly increasing accuracy by opening the hitting window wide and starting game play.
[0154]
・ Random color deletion: ON | OFF
When this option is enabled, the game object begins to lose color selectively or randomly during game play, making it more difficult to match the game object to the color-coded piano keyboard. Colors can be deleted from both game objects and keyboard keys on the screen.
[0155]
・ Conversion to notation: On | Off
Enabling this option causes the game object to be converted to notation in stages.
[0156]
・ Loop Song: On | Off
Enable or disable song looping in gameplay.
[0157]
・ Lyrics: On | Off
If the song file contains lyrics and this option is enabled, the lyrics will appear on the screen.
[0158]
・ Save songs automatically: On | Off
This option enables or disables automatic saving of all changes to the currently loaded song. If autosave of the song is enabled, the user is prompted to enter a new file name for the song. song is. amm internal song file format.
[0159]
・ SFX volume and music volume
These options are realized in the form of a slider, allowing the volume level of both sound effects and music to be adjusted.
[0160]
・ Partial extraction
This is a hybrid truck option. The user selects a clef (track) to be merged.
[0161]
・ Preparing a part to sing along with the song
This option simply breaks the text into bars for those who sing along to the song without the notation on the same score.
[0162]
・ Preparation of lead score for fake book
The “Fake” book is a collection of popular songs with lyrics, chord names for jazz players (Gm7), and possibly the main melody, and jazz players use melody and chord structure as a skeleton, while rhythm and harmonics. Impromptu performance part. MidiNotate software allows to extract lyrics, main melody, and chord names from MIDI files and print them in this manner.
[0163]
・ Code name notation
There are many ways to represent code names. An example is the A major. This can be written simply as A, or it may be written as Amaj.
[0164]
All of the above options can be combined into groups. For example, there are a large number of Boolean options (2 states, on | off) implemented as check boxes in the game's GUI.
[0165]
D. Gameplay
This is where all operations occur. Gameplay is an actual game.
[0166]
1. Keyboard shortcut
There are a number of example keyboard shortcuts linked to different options, which can be called during game play without stopping the game flow. Other options are possible. These options are listed below.
[0167]
Tempo adjustment Left arrow | Right arrow
・ Slash window adjustment Up arrow | Down arrow
・ Warning line adjustment Shift + Up arrow | Shift + Down arrow
-Overall volume adjustment Gray + | Gray-
・ Metronome M
Some adjustments are not available for some issues. For example, tempo adjustment cannot be used during game play on a tempo task.
[0168]
2. Sound removed
A mark is placed on the game field whenever the performer removes the sound. The user can scroll up and down the entire song from the freeze menu to see where it was removed.
[0169]
3. Notation
The present invention can convert a MIDI file into a notation that is displayed on the screen during game play. Examples of suitable software for performing this function can be found below.
[0170]
・ Notation Technologies
http://www.notationtechnologies.com/
・ Notation Software, Inc.
http://www.notation.com/
4). Game object
This invention features a number of game objects, also known as sprites. All game objects feature 32-bit color and alpha blending support. Game objects are two-dimensional, but are drawn on the screen using 3D techniques to take advantage of 3D drawings facilitated by hardware. This allows easy scaling, rotation, and alpha blending.
[0171]
5. Piano keyboard on screen
There are three different sizes of piano keys on the screen: 2 octaves, 4 octaves, and 5 octaves. Depending on the selected keyboard size, the size of the game object changes accordingly and maintains a width proportional to the key width of the keyboard on the screen.
[0172]
6). Scoring points
This game gives the user a score for playing the notes correctly, playing with different dynamic options enabled, playing different tasks, and so on.
[0173]
E. Freeze menu
The freeze menu can be called only during game play. It allows you to change some of the game parameters, create annotations on the screen, and save the currently playing song along with all annotations, highlighting and fingering information in an internal file format To do.
[0174]
1. call
The freeze menu is called by pressing the ESC key on the keyboard during game play. Once ESC is pressed, the following occurs: In other words, the entire game freezes and a vertical scroll bar appears on the screen, allowing the entire song to be scrolled back to the previously played portion or forward. Note that scrolling, annotation, highlighting or fingering is not available when the game screen is in rotation. Once the game is frozen, a translucent help box is displayed with instructions on what can be done on the screen. For example, a typical text would say "Click on an empty space to create an annotation on the game screen".
[0175]
2. Highlighting and fingering
When the user clicks on any game object, a pop-up menu similar to that shown below appears adjacent to the object.
[0176]
[Table 1]

[0177]
From within the pop-up menu, the user can select the highlight color or fingering numbers for both hands simply by clicking on the grid (as described above). Whatever selection is made on the pop-up menu, it is applied to the clicked object. The highlighting should be done in the form of a afterglow effect around the object. The colors shown on the above pop-up menu are for illustrative purposes only. In addition, if a fingering or highlight is already assigned to the object, they can be released by clicking on the “x” button as shown above.
[0178]
3. Annotating
The user can make notes on the game screen. These annotations will be done later. Become part of the amm file. The annotation once added on the game screen is stored in a file as necessary (saved). Users can add and remove annotations. Annotations are added to the screen in the form of speech bubbles. There are many balloon shapes to be selected. To add an annotation to the game screen, the user clicks on an empty space on the screen. Once clicked, a pop-up menu appears where the shape of the balloon and its arrow pointer direction can be selected. Next, text is entered via a text edit box. Annotations can be removed from the game screen. To remove an annotation, you must click on an existing annotation and then click on the “x” button that appears next to it. The game also allows editing of already entered annotations. To edit an annotation, as with deletion, the user clicks on the annotation already attached and then clicks the “Edit” button that appears next to it. An annotation is edited in the same manner as the input.
[0179]
4). Freeze menu option
The number of options in the freeze menu depends on the game mode. See Appendix B for preferred availability of freeze menu options in different game modes. The freeze menu options are listed below. It should be noted that a significant number of these options are a subset of the starting options defined in section C above. The specification of options already specified is omitted here.
[0180]
・ Fingering: On | Off
Object: Long | Short | Notation
・ Notation: Temporary sign |
・ Theme selection
・ Color: ON | OFF
-Track selection (range preview, including valid keys)
・ Selecting instruments for trucks
・ Accompaniment track selection
・ Selecting instruments for accompaniment tracks
・ Transposition of the selected track
・ Metronome: On | Off
・ Tempo selection
・ Slash window adjustment
・ Background: On | Off | Opacity%
・ Bar: ON | OFF
・ Object guide: ON | OFF
・ Statistics (overlay): ON | OFF
・ Start key selection
-Gradual tempo adjustment: On | Off
-Gradual window closing: On | Off
・ Random color deletion: ON | OFF
・ Conversion to notation: On | Off
・ Loop Song: On | Off
・ Lyrics: On | Off
・ SFX volume and music volume
・ Save songs automatically: On | Off
Turn the auto-save feature on or off. If this option is enabled, the song is automatically saved to a previously identified file as having been changed.
[0181]
・ Save the song with a name
Prompt for file name. The current song. Save to amm file.
[0182]
F. Game setup
Game setup can be called from the main menu. Below are the options.
[0183]
Menu option: Advanced | Beginner
This menu option allows you to choose between two different sets of menu options in the game. The advanced set is designed for users who want to get the most out of the game and need maximum flexibility, while the beginner set is too complex for the average user or child Eliminates many menu options.
[0184]
・ Gamma correction
This option allows you to adjust the gamma value of the screen.
[0185]
・ Select Midi in
This menu option allows selection of a MIDI input port for the game. This game enumerates all possible MIDI ports on the system and makes them available for selection.
[0186]
・ Select Midi out
This menu option allows selection of a MIDI output port for the game. This game enumerates all possible MIDI ports on the system and makes them available for selection.
[0187]
Game detail level: Low | Medium | High
The game detail level enables and disables some of the CPU-intensive game visual effects. If the level of detail is low, it will be possible to run the game on a low cost computer at the expense of visual effects and overall visual enjoyment.
[0188]
・ Music and SFX volume levels
These menu options in the form of sliders adjust the overall sound mixer values for the game. The SFX volume option is available for all. The overall volume level of the wav sound (sound card PCM channel) can be adjusted, while the music volume allows the overall volume level of the internal sound card synthesizer to be adjusted. Both sliders can be assigned values between 0% and 100%.
[0189]
・ Keyboard range setting
This is an option to determine the size of the keyboard connected to the computer. The user is asked to press the leftmost and rightmost keys on the MIDI keyboard, and the game calculates the octave size of the keyboard.
[0190]
・ Velocity test
This option allows the user's MIDI keyboard to be tested to determine if it is sensitive to velocity. The user is asked to tap a key on the MIDI keyboard twice, first gently and secondly. Based on the difference in the values of the received MIDI velocity messages, the program determines whether the user has a touch sensitive keyboard.
[0191]
Although the invention has been described and illustrated in detail, it is clearly understood that it is for illustration only and should not be construed as limiting. The spirit and scope of the present invention is limited only by the terms of the appended claims.
[0192]
Appendix A
CD-ROM contents
The following is a description of the contents of the CD-ROM filed as an appendix to this application.
[0193]
The CD-ROM includes a plurality of directories and files as shown in the following directory printout from the CD-ROM.
[0194]
The file extension description for most files appropriately specifies how to open them to those skilled in the art. The file can be opened using Microsoft Word, Microsoft Excel, or Adobe Acrobat Reader. . A vsd file can be read using a video.
[0195]
Software for opening any other file that may not be self-evident can be found at http://extsearch.com, or a similar file extension directory search engine.
[0196]
[Table 2]

[Brief description of the drawings]
[0197]
FIG. 1 is a block diagram of a hardware architecture for carrying out some aspects of the present invention.
FIG. 2 is a diagram illustrating an exemplary relationship between a virtual keyboard, game object paths, and their treble clef in standard notation.
FIG. 3 is a block diagram of exemplary hardware and software modules for carrying out the invention.
FIG. 4 is a diagram showing a format of a MIDI file.
FIG. 5 is a diagram illustrating a format of a header chunk of a MIDI file.
FIG. 6 is a diagram showing a format of a track chunk of a MIDI file.
FIG. 7 is a diagram showing a format of a META event from a MIDI file.
FIG. 8 is a diagram illustrating an example of a portion of an MTrk chunk of MIDI data and exemplary semantics.
FIG. 9 is an exemplary flow diagram of data showing a MIDI message for the sequential performance of three sounds.
FIG. 10 is a flowchart of an exemplary process for interpreting the MIDI data stream of FIG. 5 and generating a game object for display.
FIG. 11 is a block diagram showing an exemplary configuration of a game module according to one aspect of the present invention.
FIG. 12 is a block diagram of an exemplary computer that can be used to implement various aspects of the invention.

Claims (35)

A device for providing music instruction,
a. Computing elements,
b. A display connected to the computing element;
c. A music keyboard connected to the computing element;
d. Software executed on the computing element, the software including a virtual keyboard displayed on the display when the software is executed on the computing element as part of a game by a user of the music keyboard A device that connects with the operation of the machine. The apparatus according to claim 1, wherein the software converts a musical note of a musical piece into a game object on the display. The apparatus of claim 2, wherein the software includes an interpreter that converts the display of the music into a command for generating a game object. 4. The apparatus of claim 3, wherein the command for generating a game object drives a game object generator. The apparatus of claim 4, wherein the game object generator is associated with a library of game objects. The apparatus of claim 4, wherein the game object generator is associated with a set of parameters for setting game object properties. The apparatus according to claim 2, wherein the game object moves with time from an initial input position on the display toward a key of a virtual keyboard corresponding to a musical note. The apparatus of claim 7, wherein the game object moves along a predefined path. A method of using a computer,
A method comprising the steps of associating a music keyboard connected to a computer with an action of a game object displayed on the display relative to a virtual keyboard displayed on the computer display. The method of claim 9, wherein a game object is generated for musical notes stored in a computer readable format. The method of claim 10, wherein a song from a library of songs is stored on the computer. The method of claim 10, wherein the computer readable format is MIDI. The method of claim 9, wherein the user is rewarded in the form of at least one of audio playback, video or graphical animation display, or scoring. The method of claim 13, wherein a reward is awarded if a user presses a key on a music keyboard within a time window spanning the moment that the game object intersects with a corresponding key on the virtual keyboard. The method of claim 14, wherein for each of a plurality of users, at least one profile of user information is stored on the computer. The method of claim 14, wherein the profile information may include a set of start options. A set of start options include screen orientation mode, annotation on / off, fingering information on / off, freeze frame on / off, object, notation, theme, color, track, track instrument, accompaniment, merge Track, track to be deleted, splitting of track into both hands, transposition of selected track, output port, metronome on / off, exercise assignment selection, tempo selection, background, measure, object guide on / off, statistical overlay, keyboard Size, number of beats or bars displayed on the screen at once, start key, gradual tempo adjustment, gradual striking window closure, random color removal, object conversion to notation, loop song, lyrics on The method of claim 16, which may be selected from the group consisting of: / off, song auto-save on / off, and volume. . The method of claim 14, wherein a plurality of methods are available for selection by a user, each method including at least one song and a plurality of difficulty levels for each song. The method of claim 18, wherein each difficulty level includes a plurality of songs and one or more tasks intended to enhance skills in each field of ability. The task is selected from the group consisting of a tempo task, a rhythm accuracy task, a first beat task, a weak beat task, a color deletion task, a note duration task, a rotation task, a visibility task, and a loudness task. The method described. A computer program product,
a. A storage medium;
b. An operation of a game object displayed on the display with respect to a virtual keyboard displayed on the display of the computer. A computer program product that includes instructions for linking to. The computer program product of claim 21, wherein the computer program further comprises instructions for generating a game object for musical notes stored in a computer readable format. 23. The computer program product of claim 22, wherein the computer program further comprises instructions for rewarding a user with at least one form of audio playback, video or graphical animation display, or scoring. 24. The computer program further comprises instructions for rewarding a user pressing a key on a music keyboard within a time window spanning the moment when the game object intersects a corresponding key on the virtual keyboard. A computer program product as described in. A method for instructing a user to read notation through a graphical user interface and interaction with an input keyboard,
a) generating a graphical user interface, the graphical user interface having a first position and including a virtual keyboard having a plurality of keys positioned substantially above the interface, wherein each tendon is an input keyboard Having a corresponding key above, the method further comprising:
b) incorporating the music file into a graphical user interface, the music file including data having a rhythm pattern corresponding to the arrangement of at least first and second notes arranged in sequence, the method further comprising: ,
c) The first game object representing the first note in the arrangement is directed upward in the first substantially straight path toward the first key on the virtual keyboard corresponding to the first note. Allowing one game object to experience a first collision with a first key;
d) The second game object representing the second note in the arrangement is directed upward in the second substantially straight path toward the second key on the virtual keyboard corresponding to the second note. Causing the two game objects to experience a second collision with the second key according to the rhythm pattern of the arrangement;
e) providing a value to the user based on the user hitting a corresponding key on the input keyboard substantially simultaneously with the first and second collisions. a) Once a predetermined threshold of user performance is met, the virtual keyboard is positioned substantially to the left of the interface by rotating the interface to a second position, approximately 90 degrees counterclockwise, Causing the first and second game objects to move toward the virtual keyboard along first and second substantially straight paths;
b) introducing a series of visible staves defining a gap, the line and the gap corresponding to a straight path along which the game object moves along the virtual keyboard, 26. The teaching method according to claim 25, wherein the key is moved along a line or in between until a corresponding key hits the virtual keyboard. 27. The method of claim 26, further comprising transforming the game object into classical notation. A method for instructing a user to read notation through a graphical user interface and interaction with an input keyboard,
a) generating a graphical user interface, the graphical user interface having a first position and including a virtual keyboard having a plurality of keys positioned substantially above the interface, wherein each tendon is an input keyboard Having a corresponding key above, the method further comprising:
b) including the step of incorporating the music file into a graphical user interface, the music file including data having a rhythm pattern corresponding to an arrangement of a plurality of notes arranged in order, each note represented by a game object And the method further comprises:
c) directing the game object upward in a substantially straight path to a key on the virtual keyboard corresponding to the note;
d) colliding the game object with the corresponding key according to the rhythm pattern of the arrangement;
e) providing a value to the user based on the user hitting a corresponding key on the input keyboard substantially simultaneously with the collision;
f) Once the predetermined threshold of user performance is met, the virtual keyboard is positioned substantially to the left of the interface by rotating the interface to a second position, approximately 90 degrees counterclockwise, Allowing the game object to continue moving toward the virtual keyboard along a substantially straight path. Further comprising the step of introducing a series of visible staves defining a gap, the line and the gap corresponding to a straight path along which the game object moves toward the virtual keyboard, the game object correspondingly 29. The teaching method according to claim 28, wherein the key is moved along a line or between until it collides with the virtual keyboard. A method of teaching music comprising converting a music file into an animated game object for display on a screen. The method of claim 30, wherein the game object flows upward on the screen. 31. The method of claim 30, further comprising transforming the game object into notation. The method of claim 30, wherein the path for the game object rotates between portrait and landscape orientations. 31. The method of claim 30, wherein the game object flows upward on the screen, transforms into notation, and rotates from portrait to landscape. 35. The method of claim 34, wherein the game object is selectively colored.

Images