JP4701411B2 - Music game device, music game system, operation article, music game program, and music game method - Google Patents

Description

  The present invention relates to a music game apparatus that displays an image following the movement of an operation article and related technology.

Japanese Patent Application Laid-Open No. 2002-263360 discloses a performance conducting game device. In this performance command game device, a light projecting unit is provided at the tip of the command stick controller, and a light receiving unit is provided below the monitor. With such a configuration, the movement of the baton controller is detected.
When the game is started, an operation guidance image for instructing the direction and timing of swinging the baton controller is displayed on the monitor and a performance sound is output. This performance sound is output regardless of the operation of the baton controller. Then, only when the player operates the baton controller in the instructed direction and timing, the baton reaction sound is output. This baton reaction sound is obtained by dividing a certain performance part by a predetermined length. Therefore, each time the player operates the baton controller in the instructed direction and timing, the corresponding baton reaction sound is output.
Japanese Patent Application Laid-Open No. 10-143151 discloses a command device. In this conductor device, by operating the mouse like a conductor rod, music parameters such as tempo, accent and strength are calculated from the locus drawn by the mouse. Then, these calculated music parameters are reflected in the output music and images. For example, when displaying a moving image of a train, the speed of the train is made to follow the calculated tempo, the change in speed is made to follow the accent, and the amount of smoke of the train is made to follow the strength.
As described above, it is clear that the performance / conducting game device of Patent Document 1 is a device whose main purpose is for the player to perform. Further, in the command device of Patent Document 2, since the main purpose is for the player to perform, the movement information of the mouse is converted into music parameters, and the music parameters are reflected in music and images.
As described above, in the conventional apparatus whose main purpose is for the player to perform, the displayed image (in the above example, the train) is not interesting, and the player is not focused on enjoying the image. .
Furthermore, the following can be said about the baton controller and the mouse that are operations by the player. Since the baton controller of patent document 1 is equipped with the light projection unit, it is necessary to provide an electronic circuit. Therefore, the cost of the baton controller becomes high and causes a failure. Furthermore, the operability is lowered. In particular, since the baton controller is moved, it is preferable to have a simple configuration without providing an electronic circuit. In addition, the mouse of Patent Document 2 can only be operated on a flat surface, and there are large restrictions on the operation, and there are the same problems as the baton controller of Patent Document 1.

Therefore, the present invention provides a music game apparatus that allows a player to enjoy an image following an operation together with the music piece while the player automatically operates the music piece without involvement of the player, and the player operates the operation object with a simple configuration. And its related technology.
According to an aspect of the present invention, a music game apparatus is a music game apparatus for automatically playing music, and a stroboscope that irradiates an operation article operated by a player at a predetermined cycle, and the stroboscope. An imaging unit that captures the operation object and generates an image signal at the time of emission and an image signal at the time of extinction when the scope is turned on and off, and a difference signal between the image signal at the time of emission and the image signal at the time of turning off A difference signal generation unit that generates a state information, a state information calculation unit that calculates state information of the operation article based on the difference signal, and a display of a guide for operating a cursor linked to the operation article. And a cursor control unit for controlling display of the cursor based on the state information of the operation article. And a follow-up image control unit that controls display of an image according to guidance by the guide when an operation of the cursor by the operation article is adapted to the guide, and the follow-up image control unit Determines whether the operation of the cursor by the operation article is suitable for the guide based on the state information of the operation article and information on the guide.
According to this configuration, when the operation of the cursor is adapted to the guide, the image display is controlled according to the guidance by the guide. In this case, since the cursor is operated in conformity with the guidance by the guide, the image display is controlled in accordance with the cursor operation. That is, since the cursor is interlocked with the operation article, the display of the image is controlled according to the operation of the operation article. In addition, an operation article that is intermittently irradiated with light by a stroboscope is photographed by an imaging unit to obtain state information of the operation article. For this reason, in order to obtain the state information of the operation article, it is not necessary to incorporate a circuit driven by the power source in the operation article. Further, this music game device automatically plays music.
As a result, it is possible to enjoy an image following the operation of the operation article together with the music piece by the player operating the operation article with a simple configuration while automatically playing the music piece without involvement of the player.
Further, since the guide is controlled at the timing based on the music, if the player operates the cursor according to the guide, the operation of the operation article also matches the music. Therefore, the player can enjoy the operation of the operation article suitable for the music.
Here, the operation of the operation article means that the operation article itself is moved (for example, moved), and does not include pressing a switch, moving an analog stick, or the like.
In the music game apparatus, the guide guides the movement position and operation timing of the cursor, and the follow-up image control unit is configured such that the guide is operated when the operation of the cursor by the operation article is adapted to the guide. The display of the image is controlled in accordance with the direction of the moving position to be guided.
According to this configuration, when the player operates the operation article and moves the cursor to the movement position guided by the guide at the operation timing guided by the guide, the direction of the movement position of the cursor guided by the guide Corresponding to the above, the display of the image is controlled. As a result, it is possible to enjoy an image that follows the movement of the cursor linked to the operation of the operation article together with the music.
In the music game apparatus, the state information calculation unit calculates a position of the operation article as the state information based on the difference signal, and the follow-up image control unit calculates the position information calculation unit. When the position of the operation article is within the area guided by the guide within the period guided by the guide, it is determined that the operation of the cursor linked to the operation article is suitable for the guide. .
According to this configuration, the accuracy of the cursor operation can be determined based on the position of the operation article that can be calculated by a simple process.
In the music game apparatus, the guide guides the movement path, movement direction, and operation timing of the cursor.
According to this configuration, when the player operates the operation article and moves the cursor in the movement direction guided by the guide at the operation timing guided by the guide along the movement route guided by the guide, The display of the image is controlled according to the guide. As a result, it is possible to enjoy an image corresponding to the movement of the cursor linked to the operation of the operation article together with the music.
In the music game apparatus, the state information calculation unit calculates a position of the operation article as the state information based on the difference signal, and the follow-up image control unit calculates the position information calculation unit. When the position of the operation article is moved in a predetermined order guided by the guide within a period guided by the guide, the cursor is operated in conjunction with the operation article. Is determined to be suitable for the guide.
According to this configuration, the accuracy of the cursor operation can be determined based on the position of the operation article that can be calculated by a simple process.
In the music game apparatus, the guide is displayed at each of a plurality of predetermined positions on the screen, and the guide control unit changes the form of the guide at a timing based on the music.
According to this configuration, the player can easily recognize the position and direction in which the cursor should be moved by changing the form of the guide.
Here, in this specification, the shape of the guide includes one or both of the shape and color of the guide.
In the music game apparatus, the guide is represented by a display that can visually recognize the movement from the first predetermined position on the screen to the second predetermined position, and the guide control unit includes: The display of the guide is controlled at a timing based on the music.
According to this configuration, the player can more clearly recognize the direction and route to move the cursor.
For example, the guide is expressed by a change in the form of a plurality of objects arranged on a path starting from the first predetermined position on the screen and ending at the second predetermined position. can do.
According to this configuration, the player can easily recognize the direction and route in which the cursor should be moved by changing the form of the plurality of objects.
For example, the guide can be expressed by moving an object from the first predetermined position on the screen to the second predetermined position.
According to this configuration, the player can easily recognize the direction and route in which the cursor should be moved by moving the object.
For example, the guide can be expressed by a change in the form of a route starting from the first predetermined position on the screen and ending at the second predetermined position.
According to this configuration, the player can easily recognize the direction in which the cursor should be moved and the route by changing the form of the route.
In the music game apparatus, the state information of the operation article calculated by the state information calculation unit includes speed information, movement direction information, movement distance information, speed vector information, acceleration information, movement trajectory information, area information, or , Position information, or a combination of two or more thereof.
According to this configuration, since various information can be used as the state information of the operation article when determining whether the operation of the cursor by the operation article is suitable for the guide, the degree of freedom of expression of the guide is large. As a result, the degree of freedom in designing the game content also increases.
The novel features of the invention are set forth in the appended claims. However, the invention itself and other features and advantages can be readily understood by reading the detailed description of specific embodiments with reference to the accompanying drawings.

FIG. 1 is a diagram showing an overall configuration of a music game system according to an embodiment of the present invention.
FIG. 2 is a perspective view of the operation article of FIG.
FIG. 3A is a top view of the reflective ball of FIG. FIG.3 (b) is a side view of the reflective ball | bowl from the arrow A direction of Fig.3 (a). FIG.3 (c) is a side view of the reflective ball | bowl from the arrow B direction of Fig.3 (a).
FIG. 4 is a longitudinal sectional view of the reflective ball of FIG.
FIG. 5 is an illustrative view showing one example of the imaging unit of FIG. 1.
FIG. 6 is a diagram showing an electrical configuration of the music game apparatus of FIG.
FIG. 7 is a block diagram of the high speed processor of FIG.
FIG. 8 is a circuit diagram showing a configuration for taking pixel data from the image sensor of FIG. 6 into a high-speed processor and an LED driving circuit.
FIG. 9A is a timing chart of the frame status flag signal FSF output from the image sensor of FIG. FIG. 9B is a timing chart of the pixel data strobe signal PDS output from the image sensor of FIG. FIG. 9C is a timing chart of pixel data D (X, Y) output from the image sensor of FIG. FIG. 9D is a timing chart of the LED control signal LEDC output from the high speed processor of FIG. FIG. 9E is a timing chart showing a lighting state of the infrared light emitting diode of FIG. FIG. 9F is a timing chart showing the exposure period of the image sensor in FIG.
FIG. 10A is an enlarged view of the frame status flag signal FSF of FIG. FIG. 10B is an enlarged view of the pixel data strobe signal PDS of FIG. FIG. 10C is an enlarged view of the pixel data D (X, Y) of FIG.
FIG. 11 is a view showing an example of a game screen displayed on the screen of the television monitor of FIG.
FIG. 12 is another exemplary view of the game screen displayed on the screen of the television monitor of FIG.
FIG. 13 is a view showing still another exemplary game screen displayed on the screen of the television monitor of FIG.
FIG. 14 is an explanatory diagram of sprites that constitute objects displayed on the screen of the television monitor of FIG.
FIG. 15 is an explanatory diagram of a panoramic screen displayed on the screen of the television monitor of FIG.
FIG. 16A is an explanatory diagram before scrolling the panoramic view screen of FIG. FIG. 16B is an explanatory diagram after the whole view screen of FIG. 15 is scrolled.
FIG. 17 is a conceptual diagram showing programs and data stored in the ROM of FIG.
FIG. 18 is a conceptual diagram showing an example of the first musical score data of FIG.
FIG. 19 is a conceptual diagram showing an example of the second musical score data of FIG.
FIG. 20A is a relationship diagram between the direction of guiding the cursor and the note number. FIG. 20B is another relationship diagram between the direction of guiding the cursor and the note number. FIG. 20C is still another relationship diagram between the direction of guiding the cursor and the note number.
FIG. 21A is an illustration of an image taken by a general image sensor and not subjected to special processing. FIG. 21B is an illustration of an image signal when the image signal of FIG. 21A is level-discriminated by a certain threshold value. FIG. 21C is an illustration of an image signal when the image signal when the image sensor is turned on via the infrared filter is level-discriminated by a certain threshold value. FIG. 21D is an exemplary diagram of an image signal when the image signal when the image sensor is turned off via the infrared filter is subjected to level discrimination based on a certain threshold value. FIG. 21E is an exemplary diagram of a difference signal between an image signal at the time of lighting and an image signal at the time of turning off.
FIG. 22 is an explanatory diagram of the coordinate calculation of the attention point of the operation article in FIG.
FIG. 23A is an explanatory diagram of the X-direction scan when calculating the point-of-interest coordinates of the operation article in FIG. 1 based on the coordinates of the pixel having the maximum luminance value. FIG. 23B is an explanatory diagram at the start of the Y-direction scan when the attention point coordinates of the operation article in FIG. 1 are calculated based on the coordinates of the pixel having the maximum luminance value. FIG. 23C is an explanatory diagram of the Y-direction scan when calculating the point-of-interest coordinates of the operation article in FIG. 1 based on the coordinates of the pixel having the maximum luminance value. FIG. 23D is an explanatory diagram of a result when calculating the attention point coordinates of the operation article based on the coordinates of the pixel having the maximum luminance value.
FIG. 24 is an explanatory diagram of attention point presence area determination processing (1) performed by the CPU 201.
FIG. 25 is an explanatory diagram of the attention point presence area determination process (2) by the CPU 201.
FIG. 26 is an explanatory diagram of registration of animation of the direction guide G, the position guide g, and the route guide rg according to the present embodiment.
FIG. 27 is a view showing an example of an animation table specified by the animation table storage position information of FIG.
FIG. 28 is a timing chart for explaining the relationship between the first musical score data, the second musical score data, the direction guide G, the position guide g, the operation determination, and the dance animation in the present embodiment.
FIG. 29 is a flowchart showing the overall processing flow of the music game apparatus of FIG.
FIG. 30 is a flowchart showing the flow of the initial setting process in step S1 of FIG.
FIG. 31 is a flowchart showing the flow of the sensor initial setting process in step S14 of FIG.
FIG. 32 is a flowchart showing the command transmission process in step S21 of FIG.
FIG. 33 is a timing chart of (a) the register setting clock CLK of FIG.
FIG. 9B is a timing diagram of the register data in FIG.
FIG. 34 is a flowchart showing the register setting process in step S23 of FIG.
FIG. 35 is a flowchart showing the flow of the state information calculation process in step S2 of FIG.
FIG. 36 is a flowchart showing the pixel data group acquisition processing in step S50 of FIG.
FIG. 37 is a flowchart showing the pixel data acquisition process in step S61 of FIG.
FIG. 38 is a flowchart showing the flow of attention point extraction processing in step S51 of FIG.
FIG. 39 is a flowchart showing a flow of attention point coordinate calculation processing in step S85 of FIG.
FIG. 40 is a flowchart showing the flow of the game process in step S3 of FIG.
FIG. 41 is a flowchart showing the flow of interrupt processing in the present embodiment.
FIG. 42 is a flowchart showing the flow of the music reproduction process in step S150 of FIG.
FIG. 43 is a flowchart showing the guide registration process in step S151 of FIG.
FIG. 44 is another exemplary view of the guide in the present embodiment.
FIG. 45 is another exemplary view of the guide in the present embodiment.
FIG. 46 is another exemplary view of the guide in the present embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals, and the description thereof is incorporated.
FIG. 1 is a diagram showing an overall configuration of a music game system according to an embodiment of the present invention. As shown in FIG. 1, the music game system includes a music game apparatus 1, an operation article 150, and a television monitor 90.
The imaging unit 13 is incorporated in the housing 19 of the music game apparatus 1. The imaging unit 13 includes four infrared light emitting diodes 15 and an infrared filter 17. The light emitting part of the infrared light emitting diode 15 is exposed from the infrared filter 17.
A DC power supply voltage is applied to the music game apparatus 1 by an AC adapter 92. However, instead of the AC adapter 92, a DC power supply voltage can be applied by a battery (not shown).
The television monitor 90 is provided with a screen 91 on the front surface thereof. The television monitor 90 and the music game apparatus 1 are connected by an AV cable 93. Note that the music game apparatus 1 is placed on the upper surface of a television monitor 90, for example, as shown in FIG.
When the player 94 turns on a power switch (not shown) provided on the back of the music game apparatus 1, a game screen is displayed on the screen 91. The player 94 operates the operation article 150 according to the game screen guide to execute the game. Here, the operation of the operation article 150 means moving (for example, moving) the operation article itself, and does not include pressing a switch, moving an analog stick, or the like.
The infrared light emitting diode 15 of the imaging unit 13 emits infrared light intermittently. Infrared light from the infrared light emitting diode 15 is reflected by a reflection sheet (described later) attached to the operation article 150 and is input to an imaging device (described later) provided inside the infrared filter 17. In this way, the operation article 150 is photographed intermittently. Therefore, the music game apparatus 1 can acquire an intermittent video signal of the operation article 150 moved by the player 94. The music game apparatus 1 analyzes this video signal and reflects the analysis result in the game. The reflective sheet used in the present embodiment is, for example, a retroreflective sheet.
FIG. 2 is a perspective view of the operation article 150 of FIG. As shown in FIG. 2, the operation article 150 is configured by fixing a reflective ball 151 to the tip of a stick 152. The reflection ball 151 reflects infrared light from the infrared light emitting diode 15. Details of the reflective ball 151 will be described.
3A is a top view of the reflective ball 151 in FIG. 2, FIG. 3B is a side view of the reflective ball 151 from the direction of arrow A in FIG. 3A, and FIG. It is a side view of the reflective ball | bowl 151 from the arrow B direction of 3 (a).
As shown in FIGS. 3A to 3C, the reflective ball 151 includes a spherical inner shell inside a spherical outer shell 153 having a transparent color (including translucent, colored transparent, and colorless transparent). 154 is fixed. A reflective sheet 155 is attached to the spherical inner shell 154. The reflection sheet 155 reflects the infrared light from the infrared light emitting diode 15.
4 is a longitudinal sectional view of the reflective ball 151 of FIG. As shown in FIG. 4, the spherical outer shell 153 is formed by fixing two hemispherical outer shells with a boss 156 and a screw (not shown). The spherical inner shell 154 is formed by fixing two hemispherical inner shells with a boss 157 inside the spherical outer shell 153. Further, a stick 152 is inserted and fixed to the reflective ball 151. Specifically, the stick 152 is sandwiched between two hemispherical outer shells constituting the spherical outer shell 153 and two hemispherical inner shells constituting the spherical inner shell 154, and the two hemispherical outer shells are connected to the boss 156. In addition, the stick 152 is attached to the reflective ball 151 by fixing the two hemispherical inner shells with the boss 157 together with the screws.
FIG. 5 is an illustrative view showing one example of the imaging unit 13 of FIG. As shown in FIG. 5, the imaging unit 13 includes a unit base 35 formed by plastic molding, for example, and a support cylinder 36 is attached in the unit base 35. A trumpet-shaped opening 41 whose inner surface has an inverted conical shape is formed on the upper surface of the support cylinder 36, and a concave lens 39 is formed inside the cylindrical portion below the opening 41, for example, by molding of transparent plastic. An optical system including a convex lens 37 is provided, and an image sensor 43 as an image sensor is fixed below the convex lens 37. Therefore, the image sensor 43 can take an image corresponding to light incident from the opening 41 through the lenses 39 and 37.
The image sensor 43 is a low-resolution CMOS image sensor (for example, 32 pixels × 32 pixels: gray scale). However, the image sensor 43 may have a larger number of pixels or may be composed of other elements such as a CCD. In the following description, it is assumed that the image sensor 43 is composed of 32 pixels × 32 pixels.
In addition, a plurality (four in the embodiment) of infrared light emitting diodes 15 whose light emission directions are all upward are attached to the unit base 35. This infrared light emitting diode 15 irradiates infrared light above the imaging unit 13. An infrared filter (a filter that transmits only infrared light) 17 is attached above the unit base 35 so as to cover the opening 41. The infrared light emitting diode 15 functions as a stroboscope because lighting / extinguishing (non-lighting) is repeated continuously as will be described later. However, “stroboscope” is a general term for devices that illuminate a moving body intermittently. Therefore, the image sensor 43 photographs an object that moves within the photographing range, that is, the operation article 150 in the embodiment. As will be described later with reference to FIG. 8, the stroboscope mainly includes an infrared light emitting diode 15, an LED drive circuit 75, and a high-speed processor 200.
Here, the imaging unit 13 is incorporated in the housing 19 so that the light receiving surface of the image sensor 43 is inclined by a predetermined angle (for example, 90 degrees) from the horizontal plane. Further, the imaging range of the image sensor 43 by the concave lens 39 and the convex lens 37 is, for example, a range of 60 degrees.
FIG. 6 is a diagram showing an electrical configuration of the music game apparatus 1 of FIG. As shown in FIG. 6, the music game apparatus 1 includes an image sensor 43, an infrared light emitting diode 15, a video signal output terminal 47, an audio signal output terminal 49, a high speed processor 200, a ROM (read only memory) 51, and a bus. 53.
A bus 53 is connected to the high speed processor 200. Further, the ROM 51 is connected to the bus 53. Therefore, since the high speed processor 200 can access the ROM 51 via the bus 53, the game program stored in the ROM 51 can be read and executed, and the image data and music data stored in the ROM 51 are read. Thus, a video signal and an audio signal can be generated and output to the video signal output terminal 47 and the audio signal output terminal 49.
The operation article 150 is irradiated with infrared light emitted from the infrared light emitting diode 15, and the infrared light is reflected by the reflection sheet 155. The reflected light from the reflection sheet 155 is detected by the image sensor 43, and thus the image signal of the reflection sheet 155 is output from the image sensor 43. The analog image signal from the image sensor 43 is converted into digital data by an A / D converter (described later) built in the high speed processor 200. Similar processing is performed when the infrared light is turned off. The high speed processor 200 analyzes the digital data and reflects the analysis result in the game process.
FIG. 7 is a block diagram of the high speed processor 200 of FIG. As shown in FIG. 7, the high-speed processor 200 includes a central processing unit (CPU) 201, a graphic processor 202, a sound processor 203, a DMA (direct memory access) controller 204, a first bus arbitration circuit 205, Second bus arbitration circuit 206, internal memory 207, A / D converter (ADC: analog to digital converter) 208, input / output control circuit 209, timer circuit 210, DRAM (dynamic random access memory) refresh control circuit 211, external memory interface Circuit 212, clock driver 213, PLL (phase-locked loop) circuit 214, low voltage detection circuit 21 Includes first bus 218, and, second bus 219, a.
The CPU 201 performs various operations and controls the entire system according to a program stored in the memory (the internal memory 207 or the ROM 51). The CPU 201 is a bus master of the first bus 218 and the second bus 219, and can access resources connected to the respective buses.
The graphic processor 202 is a bus master of the first bus 218 and the second bus 219, generates a video signal based on data stored in the internal memory 207 or the ROM 51, and outputs the video signal to the video signal output terminal 47. . The graphic processor 202 is controlled by the CPU 201 through the first bus 218. The graphic processor 202 has a function of generating an interrupt request signal 220 to the CPU 201.
The sound processor 203 is a bus master of the first bus 218 and the second bus 219, generates an audio signal based on data stored in the internal memory 207 or the ROM 51, and outputs the audio signal to the audio signal output terminal 49. . The sound processor 203 is controlled by the CPU 201 through the first bus 218. Further, the sound processor 203 has a function of generating an interrupt request signal 220 to the CPU 201.
The DMA controller 204 manages data transfer from the ROM 51 to the internal memory 207. The DMA controller 204 has a function of generating an interrupt request signal 220 to the CPU 201 in order to notify the completion of data transfer. The DMA controller 204 is a bus master for the first bus 218 and the second bus 219. The DMA controller 204 is controlled by the CPU 201 through the first bus 218.
The internal memory 207 includes necessary ones among a mask ROM, an SRAM (static random access memory), and a DRAM. When it is necessary to hold SRAM data by a battery, the battery 217 is required. When a DRAM is installed, an operation for holding stored contents called refresh is required periodically.
The first bus arbitration circuit 205 receives a first bus use request signal from each bus master of the first bus 218, performs arbitration, and issues a first bus use permission signal to each bus master. Each bus master is permitted to access the first bus 218 by receiving the first bus use permission signal. Here, the first bus use request signal and the first bus use permission signal are shown as a first bus arbitration signal 222 in FIG.
The second bus arbitration circuit 206 receives a second bus use request signal from each bus master of the second bus 219, performs arbitration, and issues a second bus use permission signal to each bus master. Each bus master is permitted to access the second bus 219 by receiving the second bus use permission signal. Here, the second bus use request signal and the second bus use permission signal are shown as a second bus arbitration signal 223 in FIG.
The input / output control circuit 209 performs communication with an external input / output device or an external semiconductor element via an input / output signal. Input / output signals are read / written from the CPU 201 via the first bus 218. The input / output control circuit 209 has a function of generating an interrupt request signal 220 to the CPU 201.
The input / output control circuit 209 outputs an LED control signal LEDC for controlling the infrared light emitting diode 15.
The timer circuit 210 has a function of generating an interrupt request signal 220 for the CPU 201 based on a set time interval. The time interval and the like are set by the CPU 201 via the first bus 218.
The ADC 208 converts the analog input signal into a digital signal. This digital signal is read by the CPU 201 via the first bus 218. Further, the ADC 208 has a function of generating an interrupt request signal 220 to the CPU 201.
The ADC 208 receives pixel data (analog) from the image sensor 43 and converts it into digital data.
The PLL circuit 214 generates a high frequency clock signal obtained by multiplying the sine wave signal obtained from the crystal resonator 216.
The clock driver 213 amplifies the high frequency clock signal received from the PLL circuit 214 to a signal strength sufficient to supply the clock signal 225 to each block.
The low voltage detection circuit 215 monitors the power supply voltage Vcc, and issues a reset signal 226 of the PLL circuit 214 and other system-wide reset signals 227 when the power supply voltage Vcc is equal to or lower than a certain voltage. Further, when the internal memory 207 is composed of SRAM and data retention by the SRAM battery 217 is required, the battery backup control signal 224 is issued when the power supply voltage Vcc is equal to or lower than a predetermined voltage. .
The external memory interface circuit 212 has a function for connecting the second bus 219 to the external bus 53, and a function for controlling the bus cycle length of the second bus by issuing a cycle end signal 228 of the second bus 219. Have.
The DRAM refresh control circuit 211 unconditionally acquires the right to use the first bus 218 at regular intervals and performs a DRAM refresh operation. The DRAM refresh control circuit 211 is provided when the internal memory 207 includes a DRAM.
Here, a configuration for capturing pixel data from the image sensor 43 to the high-speed processor 200 will be described in detail with reference to FIGS.
FIG. 8 is a circuit diagram showing a configuration and an LED driving circuit for fetching pixel data from the image sensor 43 in FIG. 6 to the high-speed processor 200. FIG. 9 is a timing chart showing an operation when the pixel data is taken into the high speed processor 200 from the image sensor 43 of FIG. FIG. 10 is an enlarged timing diagram showing a part of FIG.
As shown in FIG. 8, since the image sensor 43 is of a type that outputs pixel data D (X, Y) as an analog signal, the pixel data D (X, Y) is an analog input port of the high-speed processor 200. Is input. The analog input port is connected to the ADC 208 in the high-speed processor 200, and thus the high-speed processor 200 acquires pixel data converted from the ADC 208 into digital data therein.
The midpoint of the analog pixel data D (X, Y) described above is determined by the reference voltage applied to the reference voltage terminal Vref of the image sensor 43. Therefore, a reference voltage generation circuit 59 composed of, for example, a resistance voltage dividing circuit is provided in association with the image sensor 43, and a reference voltage having a constant magnitude is always supplied from the circuit 59 to the reference voltage terminal Vref.
Each digital signal for controlling the image sensor 43 is supplied to or output from the I / O port of the high-speed processor 200. Each of these I / O ports is a digital port capable of controlling input / output, and is connected to the input / output control circuit 209 by this high speed processor 200.
More specifically, a reset signal reset for resetting the image sensor 43 is output from the output port of the high speed processor 200 and is supplied to the image sensor 43. Further, the image sensor 43 outputs a pixel data strobe signal PDS and a frame status flag signal FSF, and these signals are given to the input port of the high speed processor 200.
The pixel data strobe signal PDS is a strobe signal as shown in FIG. 9B for reading each pixel signal D (X, Y) described above. The frame status flag signal FSF is a flag signal indicating the state of the image sensor 43, and defines the exposure period of the image sensor 43 as shown in FIG. That is, the low level shown in FIG. 9A of the frame status flag signal FSF indicates the exposure period, and the high level shown in FIG. 9A indicates the non-exposure period.
The high speed processor 200 outputs a command (or command + data) to be set in a control register (not shown) of the image sensor 43 from the I / O port as register data, and repeats, for example, a high level and a low level. A set clock CLK is output and supplied to the image sensor 43.
As the infrared light emitting diode 15, four infrared light emitting diodes 15a, 15b, 15c and 15d connected in parallel with each other are used as shown in FIG. As described above, the four infrared light emitting diodes 15a to 15d irradiate infrared light in the same direction as the viewpoint direction of the image sensor 43 so as to illuminate the operation article 150, and the image sensor 43. Arranged to surround. However, these individual infrared light-emitting diodes 15a to 15d are simply referred to as infrared light-emitting diodes 15 unless it is particularly necessary to distinguish them.
The infrared light emitting diode 15 is turned on or off (not lit) by the LED drive circuit 75. The LED drive circuit 75 receives the above-described frame status flag signal FSF from the image sensor 43, and this flag signal FSF is given to the base of the PNP transistor 77 through a differentiation circuit 67 including a resistor 69 and a capacitor 71. A pull-up resistor 79 is further connected to the PNP transistor 77, and the base of the PNP transistor 77 is normally pulled up to a high level. When the frame status signal FSF becomes low level, the low level is input to the base via the differentiation circuit 67, so that the PNP transistor 77 is turned on only when the flag signal FSF is low level.
The emitter of the PNP transistor 77 is grounded via resistors 73 and 65. The connection point between the emitter resistors 73 and 65 is connected to the base of the NPN transistor 81. The collector of the NPN transistor 81 is commonly connected to the anodes of the infrared light emitting diodes 15a to 15d. The emitter of the NPN transistor 81 is directly connected to the base of another NPN transistor 61. The collector of the NPN transistor 61 is commonly connected to the cathodes of the infrared light emitting diodes 15a to 15d, and the emitter is grounded.
In this LED drive circuit 75, only when the LED control signal LEDC output from the I / O port of the high speed processor 200 is active (high level) and the frame status flag signal FSF from the image sensor 43 is low level. The infrared light emitting diode 15 is turned on.
As shown in FIG. 9A, when the frame status flag signal FSF becomes low level, the PNP transistor 77 is turned on during the low level period (although there is actually a delay due to the time constant of the differentiation circuit 67). Therefore, when the LED control signal LEDC shown in FIG. 9D is output from the high speed processor 200 at a high level, the base of the NPN transistor 81 becomes high level, and the transistor 81 is turned on. When the transistor 81 is turned on, the transistor 61 is turned on. Therefore, a current flows from the power source (indicated by small white circles in FIG. 8) through each of the infrared light emitting diodes 15a to 15d and the transistor 61. Accordingly, as shown in FIG. 9E, each of the infrared light emitting diodes 15a to 15d Illuminated.
In this way, in the LED drive circuit 75, the infrared light-emitting diode 15 is activated only when the LED control signal LEDC in FIG. 9D is active and the frame status flag signal FSF in FIG. 9A is at a low level. Since it is lit, the infrared light emitting diode 15 is lit only during the exposure period of the image sensor 43 (see FIG. 9F).
Therefore, useless power consumption can be suppressed. Further, since the frame status flag signal FSF is coupled by the capacitor 71, even if the flag signal FSF is stopped at a low level due to the runaway of the image sensor 43, the transistor 77 is always turned off after a certain time. The infrared light emitting diode 15 is also always turned off after a certain time.
As described above, by changing the duration of the frame status signal FSF, the exposure time of the image sensor 43 can be set or changed arbitrarily and freely.
Furthermore, by changing the duration and period of the frame status signal FSF and the LED control signal LEDC, the light emission period, non-light emission period, light emission / non-light emission period, etc. of the infrared light emitting diode 15, that is, the stroboscope can be arbitrarily and freely set. Can be changed or set.
As described above, when the operation article 150 is irradiated with the infrared light from the infrared light emitting diode 15, the image sensor 43 is exposed by the reflected light from the operation article 150. In response, the image sensor 43 outputs the pixel data D (X, Y) described above. More specifically, the image sensor 43 performs the pixel data strobe PDS shown in FIG. 9B during a period in which the frame status flag signal FSF in FIG. 9A is at a high level (non-lighting period of the infrared light emitting diode 15). In synchronism with this, analog pixel data D (X, Y) is output as shown in FIG.
The high speed processor 200 acquires digital pixel data through the ADC 208 while monitoring the frame status flag signal FSF and the pixel data strobe PDS.
However, as shown in FIG. 10C, the pixel data is output in the order of row 0, row 1,... Row 31. However, as will be described later, the first pixel of each row is dummy data.
Next, the game contents by the music game apparatus 1 will be described with specific examples.
FIG. 11 is a view showing an example of a game screen displayed on the screen 91 of the television monitor 90 of FIG. The game screen shown in FIG. 11 is a game start screen. As illustrated in FIG. 11, as a game start screen, a background 91, position guides G1 to G4, evaluation objects 107 to 109, a cursor 105, a dance object 106, and masks 141 and 142 are displayed on a screen 91. Then, automatic performance of the music is started.
In the example of the present embodiment, the position guides G1 to G4 are represented by flower figures, the evaluation objects 107 to 109 are represented by heart figures, and the dance object 106 is danced by men and women. The cursor 105 is expressed by a graphic imitating the operation article 150. Here, when the position guides G1 to G4 are expressed comprehensively, they are expressed as “position guide G”.
The cursor 105 represents the position of the operation article 150 on the screen 91, and moves on the screen 91 following the movement of the operation article 150. Therefore, from the viewpoint of the player 94, the operation of the operation article 150 and the operation of the cursor 105 are synonymous. The position guide G guides the operation timing and movement position of the cursor 105 (the operation article 150) at the timing of the music to be automatically played. Direction guides g1 to g5 described later guide the operation timing and moving direction of the cursor 105 (operation article 150) at the timing of music to be automatically played. The route guides rg1 to rg10 described later guide the operation timing, moving direction, and moving route of the cursor 105 (operation article 150) at the timing of the music to be automatically played. The evaluation objects 107 to 109 visually display the evaluation of the operation of the cursor 105 (operation article 150) by the player 94. Here, when the direction guides g1 to g5 are comprehensively expressed, they are expressed as “direction guide g”. In addition, when the route guides rg1 to rg10 are comprehensively expressed, they are described as “route guide rg”.
FIG. 12 is another exemplary view of the game screen displayed on the screen 91 of the television monitor 90 of FIG. As shown in FIG. 12, the animation in which the flower gradually opens according to the position guide G indicates a position where the cursor 105 is moved. As a result, the player 94 is instructed to move the cursor 105 to the area where the position guide G in which the flower opening animation is performed is displayed. The player 94 moves the operation article 150 to move the cursor 105 to the area where the position guide G in which the flower opening animation is displayed is displayed. After the flower opening animation, the position guide G is expressed by an animation in which the flower is deflated. Further, the direction of the position guide G that is performing an animation in which a flower opens indicates the direction in which the cursor 105 is moved. Thereby, the player 94 is instructed to move the cursor 105 in the direction of the position guide G in which the flower opening animation is performed.
In addition to the above, the direction in which the cursor 105 is moved is also guided by the direction guides g1 to g5. That is, the direction guides g1 to g5 appear in the order of the direction guide g1, the next direction guide g2, the next direction guide g3, the next direction guide g4, and the next direction guide g5. Therefore, the direction in which the cursor 105 is moved is guided by the appearance direction of the direction guides g1 to g5. Here, each of the direction guides g1 to g5 is a figure that represents a small sphere at the beginning, and the sphere gradually increases with time. Is executed. Therefore, the direction in which the spherical figure appears is the direction in which the cursor 105 is moved.
The player 94 must move the cursor 105 to an area where the position guide G is displayed during a predetermined time when the flower by the position guide G opens. That is, the position guide G guides the operation timing of the cursor 105 by an animation in which a flower opens. In addition, the player 94 must move the cursor 105 to an area where the position guide G where the flower is opened is displayed for a predetermined time after the figure of the sphere by the last direction guide g appears. . That is, the operation timing of the cursor 105 is also guided by the direction guide g.
The position guide G also has a function of notifying the operation direction of the cursor 105. That is, when the flower bud starts to open by the position guide G, the player 94 can know the direction in which the cursor 105 is moved next. Further, the direction guide g also has a function of notifying the operation direction of the cursor 105. That is, since the direction guide g appears before the operation timing arrives, the player 94 can also know the direction in which the cursor 105 is moved next.
The above points will be described with specific examples. In the example of FIG. 12, the position where the cursor 105 is moved is guided by an animation in which the flower gradually opens by the position guide G2. Thereby, the player 94 is instructed to move the cursor 105 to the area where the position guide G2 in which the flower opening animation is performed is displayed. Further, the direction of the position guide G2 in which the flower opening animation is performed is the direction in which the cursor 105 is moved. Thereby, the player 94 is instructed to move the cursor 105 in the direction of the position guide G2 in which the flower opening animation is performed. Furthermore, a spherical figure formed by the direction guides g1 to g5 appears from the position guide G1 toward the position guide G2. As described above, the direction guides g1 to g5 guide the movement of the cursor 105 from the position guide G1 to the position guide G2.
The player 94 must move the cursor 105 to a region where the position guide G2 is displayed during a predetermined time when the flower is opened by the position guide G2. In addition, the player 94 must move the cursor 105 to the area where the position guide G2 where the flower is opened is displayed for a predetermined time after the sphere figure by the last direction guide g5 appears. . That is, the operation timing of the cursor 105 is also guided by the direction guide g.
The player 94 moves the cursor 105 from the position guide G1 to the position guide G2 by appropriately operating the operation article 150 according to the guidance by the position guide G2 and the direction guides g1 to g5. Accordingly, an animation in which all of the evaluation objects 107 to 109 blink is executed. When the cursor 105 is operated at the most appropriate timing, an animation in which all of the evaluation objects 107 to 109 blink is executed, and the cursor 105 is operated at an allowable range timing, although not the most appropriate timing. In this case, an animation in which only the evaluation object 108 blinks is executed. Note that the position guides G1, G3, and G4 are flower bud shapes because they are not time zones for guiding the operation timing and movement position of the cursor 105. Further, the operation timing and the operation direction of the cursor 105 are guided between the position guide G2 and the position guide G4, between the position guide G4 and the position guide G3, and between the position guide G3 and the position guide G1. Since it is not a time zone, the direction guide g does not appear.
Since the player 94 appropriately operates the cursor 105 according to the guidance by the position guide G2 and the direction guides g1 to g5, the moving direction of the cursor 105 (from the position guide G1 to the position guide G2, that is, toward the screen 91). Animation to perform the dance in the direction corresponding to (right direction). For example, the dance object 106 performs an animation that rotates counterclockwise, and the background 110 scrolls leftward toward the screen 91. As a result, the dance object 106 appears to have rotated counterclockwise while moving in the right direction even though the position of the dance object 106 is at the center of the screen 91.
FIG. 13 is a view showing still another exemplary game screen displayed on the screen 91 of the television monitor 90 of FIG. In FIG. 13, the four position guides G1 to G4 have an animation in which a flower opens at the same time. With this as an opportunity, the player 94 is instructed to move the cursor 105 in the route and direction guided by the route guides rg1 to rg10. In this case, the appearance positions of the route guides rg1 to rg10 indicate the guide route of the cursor 105. Further, the route guides rg1 to rg10 appear in the order of the route guide rg1, then the route guide rg2, then the route guide rg3, then the route guide rg4,..., And finally the route guide rg10. Therefore, the direction in which the cursor 105 is moved is guided by the direction in which the route guides rg1 to rg10 appear. Here, each of the route guides rg1 to g10 is a figure that represents a small sphere at the beginning of the appearance, and the sphere gradually increases with time. Is executed. In FIG. 13, it is guided that the cursor 105 is moved counterclockwise along the route guides rg1 to g10, starting from the vicinity of the position guide G3.
When the player 94 operates the operation article 150 and appropriately operates the cursor 105 according to the position guides G1 to G4 and the route guides rg1 to rg10, the dance object 106 displays an animation corresponding to the route guides rg1 to rg10 (for example, , An animation that rotates greatly counterclockwise).
As described above, the objects such as the dance object 106 shown in FIGS. 12 and 13 are images of a certain frame of animation. For example, a series of dance objects 106 are prepared for dance animation. For example, a series of flower graphic object images are prepared for the animation of the position guide G. For example, a series of spherical object images are prepared for animation of the direction guide g and the route guide rg.
Here, the dance object 106, the position guide G, the evaluation objects 107 to 109, the cursor 105, the direction guide g, and the route guide rg on the game screen shown in FIGS. 11 to 13 are composed of one or more sprites. The sprite is a rectangular pixel set and can be arranged at an arbitrary position on the screen 91. The position guide G, the evaluation objects 107 to 109, the cursor 105, the direction guide g, and the route guide rg may be collectively referred to as an object (or an object image).
FIG. 14 is an explanatory diagram of sprites that constitute objects displayed on the screen 91. As shown in FIG. 14, the dance object 106 of FIG. 11 is composed of, for example, 12 sprites SP0 to SP11. Each of the sprites SP0 to SP11 includes, for example, 16 pixels × 16 pixels. When the dance object 106 is arranged on the screen 91, for example, the coordinate on which the center of the upper left corner sprite SP0 is arranged is designated. Based on the designated coordinates and the sizes of the sprites SP0 to SP11, the coordinates for arranging the centers of the sprites SP1 to SP11 are calculated.
Next, scrolling of the background 110 will be described. First, the panoramic view screen will be described.
FIG. 15 is an explanatory diagram of a panoramic screen displayed on the screen 91 of the television monitor 90 of FIG. As shown in FIG. 15, the panoramic view screen 140 includes, for example, 32 × 32 blocks “0” to “1023”. Each of the blocks “0” to “1023” is, for example, a rectangular element composed of 8 pixels × 8 pixels. Corresponding to the blocks “0” to “1023”, an array PA [0] to an array PA [1023] and an array CA [0] to CA [1023] are prepared. Here, when the block “0” to the block “1023” are comprehensively expressed, it is simply expressed as “block”, and when the array PA [0] to the array PA [1023] is comprehensively expressed, “ When it is expressed as “array PA” and array CA [0] to array CA [1023] is comprehensively expressed, it is expressed as “array CA”.
Now, data (pixel pattern data) designating the pixel pattern of the corresponding block is substituted into the array PA. This pixel pattern data is color information of each pixel of 8 pixels × 8 pixels constituting the block. Also, information for designating the color palette and depth value used for the corresponding block is substituted into the array CA. The color palette includes a predetermined number of color information. The depth value is information representing the depth of the pixel, and when a plurality of pixels are present at the same position, only the pixel having the largest depth value is displayed.
FIG. 16A is an explanatory diagram before scrolling the panoramic view screen 140, and FIG. 16B is an explanatory diagram after scrolling the panoramic view screen 140. As shown in FIG. 16A, since the size of the screen 91 of the television monitor 90 is 256 pixels × 224 pixels, a range of 256 pixels × 224 pixels of the panoramic screen 140 is displayed on the screen 91. . Here, consider that the center position of the panoramic screen 140 is scrolled to the left by k pixels. Then, since the width in the horizontal direction (horizontal direction) of the panoramic view screen 140 is the same as the width in the horizontal direction of the screen 91, as shown in FIG. Portion) is displayed at the right end of the screen 91. That is, conceptually, when scrolling in the horizontal direction, it can be considered that the same plurality of panoramic screens 140 are connected in the horizontal direction.
For example, if the portion outside the range of the screen 91 (shaded portion) is the block “64”, the block “96”,..., The block “896”, the block “928” in FIG. Images determined by the array PA [64],... PA [928] and the array CA [64],... CA [928] are displayed on the right end of the screen 91. Therefore, in order to make the background continuously continuous by left-scrolling the panoramic view screen 140, the background is substituted into the arrays PA and CA corresponding to the blocks included in the portion outside the range of the screen 91 (shaded portion). Data needs to be updated. Then, an image determined by the updated array PA and array CA is displayed on the right end of the screen 91.
In order to make the background appear smooth and continuous, it is necessary to update the data of the corresponding array PA and array CA before being displayed on the right end of the screen 91. Then, when the data is being displayed at the left end of the screen 91, it is necessary to update the data in the corresponding array PA and array CA, and the display at the left end of the screen 91 becomes discontinuous. Therefore, in order to avoid such an inconvenience, a mask 141 is placed on the left end of the screen 91 as shown in FIGS. For the same reason, a mask 142 is also provided at the right end.
Note that the scrolling process in the right direction is the same as the scrolling process in the left direction. In this embodiment, since the scroll range in the vertical direction (vertical direction) of the panoramic screen 140 is ± 16 pixels, the upper and lower ends of the screen 91 are not masked.
As described above, the background 110 is scrolled by scrolling the panoramic view screen 140.
Next, details of the game processing by the music game apparatus 1 will be described. FIG. 17 is a conceptual diagram showing programs and data stored in the ROM 51 of FIG. As shown in FIG. 17, the ROM 51 stores a game program 300, image data 301, and music data 304. The image data 302 includes object image data (including image data such as a position guide G, a direction guide g, a route guide rg, evaluation objects 107 to 109, and a cursor 105) and background image data. The music data 304 includes first musical score data 305, second musical score data 306, and sound source data (waveform data) 307.
The first musical score data 305 shown in FIG. 17 is data in which music control information is arranged in time series.
FIG. 18 is a conceptual diagram showing an example of the first musical score data 305 of FIG. As shown in FIG. 18, the music piece control information includes a command, note number / standby time information, instrument designation information, velocity, and gate time.
Note-on is a command for making a sound, and standby is a command for setting a standby time. The waiting time is the time until the next command is read (the time from one note to the next note). The note number is information for designating the pitch (pitch) of the sound. The standby time information is information that specifies a standby time to be set. The instrument designation information is information for designating which instrument tone is used. The velocity is information on the strength of the sound, that is, information on the volume. Gate time is the length of time a sound is produced.
Returning to FIG. 17, the second musical score data 306 is data in which guide control information is arranged in time series. The second musical score data 306 is used when guides (position guide G, direction guide g, and route guide rg) are displayed on the screen 91. That is, the first musical score data 305 is musical score data for automatically playing a musical piece, while the second musical score data 306 is musical score data for displaying a guide at a timing suitable for the musical piece.
FIG. 19 is a conceptual diagram showing an example of the second musical score data 306 of FIG. As shown in FIG. 19, the guide control information includes a command, note number / standby time information, and instrument designation information.
In the second musical score data 306, the musical instrument designation information is not a number indicating the musical instrument (tone) that plays the music, but the second musical score data 306 is a guide (position guide G, direction guide g, and route guide rg). Is a number indicating that the musical score data is to appear.
Therefore, note-on is not a command for making a sound, but a command for instructing the start of the animation of the position guide G or instructing the start of appearance of the direction guide g and the route guide rg. The note number is not information for designating the pitch (pitch) of sound, but indicates which position guide G animation is started, and indicates where the direction guide g and route guide rg appear. Information. This point will be described in detail.
FIG. 20A to FIG. 20C are relationship diagrams between the direction in which the cursor 105 is guided and the note number. As shown in FIGS. 20A to 20C, the direction of the arrow indicates the direction in which the cursor 105 is guided, the start point of the arrow indicates the position of the position guide G that has guided the cursor 105 immediately before, The end point of the arrow indicates the position of the position guide G that guides the cursor 105. For example, as shown in FIG. 20A, the note number “55” is used when the cursor 105 is guided from the position guide G1 to the position guide G2, and the note number indicated by the score data pointer is “55”. In this case, a position guide G and a direction guide g shown in FIG. 12 appear. Further, for example, as shown in FIG. 20C, the note number “57” is used when guiding the rotation of the cursor 105 counterclockwise starting from the position guide G3, and the score data pointer is When the indicated note number is “57”, the position guide G and the route guide rg shown in FIG. 13 appear.
For example, the note number “81” is dummy data arranged at the head of the second musical score data 306 (see FIG. 19), and is not information for controlling the display of the guide. In this way, the first musical score data 305 and the second musical score data 306 are aligned at the beginning. Further, for example, the note number “79” is data indicating the end of music and is arranged at the end of the second musical score data 306 (see FIG. 19). Note number “79” is not information for controlling display of the guide.
Next, main processing executed by the high speed processor 200 will be described.
[Pixel Data Group Acquisition Processing] The CPU 201 acquires digital pixel data obtained by converting analog pixel data output from the image sensor 43, and substitutes it into the array P [X] [Y]. The horizontal direction (lateral direction, row direction) of the image sensor 43 is taken as the X axis, and the vertical direction (longitudinal direction, column direction) is taken as the Y axis.
[Difference Data Calculation Processing] The CPU 201 calculates the difference between the pixel data P [X] [Y] when the infrared light emitting diode 15 is turned on and the pixel data P [X] [Y] when the infrared light emitting diode 15 is turned off. The difference data is substituted into the array Dif [X] [Y]. Here, the effect of obtaining the difference will be described with reference to the drawings. Here, the pixel data represents luminance. Therefore, the difference data also represents luminance.
FIG. 21A is an illustration of an image taken by a general image sensor that is not subjected to special processing, and FIG. 21B is a level discrimination of the image signal of FIG. 21A with a certain threshold. FIG. 21C is a diagram illustrating an image signal when the image sensor 43 is turned on via the infrared filter 17, and FIG. 21C is a diagram illustrating the image signal when the image signal is discriminated by a certain threshold. FIG. 21D is an exemplary diagram of an image signal when the image signal when the image sensor 43 is turned off via the infrared filter 17 is discriminated by a certain threshold, and FIG. 21E shows the image signal when the image sensor 43 is turned on. It is an illustration figure of the difference signal with the image signal at the time of light extinction.
As described above, the operation article 150 is irradiated with infrared light, and an image of reflected infrared light incident on the image sensor 43 via the infrared filter 17 is captured. When the operation article 150 is photographed with a stroboscope using a general light source in a general indoor environment, a general image sensor (corresponding to the image sensor 43 in FIG. 5) is shown in FIG. As shown in FIG. 5, in addition to the image of the operation article 150, not only a light source such as a fluorescent light source, an incandescent light source, and sunlight (window), but also all images of the room. Therefore, processing the image of FIG. 21A to extract only the image of the operation article 150 requires a fairly high speed computer or processor. However, such a high-performance computer cannot be used in an inexpensive device. Therefore, it is conceivable to reduce the burden by performing various processes.
Note that the image in FIG. 21A is originally an image represented by black and white gradations, but the illustration thereof is omitted. In FIGS. 21A to 21E, the reflection sheet 155 of the operation article 150 is photographed.
FIG. 21B is an image signal when the image signal of FIG. 21A is level-discriminated by a certain threshold value. Such level discrimination processing can be executed either by a dedicated hardware circuit or by software. However, by any method, if level discrimination that cuts pixel data of a certain amount or less is executed, the operation article Low luminance images other than 150 and the light source can be removed. In the image of FIG. 21B, the processing of the image other than the operation article 150 and the light source in the room can be omitted, and thus the burden on the computer can be reduced. However, the high-intensity image including the light source image is still captured. Therefore, it is difficult to separate the operation article 150 from other light sources.
Therefore, as shown in FIG. 5, the infrared filter 17 is used to prevent the image sensor 43 from capturing an image other than an infrared image. Thereby, as shown in FIG. 21 (c), the image of the fluorescent light source that hardly contains infrared light can be removed. However, sunlight and incandescent lamps are still included in the image signal. Therefore, in order to further reduce the burden, the difference between the pixel data when the infrared stroboscope is turned on and the pixel data when the infrared stroboscope is turned off is calculated.
Therefore, the difference between the pixel data of the image signal when turned on in FIG. 21C and the pixel data of the image signal when turned off in FIG. 21D was calculated. Then, as shown in FIG.21 (e), the image only for the difference can be acquired. The image based on the difference data includes only the image obtained by the operation article 150, as is clear from comparison with FIG. Therefore, the state information of the operation article 150 can be acquired while reducing the processing. The state information is, for example, one of speed information, moving direction information, moving distance information, speed vector information, acceleration information, moving trajectory information, area information, or position information, or a combination of two or more thereof. , Etc.
For the reasons described above, the CPU 201 calculates the difference between the pixel data when the infrared light emitting diode 15 is turned on and the pixel data when the infrared light emitting diode 15 is turned off, and obtains difference data.
[Attention Point Extraction Processing] The CPU 201 obtains the coordinates of the attention point of the operation article 150 based on the calculated difference data Dif [X] [Y]. This point will be described in detail.
FIG. 22 is an explanatory diagram of the coordinate calculation of the attention point of the operation article 150. The image sensor 43 shown in FIG. 22 is assumed to be 32 pixels × 32 pixels.
As shown in FIG. 22, the CPU 201 scans differential data for 32 pixels in the X direction (horizontal direction, horizontal direction, row direction), increments the Y coordinate, and stores differential data for 32 pixels in the X direction. And the Y coordinate is incremented, and the difference data for 32 pixels is scanned in the X direction while incrementing the Y coordinate.
In this case, the CPU 201 obtains the maximum brightness value difference data from the scanned difference data of 32 pixels × 32 pixels, and compares the maximum brightness value with a predetermined threshold Th. Then, when the maximum luminance value is larger than the predetermined threshold Th, the CPU 201 calculates the coordinates of the attention point of the operation article 150 based on the coordinates of the pixel having the maximum luminance value. This point will be described in detail.
FIG. 23A is an explanatory diagram of an X-direction scan when calculating the point-of-interest coordinates of the operation article 150 based on the coordinates of the pixel having the maximum luminance value, and FIG. 23B has the maximum luminance value. FIG. 23C is an explanatory diagram at the start of the Y-direction scan when calculating the point-of-interest coordinates of the operation article 150 based on the pixel coordinates, and FIG. 23C shows the operation article based on the coordinates of the pixel having the maximum luminance value. FIG. 23D is an explanatory diagram of Y-direction scanning when calculating 150 attention point coordinates, and FIG. 23D is a result of calculating the attention point coordinates of the operation article 150 based on the coordinates of the pixel having the maximum luminance value. It is explanatory drawing.
As shown in FIG. 23A, the CPU 201 scans the difference data in the X direction around the coordinates of the pixel having the maximum luminance value, and detects a pixel having a luminance value larger than the predetermined threshold Th. . In the example of FIG. 23A, X = 11 to 15 are pixels that exceed a predetermined threshold Th.
Next, as shown in FIG. 23B, the CPU 201 obtains the center of X = 11-15. Then, the X coordinate of the center is Xc = 13.
Next, as shown in FIG. 23C, the difference data is scanned in the Y direction around the X coordinate (= 13) obtained in FIG. 23B, and is larger than a predetermined threshold Th. Detect pixels with luminance values. In the example of FIG. 23C, Y = 5 to 10 are pixels that exceed a predetermined threshold Th.
Next, as shown in FIG. 23D, the CPU 201 obtains the center of Y = 5 to 10. Then, the Y coordinate of the center is Yc = 7.
The CPU 201 converts the coordinates (Xc, Yc) = (13, 7) of the attention point calculated as described above into coordinates (xc, yc) on the screen 91. The CPU 201 executes the process for obtaining the coordinates (xc, yc) of the attention point as described above every time the frame is updated. Then, the CPU 201 assigns xc and yc to the arrays Px [M] and Py [M], respectively. “M” is an integer, and is incremented by one each time the frame displayed on the screen 91 is updated.
[Attention Point Existing Area Determination Process (1)] The CPU 201 determines in which area a1 to a4 on the screen 91 the attention point of the operation article 150 exists. This point will be described in detail.
FIG. 24 is an explanatory diagram of attention point presence area determination processing (2) performed by the CPU 201. As shown in FIG. 24, the screen 91 includes a predetermined area a1 including the position guide G1, a predetermined area a2 including the position guide G2, a predetermined area a3 including the position guide G3, and a predetermined area a4 including the position guide G4. Set. The CPU 201 determines a region to which the attention point (xc, yc) of the operation article 150 belongs from among the predetermined regions a1 to a4, and stores the determination result in the array J1 [M]. The CPU 201 executes the determination process as described above every time the frame displayed on the screen 91 is updated.
[Attention Point Existing Area Determination Process (2)] The CPU 201 determines in which areas A1 to A4 on the screen 91 the attention point of the operation article 150 exists. This point will be described in detail.
FIG. 25 is an explanatory diagram of the attention point presence area determination process (2) by the CPU 201. As shown in FIG. 25, areas A1 to A4 obtained by dividing the screen 91 into four parts are set. The CPU 201 determines the area to which the attention point (xc, yc) of the operation article 150 belongs from the areas A1 to A4, and stores the determination result in the array J2 [M]. The CPU 201 executes the determination process as described above every time the frame displayed on the screen 91 is updated.
[Cursor control processing] The CPU 201 registers the coordinates (xc, yc) of the current point of interest of the operation article 150 as the coordinates of the cursor 105 to be displayed in the next frame (stored in the internal memory 207).
[Guide Type Registration Processing] The CPU 201 arranges the note numbers (see FIG. 19 and FIGS. 20A to 20C) read from the second score data 306 according to the guide score data pointer in the array NN “0”. Or substituted into the array NN [1]. As described above, the number of elements in the array is set to two when the guidance by a certain position guide G and the guidance by another position guide G are started at different timings, but are executed repeatedly for a certain period of time. Because there is also. Note that the musical score data pointer for guide is a pointer indicating a reading position of the second musical score data 306.
[Guide Control Processing] The CPU 201 refers to the array NN [J] (guide display number J = 0, 1) and refers to the animation information and position of the direction guide G according to the note number assigned to the array NN [J]. The animation information of the guide g and the animation information of the route guide rg are registered. This point will be described in detail.
FIG. 26 is an explanatory diagram of an animation registration process for the direction guide G, the position guide g, and the route guide rg. As shown in FIG. 26, the note number and animation information (storage position information of the animation table of the position guide G, display coordinates of the position guide G on the screen 91, display timing information of the position guide G, direction guide g / path guide) rg animation table storage position information, direction guide g / route guide rg display coordinate information on screen 91, and direction guide g / route guide rg display timing information) are associated with ROM 51 or Prepared in the internal memory 207.
The note numbers in this table are note numbers for controlling the display of guides, and are the note numbers shown in FIGS. 20 (a) to 20 (c). For example, when the note number assigned to the array NN [J] is “55”, the CPU 201 refers to this table and refers to the animation information associated with the note number “55” (animation table of the position guide G). Storage position information, display coordinates of the position guide G on the screen 91, display timing information of the position guide G, storage position information of the animation table of the direction guide g, display coordinate information of the direction guide g on the screen 91, and direction The display timing information of the guide g) is registered (stored in a predetermined area of the internal memory 207).
Here, the display timing information is information indicating when the object is displayed on the screen 91. For example, for the guide number “55”, since the display timing information of the position guide G is “0”, the position guide G2 is displayed at the coordinates (x1, y1) in the frame next to the currently displayed frame. Means. For example, for the guide number “55”, since the display timing information of the direction guide g is 0, 6, 12,..., 24, the position guide g1 is set to the coordinates ( x3, y1), the position guide g2 is displayed at coordinates (x4, y1) after 6 frames,..., and the position guide g5 is displayed at coordinates (x7, y1) after 24 frames.
FIG. 27 is a view showing an example of an animation table specified by the animation table storage position information of FIG. As shown in FIG. 27, the animation table includes storage position information of animation image data (a plurality of object image data arranged in time series), a number of objects to be animated arranged in time series, and how many frames continuous. This is a table in which information (the number of sustained frames) indicating whether to display each object, the object size, the color palette information, the depth value information, and the sprite size are associated with each other. The animation image data is pixel pattern data. Here, the pixel pattern data, the color palette, and the depth value relate to the sprites constituting the object, and their meanings are the same as those related to the blocks described in FIG.
The animation table indicated by the animation table storage position information address0 is an example of the animation table of the position guide G. The animation table indicated by the animation table storage position information address1 is an example of the animation table of the direction guide g, and the animation table storage position. The animation table indicated by the information address 2 is an example of the animation table of the route guide rg, and the animation table indicated by the animation table storage position information address 3 is an animation of the position guide G used when the operation of the cursor 105 by the player 94 is successful. It is an illustration of a table.
[Dance Control Processing] The CPU 201 determines whether or not the operation of the cursor 105 by the player 94 is operated in conformity with the position guide G and the direction guide g or the position guide G and the route guide rg. Specifically, it is as follows.
The CPU 201 moves the cursor 105 (that is, the attention of the operation article 150) to the area currently guided by the position guide G and the direction guide g based on the determination result J1 [M] by the attention point presence area determination process (1). It is determined whether or not (point) exists (see FIG. 24). For example, when the area where the position guide G and the direction guide g are currently guiding is the area a2, and the area where the cursor 105 exists is the area a2, the CPU 201 determines that the cursor 105 is the position guide G and the direction guide g. Judged that it was operated in conformity with
In addition, the CPU 201 moves the cursor 105 (that is, the operation) along the route currently guided by the position guide G and the route guide rg based on the determination result J2 [M] by the attention point presence region determination process (2). It is determined whether or not the attention point of the object 150 has moved (see FIG. 25). Here, the route guided by the guide number “53” in FIG. 20B is area A3 → area A1 → area A2 → area A4 in FIG. Further, the route guided by the guide number “57” in FIG. 20C is the area A4 → the area A2 → the area A1 → the area A3 in FIG. Accordingly, for example, when the route currently guided by the position guide G and the route guide rg is a route corresponding to the guide number “53”, the CPU 201 determines that the route to which the cursor 105 has moved is the region A3 → the region A1 → In the case of region A2 → region A4, it is determined that the cursor 105 has been operated in conformity with the position guide G and the route guide rg.
As a result of the above, if the CPU 201 determines that the operation of the cursor 105 by the player 94 is operated in conformity with the position guide G and the direction guide g or the position guide G and the route guide rg, the CPU 201 Dance animation information corresponding to the direction guide g or the position guide G and the route guide rg is registered (stored in a predetermined area of the internal memory 207). Similar to the table of FIG. 26, a table in which the note number for controlling the display of the guide (see FIGS. 20A to 20C) and dance animation information is associated is prepared in the ROM 51 or the internal memory 207. . However, for the note numbers having the same guidance direction (for example, note numbers “55” and “67”), the dance animation information is also the same.
The dance animation information is the same as the animation information of FIG. 26, and includes storage position information of the dance animation table, display coordinates on the screen 91 of the dance object 106, and display timing information of the dance object 106. Also, the dance animation table is the same as the animation table of FIG. 27, storage position information of dance animation image data (image data of a plurality of dance objects 106 arranged in time series), and a dance object that performs animation. 106 numbers arranged in chronological order, information indicating how many frames each dance object 106 is continuously displayed (number of continuous frames), the size of the dance object 106, color palette information, depth value information, and , Sprite size. The dance animation image data is pixel pattern data.
Note that the second musical score data 306 in FIG. 17 can include a note number for instructing to perform a fast-speed dance animation and a note number for instructing to perform a slow-speed dance animation. In this case, a dance animation table with a high speed and a dance animation table with a low speed are also prepared. In this case, a table in which the note number for controlling the display of the guide, the note number for controlling the dance speed, and the dance animation information are associated is prepared in the ROM 51 or the internal memory 207. Similarly, a dance animation table is prepared for each dance speed.
If it is determined that the operation of the cursor 105 by the player 94 is performed in conformity with the position guide G and the direction guide g or the position guide G and the route guide rg, the CPU 201 determines that the operation is successful. The storage position information (address 3 in the example of FIG. 27) of the animation table of the position guide G to be used is registered.
On the other hand, when determining that the operation of the cursor 105 by the player 94 is performed in conformity with the position guide G and the direction guide g or the position guide G and the route guide rg, the CPU 201 determines whether the evaluation objects 107 to 109 have been operated. Animation information is registered (stored in a predetermined area of the internal memory 207). This animation information is the same as the animation information of FIG. Accordingly, the animation information includes storage position information of the animation table for the evaluation objects 107 to 109. This animation table is the same as the animation table of FIG.
Further, when it is determined that the operation of the cursor 105 by the player 94 is operated in conformity with the position guide G and the direction guide g or the position guide G and the route guide rg, the CPU 201 determines that the position guide G and the direction guide. or scroll control corresponding to the position guide G and the route guide rg. Specifically, the CPU 201 changes the center position of the panoramic screen 140 corresponding to the position guide G and direction guide g, or the position guide G and path guide rg, and the note number that controls the dance speed. (See FIGS. 16A and 16B), the background 110 is scrolled. Further, when scrolling the panoramic screen 140 in the horizontal direction, the CPU 201 changes the data of the corresponding array PA and array CA.
FIG. 28 is a timing chart for explaining the relationship between the first musical score data 305, the second musical score data 306, the direction guide G, the position guide g, the operation determination, and the dance animation. In FIG. 28, the thick line indicates the processing execution period, the black circle at the left end of the thick line indicates the start of the process, and the black circle on the right side of the thick line indicates the end of the process.
As shown in FIG. 28, the reading start time of the second score data 306 is earlier by a predetermined time t than the reading start times T1, T2,... Of the first score data 305. Therefore, the direction guide g is displayed earlier by a predetermined time t than the time T1 to T3 when the corresponding note number of the first score data 305 is read, and the corresponding note number of the first score data 305 is read. Displayed from time T1 to T3 (for example, for 60 frames). Similarly, the position guide G starts the animation by a predetermined time t earlier than the time T1 to T3 when the corresponding note number of the first score data 305 is read, and the corresponding note number of the first score data 305 is read. When is read, the animation continues until a little after T1 to T3.
The CPU 201 determines whether or not the operation of the cursor 105 is performed in conformity with the direction guide g and the position guide G after a certain period of time has elapsed from the start of display of the direction guide g (for example, after 30 frames have elapsed). Start and end at T1 to T3 when the corresponding note number of the first musical score data 305 is read. If the CPU 201 determines that the operation of the cursor 105 is performed in conformity with the direction guide g and the position guide G, the CPU 201 registers dance animation information at the end of the determination period. Accordingly, in this case, a dance animation is executed based on the registered dance animation information.
The reason why the start point of reading the second score data 306 is earlier than the start point of reading T1, T2,... Of the first score data 305 by a predetermined time t is as follows. In other words, the player 94 starts the operation of the operation article 150 after starting the guidance by the direction guide g and the position guide G, and therefore adjusts the time error so that the direction guide g and the position guide G are displayed from the timing of the music. It was too early.
The display timing of the route guide rg is the same as the display timing of the direction guide G. However, for example, whether or not the operation of the cursor 105 is suitable for the route guide rg is determined from the start to the end of the guidance by the route guide rg (for example, during 60 frames).
[Image Display Processing] The CPU 201 displays the information necessary for drawing in the vertical blanking period based on the information registered by the cursor control processing, the guide control processing, and the dance control processing. To give. Then, the graphic processor 202 generates a video signal based on the given information and outputs it to the video signal output terminal 47. As a result, a game screen including the position guide G and the background 110 is displayed on the screen 91 of the television monitor 90. More specifically, it is as follows.
The CPU 201 calculates display coordinates of each sprite constituting the cursor 105 based on the coordinate information registered in the cursor control process (coordinate information of the attention point of the operation article 150). Then, the CPU 201 provides the graphic processor 202 with display coordinate information, color palette information, depth value, size information, and pixel pattern data storage position information of each sprite constituting the cursor 105. Based on these pieces of information, the graphic processor 202 generates a video signal representing the cursor 105 and outputs it to the video signal output terminal 47.
Further, the CPU 201 constructs an animation image of a guide (position guide G, direction guide g, route guide rg) by referring to the animation table based on the animation table storage position information included in the animation information registered in the guide control process. The size information of the object to be processed and the size information of the sprite constituting the object are acquired. Then, the CPU 201 calculates the display coordinates of each sprite constituting the object based on these pieces of information and the display coordinate information included in the registered animation information. Further, the CPU 201 based on the object number of the position guide G to be displayed, the size information of the object and sprite included in the animation table, and the animation image data storage position information of the position guide G included in the animation table. The pixel pattern data storage position information of each sprite constituting the object is calculated.
Further, the CPU 201 refers to the animation table, and displays color palette information, depth value, and size information of each sprite constituting the position guide G, pixel pattern data storage position information and display coordinate information of each sprite. At the same time, it is given to the graphic processor 202. In this case, the CPU 201 gives the above information to the graphic processor 202 according to the display timing information of the position guide G included in the registered animation information and the number of sustained frames in the animation table.
Regarding the direction guide g and the route guide rg, the acquisition and content of the information that the CPU 201 gives to the graphic processor 202 is the same as that of the position guide G. However, the direction guides g1 to g4 and the route guides rg1 to rg10 are sequentially displayed at a plurality of positions indicated by the display coordinate information included in the animation information at the timing indicated by the display timing information included in the animation information. At the start of display of each of the direction guides g1 to g4 and each of the route guides rg1 to rg10, the CPU 201 refers to the display coordinate information and the display timing information included in the registered animation information, and refers to the direction guides g1 to g1. Information on g4 and route guides rg1 to rg10 is given to the graphic processor 202.
Based on the information given as described above, the graphic processor 202 generates a video signal representing a guide (position guide G, direction guide g, route guide rg) and outputs it to the video signal output terminal 47. .
In addition, the CPU 201 refers to the dance animation table based on the dance animation table storage position information included in the dance animation information registered in the dance control process, and size information of the dance object 106 constituting the dance animation image, and the dance The size information of sprites constituting the object 106 is acquired. Then, the CPU 201 calculates the display coordinates of each sprite constituting the dance object 106 based on this information and the display coordinate information included in the registered dance animation information. Further, the CPU 201, based on the number of the dance object 106 to be displayed, the size information of the dance object 106 and the sprite included in the dance animation table, and the dance animation image data storage position information included in the dance animation table, The pixel pattern data storage position information of each sprite constituting the dance object 106 is calculated.
Further, the CPU 201 refers to the dance animation table, and displays the color palette information, depth value, and size information of each sprite constituting the dance object 106, pixel pattern data storage position information and display coordinates of each sprite. It is given to the graphic processor 202 together with the information. In this case, the CPU 201 gives the above information to the graphic processor 202 in accordance with the display timing information included in the registered dance animation information and the number of sustained frames in the dance animation table.
Furthermore, the CPU 201 acquires information necessary for generating a video signal based on the animation information and the animation table for the evaluation objects 107 to 109 registered in the dance control process, and provides the graphic processor 202 with the information. In this case, the acquisition and contents of information given to the graphic processor 202 by the CPU 201 are the same as in the case of the dance object 106.
Based on the information given as described above, the graphic processor 202 generates a video signal representing the dance object 106 and the evaluation objects 107 to 109 and outputs the video signal to the video signal output terminal 47.
[Music Playback] Music playback is performed by interrupt processing. The CPU 201 reads and interprets the music control information in FIG. 18 while incrementing the musical score data pointer for music. Note that the musical score data pointer for music is a pointer indicating the reading position of the first musical score data 305.
If the command included in the read music control information is note-on, the CPU 201 indicates the pitch (pitch) indicated by the note number included in the music control information and the instrument (timbre) indicated by the instrument designation information. Is given to the sound processor 203 at the head address where the waveform data corresponding to the is stored. Further, if the command included in the read music control information is note-on, the CPU 201 gives the sound processor 203 a head address where necessary envelope data is stored. Furthermore, if the command included in the read music control information is note-on, the CPU 201 determines the pitch control information corresponding to the pitch (pitch) indicated by the note number included in the music control information, and Volume information included in the music piece control information is given to the sound processor 203.
Here, the pitch control information will be described. The pitch control information is used for pitch conversion performed by changing the period for reading waveform data. That is, the sound processor 203 reads and accumulates pitch control information at regular intervals. Then, the sound processor 203 processes this accumulated result to obtain an address pointer for waveform data. Therefore, if the pitch control information is set to a large value, the address pointer is incremented quickly, and the waveform data frequency is increased. If the pitch control information is set to a small value, the address pointer is incremented slowly. Therefore, the frequency of the waveform data is lowered. In this way, the sound processor 203 performs the pitch conversion of the waveform data.
The sound processor 203 reads the waveform data stored at the position indicated by the given head address from the ROM 51 while incrementing the address pointer based on the given pitch control information. The sound processor 203 then multiplies the sequentially read waveform data by envelope data and volume information to generate an audio signal. In this way, an audio signal having the tone color, pitch (pitch), and volume of the musical instrument indicated by the first musical score data 305 is generated and output to the audio signal output terminal 49.
On the other hand, the CPU 201 manages the gate time included in the read music control information. Accordingly, the CPU 201 instructs the sound processor 203 to end the sound generation of the corresponding musical sound when the gate time has elapsed. In response to this, the sound processor 203 ends the sound generation of the instructed musical sound.
As described above, the music is reproduced based on the first musical score data 305 and is generated from a speaker (not shown) of the television monitor 90.
Now, the overall processing flow of the music game apparatus 1 of FIG. 1 will be described using a flowchart.
FIG. 29 is a flowchart showing the overall processing flow of the music game apparatus 1 of FIG. As shown in FIG. 29, in step S1, the CPU 201 executes initial setting of the system. In step S <b> 2, the CPU 201 calculates state information of the operation article 150. In step S3, the CPU 201 executes game processing based on the state information of the operation article 150 calculated in step S2. In step S4, the CPU 201 determines whether “M” is smaller than a predetermined value “K”. If “M” is greater than or equal to the predetermined value “K”, the CPU 201 proceeds to step S5, substitutes “0” for “M”, and proceeds to step S6. On the other hand, when “M” is smaller than the predetermined value “K”, the CPU 201 proceeds from step S4 to step S6. This “M” will become clear in the following description.
In step S6, the CPU 201 determines whether to wait for a video synchronization interrupt. The CPU 201 gives image information for updating the display screen of the television monitor 90 to the graphic processor 202 after the start of the vertical blanking period (step S7). Therefore, when the calculation process for updating the display screen is completed, the process is not allowed to proceed until there is a video synchronization interrupt. If “YES” in the step S6, that is, if the video synchronization interruption is waited (no interruption by the video synchronization signal), the process returns to the same step S6. On the other hand, if “NO” in the step S6, that is, if not waiting for a video synchronization interrupt (if there is an interrupt due to a video synchronization signal), the process proceeds to a step S7. In step S7, the CPU 201 gives image information necessary for generating a game screen (see FIGS. 11 to 13) to the graphic processor 202 in the vertical blanking period based on the result of the game process in step S3 (see FIG. 11). Image display processing).
FIG. 30 is a flowchart showing the flow of the initial setting process in step S1 of FIG. As shown in FIG. 30, in step S10, the CPU 201 initializes a musical score data pointer for guide. In step S11, the CPU 201 sets a guide execution standby counter to “0”.
In step S12, the CPU 201 initializes a musical score data pointer for music. In step S13, the CPU 201 sets an execution standby counter for music to “t”.
In step S <b> 14, the CPU 201 executes an initial setting process for the image sensor 43. In step S15, the CPU 201 initializes various flags and various counters.
In step S16, the CPU 201 sets the timer circuit 210 as an interrupt source for sound generation. Note that the processing by the sound processor 203 is executed by the interrupt processing, and sound is output from the speaker of the television monitor 90.
FIG. 31 is a flowchart showing the flow of the sensor initial setting process in step S14 of FIG. As shown in FIG. 31, in the first step S20, the high speed processor 200 sets a command “CONF” as setting data. However, the command “CONF” is a command for notifying the image sensor 43 that the setting mode for transmitting the command from the high speed processor 200 is entered. Then, in the next step S21, command transmission processing is executed.
FIG. 32 is a flowchart showing the command transmission process in step S21 of FIG. As shown in FIG. 32, in the first step S30, the high speed processor 200 sets the setting data (command “CONF” in the case of step S21) to the register data (I / O port), and in the next step S31, the register Set clock CLK (I / O port) is set to low level. Thereafter, after waiting for a specified time in step S32, the register setting clock CLK is set to a high level in step S33. Further, after waiting for the specified time in step S34, the register setting clock CLK is set to the low level again in step S35.
In this way, as shown in FIG. 33, the command (command or command + data) transmission process is performed by setting the register setting clock CLK to the low level, the high level, and the low level while waiting for the specified time. Done.
Returning to the description of FIG. In step S22, the pixel mode is set and the exposure time is set. In this embodiment, since the image sensor 43 is, for example, a 32 pixel × 32 pixel CMOS image sensor as described above, the pixel mode register of the setting address “0” has 32 pixels × 32 pixels. “0h” is set. In the next step S23, the high speed processor 200 executes a register setting process.
FIG. 34 is a flowchart showing the register setting process in step S23 of FIG. As shown in FIG. 34, in the first step S40, the high speed processor 200 sets the command “MOV” + address as the setting data, and in the next step S41, executes the command transmission process described earlier in FIG. And send it. Next, in step S42, the high speed processor 200 sets the command “LD” + data as setting data, executes command transmission processing in the next step S43, and transmits it. In step S44, the high speed processor 200 sets the command “SET” as the setting data, and transmits it in the next step S45. Note that the command “MOV” is a command indicating that the address of the control register is transmitted, the command “LD” is a command indicating that the data is transmitted, and the command “SET” is for actually setting the data to the address. Command. This process is repeatedly executed when there are a plurality of control registers to be set.
Returning to the description of FIG. In step S24, the set address is set to “1” (indicating the address of the exposure time setting register), and “FFh” indicating the maximum exposure time is set as data to be set. In step S25, the register setting process shown in FIG. 34 is executed. Similarly, in step S26, the setting address is set to “2” (indicating the high nibble address of the exposure time setting register), and the high nibble data “Fh” of “FFh” indicating the maximum exposure time is set as data to be set. In step S27, register setting processing is executed.
Thereafter, a command “RUN” is set in step S28 to indicate the end of setting and to cause the image sensor 43 to start outputting data, and is transmitted in step S29. In this way, the sensor initial setting process in step S14 shown in FIG. 30 is executed. However, the specific examples shown in FIGS. 31 to 34 can be appropriately changed according to the specifications of the image sensor 43 used.
FIG. 35 is a flowchart showing the flow of the state information calculation process in step S2 of FIG. As shown in FIG. 35, in step S50, the CPU 201 acquires digital pixel data from the ADC 208. The digital pixel data is obtained by converting analog pixel data from the image sensor 43 to digital by the ADC 208.
In step S51, attention point extraction processing is executed. Specifically, the CPU 201 calculates a difference between pixel data when the infrared light emitting diode 15 emits light and pixel data when the infrared light emitting diode 15 is turned off, and obtains difference data. Then, the CPU 201 detects the maximum value of the difference data and compares it with a predetermined threshold Th. Further, when the maximum value of the difference data exceeds a predetermined threshold Th, the CPU 201 converts the coordinates of the pixel having the difference data of the maximum value into the coordinates on the screen 91 of the television monitor 90, and performs the operation. The coordinates of the attention point of the object 150 are used.
In step S52, the CPU 201 determines in which region a1 to a4 in FIG. 24 the point of interest of the operation article 150 exists, and stores the determination result in the array J1 [M].
In step S53, the CPU 201 determines in which region A1 to A4 in FIG. 25 the point of interest of the operation article 150 exists, and stores the determination result in the array J2 [M].
FIG. 36 is a flowchart showing the pixel data group acquisition processing in step S50 of FIG. As shown in FIG. 36, in the first step S60, the CPU 201 sets “−1” for X and “0” for Y as the element number of the pixel data array. The pixel data array in the present embodiment is a two-dimensional array with X = 0 to 31 and Y = 0 to 31, but dummy data is output as data of the first pixel in each row as described above. “−1” is set as an initial value. In a succeeding step S61, a pixel data acquisition process is executed.
FIG. 37 is a flowchart showing the pixel data acquisition process in step S61 of FIG. As shown in FIG. 37, in the first step S70, the CPU 201 checks the frame status flag signal FSF from the image sensor 43, and whether or not the up edge (from low level to high level) has occurred in step S71. to decide. When the up edge of the flag signal FSF is detected in step S71, the CPU 201 instructs the start of conversion of the analog pixel data input to the ADC 208 to digital data in the next step S72. Thereafter, the pixel strobe PDS from the image sensor 43 is checked in step S73, and it is determined in step S74 whether or not an up edge from the low level to the high level of the strobe signal PDS has occurred.
If “YES” is determined in the step S74, the CPU 201 determines whether or not X = −1, that is, whether or not it is the first pixel in a step S75. As described above, since the first pixel of each row is set as a dummy pixel, if “YES” is determined in this step S75, the pixel data at that time is not acquired in the next step S77, and the element The number X is incremented.
If “NO” is determined in the step S75, the pixel data is the second and subsequent pixel data in the row, so that the pixel data at that time is acquired in the steps S76 and S78, and the pixel is stored in a temporary register (not shown). Store the data. Thereafter, the process proceeds to step S62 in FIG.
In step S62 of FIG. 36, the pixel data stored in the temporary register is substituted into the pixel data array P [Y] [X].
In the following step S63, X is incremented. When X is less than 32, the processes from S61 to S63 are repeated. When X is 32, that is, when the acquisition of pixel data has reached the end of the row, “−1” is set to X in subsequent step S65, Y is incremented in step S66, and the beginning of the next row is started. Repeat the pixel data acquisition process.
If Y is 32 in step S67, that is, if the acquisition of pixel data has reached the end of the pixel data array P [Y] [X], the process proceeds to step S51 in FIG.
FIG. 38 is a flowchart showing the flow of attention point extraction processing in step S51 of FIG. As shown in FIG. 38, in step S80, the CPU 201 calculates the difference between the pixel data when the infrared light emitting diode 15 is turned on and the pixel data when the infrared light emitting diode 15 is turned off from the image sensor 43. Calculate to obtain difference data. In step S81, the CPU 201 assigns the calculated difference data to the array Dif [X] [Y]. Here, in the embodiment, since the image sensor 43 of 32 pixels × 32 pixels is used, X = 0 to 31 and Y = 0 to 31.
In step S82, the CPU 201 scans all elements of the array Dif [X] [Y]. In step S83, the CPU 201 detects the maximum value of all the elements in the array Dif [X] [Y]. If the maximum value is greater than the predetermined threshold Th, the CPU 201 proceeds to step S85, and if not greater than the predetermined threshold Th, the CPU 201 proceeds to step S4 in FIG. 29 (step S84).
In step S85, the CPU 201 calculates the coordinates (Xc, Yc) of the attention point of the operation article 150 based on the coordinates of the maximum value. In step S86, the CPU 201 increments the value of the number of times M by one (M = M + 1).
In step S <b> 87, the CPU 201 converts the coordinates (Xc, Yc) of the target point on the image sensor 43 into coordinates (xc, yc) on the screen 91 of the television monitor 90. In step S88, the CPU 201 assigns xc to the array Px [M] as the x coordinate of the Mth point of interest, and substitutes yc to the array Py [M] as the y coordinate of the Mth point of interest.
FIG. 39 is a flowchart showing a flow of attention point coordinate calculation processing in step S85 of FIG. As shown in FIG. 39, in step S100, the CPU 201 substitutes the maximum X and Y coordinates obtained in step S83 for “m” and “n”, respectively. In step S101, the CPU 201 increments “m” by one (m = m + 1). If the difference data Dif [m] [n] is greater than the predetermined threshold Th, the CPU 201 proceeds to step S103, otherwise proceeds to step S104 (step S102). In step S103, the CPU 201 assigns “m” at that time to “mr”. In this way, while repeating steps S101 to S103, scanning is performed in the positive direction of the X axis from the maximum value, and the X coordinate of the extreme difference data whose value exceeds the predetermined threshold Th is obtained.
In step S104, the CPU 201 assigns the maximum X coordinate obtained in step S83 to “m”. In step S105, the CPU 201 decrements “m” by one. If the difference data Dif [m] [n] is greater than the predetermined threshold Th, the CPU 201 proceeds to step S107, otherwise proceeds to step S108 (step S106). In step S107, the CPU 201 assigns “m” at that time to “ml”. In this way, while repeating steps S105 to S107, scanning is performed in the negative direction of the X axis from the maximum value, and the X coordinate of the extreme difference data whose value exceeds the predetermined threshold Th is obtained.
In step S108, the CPU 201 calculates the center coordinate between the X coordinate mr and the X coordinate ml and sets it as the X coordinate (Xc) of the target point. In step S109, the CPU 201 assigns “Xc” obtained in step S108 and the maximum Y coordinate obtained in step S83 to “m” and “n”, respectively. In step S110, the CPU 201 increments “n” by one (n = n + 1). If the difference data Dif [m] [n] is greater than the predetermined threshold Th, the CPU 201 proceeds to step S112, otherwise proceeds to step S113 (step S111). In step S112, the CPU 201 assigns “n” at that time to “md”. In this way, while repeating steps S110 to S112, scanning is performed in the positive direction of the Y axis from the maximum value, and the Y coordinate of the extreme difference data whose value exceeds the predetermined threshold Th is obtained.
In step S113, the CPU 201 assigns the maximum Y coordinate obtained in step S83 to “n”. In step S114, the CPU 201 decrements “n” by one. If the difference data Dif [m] [n] is larger than the predetermined threshold Th, the CPU 201 proceeds to step S116, otherwise proceeds to step S117 (step S115). In step S116, the CPU 201 assigns “n” at that time to “mu”. In this way, while repeating steps S114 to S116, scanning is performed from the maximum value in the negative direction of the Y axis, and the Y coordinate of the extreme difference data whose value exceeds the predetermined threshold Th is obtained.
In step S117, the CPU 201 calculates the center coordinate between the Y coordinate md and the Y coordinate mu, and sets it as the Y coordinate (Yc) of the target point. As described above, the coordinates (Xc, Yc) of the attention point of the operation article 150 are calculated.
FIG. 40 is a flowchart showing the flow of the game process in step S3 of FIG. As shown in FIG. 40, in step S120, the CPU 201 checks the music end flag (see step S196 in FIG. 43). If the music ends, the game process ends. If not, the step ends. Proceed to S121
In step S <b> 121, the CPU 201 registers the x coordinate Px [M] and the y coordinate Py [M] of the attention point of the operation article 150 as display coordinates on the screen 91 of the cursor 105.
CPU201 repeats the process between step S122 and step S144 twice. Here, “j” means the guide display number J (see FIG. 43).
In step S123, the CPU 201 checks the guide start flag GF [j] (see step S194 in FIG. 43). If the guide start flag GF [j] is on, the CPU 201 proceeds to step S125, and if it is off, the CPU 201 proceeds to step S144 (step S124). In step S125, the CPU 201 checks the frame counter C [j]. When the frame counter C [j] is greater than “0”, the CPU 201 proceeds to step S128, and when the frame counter C [j] is “0”, the CPU 201 proceeds to step S127 (step S126). In step S127, the CPU 201 registers the animation information of the position guide G and the animation information of the direction guide g or the route guide rg according to the note number NN [j]. Only when the frame counter C [j] is “0”, such animation information is registered because once animation information is registered, animation is executed according to the registration information. , It is registered only at the start of the animation.
In step S128, the CPU 201 checks the note number NN [j], and if it is a note number instructing the turning of the cursor 105 (see FIG. 20B and FIG. 20C), the CPU 201 proceeds to step S131. If it is any other note number (see FIG. 20A), the process proceeds to step S129. In step S129, the CPU 201 checks the frame counter C [j]. If the frame counter [j] is equal to or greater than the prescribed frame number f1, the CPU 201 proceeds to step S131, otherwise proceeds to step S141 (step S130). In step S131, the CPU 201 determines whether or not the operation of the cursor 105 is adapted to the guide (position guide G / direction guide g / route guide rg) (success determination).
As is clear from step S128 and step S131, when the note number NN [j] is a note number for instructing the turning of the cursor 105, the position guide G and the route are independent of the frame counter C [j]. Since the display of the guide rg is started (the frame counter C [j] is “0”), the success of the operation of the cursor 105 is determined. On the other hand, when the note number NN [j] is other than the note number for instructing the turning of the cursor 105, from the start of display of the position guide G and the direction guide g (the frame counter C [j] is “0”) After the number of frames f1 (for example, 30 frames) has passed, the success determination of the operation of the cursor 105 is performed (see FIG. 28).
As a result of the determination in step S131, the CPU 201 proceeds to step S133 if the operation of the cursor 105 is successful, and proceeds to step S140 if not successful (step S132). In step S133, the CPU 201 refers to the note number NN [j] and the dance speed flag DF (see step S193, step S190, and step S192 in FIG. 43) and registers dance animation information. If the operation is successful, the CPU 201 refers to the note number NN [j] and the dance speed flag DF to scroll the background 110, changes the center position of the panoramic screen 140, and sets the arrays PA and CA. Change the corresponding data. Further, the CPU 201 registers the animation table storage position information of the position guide G used when the operation is successful.
In step S134, the CPU 201 checks the note number NN [j] and proceeds to step S137 if it is a note number instructing to turn the cursor 105, and proceeds to step S135 if it is any other note number. move on. In step S135, the CPU 201 checks the frame counter C [j]. If the frame counter [j] is equal to or greater than the prescribed frame number f2, the CPU 201 proceeds to step S137, otherwise proceeds to step S138 (step S136). In step S137, the CPU 201 adds “3” to the score S. On the other hand, “1” is added to the score S in step S138.
In step S137, “3” is added to the score S, and in step S138, “1” is added to the score S for the following reason. From the start of display of the position guide G and the direction guide g (the frame counter C [j] is “0”), within a predetermined period (for example, 10 frames) after a specified number of frames f2 (for example, 50 frames) has elapsed. When the cursor 105 exists in the area of the position guide G, “3” is added assuming that the cursor 105 is operated at the best timing. On the other hand, if the cursor 105 is present in the area of the position guide G from when the specified frame number f1 has elapsed and before the specified frame number f2 has elapsed, “1” is added as normal success. Further, when an operation suitable for the position guide G and the route guide rg (guide for instructing the turning of the cursor 105) is performed, “3” is uniformly added.
In step S139, the CPU 201 checks the frame counter C [j]. If the frame counter C [j] is equal to the specified frame number f3 (for example, 60 frames), the CPU 201 proceeds to step S142, otherwise proceeds to step S141 (step S140). In step S141, the CPU 201 increments the frame counter C [j] by one. On the other hand, in step S142, the CPU 201 sets “0” to the frame counter C [j]. In step S143, the CPU 201 turns off the guide start flag GF [j]. The prescribed frame number f3 defines the end of success determination.
FIG. 41 is a flowchart showing the flow of interrupt processing. As shown in FIG. 41, in step S150, the CPU 201 executes a music reproduction process. In step S151, the CPU 201 executes a guide (position guide G, direction guide g, route guide rg) registration process.
FIG. 42 is a flowchart showing the flow of the music reproduction process in step S150 of FIG. As shown in FIG. 42, in step S160, the CPU 201 checks an execution standby counter for music. If the value of the execution standby counter for music is “0”, the process proceeds to step S162, and if not “0”, the process proceeds to step S170 (step S161). In step S170, the CPU 201 decrements the execution standby counter for music.
On the other hand, in step S162, the CPU 201 reads and interprets the command indicated by the musical score data pointer for music. If the command is note-on, the process proceeds to step S164 (step S163). On the other hand, if the command is not note-on, that is, if it is standby, the process proceeds to step S165. In step S165, the CPU 201 sets a standby time in the music execution standby counter.
In step S164, the CPU 201 causes the sound processor 203 to start sounding according to the read note number. In step S166, the CPU 201 increments the musical score data pointer for music.
In step S167, the CPU 201 checks the remaining sound generation time for the note number being sounded. If the remaining sound generation time is “0”, the process proceeds to step S169; otherwise, the process proceeds to step S151 in FIG. 41 (step S168). In step S <b> 169, the CPU 201 causes the sound processor 203 to execute a sound generation end process for the note number whose remaining sound generation time is “0”.
FIG. 43 is a flowchart showing the guide registration process in step S151 of FIG. As shown in FIG. 43, in step S180, the CPU 201 checks a guide execution standby counter. If the value of the execution waiting counter for guide is “0”, the process proceeds to step S182, and if not “0”, the process proceeds to step S198 (step S181). In step S198, the CPU 201 decrements the guide execution standby counter.
On the other hand, in step S182, the CPU 201 reads and interprets the command indicated by the musical score data pointer for guide. If the command is note-on, the CPU 201 proceeds to step S184 (step S183). On the other hand, if the command is not note-on, that is, if it is standby, the CPU 201 proceeds to step S197. In step S197, the CPU 201 sets a standby time in a guide execution standby counter.
The CPU 201 proceeds to step S196 if the note number means the end of music, and proceeds to step S185 otherwise (step S184). In step S196, the CPU 201 turns on the music end flag.
On the other hand, if the note number indicates the start of music, the CPU 201 proceeds to step S195, otherwise proceeds to step S186 (step S185). If the guide display number J is “1”, the CPU 201 sets the guide display number J to “0” in step S188, and if the guide display number J is not “1” (if it is “0”). ) In step S187, the guide display number J is set to “1”. It should be noted that the guidance by a certain position guide G and the guidance by another position guide G are started at different timings. However, there are cases where the guidance is executed overlappingly for a certain period. The game process of step S3 of 29 is executed.
The CPU 201 proceeds to step S190 if the note number is a note number instructing to perform a dance animation at a high speed, and proceeds to step S191 otherwise (step S189). In step S190, the CPU 201 sets the dance speed flag DF to “1” (fast dance animation).
On the other hand, the CPU 201 proceeds to step S192 if the note number is a note number instructing to perform a dance animation at a low speed, and proceeds to step S193 otherwise (step S191). In step S192, the CPU 201 sets the dance speed flag DF to “0” (slow speed dance animation).
If the note number is not one of the note number indicating the end of the song, the note number indicating the start of the song, the note number indicating the fast-speed dance animation, and the note number indicating the slow-speed dance animation, Since the note number is a note number indicating the type of guide (FIGS. 20A to 20C), the CPU 201 assigns the note number to the array NN [J] in step S193. . In step S194, the CPU 201 turns on the guide start flag GF [J].
In step S195, the CPU 201 increments the musical score data pointer for guide.
Next, another example of the direction guide g will be described. FIG. 44 is a view showing an example of a game screen to which another example of the direction guide g is applied. As shown in FIG. 44, a strip-shaped direction guide g20 is displayed so as to extend from the position guide G1 to the position guide G2. The direction guide g20 extends from the position guide G1 to the position guide G2 over time. The direction in which the cursor 105 is operated is guided by the direction in which the direction guide g20 extends. Further, if the predetermined time from the time when the direction guide g20 reaches the position guide G2 is a period for determining the success of the operation of the cursor 105, the operation timing of the cursor 105 can be guided by the direction guide g20. It can be said that the direction guide g20 is expressed by gradually changing the color of the path from the position guide G1 to the position guide G2.
FIG. 45 is an illustration of a game screen to which still another example of the direction guide g is applied. As shown in FIG. 45, a direction guide g30 is displayed between the position guide G and another position guide G on the game screen. The direction guide g30 includes five partial paths g31 to g35. In the example of FIG. 45, the colors of the five partial paths g31 to g35 change in order from the position guide G1 to the position guide G2. The color change is indicated by hatching. By doing in this way, the operation direction of the cursor 105 can be guided by the direction of the color change of the partial paths g31 to g35. Further, in this example, if the predetermined time from the time when the color of the partial path g35 adjacent to the position guide G2 that guides the moving position of the cursor 105 changes is set as the period for determining the success of the operation of the cursor 105, the direction guide g30. Thus, the operation timing of the cursor 105 can be guided.
FIG. 46 is a view showing an example of a game screen to which still another example of the direction guide g is applied. As shown in FIG. 46, a direction guide g40 is displayed on the game screen between the position guide G1 and the position guide G2. The direction guide g40 moves from the position guide G1 to the position guide G2 over time. The direction in which the cursor 105 is operated is guided by the direction in which the direction guide g40 moves. Further, if the predetermined time from the time when the direction guide g40 reaches the position guide G2 is set as a period for determining the success of the operation of the cursor 105, the operation timing of the cursor 105 can be guided by the direction guide g40.
As described above, in the present embodiment, when the operation of the cursor 105 is adapted to a guide (position guide G, direction guide g, route guide rg), an image (in the above example, according to the guide by the guide). The display of the dance object 106 and the background 110) is controlled. In this case, since the operation of the cursor 105 is performed in conformity with the guidance by the guide, the display of the image is controlled according to the operation of the cursor 105. That is, since the cursor 105 is interlocked with the operation article 150, display of an image is controlled according to the operation of the operation article 150. Further, the operation article 150 that is intermittently irradiated with light by a stroboscope is photographed by the imaging unit 13 to obtain state information of the operation article 150. For this reason, in order to obtain the state information of the operation article 150, it is not necessary to incorporate a circuit driven by the power source in the operation article 150. Furthermore, this music game apparatus 1 automatically plays music.
As a result, while the music piece is automatically played without the player 94 being involved, when the player 94 operates the operation article 150 having a simple configuration, the user can enjoy an image that follows the operation of the operation article 150 together with the music piece. Can do.
In addition, since the guide is controlled at a timing based on the music, if the player 94 operates the cursor 105 according to the guide, the operation of the operation article 150 also matches the music. Therefore, the player 94 can enjoy the operation of the operation article 150 suitable for the music.
Here, for example, the high speed processor 200 scrolls the background 110 to the left, and rotates the dance object 106 counterclockwise corresponding to the note numbers “55” and “67”, and the dance animation. Prepare a table. Further, for example, the high speed processor 200 corresponds to the note numbers “45” and “64”, and scrolls the background 110 to the right, and the dance animation information and the dance animation table for rotating the dance object 106 clockwise. prepare. In addition, for example, the high speed processor 200 scrolls the background 110 downward corresponding to the note numbers “76” and “77”, and also dance information and dance animation table for rotating the dance object 106 counterclockwise. Prepare. Also, for example, the high speed processor 200 generates dance animation information and a dance animation table for scrolling the background 110 upward and rotating the dance object 106 clockwise corresponding to the note numbers “65” and “74”. prepare. Also, for example, corresponding to the note number “53”, dance animation information and a dance animation table for rotating the dance object 106 greatly in the clockwise direction are prepared. Also, for example, dance animation information and a dance animation table for rotating the dance object 106 greatly counterclockwise are prepared corresponding to the note number “57”.
Note that such a note number (see FIGS. 20A to 20C) is a note number that controls the display of the guide, so the background 110 and the dance object 106 are controlled according to the guide. It turns out that. In other words, the background 110 and the dance object 106 are controlled according to the operation of the operation article 150.
In the present embodiment, the position guide G guides the movement position and operation timing of the cursor 105. Then, when the operation of the cursor 105 by the operation article 150 is adapted to the guidance of the position guide G, the high speed processor 200 corresponds to the direction of the moving position guided by the position guide G (in the above example, the dance Control display of object 106 and background 110).
Therefore, when the player 94 operates the operation article 150 to move the cursor 105 to the movement position guided by the position guide G at the operation timing guided by the position guide G, the movement guided by the position guide G is performed. The display of the image is controlled corresponding to the direction of the position. As a result, it is possible to enjoy an image that follows the movement of the cursor 105 linked to the operation of the operation article 150 together with the music (see FIG. 12).
Furthermore, in the present embodiment, the route guide rg guides the moving route, moving direction, and operation timing of the cursor 105. Accordingly, the player 94 operates the operation article 150 to move the cursor 105 along the movement route guided by the route guide rg in the moving direction guided by the route guide rg at the operation timing guided by the route guide rg. In this case, display of an image (in the above example, a dance object) is controlled according to the route guide rg. As a result, it is possible to enjoy an image corresponding to the movement of the cursor 105 linked to the operation of the operation article 150 together with the music (see FIG. 13).
Furthermore, according to the present embodiment, when the position of the point of interest of the operation article 150 exists within the area guided by the position guide G within the period guided by the position guide G, the operation article 150 is interlocked with the operation article 150. It is determined that the operation of the cursor 105 is suitable for the guidance of the position guide G (see FIG. 24). Further, when the position of the attention point of the operation article 150 moves in a predetermined order guided by the route guide rg within a period guided by the route guide rg, a plurality of predetermined areas guided by the route guide rg are operated. It is determined that the operation of the cursor 105 linked to the object 150 is suitable for the guidance of the route guide rg (see FIG. 25). As described above, the accuracy of the operation of the cursor 105 can be determined based on the position of the attention point of the operation article 150 that can be calculated by simple processing.
Further, in the present embodiment, the position guide G is displayed at each of a plurality of predetermined positions on the screen 91. Then, the high speed processor 200 changes the form of the position guide G at the timing based on the music (in the example of FIG. 12, an animation in which a flower opens). Therefore, the player 94 can easily recognize the position and direction in which the cursor 105 should be moved by the change in the form of the position guide G.
Further, in the present embodiment, the direction guide g and the route guide rg are expressed by a display that can visually recognize the movement from the first predetermined position on the screen 91 to the second predetermined position. Is done. Thus, in addition to the position guide G, the operation of the cursor 105 is guided by the direction guide g and the route guide rg. Therefore, the player 94 can more clearly recognize the direction and route in which the cursor 105 should be moved. More specifically, it is as follows.
The direction guide g and the route guide rg have a plurality of objects (see FIG. 12 and FIG. 12) arranged on a route starting from the first predetermined position on the screen 91 and ending at the second predetermined position. In the example of FIG. 13, it is expressed by a change in the form of a sphere figure). In this case, the player 94 can easily recognize the direction and route in which the cursor 105 should be moved by changing the form of the plurality of objects.
The direction guide g is expressed by the movement of an object (a bird figure in the example of FIG. 46) from a first predetermined position on the screen 91 to a second predetermined position. In this case, the player 94 can easily recognize the direction and route in which the cursor 105 should be moved by moving the object.
Further, the direction guide g is represented by a change in the form of a route starting from the first predetermined position on the screen 91 and ending at the second predetermined position (FIGS. 44 and 45). reference). In this case, the player 94 can easily recognize the direction and path in which the cursor 105 should be moved by changing the form of the path.
Furthermore, in the present embodiment, the high speed processor 200 uses the speed information, the movement direction information, the movement distance information, the velocity vector information, the acceleration information, the movement trajectory information, the area information, or the position information as the state information of the operation article 150. Any of the information, or a combination of two or more thereof can be calculated. As described above, as the state information of the operation article 150 when determining whether or not the operation of the cursor 105 by the operation article 150 is suitable for the guide (position guide G, direction guide g, route guide rg), Since information can be used, the degree of freedom of expression of the guide is increased, and as a result, the degree of freedom of design of the game content is also increased.
Furthermore, in the present embodiment, it is possible to obtain state information of the operation article 150 by intermittently irradiating the operation article 150 with the reflection sheet 155 with infrared light and photographing the operation object 150. For this reason, in order to obtain the state information of the operation article 150, it is not necessary to incorporate a circuit driven by the power source in the operation article 150. Therefore, the operability and reliability of the operation article 150 can be improved, and the cost can be reduced.
The present invention is not limited to the above-described embodiment, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.
(1) In the embodiment, the dance object 106 and the background 110 are given as examples of images (following images) controlled to follow the operation article 150. However, the present invention is not limited to this, and an arbitrary object can be selected as the follow-up image. In addition, instead of expressing the movement of the dance object 106 by scrolling the background 110, the dance object 106 itself can be moved up, down, left, and right.
(2) In the embodiment, the linear operation of the cursor 105 is guided by both the position guide G and the direction guide g, but can be guided by either one. When the operation of the cursor 105 is guided only by the direction guide g, the still image position guide G is preferably arranged at the start point and the end point of the direction guide g. In addition, the turning operation of the cursor 105 is guided by both the position guide G and the route guide rg, but it is also possible to guide only by the route guide rg. Further, the guides (position guide G, direction guide g, route guide rg) are expressed by animation, but the present invention is not limited to this. Further, the mode of the guide is not limited to the above.
(3) In the embodiment, the operation article 150 including the stick 152 and the reflective ball 151 is adopted as the operation article. However, the shape of the operation article is not limited to this as long as it includes a reflector.
(4) In the embodiment, the coordinates of the attention point of the operation article 150 are calculated as shown in FIGS. 23A to 23D. However, the pixel having the maximum luminance value exceeding the predetermined threshold Th. (See step S83 in FIG. 38) can be converted into coordinates on the screen 91 and used as the coordinates of the target point.
(5) Although any type of processor can be used as the high-speed processor 200 of FIG. 6, it is preferable to use a high-speed processor that has already been applied for a patent. This high speed processor is disclosed in detail, for example, in Japanese Patent Laid-Open No. 10-307790 and US Pat. No. 6,070,205 corresponding thereto.
Although the present invention has been described in detail with reference to the embodiments, it will be apparent to those skilled in the art that the present invention is not limited to the embodiments described herein. The present invention can be implemented as modified and changed modes without departing from the spirit and scope of the present invention defined by the description of the scope of claims. Therefore, the description of the present application is for illustrative purposes and does not have any limiting meaning to the present invention.

Claims (16)

A music game device for automatically playing music,
Imaging means for imaging an object to be moved by the player,
And status-information obtaining means based on the captured image, it calculates the status information of the object by the image pickup means,
Guide control means for guiding the movement position of the cursor linked to the subject by changing the form of the position guide displayed at each of a plurality of predetermined positions on the screen at a timing based on the music ;
Cursor control means for controlling display of the cursor based on the state information of the subject ;
Based on the state information of the subject and information on the position guide, it is determined whether the operation of the cursor by the subject is suitable for the position guide, and the operation of the cursor by the subject is determined by the position. when adapted to guide, in correspondence with the direction of the movement position of the position guide for guiding the music game apparatus comprising: a follow-up image control means for controlling the display of the tracking image.   The plurality of position guides include a first guide and a second guide,
  The guide control means guides the cursor from the position of the first guide to the position of the second guide by the second guide,
  The music game apparatus according to claim 1, wherein the first guide is a guide that guides the cursor immediately before the guidance by the second guide. The following image control means, operation of the cursor by the subject, when adapted to the position guide, to operate in a direction corresponding to the direction of the movement position of the position guide for guiding the tracking image to control the display of the music game apparatus according to claim 1 or 2, wherein. The state information calculation means calculates the position of the subject as the state information based on the captured image,
The following image control means, the position of the object in which the state information calculating unit has calculated, within the period in which the position guide to guide, when present in a region where the position guide to guide, in conjunction with the subject operation of the cursor that is, determines that conforms to the position guide, the music game apparatus according to any one of claims 1 to 3. The guide control means controls a route guide for guiding a movement route of the cursor at a timing based on the music ,
The following image control means, operation of the cursor by the subject, when adapted to the route guide, in response to the path which the path guide for guiding, controlling the display of the follow-up images, from claim 1 5. The music game apparatus according to any one of 4 . The state information calculation means calculates the position of the subject as the state information based on the captured image,
The following image control means, the position of the object in which the state information calculating unit has calculated, within the period in which the path guide for guiding a plurality of predetermined areas in which the path guide for guiding the path guide for guiding The music game apparatus according to claim 5 , wherein when the movement is performed in a predetermined order, it is determined that a cursor operation linked to the subject is suitable for the route guide. The plurality of position guides include a fourth guide and a fifth guide,
The guide control means, the display of the fourth guide from said fifth position in the guide direction guide movement Ru expressed as visually recognizable to the position of the controls at the timing based on the musical composition, The music game device according to claim 1. The direction guide is represented by a change in the form of a plurality of objects arranged on a path starting from a first predetermined position on the screen and ending at a second predetermined position. The music game apparatus according to claim 7. The music game device according to claim 7, wherein the direction guide is expressed by movement of an object from a first predetermined position on the screen to a second predetermined position. 8. The music game apparatus according to claim 7, wherein the direction guide is expressed by a change in a route form starting from a first predetermined position on the screen and ending at a second predetermined position. . The state information of the subject calculated by the state information calculation means is any of speed information, moving direction information, moving distance information, speed vector information, acceleration information, moving trajectory information, area information, or position information. The music game device according to claim 1, or a combination of two or more thereof.   The music game device according to claim 1, wherein the follow-up image is an image for performing a dance animation. A music game system for automatically playing music,
Imaging means for imaging an object to be moved by the player,
And status-information obtaining means based on the captured image, it calculates the status information of the object by the image pickup means,
Guide control means for guiding the movement position of the cursor linked to the subject by changing the form of the position guide displayed at each of a plurality of predetermined positions on the screen at a timing based on the music ;
Cursor control means for controlling display of the cursor based on the state information of the subject ;
Based on the state information of the subject and information on the position guide, it is determined whether or not the operation of the cursor by the subject is suitable for the position guide, and the operation of the cursor by the subject is determined by the position guide. And a follow-up image control means for controlling the display of the follow-up image in accordance with the direction of the moving position guided by the position guide when the position guide is adapted. An operation article operated by a player as a subject of the music game device according to any one of claims 1 to 12 ,
A rod-shaped gripping part gripped by the player;
A reflection part that is provided at one end of the gripping part and retroreflects the received light ;
The reflective part is an operation article that is fixed inside a transparent outer shell . The computer of the music game device that analyzes the captured image of the subject that is imaged by the imaging device and is moved by the player, reflects the analysis result in the game process, and controls the video displayed on the display device.
A step of automatically playing music,
Obtaining the captured image from the imaging device and calculating state information of the subject based on the captured image ;
A step of guiding a moving position of a cursor linked to the subject by changing a form of a position guide displayed at each of a plurality of predetermined positions on the screen of the display device at a timing based on the music; When,
Controlling the display of the cursor on the screen of the display device based on the state information of the subject ;
A step of, based on the information about the status information and the position guide of the subject, the operation of the cursor by the subject to determine whether they comply with the said location guide,
A step operation of the cursor by the subject, when adapted to the position guide, which corresponds to the direction of the movement position of the position guide for guiding, controlling the display of the tracking image on the screen of the display device A music game program that lets you run. A music game method executed by a computer of a music game apparatus that analyzes a captured image of a subject moved by a player, which is captured by an imaging apparatus, reflects the analysis result in game processing, and controls a video displayed on the display apparatus There,
A step of automatically playing music,
Obtaining the captured image from the imaging device and calculating state information of the subject based on the captured image;
A step of guiding a moving position of a cursor linked to the subject by changing a form of a position guide displayed at each of a plurality of predetermined positions on the screen of the display device at a timing based on the music; When,
Controlling the display of the cursor on the screen of the display device based on the state information of the subject ;
Determining whether an operation of the cursor by the subject is compatible with the position guide based on the state information of the subject and information on the position guide;
A step operation of the cursor by the subject, when adapted to the position guide, which corresponds to the direction of the movement position of the position guide for guiding, controlling the display of the tracking image on the screen of the display device , Including music game methods.

Images