JPH07152369A - Multi-media controller - Google Patents

Abstract

PURPOSE:To realize a multi-media controller capable of harmonizing a visual effect and an acoustic effect and cultivating sound feeling and rhythmic feeling together with game feeling. CONSTITUTION:When either one of instrument sounds allocated at every plural character picture displayed on a display means 20 is specified by a touch pen 5, and a pad 8 is performance operated, a musical sound synthesis circuit 15 pronounciation controls the specified instrument sound according to pad operation information supplied from a CPU 11, and a display control circuit 14 animation controls the performance operation of the specified character according to the pad operation information. Thus, the specified instrument sound is pronounced with performance operated by the specified character picture displayed according to the pad operation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia control device suitable for use in, for example, musical educational toys.

[0002]

2. Description of the Related Art Conventionally, various types of devices have been put into practical use for controlling video and audio according to the operation of an operator. This type of device is called a video game or a TV game, and a shooting game for asking a player's reflexes is known. A video game includes a computer section that generates a video signal corresponding to a game operation, and a display that displays a video signal supplied from the computer section. In the computer section, a well-known ROM
The image information and control information stored in the pack are sequentially read out, for example, a still image serving as a screen background is displayed on the display, and a corresponding character is moved on the still image according to a game operation to display a moving image, A sound effect is produced according to the movement.

[0003]

In such a video game, the visual effect of the image is the main one, and the sound effect often plays a secondary role. In addition, the game has poor interactivity with the player, and in many cases, the game progresses in accordance with a predetermined story content. For this reason, there is a demand for a multimedia control device that allows a player to proceed with a game freely, and that combines visual effects and acoustic effects. In particular, as an educational toy for infants, a multimedia control device that fuses a visual effect and a sound effect and can cultivate a sense of sound and a rhythm like a game is desired. The present invention has been made in view of the above circumstances, and fuses visual effects and acoustic effects to cultivate a sense of sound and rhythm in a game sense, and further, tones of musical instruments,
It is an object of the present invention to provide a multimedia control device capable of learning a playing method.

[0004]

According to a first aspect of the present invention, there is provided a performance operator for generating an operation signal corresponding to a performance operation, and a musical instrument sound assigned to each of a plurality of characters displayed on the screen. Instrument sound designating means for designating any one,
A sound generation control means for controlling the sounding of a musical instrument sound designated by the musical instrument sound designating means in accordance with the operation signal, and a playing operation of a character corresponding to the musical instrument sound designated by the musical instrument sound designating means in accordance with the operating signal. And an image control unit for controlling.

According to the second aspect of the present invention, one of the performance operator for generating an operation signal according to the performance operation and the musical instrument sound assigned to each of the plurality of characters displayed on the screen is designated. The musical instrument sound designating means and the means for storing the performance pattern for each of the plurality of characters are provided, and the musical instrument sound designated by the musical instrument sound designating means is controlled to be sounded according to the corresponding performance pattern. When a performance operation is performed, sound generation control means for controlling the sound generation of the musical instrument sound designated by the musical instrument sound designating means in accordance with the operation signal, and means for storing an image pattern representing the performance movement of each of the plurality of characters. While reading out the image pattern of the first character corresponding to the musical instrument sound designated by the musical instrument sound designating means and displaying the playing action of the first character as a moving image. When the performance operator is operated for performance, image control means for controlling a moving image of the performance operation of the first character in synchronization with the operation signal, and when the performance operator is operated for performance, the musical instrument sound is generated. The present invention is characterized by further comprising performance determining means for determining whether or not the performance pattern corresponding to the musical instrument sound designated by the designating means and the performance operation are coincident.

[0006]

According to the present invention, when a musical instrument sound assigned to each of a plurality of characters, that is, one of the timbres, is designated by the musical instrument tone designating means and the performance operator is operated, the sound producing control means is designated. While controlling the sounding of the generated musical instrument sound according to the operation signal supplied from the performance operator, the image control means performs moving image control of the performance operation of the character according to the operation signal. As a result, it becomes possible to learn the shape of the musical instrument, the timbre, and the playing method by combining the visual effect and the acoustic effect.

[0007]

Embodiments of the present invention will be described below with reference to the drawings. A. Outline of Embodiment FIG. 1 is a diagram showing an outline of a multimedia control apparatus according to an embodiment of the present invention. As shown in FIG. 1B, the multimedia control device 1 generates a video signal Vs and an audio signal As composed of stereo components of L channel and R channel, and is connected via a cable C. It is supplied to the television receiver 20. The television receiver 20 reproduces an image in accordance with the video signal Vs, and at the same time, synchronizes with the image and outputs the audio signal A.
A sound and a voice corresponding to s are produced from the speaker.

The multimedia control device 1 is shown in FIG.
As shown in FIG. 2, it is composed of a lid 1a that can be opened and closed and a tablet portion 1b. It should be noted that FIG. 1A shows the unfolded state. In the housed state, the lid 1a is placed on the tablet portion 1b, and the device 1 has a structure that is housed in a bag shape. In FIG. 1B, reference numeral 2 denotes a ROM cassette, which is a control program described later, automatic performance data and image patterns, or ADP in which voices and instrument sounds are compression-coded.
CM data and the like are stored. 3 is a picture book. The pictures drawn on each page of the picture book 3 are stored so that the image patterns of the ROM cassette 2 substantially correspond to each other.

Reference numeral 4 denotes a page detection sensor arranged on the upper portion of the lid 1a, which detects the presence or absence of a cutout portion formed on each page of the picture book 3 to discriminate the currently opened page. 5
Is a touch pen that generates magnetic flux. The magnetic flux generated by the touch pen 5 is generated in the back surface of the lid 1a and the tablet portion 1b.
The pen position detection matrix 5a laid in the upper surface (see FIG. 2).
The current position of the pen 5 is determined based on the detected value of the pen 5). Reference numeral 6 is a power switch for setting the device 1 on and off, and reference numeral 7 is a keyboard sheet placed on the tablet portion 1b. This keyboard sheet is a pen position detection matrix 5
It corresponds to the arrangement position of a, and the pitch and note length can be designated according to the pointing operation of the touch pen 5. Reference numeral 8 is a pad operated during rhythm performance, and 9 is a joy pad used as a selection operator. Reference numeral 10 is a microphone for voice input.

B. Configuration of Embodiment Next, the configuration of the embodiment will be described with reference to FIG.
It should be noted that parts common to the respective parts shown in FIG. 1 are denoted by the same reference numerals, and the description thereof will be omitted. In FIG. 2, 5a is a pen position detection matrix laid on the back surface of the lid 1a and the upper surface of the tablet portion 1b described above. The matrix 5a detects a potential that is electromagnetically induced according to the magnetic flux of the touch pen 5 to generate pointing position data of the pen 5 and supplies the pointing position data to the bus. 11 is a CPU,
Each part of the apparatus is controlled based on the control program read from the ROM cassette 2. The operation of the CPU 11 will be described later.

Reference numeral 12 denotes a RAM used as a work area for the CPU 11, in which various register data and the like are temporarily stored. Reference numeral 13 is a tone data input / output terminal, and an external M
While outputting the MIDI data supplied from the IDI musical instrument to the bus, the MI supplied from the musical sound synthesis unit 15 described later.
Send DI data to an external sound source. A display control circuit 14 converts the image pattern read from the ROM cassette 2 into a video signal Vs according to an image control signal supplied from the CPU 11 and outputs the video signal Vs. 20 is a video signal Vs
It is a display means such as a television receiver for reproducing the video.

The tone synthesis circuit 15 has a sound source configured by a well-known waveform memory reading method or an arithmetic method,
A musical tone signal obtained by synthesizing a musical tone in accordance with the performance data supplied from the CPU 11 is supplied to the next stage. Reference numeral 16 is a reverberation / volume control circuit, which applies reverb to the tone signal output from the tone synthesis circuit 15 to a depth corresponding to the control signal supplied from the CPU 11 and adjusts the volume thereof. Reference numeral 17 denotes a microphone reverberation / volume control circuit that applies a reverb having a depth corresponding to a control signal supplied from the CPU 11 to an audio signal input from the microphone 10 and adjusts the volume thereof.

Reference numeral 18 denotes a digital voice processing circuit, which is C
Under the instruction of the PU 11, the ADPCM data read from the ROM cassette 2 is decoded to generate an analog audio signal. The processing circuit 18 samples the voice input from the microphone 10 for a certain period of time, encodes the voice into ADPCM data, and writes the ADPCM data in a predetermined storage area of the RAM 12 via a bus under the control of the CPU 11. A mixer 19 mixes the outputs of the reverberation / volume control circuits 16 and 17 and the digital audio processing circuit 18, converts them into an audio signal As consisting of stereo components of the L channel and the R channel, and outputs the audio signal As. Reference numeral 21 is a sound output terminal, which supplies the L channel and R channel audio signals As to an external sound system such as the television receiver 20.

C. Operation of Embodiment Next, the operation of the embodiment having the above configuration will be described with reference to FIGS.
Will be described with reference to. Here, the operation of the main routine will be described with reference to FIGS. 3 and 4, and then the operation of the icon processing called in the main routine will be described with reference to FIGS. 5 to 9.

(1) Operation of Main Routine First, when the multimedia control device 1 according to this embodiment is powered on, the CPU 11 loads a predetermined control program from the ROM cassette 2 to execute the main routine shown in FIG. It is activated and the process proceeds to step Sa1. In step Sa1, various registers and flags stored in the RAM 3 are reset and initialized, and the process proceeds to the next step Sa2. When the process proceeds to step Sa2, the CPU 11
Reads the initial image pattern and the automatic performance data corresponding thereto from the ROM cassette 2 sequentially. It should be noted that the automatic performance data mentioned here is pattern data for a predetermined measure that indicates the sounding timing of each rhythm instrument sound corresponding to the specified rhythm type.

Next, when the process proceeds to step Sa3, the CPU
Reference numeral 11 transfers the thus-read initial image pattern and automatic performance data to the display control circuit 14 and the tone synthesis circuit 15, respectively, via the bus. As a result, the display control circuit 14 converts the initial image pattern into the video signal Vs and supplies the video signal Vs to the display means (television receiver) 20.
On the other hand, the musical tone synthesizing circuit 15 synthesizes musical tone signals for each rhythm instrument sound in accordance with the automatic performance data. The tone signal is supplied to the sound system from the sound output terminal 21 via the mixer 19 after being subjected to reverberation or volume control.
As a result, an automatic rhythm performance is performed in synchronization with the opening screen display.

Next, the CPU 11 proceeds to step Sa4 and determines whether or not the page of the picture book 3 set on the back surface of the lid 1a (see FIG. 1A) has been turned. Here, if the output of the page detection sensor 4 does not change, it is considered that the opening page remains and the judgment result is "N.
O ”, the process returns to step Sa3, and the above-mentioned opening state is maintained. Then, for example, when the “animal stage” page shown in FIG. 4 is opened, the page detection sensor 4 detects this. As a result, the determination result of step Sa4 becomes "YES", and the processing of the CPU 11 proceeds to step Sa5.

At step Sa5, the image pattern and the automatic performance data of the corresponding page are sequentially read from the ROM cassette 2. Then, in step Sa6, the read image pattern and the automatic performance data are transferred to the display control circuit 14 and the musical tone synthesis circuit 15, respectively. As a result, the "animal stage" screen similar to the picture book page shown in FIG. 4 is displayed as a still image, while each rhythm instrument sound corresponding to the automatic performance data is synthesized to perform the stage opening performance.

Next, the processing of the CPU 11 is performed in step Sa.
In step 7, it is determined whether or not the “icon” on the picture book page shown in FIG. 4 is pointed by the touch pen 5. On this page, "1 banana", "2 bananas" and "3" are drawn at the bottom of the right page.
"Book Banana" and "4 Bananas" are icon EC ~
It becomes EC4. Here, when none of the icons EC1 to EC4 is pointed, the determination result is "N.
"O" and the "Animal Stage" screen continues to be displayed along with the stage opening performance. On the other hand, icon E
When any one of C1 to EC4 is pointed, the determination result here is “YES”, and the process proceeds to step Sa8.

At step Sa8, an event corresponding to the type of the icon pointed to is executed. This event is a process for educating the sense of sound, the sense of rhythm and the like in a game sense, and the details thereof will be described later. Then, each time the event for each icon is completed, the CPU 11 advances the process to step Sa9, and repeats steps Sa6 to Sa8 described above until a page change occurs. As described above, in the main routine, the image corresponding to each page of the picture book 3 and the performance are reproduced in synchronization with each other, and the multimedia that emphasizes the interactivity with the player is realized.

(2) Operation of Icon Processing Routine Operation when Icon EC1 is Pointed In the picture book page shown in FIG. 4, the icon EC1 is displayed.
When ("one banana") is pointed by the touch pen 5, the pen position detection matrix 5a gives the pointing position data to the CPU 11 in response. As a result, the CPU 11 recognizes that the icon EC1 is pointed, activates the icon EC1 processing routine shown in FIG. 5, and advances the process to step Sb1. A micro switch is provided at the pen tip of the touch pen 5, and the pen tip detects the pen position detection matrix 5.
The pointing position data is fetched by the CPU 11 when the switch is pressed on a and the switch is turned on.

In step Sb1, a plurality of image data corresponding to the performance movements of the respective characters A to E of "elephant", "flamingo", "giraffe", "crocodile" and "lion" are read out from the ROM cassette 2, and By switching a plurality of images, these characters A to E are respectively displayed in FIG.
As shown in, a moving image is displayed so that the percussion instrument performance is performed.
In addition, the percussion instrument sounds (elephant: bass drum, flamingo: tambourine, giraffe: claves, crocodile: bongo, lion: cymbal) assigned to each character in synchronization with this percussion instrument performance are synthesized to create a “jungle band”. To start playing.

Next, in step Sb2, it is determined whether or not any of the characters A to E is pointed. Here, for example, character A (“elephant”)
Is pointed. Then, the determination result becomes "YES", and the process proceeds to the next step Sb3. In step Sb3, the pointed character A,
That is, the movement control is performed so that the image of the "elephant" advances one step on the screen, and the background image is controlled such that the "tulip" displayed in front of the selected character has an open shape. Then, when proceeding to step Sb4, only the performance data corresponding to the character A (“elephant”) is synthesized with a musical tone. In this case, volume control is performed so that the volume of the "bass drum sound", which is the musical instrument of the character A ("elephant"), is increased to enhance the solo performance.

When the solo performance of the designated character is completed for the predetermined phrase in this way, the CPU 11 executes step Sb5.
Then, it is determined whether or not the reproduction has been completed (pointed) for all the characters A to E. Here, if the solo performances of all the characters A to E have not been completed,
The determination result is “NO”, and the above-described steps Sb1 to Sb1.
Repeat Sb4. On the other hand, when the reproduction is completed for all the characters A to E, the determination result is “YES”, and the process proceeds to the next step Sb6. In step Sb6, the rhythm pattern is updated in order to change the content of the solo performance, assuming that the solo performance of each of the characters A to E has completed one cycle.

Then, when the operation proceeds to step Sb7, the ROM
All rhythm patterns stored in the cassette 2 (for example, 5
It is determined whether all performance data of type) have been reproduced. Here, when all types of rhythm patterns are not reproduced, the determination result is "NO", and steps Sb1 to Sb6 are repeated again based on the rhythm pattern updated in step Sb6. It should be noted that when a solo performance is performed for each of the characters A to E for all rhythm patterns, the determination result here is “YES”, and the CPU
11 returns the processing to the above-mentioned main routine.

As described above, in the icon EC1 processing,
The animal (character) selected by the player advances one step forward on the screen, and the percussion instrument possessed by the animal is played alone. At this time, since the motion of the animal (character) is controlled in motion in synchronization with the performance sound, it is as if the picture book 3
The visual and acoustic effects as if the animal depicted in FIG.

Operation When Icon EC2 is Pointed In the picture book page shown in FIG. 4, when the player points the icon EC2 (“two bananas”) with the touch pen 5, the pen position detection matrix 5a responds accordingly. And gives pointing position data to the CPU 11.
As a result, the CPU 11 recognizes that the icon EC2 is pointed, activates the icon EC2 processing routine shown in FIG. 6, and advances the processing to step Sc1.
In step Sc1, the ADPCM data read from the ROM cassette 2 is processed by the digital audio processing circuit 18 (see FIG. 2).
See). As a result, audio playback is performed,
The narration instruction "Please touch your favorite animal" is given.

Then, when the operation proceeds to step Sc2, the CPU
11 waits until the player points to one of the characters A to E in response to this narration instruction. Here, for example, by the player, the character A
When ("elephant") is pointed, the determination result is "YES", and the process proceeds to the next step Sc3. In step Sc3, the movement of the pointed "elephant" image is controlled so as to move one step forward on the screen, and the background image control is performed so that the "tulip" displayed in front of the "elephant" has an open shape. As a result, the image shown in FIG. 9 is displayed on the television receiver 20.

Then, in step Sc4, the performance data of the percussion instrument corresponding to the designated character A is read out. In this case, the drum sound is rhythmically played for one bar,
The performance operation of the character A is displayed as a moving image in synchronization with this performance. Next, when the rhythm performance for one bar and the moving image display are completed, the CPU 11 advances the processing to step Sc5, transfers the ADPCM data read from the ROM cassette 2 to the digital voice processing circuit 18, Please narrate. " Then, in order to further encourage the player to operate the pad, the voice narration of "Ichi-san, yes" is reproduced.

When the player hits the pad 8 (see FIG. 2) according to this narration to make a rhythm corresponding to one bar, the CPU 11 detects the pad operation in step Sc6 and generates pad operation information. To do. Then, in step Sc7, the playing motion of the character A is displayed as a moving image in accordance with the pad operation information, and the drum sound is synthesized in synchronization with the pad operation information. That is, the character A is caused to perform a drum performance operation in response to a pad operation, and a drum sound is produced in synchronization with the operation. Next, step Sc8
Proceeding to FIG. 7, it is determined in step Sc4 whether or not the proper rhythm pattern played and the pad operation timing by the player match. Here, if there is a predetermined error between the rhythm pattern and the pad operation timing, the determination result is “NO”, and the process proceeds to step Sc9.

In step Sc9, the ADPCM data read from the ROM cassette 2 is transferred to the digital voice processing circuit 18, and the narration "Please tap the pad again" is voice-synthesized. After this, the CPU 11
Repeats step Sc4 and subsequent steps described above again. on the other hand,
When the rhythm pattern and the pad operation timing match, the determination result of step Sc8 is “YES”, and the process proceeds to step Sc10. When the rhythm pattern matches the pad operation timing and the pad operation can be cleared, the fanfare sound is synthesized. In step Sc10, it is determined whether or not the pad operation has been cleared for all the characters A to E. Here, if it has not been cleared, the determination result is “NO”, and the routine proceeds to the subsequent Step Sc11.

In step Sc11, for example, the character A ("elephant") described above is followed by the next character B.
(“Flamingo”) is specified, and the above-mentioned step Sc3 is performed.
Repeat the above. When the pad operation for all the characters A to E is cleared, the determination result of step Sc10 becomes "YES", and the process proceeds to step Sc12. At step Sc12, all the characters A to E, that is,
RO image data corresponding to performances of "elephant,""flamingo,""giraffe,""crocodile," and "lion."
The characters are read out from the M cassette 2 and displayed as a moving image so that the characters A to E perform the percussion ensemble. Further, in synchronization with this moving image display, the percussion instrument sound assigned to each character is synthesized into a musical sound, and a "jungle band" is played.

As described above, in the icon EC2 process, a percussion instrument participation event is performed, and when the pad is struck in accordance with the rhythm of the selected animal (character) in accordance with the narration instruction, the character responds to the pad operation and the character plays the musical instrument. A peculiar performance operation is performed, and the percussion instrument sound of the character is rhythmically pronounced. When the player hits one bar in good timing, the fanfare sounds and the character moves to the next character. Therefore, it is possible to cultivate an accurate sense of rhythm while playing as if playing a game by using multimedia that combines visual effects and acoustic effects.

When the icon EC3 ("3 bananas") shown in FIG. 4 is pointed, a percussion instrument participation event is performed as in the icon EC2 process.
In the icon EC3 processing, when a pad is hit according to the rhythm (two measures) of the selected animal (character) according to the narration instruction, the character performs a performance operation in response to the pad operation, and the corresponding percussion sound is rhythmically pronounced. To be done. Also in this case, if the rhythm for two measures is timed, the fanfare sounds and the rhythm of the character can be cleared.

Further, when the icon EC4 ("4 bananas") is pointed, a free performance event is executed. In this free performance event, any character is selected while all the characters A to E are playing the percussion instrument, and the selected character plays according to the pad operation of the player. Therefore, also in the processing of these icons EC3 and EC4, the percussion instrument performance by the actual pad operation is added to the visual effect as if the animal depicted in the picture book is playing the percussion instrument, and the sense of sound and the rhythm are cultivated as if playing a game. It will be possible. In the above embodiments, all musical instruments were rhythm musical instruments, but musical instrument sounds and image patterns are not limited to this,
For example, as shown in FIG. 10, a wind instrument may be assigned to the character, and needless to say, a string instrument, a keyboard instrument, or the like may be used.

[0036]

As described above, according to the present invention, when one of the instrument sounds assigned to each of a plurality of characters is designated by the instrument sound designating means and the performance operator is operated, While the sounding control means controls the sounding of the designated musical instrument sound in response to the operation signal supplied from the performance operator, the image control means controls the moving performance of the character in response to the operation signal. You can comprehensively learn tones and playing methods.

[Brief description of drawings]

FIG. 1 is a diagram showing a schematic structure of a multimedia control device 1 according to the present invention.

FIG. 2 is a block diagram showing an electrical configuration of the embodiment.

FIG. 3 is a flowchart showing an operation of a main routine in the embodiment.

FIG. 4 is a diagram showing an example of a picture book page in the embodiment.

FIG. 5 is a flowchart showing an operation of an icon EC1 processing routine in the embodiment.

FIG. 6 is a flowchart showing an operation of an icon EC2 processing routine in the embodiment.

FIG. 7 is a flowchart showing an operation of an icon EC2 processing routine in the embodiment.

FIG. 8 is a diagram for explaining the operation of an icon EC1 processing routine in the embodiment.

FIG. 9 is a diagram for explaining the operation of an icon EC2 processing routine in the embodiment.

FIG. 10 is a diagram showing an example of an image pattern when a wind instrument is assigned to a character.

[Explanation of symbols]

2 ... ROM cassette (sound control means, image control means),
5 ... Touch pen (instrument sound designating means), 5a ... Pen position detection matrix (instrument sound designating means), 8 ... Pad (performance operator), 11 ... CPU (instrument sound designating means, sound generation control means, image control means), 12 ... RAM, 14 ... Display control circuit (image control means), 15 ... Tone synthesis circuit (sound control means).

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G06T 13/00 G09B 15/00 E G09G 5/00 9471-5G G10H 1/00 Z 8622-5H

Claims (2)

[Claims] 1. A performance operator for generating an operation signal according to a performance operation, a musical instrument sound designating means for designating any one of musical instrument sounds assigned to each of a plurality of characters displayed on a screen, and the musical instrument sound designating means. Sound control means for controlling the sound of the musical instrument sound designated by the musical instrument in accordance with the operation signal, and image control means for controlling the moving image of the performance performance of the character corresponding to the musical instrument sound designated by the musical instrument sound designating means in response to the operational signal. And a multimedia control device. 2. A performance operator for generating an operation signal according to a performance operation, a musical instrument sound designating means for designating one of musical instrument sounds assigned to each of a plurality of characters displayed on the screen, and each of the plurality of characters. Of the musical instrument sound designating means, the musical instrument sound designated by the musical instrument sound designating means is controlled in accordance with the corresponding musical performance pattern, while the musical instrument is operated when the musical performance operator is operated. The musical instrument sound designating means is provided with sound producing control means for controlling the pronunciation of the musical instrument sound designated by the sound designating means in accordance with the operation signal, and means for storing an image pattern representing a performance action for each of the plurality of characters. While the image pattern of the first character corresponding to the musical instrument sound is read out and the performance operation of the first character is displayed as a moving image, when the performance operator is operated. An image control means for controlling a moving image of the performance movement of the first character in synchronization with the operation signal; and a performance corresponding to the musical instrument sound designated by the musical instrument sound designating means when the musical performance operator is operated for performance. A multimedia control device comprising: a performance determining means for determining whether or not the pattern matches the performance operation.