JP2006178268A - Musical sound controller and musical sound control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize a musical sound controller which creates a new pleasure to familiarize a user with music. <P>SOLUTION: A CPU 4 generates control parameters 1 to 3 changing according to the hue of image data imaged by an imaging input section consisting of a camera 7 and an A/D converter 3. The CPU selects chord data pit producing sound according to the control parameter 1 and generates the sound of the selected chord data pit when the control parameter 1 of the specified value or above is generated. During the sound production of the chord, the CPU performs the pitch control complying with the control parameter 2 and the sound image control complying with the control parameter 3. The CPU silences the chord during the sound production when the control parameter 2 of the specified value or above is generated. As a result, such a new pleasure which familiarizes the user with the music can be created. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a musical tone control device and a musical tone control program suitable for use in, for example, a DTM device using a personal computer or an electronic musical instrument.

  A technique for controlling a musical sound generated using a pointing device such as a mouse is known. For example, in Patent Document 1, characteristics (pitch, volume, timbre, and effect) of a musical tone to be generated are changed in accordance with two-dimensional position information generated by a pointing device operation. Techniques have been disclosed for improving the operability for changing the characteristics of sounds and expanding the range of performance expressions.

JP 2001-195069 A

By the way, the technique disclosed in Patent Document 1 is premised on a user who has a certain degree of musical knowledge. If such a user can predict a change in musical sound corresponding to a pointing operation, the range of performance expression is widened. Become. However, there is a problem that it becomes unsuitable for the purpose of getting used to music, such as a music learning toy for infants.
Therefore, the present invention has been made in view of the above-described circumstances, and an object thereof is to provide a musical sound control device and a musical sound control program capable of creating new enjoyment that makes it easy to get used to music.

  In order to achieve the above object, according to the first aspect of the present invention, the control parameter generating means for generating a control parameter that changes in accordance with the hue of the captured image and the control parameter generated by the control parameter generating means are supported. The tone generation means for generating or muting the type of musical sound selected in accordance with the change of the control parameter, and the control parameter generation means for generating the tone generation form of the tone generated by the musical sound generation means It is characterized by comprising sound generation form control means for controlling in accordance with parameters.

  According to the second aspect of the present invention, the control parameter generating means for generating at least one control parameter that changes in accordance with the hue of the captured image, and the value of the control parameter generated by the control parameter generating means. A tone generating means for sounding or muting at least one of the selected chords and distributed chords according to the value of at least one control parameter generated by the control parameter generating means; and the tone generating means Pitch sound image control means for controlling the pitch and sound image of the musical sound in accordance with at least one control parameter generated by the control parameter generating means.

  According to the third aspect of the present invention, there is provided a control parameter generation process for generating a control parameter that changes in accordance with the hue of a captured image, and a type selected according to the control parameter generated by the control parameter generation process. A musical tone generation process for generating or muting a musical sound according to a change in the control parameter, and a musical sound generation form generated by the musical sound generation process are controlled according to the control parameter generated by the control parameter generation process. The sound generation form control processing is executed by a computer.

  The invention according to claim 4 corresponds to a control parameter generation process for generating at least one control parameter that changes according to the hue of the captured image, and a value of the control parameter generated by the control parameter generation process. A tone generation process for generating or muting at least one tone of a chord and a distributed chord selected in accordance with a value of at least one control parameter generated in the control parameter generation process; and A pitch sound image control process for controlling a pitch and a sound image of a musical sound to be performed according to at least one control parameter generated by the control parameter generation process is executed by a computer.

  In the first and third aspects of the invention, when a control parameter that changes in accordance with the hue of the captured image is generated, the type of musical sound that is selected corresponding to the control parameter is changed in accordance with the change in the control parameter. While sounding or muting is performed, the sounding form of the generated musical sound is controlled according to the control parameter, so that new enjoyment that makes it easy to get used to music can be created.

  According to the second and fourth aspects of the present invention, when at least one control parameter that changes in accordance with the hue of the captured image is generated, a chord and a distributed chord selected corresponding to the value of the generated control parameter At least one musical sound is sounded or muted according to the value of at least one control parameter, while the pitch and sound image of the musical sound to be generated are controlled according to at least one control parameter, respectively, so that the user can become familiar with music. Such new pleasure can be created.

Embodiments of the present invention will be described below with reference to the drawings.
A. Configuration (1) Overall Configuration FIG. 1 is a block diagram showing the overall configuration of a musical tone control apparatus according to an embodiment of the present invention. In this figure, the keyboard 1 generates performance information including a key-on / key-off signal, a key number (key number), velocity, and the like corresponding to a press / release key operation (performance operation). The switch unit 2 is a timbre switch for setting the tone color of the musical tone to be generated, a pattern selection switch for selecting an accompaniment pattern, an A button for instructing the on / off operation of a chord, and the on / off operation of the accompaniment sound. B button for instructing the switch, and a switch event corresponding to the type of switch (button) to be operated is generated.

The display unit 3 is composed of an LCD panel or the like, and displays a setting state and an operation mode of each unit of the musical instrument in accordance with a display control signal supplied from a CPU 4 to be described later, or a camera 7 and an A / D converter 8 to be described later. An image (or video) input by the imaging input unit is displayed on the screen.
The CPU 4 operates in the instrument mode or the imaging input mode. When operating in the musical instrument mode, the CPU 4 sets the operating state of each part of the musical instrument based on the switch event generated by the switch unit 2, and sets musical tone parameters (for example, a sound generation instruction command, a mute instruction command, etc.) according to performance information. It is generated and sent to the tone generator 9. On the other hand, when the CPU 4 operates in the image input mode, the CPU 4 sends a chord or accompaniment sound / mute to the musical sound generation unit 9 in response to an on / off operation of the A button or the B button provided in the switch unit 2, and will be described later. Control parameters that change the sound generation mode according to image data input by the imaging input unit including the camera 7 and the A / D converter 8 (for example, a pan control value that changes the sound image, a bend control value that changes the volume, etc.) Is transmitted to the musical sound generation unit 9. The processing operation of the CPU 4 under the imaging input mode related to the gist of the present invention will be described in detail later.

The ROM 5 includes a program area and a data area. Various control programs loaded on the CPU 4 are stored in the program area of the ROM 5. The various control programs referred to here include a main routine, variable generation processing, switch processing, chord sound generation processing, accompaniment sound generation processing, and accompaniment progression processing described later. In the data area of the ROM 5, chord data and accompaniment pattern data are stored, and the data structure will be described later.
The RAM 6 includes a work area for storing various register / flag data, an image data area for storing image data input by an imaging input unit including the camera 7 and the A / D converter 8, and an image buffer. The configuration of main data stored in the work area and the image data area will be described later.

  The imaging input unit composed of the camera 7 and the A / D converter 8 controls the image signal of the subject imaged by the camera 7 composed of a CCD or the like by the A / D converter 8 under a control of the CPU 4 at predetermined intervals. The image data is converted into image data and stored in the image buffer of the RAM 6. The tone generator 9 is configured by a well-known waveform data readout method. For example, under the imaging input mode, the chord data (stored in the ROM 5) (in accordance with an instruction to generate (or mute) a chord (or accompaniment tone) from the CPU 4) (Or accompaniment pattern data) is reproduced based on the chord (or accompaniment sound), and tone data obtained by changing the sounding mode of the reproduced chord (or accompaniment sound) according to the control parameter supplied from the CPU 4 is output. . The D / A converter 10 converts the musical tone data output from the musical tone generator 9 into a musical tone signal in an analog signal format and outputs the result. The sound system 11 performs filtering such as noise removal on the musical sound signal supplied from the D / A converter 10, then amplifies the signal and emits the sound from a speaker (not shown).

(2) Data Configuration Next, referring to FIGS. 2 to 3, the configuration of chord data and accompaniment pattern data stored in the data area of the ROM 5, the configuration of main register flag data provided in the work area of the RAM 6, and The configuration of image data stored in the image data area of the RAM 6 will be described.

  FIG. 2A shows the structure of chord data stored in the data area of the ROM 5. In the data area of the ROM 5, a plurality (for the number of MAXPIT) of chord data pit [0] to [MAXPIT-1] are stored. One chord data pit [n] is composed of a plurality of chord constituent sounds [0] to [WN-1] (WN is the number of chord constituent sounds). The pitch of the chord constituent sound specified by the chord data pit [n] [m] is expressed by a MIDI note number.

  FIG. 2B is a diagram showing the structure of accompaniment pattern data stored in the data area of the ROM 5. The accompaniment pattern data is composed of a plurality of types of accompaniment patterns apat [0] to [MAXPAT-1]. One accompaniment pattern apat includes a header and a plurality of note data anote. The header has a resolution res expressed by the number of notes maxnote included in the pattern, the number of ticks of the quarter note length (number of tempo clocks), the beat beat expressed by the number of quarter notes in one measure, and the number of measures. The pattern length len is represented. The note data annote [0] to [maxnote-1] has a sound generation start time time expressed by the number of ticks, and a note-on event com indicating the pitch and volume to be sounded.

  FIG. 3A is a diagram showing a configuration of main register / flag data provided in the work area of the RAM 6. In this figure, a flag hmny is a flag for instructing the sounding or muting of a chord. In the case of “1”, the sounding is instructed, and in the case of “2”, the sounding is instructed. At the time of initialization, an initial value “0” is set in the flag hmny. The flag drum is a flag for instructing the sounding or muting of the accompaniment sound. When the flag is “1”, the sounding is instructed. When the flag is “2”, the muting is instructed. At the time of initialization, the initial value “0” is set in the flag drum. The control parameters param1 to param3 are generated according to image data input by the imaging input unit including the camera 7 and the A / D converter 8, and are used as parameters for changing the sound generation mode. The purpose of these control parameters param1 to param3 will be described later.

  The number of the selected chord data pit is temporarily stored in the register pitn. At the time of non-sounding, “−1” is stored in the register pitn. The wait time is temporarily stored in the register wait. The register piti temporarily stores an index value indicating the chord constituent sound being generated in the selected chord data pit. The number of the selected accompaniment pattern apat is temporarily stored in the register patn. At the time of non-sounding, “−1” is stored in the register patn. The register notei temporarily stores an index value indicating the note data anote that is being sounded in the selected accompaniment pattern apat. The register basetime temporarily stores the current time (in msec units) when accompaniment sounding is started. The register tickmeasprev temporarily stores the number of ticks from the beginning of the pattern in the accompaniment progression process described later. The register clri temporarily stores an index value indicating a register for storing an average value in image data to be described later.

  FIG. 3B is a diagram showing a configuration of image data stored in the image data area of the RAM 6. In the image data area of the RAM 6, registers grf [0] to [MAXCLR-1] corresponding to the image data captured by the imaging input unit including the camera 7 and the A / D converter 8 are provided. An arbitrary register grf [clri] specified by the index value clri includes a register grf [clri] [0] in which an average value of the red component R in the captured image data is stored, and an average value of the green component G. A register grf [clri] [1] to be stored and a register grf [clri] [2] to store the average value of the blue component B are provided.

B. Operation Next, the operation of the embodiment will be described with reference to FIGS. In the following, the operation of the main routine will be described first, and then the operations of variable generation processing, switch processing, chord sound generation processing, accompaniment sound generation processing, and accompaniment progression processing called from the main routine will be described.

(1) Operation of main routine In response to the power-on of the musical tone control apparatus having the above-described configuration, the CPU 4 executes the main routine shown in FIG. 4 and proceeds to step SA1, where various registers stored in the work area of the RAM 6 An initial process for initializing flags is executed. Then, the process proceeds to step SA2, and the control parameters param1 to 3 for changing the sound generation mode are generated based on the image data captured by the imaging input unit including the camera 7 and the A / D converter 8, and the generated control parameters param1, In response to 2, variable generation processing for generating a flag hmny for instructing chord pronunciation / mute is executed. Subsequently, in step SA3, a flag hmny for instructing the sounding / mute of chords is set according to the on / off operation of the A button, or the flag drum for instructing sounding / muteing of the accompaniment sound according to the on / off operation of the B button. Execute the switch process to be set.

  Next, in step SA4, a chord sound generation process is executed. In the chord sound generation process, as will be described later, the chord data pit [pitn] to be sounded is selected according to the control parameter param1 generated based on the captured image data content, and the selected chord data pit [pitn] When the button is turned on or when an image having a large red component is captured, the sound is generated. During sound generation, the volume and sound image are controlled according to the control parameters param2 and param3 generated based on the captured image data contents, and the A button When an off operation or an image with a large amount of green component is captured, the chord being sounded is muted.

  Then, the process proceeds to step SA5, where accompaniment sound generation processing for instructing the sound generation or muting of the accompaniment sound is executed in accordance with the ON / OFF operation of the B button. Next, in step SA6, for example, other processing such as generating musical tone parameters (for example, a sound generation instruction command, a mute instruction command, etc.) corresponding to the performance information generated by the keyboard 1 and sending it to the musical sound generation unit 9 is performed. Is executed, the process returns to step SA2 described above. Thereafter, the CPU 4 repeats the operations in steps SA2 to SA6 described above until the apparatus power is turned off.

(2) Operation of Variable Generation Processing Next, the operation of variable generation processing will be described with reference to FIG. When this process is executed through the above-described step SA2 (see FIG. 4), the CPU 4 proceeds to step SB1 shown in FIG. In step SB1, image data for a plurality of frames picked up by the image pickup input unit including the camera 7 and the A / D converter 8 is temporarily transferred to an image buffer (not shown) of the RAM 6, and the plurality of frames in the image buffer are stored. Average values of the red component R, the green component G, and the blue component B, that is, the hue is extracted from the image data. The average values of the extracted red component R, green component G, and blue component B are stored in the registers grf [clri] [0] to [clri] [2] designated by the index value clri of the image data area of the RAM 6. Store. Thereafter, the index value clri is updated.

Next, in step SB2, the average value of the red component R stored in the register grf [] [0] is set as the control parameter param1, the average value of the green component G stored in the register grf [] [1] is set as the control parameter param2. The average value of the blue component B stored in the register grf [] [2] is set in the control parameter param3.
Next, in step SB3, it is determined whether or not the control parameter param1 is larger than a preset threshold value THR1. If it is greater than the threshold value THR1, the determination result is “YES”, the process proceeds to step SB4, the flag hmny is set to “1” to indicate a chord pronunciation, and then the process proceeds to step SB5. Therefore, the chord pronunciation is automatically instructed when the red component R of the captured image is large.

  On the other hand, if the control parameter param1 is smaller than the threshold value THR1, the determination result in step SB3 is “NO”, and the process proceeds to step SB5. In step SB5, it is determined whether or not the control parameter param2 is larger than a preset threshold value THR2. If it is greater than the threshold value THR2, the determination result is “YES”, the process proceeds to step SB6, the flag hmny is set to “2” and the chord mute is instructed, and then this process is completed. Therefore, when there is a large amount of the green component G in the image input, the chord mute is automatically instructed. On the other hand, if the control parameter param2 is smaller than the threshold value THR2, the determination result in step SB5 is “NO”, and this process is completed.

  As described above, in the variable generation process, the control parameters param1 to 3 that change the sound generation mode based on the image data captured by the imaging input unit including the camera 7 and the A / D converter 8 are generated, and the generated control parameters are generated. A flag hmny for instructing the sounding / muteing of chords is generated in accordance with param1 and param2.

(3) Operation of Switch Processing Next, the operation of switch processing will be described with reference to FIG. When this process is executed through the above-described step SA3 (see FIG. 4), the CPU 4 proceeds to step SC1 shown in FIG. In steps SC1 to SC8, processing corresponding to the operated switch type is executed. In the following description, the operation will be described separately for the case where the A button is turned on, the case where the A button is turned off, the case where the B button is turned on, and the case where the B button is turned off.

<When button A is turned on>
When the A button is turned on, the determination result in step SC1 is “YES”, the process proceeds to step SC2, and “1” is set in the flag hmny to instruct the generation of a chord.

<When button A is turned off>
When the A button is turned off, the determination result in step SC3 is “YES”, the process proceeds to step SC4, and “2” is set in the flag hmny to instruct the muting of the chord.

<When button B is turned on>
When the B button is turned on, the determination result in step SC5 becomes “YES”, and the process proceeds to step SC6, where “1” is set in the flag drum to instruct the pronunciation of the accompaniment sound.

<When button B is turned off>
When the B button is turned off, the determination result in step SC7 becomes “YES”, and the process proceeds to step SC8, where “2” is set in the flag drum to instruct to mute the accompaniment sound. Thereafter, the process proceeds to step SC9, and after executing other switch processing such as setting the tone color of the generated musical tone according to the operation of the tone color switch, this processing is completed.
As described above, in the switch processing, the flag hmny for instructing the sounding / muteing of the chord is set according to the on / off operation of the A button, or the flag drum for instructing the sounding / muteing of the accompaniment sound according to the on / off operation of the B button. Is set.

(4) Operation of Chord Sound Generation Process Next, the operation of the chord sound generation process will be described with reference to FIG. When this process is executed via step SA4 (see FIG. 4) described above, the CPU 4 proceeds to step SD1 shown in FIG. After step SD1, it is determined whether or not sound generation is instructed in a non-sounding state (step SD1), whether mute is instructed during sounding (step SD5), and whether sounding is in progress (step SD10). In the following, corresponding to these state determinations, the operation in each case will be described separately when sound generation is instructed in an unsound state, mute is instructed during sound generation, and when sound generation is in progress.

<When pronunciation is instructed in an unsound state>
In step SD1, it is determined whether or not the flag hmny is “1” and the value of the register pitn is “−1”, that is, whether or not sound generation is instructed. Here, for example, in the state of no sound, the flag hmny is set to “1” in response to the ON operation of the A button, or the flag hmny is set to “1” based on the control parameter param1 obtained by capturing an image with many red components. It is assumed that “1” is set. If it does so, the judgment result of the said step SD1 will become "YES", and will progress to step SD2. In step SD2, a remainder value obtained by dividing the value of the control parameter param1 by the number MAXPIIT of the chord data pit is stored as a chord number in a register pitn (referred to as a chord number pitn), and the chord specified by the chord number pitn is stored. The pointer n for searching for a chord constituent sound of the data pit [pitn] is reset to zero.

  Next, in step SD3, it is determined whether or not the pointer n is smaller than the number of constituent sounds WN of the chord data pit [pitn]. That is, it is determined whether all the chord constituent sounds of the chord data pit [pitn] have been generated. If not all are pronounced, the determination result is “YES”, and the process proceeds to step SD4. In step SD4, the chord data pit [pitn] [n] (pitch of the chord constituent sound) designated by the pointer n is read from the data area of the ROM 5 (see FIG. 2), stored in the register p, and pointer to the register ch. n is stored, and the volume value “110” is set in the register v.

Then, the musical tone generation unit 9 is instructed to sound the musical tone having the pitch stored in the register p from the sound generation channel specified by the register ch with the volume value of the register v. Thereafter, the pointer n is incremented and incremented, and the process returns to step SD3.
Thereafter, steps SD3 to SD4 are repeated, and the chord constituent sound in the chord data pit [pitn] is incremented while the pointer n is incremented until the number of constituent sounds WN of the chord data pit [pitn] specified by the chord number pitn is exceeded. Pronounce all. When all the sounds are generated, the determination result in step SD3 is “NO”, and this process is once completed.

<When mute is instructed during pronunciation>
In this case, the determination result in step SD1 is “NO”, the process proceeds to step SD5, the flag hmny is “2”, and the value of the register pitn is not “−1”, that is, mute is instructed during sound generation. It is judged whether it is done. Here, for example, during sound generation, the flag hmny is set to “2” according to the OFF operation of the A button, or the flag hmny is set to “2” based on the control parameter param2 obtained by capturing an image with a lot of green hues. ”Is set. If it does so, the judgment result of the said step SD5 will become "YES", and will progress to step SD6.

  In step SD6, the pointer n is reset to zero, and in the subsequent step SD7, it is determined whether or not the pointer n is smaller than the number of constituent sounds WN of the chord data pit [pitn]. That is, it is determined whether all the chord constituent sounds of the chord data pit [pitn] have been muted. If all the sound is not muted, the determination result is “YES”, and the flow proceeds to step SD8. In step SD8, the chord data pit [pitn] [n] (pitch of the chord constituent sound) designated by the pointer n in the chord data pit [pitn] in the sounding state is stored in the register p and the pointer is stored in the register ch. n is stored, and the musical tone generation unit 9 is instructed to mute the musical tone of the pitch stored in the register p from the sound generation channel specified by the register ch. Thereafter, the pointer n is incremented and incremented, and the process returns to step SD7.

  Thereafter, steps SD7 to SD8 are repeated, and the chord constituent sound in the chord data pit [pitn] is incremented while incrementing the pointer n until the number of constituent sounds WN of the chord data pit [pitn] specified by the chord number pitn is exceeded. Mute all. When all the sounds are muted, the determination result in step SD7 is “NO”, the process proceeds to step SD9, the value “−1” representing the muted state is stored in the chord number pitn, and this processing is once completed.

<When sounding>
In this case, each determination result in steps SD1 and SD5 is “NO”, and the process proceeds to step SD10 to determine whether the value of the register pitn is not “−1”, that is, whether sounding is being performed. If the sound is being generated, the determination result is “YES”, the process proceeds to step SD11, and the pointer n is reset to zero. Subsequently, in step SD12, it is determined whether or not the pointer n is smaller than the number of constituent sounds WN of the chord data pit [pitn]. If the pointer n does not exceed the number of constituent sounds WN, the determination result is “YES”, and the flow proceeds to step SD13.

  In step SD13, the tone generator 9 is instructed to perform bend control for volume control according to the control parameter param2 and pan control for sound image control according to the control parameter param3. Specifically, a pitch bend change message corresponding to the control parameter param2 and a pan control change message corresponding to the control parameter param3 are generated and supplied to the tone generation unit 9. Thereafter, the pointer n is incremented and incremented, and the process returns to step SD12.

  Thereafter, steps SD12 to SD13 are repeated, and the volume and sound image of the constituent sound being sounded are controlled according to the control parameter param2 and the control parameter param3 while the pointer n is incremented until the number of constituent sounds WN is exceeded. When the control of the volume and the sound image is completed for all the chord constituent sounds that are sounding, the determination result in step SD12 is “NO”, and this processing is once completed.

  As described above, in the chord sound generation process, the chord data pit [pitn] to be generated is selected according to the control parameter param1 generated based on the captured image data content, and the selected chord data pit [pitn] is selected by the A button. The sound is generated when an on operation or an image having a large red component is picked up. During sound generation, the sound volume and sound image are controlled according to the control parameters param2 and param3 generated based on the content of the picked up image data, and the A button is turned off. Alternatively, when an image with a large amount of green component is captured, the chord being sounded is muted.

(5) Accompaniment Sound Generation Operation Next, the accompaniment sound generation operation will be described with reference to FIG. When this process is executed via the above-described step SA5 (see FIG. 4), the CPU 4 advances the process to step SE1 shown in FIG. 8, the flag drum is “1”, and the value of the register patn is “ -1 ", that is, whether or not the generation of the accompaniment sound is instructed in the unsound state. If the sound of the accompaniment sound is not instructed in the unsound state, the determination result is “NO”, and the process proceeds to step SE3 described later.
On the other hand, for example, if the flag drum is set to “1” in response to the ON operation of the B button in the non-sounding state, the determination result here is “YES”, and the process proceeds to step SE2.

  In step SE2, the remainder value obtained by dividing the value of the control parameter param3 by the number of accompaniment patterns apat MAXPAT is stored as an accompaniment pattern number in a register patn (referred to as accompaniment pattern number patn), and designated by this accompaniment pattern number patn. A register notei (referred to as index notei) holding an index value for specifying note data notes [0] to [maxnote-1] constituting the accompaniment pattern apat [patn] is reset to zero. Further, the current time expressed in msec units is stored in the register basetime as the accompaniment start time, and the register tickmeasprev is reset to zero to complete this processing.

In step SE3, it is determined whether or not the flag drum is “2” and the value of the register patn is not “−1”, that is, whether or not the accompaniment sound mute is instructed during sound generation. If mute of the accompaniment sound is not instructed during sound generation, the determination result is “NO”, and this process is completed.
On the other hand, for example, if the flag drum is set to “2” in response to the OFF operation of the B button while the accompaniment sound is being generated, the determination result here is “YES”, and the process proceeds to step SE4, where the accompaniment pattern number is set. “−1” is set to patn to set to a non-sounding state (accompaniment progress stop state), and this processing is finished.

(6) Accompaniment Progression Process Next, the accompaniment progression process will be described with reference to FIG. The accompaniment progression process is executed at predetermined intervals by timer interruption. At the execution timing, the CPU 4 advances the process to step SF1 shown in FIG. 9, and determines whether the accompaniment pattern number patn is not “−1”, that is, whether the accompaniment is in progress. If the accompaniment pattern number patn is set to “−1” and the sound is not sounded (accompaniment progression stop state), the determination result is “NO”, and this process is completed without performing anything. On the other hand, if the accompaniment is in progress, the determination result in step SF1 is “YES”, and the process proceeds to step SF2.

In step SF2, each value is substituted into the following equation (1) to convert the current time into an elapsed time tick from the accompaniment start time stored in the register basetime.
tick = (current time−basetime) × apat [patn]. res × tempo / (60 × 1000) (1)
In the above equation (1), apat [patn]. Res represents the resolution res in the accompaniment pattern apat [patn] selected according to the value of the control parameter param3 in step SE2 (see FIG. 8) of the accompaniment sounding process described above, and tempo represents the playback tempo (1 minute). Middle quarter note number).

In step SF2, each value is substituted into the following equation (2) to convert the elapsed time tick into a position tickmeas in the accompaniment pattern apat [patn].
tickmeas = (tick / (apat [patn] .res × apat [patn] .beat × apat [patn] .len)) remainder (2) In the above equation (2), apat [patn]. res represents the resolution res in the selected accompaniment pattern apat [patn]. apat [patn]. beat represents a beat beat in the selected accompaniment pattern apat [patn]. apat [patn]. len represents the pattern length len in the selected accompaniment pattern apat [patn].

  Next, in step SF3, it is determined whether or not the current position tickmeas converted from the elapsed time tick is smaller than the previous position tickmeasprev calculated last time. If the current position tickmeas does not exceed the previous position tickmeasprev, the determination result is “YES”, and the flow proceeds to step SF4. In step SF4, the index notei specifying the note data notes [0] to [maxnote-1] constituting the selected accompaniment pattern apat [patn] is reset to zero and the process proceeds to step SF5.

  On the other hand, if the current position tickmeas exceeds the previous position tickmeasprev, the determination result in step SF3 is “NO”, and the process proceeds to step SF5. In step SF5, it is determined whether or not the index notei is smaller than the number of notes maxnote included in the selected accompaniment pattern apat [patn], that is, whether or not all the note data in the pattern has been reproduced. If all the note data in the pattern has been reproduced, the determination result is “NO”, the process proceeds to step SF9, the current position tickmeas is updated to the previous position tickmeasprev, and this process is terminated.

On the other hand, if all the note data in the pattern has not been reproduced, the determination result in step SF5 is “YES”, and the flow advances to step SF6.
In step SF6, the note data apat [patn]. Note [notei] is stored in the register note. As a result, a note-on event “com” indicating the sound generation start time “time” of the note data “note [notei]” and the pitch and volume to be sounded is stored in the register note.

  In step SF7, it is determined whether or not the current position tickmeas is equal to or longer than the sound generation time time in the register note, that is, the sound generation timing. If the current position tickmeas is smaller than the sound generation time time, it is determined that the sound generation timing has not been reached, the determination result is “NO”, the process proceeds to step SF9, the current position tickmeas is updated to the previous position tickmeasprev, and this processing is performed. Finish.

  On the other hand, if the current position tickmeas is under the sounding timing equal to or longer than the sounding time time in the register note, the determination result in step SF7 is “YES”, the process proceeds to step SF8, and the note-on stored in the register note After sending the event com to the musical tone generation unit 9 to instruct the sound generation of the accompaniment sound, the index notei is incremented, and the process returns to step SF5 described above. Thereafter, by repeatedly executing the above steps SF5 to SF8, the sounds constituting the designated accompaniment pattern are sequentially reproduced.

As described above, in the present embodiment, the control parameters param 1 to 3 that change according to the hue of the image data captured by the imaging input unit including the camera 7 and the A / D converter 8 are generated. Then, the chord data pit to be sounded is selected according to the control parameter param1, and the selected chord data pit is sounded when the control parameter param1 exceeding a certain value is generated, and the sound volume according to the control parameter param2 is generated during the sounding of the chord. Control and sound image control according to the control parameter param3 are performed, and when a control parameter param2 of a certain value or more is generated, the chord being sounded is muted.
As described above, since the chord performance can be performed according to the control parameters param 1 to 3 that change in accordance with the hue of the captured image data, it is possible to create new enjoyment that makes it easy to get used to music.

(7) Operation of Modified Example (Arpeggiator Processing) Next, a modified example will be described with reference to FIG. FIG. 10 is a flowchart showing an arpeggiator process performed in place of the above-described chord sound generation process (see FIG. 7). The arpeggiator process is executed at predetermined intervals by timer interruption. At the execution timing, the CPU 4 advances the process to step SG1 shown in FIG.
In step SG1 and subsequent steps, state determination is performed as to whether or not a chord sound is instructed (steps SG1 and SG2) and whether mute is instructed during sound generation (step SG6). Therefore, in the following, the description of the operation will be divided into a case where pronunciation of a chord is instructed and a case where mute is instructed during pronunciation.

<When chord pronunciation is instructed>
In this case, since the flag hmny is “1”, the determination result in step SG1 is “YES”, and the process proceeds to step SG2. In step SG2, it is determined whether or not the value of the register pitn (chord number) is “−1”, that is, the sound is not generated.
If chord pronunciation is instructed in the unsounded state, the determination result in step SG2 is “YES”, and the process proceeds to step SG5 described later in order to perform the process of starting the first sound.

  On the other hand, if it is already sounding, the determination result in step SG2 is “NO”, and the process proceeds to step SG3. In step SG3, the counter wait for counting the waiting time is subtracted by the elapsed time, and in the subsequent step SG4, whether or not the waiting time of the counter wait subtracted by the elapsed time is greater than “0”, that is, the next one sound generation. Determine if it is timing. When the waiting time of the counter wait after subtracting the elapsed time is larger than “0” and the sound generation timing has not been reached, the determination result of the above step SG4 becomes “YES”, and this processing is completed without doing anything.

  On the other hand, when the waiting time of the counter wait after subtracting the elapsed time becomes smaller than “0” and the sound generation timing is reached, the determination result in step SG4 is “NO”, and the process proceeds to step SG4-1. In step SG4-1, the chord data pit [pitn] [piti] (pitch of the constituent sound) specified by the index value stored in the register piti is stored in the register p in the chord data pit [pitn] being generated. At the same time, “1” is stored in the register ch, and the musical tone generation unit 9 is instructed to mute the musical tone of the pitch stored in the register p from the sound generation channel specified by the register ch. As a result, the chord constituent sound being generated is muted. Thereafter, the process proceeds to step SG5 in order to perform processing for starting the next sound generation.

In step SG5, the following processing is performed. First, pan control is performed according to the random numbers generated in the range of 0 to 127, and then the range of 0 to (WN−1), that is, the number of constituent sounds WN of the currently selected chord data pit [pitn]. The random number generated in the range is stored in the register piti as an index value. Next, a remainder value obtained by dividing the control parameter param1 by the chord data number MAXPIT is stored in the register pitn as a chord number.
Next, in the chord data pit [pitn] specified by the new chord number stored in the register pitn, the pitch pit [pitn] [piti] of the constituent sound specified by the index value stored in the register piti is obtained. The data is read from the data area of the ROM 5 (see FIG. 2) and stored in the register p, “1” is stored in the register ch, and the volume value “110” is set in the register v.
Then, after instructing the musical sound generation unit 9 to generate the musical tone of the pitch stored in the register p from the sound generation channel specified by the register ch with the volume value of the register v, a waiting time corresponding to the control parameter param2 is set. The counter wait is set and the process is finished.

<When mute is instructed during pronunciation>
In this case, since the flag hmny is “2” and the value of the register pitn (chord number) is not “−1”, the determination result in step SG6 is “YES”, and the process proceeds to step SG7, where sound is being generated. In the chord data pit [pitn], the chord data pit [pitn] [piti] (pitch of the constituent sound) specified by the index value stored in the register piti is stored in the register p and “1” is stored in the register ch. , And the musical tone generator 9 is instructed to mute the musical tone of the pitch stored in the register p from the sound generation channel specified by the register ch, and then “−1” is stored in the register pitn. This process is finished. As a result, the chord constituent sound being sounded is muted and the sounding state is set to the unsounding state.

  As described above, in the modified example, since the arpeggio can be played according to the control parameters param 1 and 2 that change according to the hue of the captured image data, it is possible to create new enjoyment that makes it easy to get used to music. It becomes possible.

  In the above-described embodiment, an example in which a chord performance is performed according to the control parameters param1 to 3 that change according to the hue of the captured image data has been described. Applying a well-known face image recognition process to recognize the face, change the sound generation form of the generated sound according to the arrangement relationship such as "brow", "eyes", "nose" and "mouth" of the recognized face, Therefore, it is possible to make it easy to become familiar with music and to create new enjoyment.

It is a block diagram which shows the whole structure of embodiment by this invention. It is a figure which shows the structure of the chord data memorize | stored in ROM5, and the structure of accompaniment pattern data. 3 is a diagram showing a configuration of main register flag data provided in a work area of a RAM 6 and a configuration of image data stored in an image data area of the RAM 6. FIG. It is a flowchart which shows operation | movement of a main routine. It is a flowchart which shows the operation | movement of a variable production | generation process. It is a flowchart which shows the operation | movement of a switch process. It is a flowchart which shows the operation | movement of a chord sound generation process. It is a flowchart which shows the operation | movement of an accompaniment sound process. It is a flowchart which shows the operation | movement of an accompaniment progress process. It is a flowchart which shows the operation | movement of the arpeggiator process by a modification.

Explanation of symbols

1 Keyboard 2 Switch part 3 Display part 4 CPU
5 ROM
6 RAM
7 Camera 8 A / D converter 9 Musical sound generator 10 D / A converter 11 Sound system

Claims (4)

Control parameter generating means for generating a control parameter that changes according to the hue of the captured image;
A musical sound generating means for generating or muting a musical sound of a type selected in accordance with a control parameter generated by the control parameter generating means;
A tone control apparatus comprising: a tone generation mode control unit that controls a tone generation mode of a tone generated by the tone generation unit according to a control parameter generated by the control parameter generation unit. Control parameter generating means for generating at least one control parameter that changes according to the hue of the captured image;
According to the value of at least one control parameter generated by the control parameter generating means, at least one of the chords and the distributed chords selected corresponding to the value of the control parameter generated by the control parameter generating means is selected. A musical sound generating means to pronounce or mute,
A pitch tone image control means for controlling a pitch and a tone image of the tone generated by the tone generation means according to at least one control parameter generated by the control parameter generation means; Control device. Control parameter generation processing for generating a control parameter that changes according to the hue of the captured image;
A musical tone generation process for generating or muting a musical tone of a type selected in accordance with the control parameter generated by the control parameter generating process, according to a change in the control parameter;
A musical tone control program for causing a computer to execute a sound generation pattern control process for controlling a sound generation pattern generated by the music sound generation process in accordance with a control parameter generated by the control parameter generation process. A control parameter generation process for generating at least one control parameter that changes according to the hue of the captured image;
According to the value of at least one control parameter generated in the control parameter generation process, at least one of the chords and the distributed chords selected corresponding to the control parameter value generated by the control parameter generation process Musical sound generation processing to pronounce or mute
A pitch sound image control process for controlling a pitch and a sound image of a musical sound generated by the musical sound generation process according to at least one control parameter generated in the control parameter generation process. Tone control program.

Images