JP4765705B2 - Music control device - Google Patents

Description

  The present invention relates to a musical tone control apparatus having a function of controlling musical tones according to the movement of a player's finger above a keyboard.

For example, Patent Document 1 discloses a technique for detecting movements of the performer's hand and fingers above the keyboard in an electronic keyboard instrument and controlling the characteristics of the musical tone according to the state of the hand before or after pressing the keyboard. And Patent Document 2 discloses. In the electronic keyboard instrument of Patent Document 1, a finger drop speed pattern is detected by an optical sensor composed of a light emitting device and a light receiving device, or a finger for each key is detected by an inductance sensor composed of an inductance coil and an oscillation circuit provided for each key. The approach speed is detected. Moreover, in the electronic keyboard instrument of patent document 2, the shape of the finger at the time of performance is detected by recognizing the captured image.
Japanese Utility Model Publication No. 3-29994 No. 7-53117

  However, with the technique according to Patent Document 1, the key position in the left-right (pitch) direction cannot be specified depending on the optical sensor, and the position in the left-right (pitch) direction cannot be specified according to the inductance sensor for each key. Although a sensor is provided for each key, the cost increases. Further, with the technique according to Patent Document 2, the position in the left-right (pitch) direction cannot be specified. As described above, the prior art cannot identify the position in the left-right direction or increases the cost to identify the position in the left-right direction, and is not practically suitable for musical sound control that specifies the position in the left-right direction. .

  In view of such circumstances, an object of the present invention is to provide a musical sound control device that can reflect the movement of a finger above a keyboard on a musical sound for each pitch with a simple configuration.

  According to the main feature of the present invention, a plurality of imaging devices (18C; A1, B1) for imaging the space above the keyboard (14K) and images captured by these imaging devices (18C) are used to detect the upper part of the keyboard (p0). To p2), the position recognition means (10; A2, B2) for recognizing the position of the performer's finger in the key release direction and the key arrangement direction of the keyboard, and the press release recognized by the position recognition means (10). Tone control means (1-3; A3 to A10, B3 to B12) for controlling a tone (second tone) based on the position in the key direction and the position in the key arrangement direction, and recognized by the position recognition means (10) Provided is a musical tone control device comprising (A3 to A6, B3 to B7) for controlling the start or mute of the musical tone (second musical tone) according to the position in the pressed and released key direction. Is done. Note that the parentheses indicate the corresponding reference symbols, terms, and the like of the embodiments added for convenience of understanding.

  In the musical tone control apparatus according to the present invention, the musical tone control means (1-3) controls the pitch of the musical tone (second musical tone) to be generated according to the position in the key arrangement direction recognized by the position recognition means (10). (A4, B7) [Claim 2] Further, the musical tone control means (1-3) performs the operation (m3, m5) of the keyboard (14K) according to the position in the key release direction or the position in the key arrangement direction recognized by the position recognition means (10). The characteristic of the musical tone (first musical tone) generated based on the control can be controlled (FIG. 8).

  In the musical tone control apparatus according to the main feature of the present invention (Claim 1), from the images picked up by the plurality of image pickup devices (18C), the finger pressing / releasing key direction position (up and down) above the keyboard (14K) (p0 to p2). (Position) and key arrangement direction position (left / right position / pitch position) are recognized (A2, B2), and the tone (second tone) is controlled according to the recognized key release position and key arrangement direction ( A3 to A6, B3 to B7), and in particular, the start or mute of the tone (second tone) is controlled according to the key pressing position (vertical position) above the keyboard (p0 to p2). (A3 to A6, B3 to B7).

  Therefore, according to the present invention, it is possible to detect the movement of the finger above the keyboard with a simple configuration without providing a sensor for each key, and to control the musical sound for each key arrangement direction position, that is, pitch. The movement of the finger above the keyboard can be reflected in the musical tone for each pitch, and in particular, the start of sound generation or mute can be controlled according to the vertical position of the finger above the keyboard.

  The musical tone control apparatus according to the present invention is configured to control the pitch of the musical tone (second musical tone) to be generated according to the key arrangement direction position of the finger above the keyboard (p0 to p2) (A4, B7). Therefore, the movement of the finger in the key arrangement direction (left and right) above the keyboard can be used for the tone pitch control.

Furthermore, in the musical tone control apparatus according to the present invention, the finger (m1, m2, m4) in the upper part (p0 to p2) of the key (m1, m2, m4) is pressed or released in the key arrangement direction (up / down position or left / right / pitch position). Since it is configured to control the characteristics of the musical tone (first musical tone) generated based on the keyboard operation (m3, m5) (FIG. 8), the keyboard is moved according to the up / down or left / right movement of the finger above the keyboard. The characteristics of the musical sound generated by the operation can be controlled (claim 3).

[System Overview]
An electronic music apparatus incorporating the function of a musical tone control device according to the present invention is a kind of computer having a music information processing function including musical tone control processing, and is dedicated to music such as an electronic musical instrument having a performance operation input device (keyboard device). A general-purpose information processing apparatus such as a personal computer having a predetermined music information processing function, to which a device or performance operation input device is externally attached, is used.

  FIG. 1 is a block diagram showing the hardware configuration of an electronic music apparatus according to an embodiment of the present invention. This electronic music apparatus includes a central processing unit (CPU) 1, a random access memory (RAM) 2, a read only memory (ROM) 3, an external storage device 4, a performance operation detection circuit 5, a setting operation detection circuit 6, and a display circuit 7. , Sound source circuit 8, effect circuit 9, finger position analysis device 10, MIDI interface (I / F) 11, communication interface (I / F) 12, etc. These elements 1 to 12 are connected to each other via a bus 13. Is done.

  The CPU 1 constitutes a data processing unit together with the RAM 2 and the ROM 3 and executes various music information processing including a musical tone control process using a clock by a timer according to a predetermined control program including a musical tone control program. The RAM 2 is used as a work area for temporarily storing various data necessary for these processes. The ROM 3 stores various control programs including a musical tone control program, preset musical tone control data, performance data, and the like in advance in order to execute these processes.

  The external storage device 4 is not only a built-in storage medium such as a hard disk (HD) but also a compact disk read only memory (CD-ROM), a flexible disk (FD), a magneto-optical (MO) disk, a digital multipurpose disk ( Various control programs and data can be stored in an arbitrary external storage device 4 including various portable external recording media such as a small memory card such as a DVD and a smart media (registered trademark).

  The performance operation detection circuit 5 detects the operation content of a performance operator (performance operation input device) 14 such as a keyboard (14K: see FIG. 2), and the setting operation detection circuit 6 sets setting operators (such as switches and mice) ( The operation content of the panel operator 15 is detected, and information corresponding to the detected content is introduced into the data processing unit. The display circuit 7 includes a display 16 such as an LCD for displaying various screens necessary for performance and setting, and other indicators / lamps. The display circuit 7 controls the display / lighting contents according to instructions from the data processing unit, and controls the operator 14. , 15 is provided for display operation.

  The tone generator circuit 8 generates a musical tone signal corresponding to the operation key based on a performance operation on the keyboard, and the effect circuit 9 has an effect imparting DSP and imparts an effect to the musical tone signal from the tone generator circuit 8. The circuit can also include software. Both circuits 8 and 9 constitute a musical tone signal generator and can control musical tone signals based on various musical tone control parameters. The sound system 17 placed after the effect circuit 9 includes a D / A converter, an amplifier, and a speaker, and generates a musical sound based on a musical sound signal from the effect circuit 9.

  The musical tone signal generators 8 and 9 generate automatic musical tone signals based on the performance data stored in the storage means 3 and 4, or in accordance with finger position information from the finger position analysis device 10. It is possible to generate a predetermined musical sound signal based on the musical sound characteristic parameters.

  The finger position analysis device 10 and a finger detection device 18 connected to the finger position analysis device 10 constitute a finger position detection unit that detects the position of each finger of the performer above the keyboard. The finger detection device 18 sends finger detection data representing the player's movement in the space above the keyboard to the finger position analysis device 10, and the finger position analysis device 10 analyzes the finger detection data and performs each finger of the player above the keyboard. Finger position information indicating the position in the left and right direction (pitch direction) (key / pitch position) and the position in the up and down direction (pressing and releasing key direction) can be introduced into the system body.

  The finger detection device 18 can be configured by an imaging device including a plurality of imaging cameras (FIG. 2: 18C), and the finger position analysis device 10 can be configured by an image analysis (image recognition) device. In this case, the upper region of the keyboard is imaged by these imaging cameras, and the image analysis device 10 analyzes the image data of the captured finger using an image processing technique, and indicates the position of the finger that is playing above the keyboard. Location information can be obtained.

  Further, another MIDI music device 30 is connected to the MIDII / F11, and MIDI performance data can be exchanged with the other music device 30 and used in this electronic music apparatus. Further, a communication network 40 such as the Internet or a local area network (LAN) is connected to the communication I / F 12 to receive a control program, various data, and the like from an external server computer 50 and store them in the external storage device 4. Can do.

[Finger detection mechanism and its installation example]
In the electronic music apparatus according to an embodiment of the present invention, a plurality of imaging cameras are used as the detection mechanism of the finger position detection unit, and the movements of the performer's fingers above the keyboard are captured by the plurality of imaging cameras, and obtained from the cameras. Based on the captured image data, it is possible to detect the pitch at which the finger is located and the movement of the finger, and to control the musical tone according to the detected pitch and movement. FIG. 2 shows an installation example of an imaging camera used as a finger detection mechanism in the electronic music apparatus according to one embodiment of the present invention.

  According to the characteristics relating to the use of image data of the electronic music apparatus, as shown in FIG. 2, the performer's fingers above the keyboard 14K are imaged by a plurality of image capturing means (cameras) 18C, and the captured image data is imaged. By processing, the position of the finger in the key release direction (vertical direction) and the key arrangement direction (horizontal direction / pitch direction) above the keyboard 14K is recognized, and the finger position information representing the recognized finger position is obtained. Control music based on. For example, according to the key arrangement direction position (pitch position) of the finger position information, the pitch of the tone to be generated (second tone) is designated, and the movement of the finger in the key release direction obtained from the finger position information Depending on the sound, a tone having a specified pitch is sounded or muted. In addition, it is possible to control the characteristics of another musical tone (first musical tone) generated by the key release operation on the keyboard 14K.

  The example of the imaging camera in FIG. 2 will be described. Two imaging cameras 18C provided on the left and right sides of the front panel of the electronic music apparatus housing, that is, near both ends of the keyboard 14, are used as a pair for the finger detection mechanism. Both imaging cameras 18C are positioned slightly above the keyboard surface, and the optical axis of each camera is horizontal (parallel to the keyboard surface) and is directed toward the center of the keyboard 14K, and each of the two cameras captures the entire keyboard. It is installed so that you can.

  Accordingly, an image analysis device is used as the finger position analysis device 10, and image processing is performed on two image data respectively captured by the two imaging cameras 18C by applying a conventionally known image processing technique. The image of each finger is analyzed to detect the position of the finger that is playing in the left-right direction (pitch direction / key arrangement direction) and the up-down direction (press / release key direction / direction perpendicular to the keyboard surface). For example, regarding the position in the left-right direction, it is possible to recognize which key on the keyboard 14K corresponds to the pitch direction position by performing analysis of two image data captured simultaneously.

  In addition, in order to make it easy to detect the position of a finger, an infrared camera capable of clearly capturing a biological image is used for each imaging camera 18C, for example, a manicure is applied to a fingernail of a player, or a finger is applied. A specific color or the like may be marked on the performer's finger, such as wearing an attachment. The imaging camera 18C only needs to be able to image a predetermined area above the keyboard 14K over the entire key area, and the position and number of cameras are not limited to the above-described example. For example, many cameras may be installed at intervals of one octave or the like, and in this way, finger position detection accuracy can be improved. Further, the optical axis of the camera is not limited to the horizontal, but may be slightly inclined from the horizontal axis.

[First Embodiment of Musical Sound Control]
In the electronic music apparatus according to one embodiment of the present invention, the movement of the player's finger above the keyboard is detected, and a musical tone having a timbre characteristic different from the musical tone based on the keyboard performance is generated according to the detected movement of the finger. Can do. 3 and 4 are diagrams for explaining a first embodiment of musical tone control in an electronic music apparatus according to an embodiment of the present invention. In the first embodiment, fingers obtained from finger position information above the keyboard are shown. Based on the up / down movement, a musical tone having other characteristics is generated prior to key-on of a musical tone having a predetermined characteristic by a key operation.

  According to the feature related to the generation of one operation and two musical tones by the electronic music apparatus, the finger position detecting means 18 and 10 detect the position and movement of the finger pressing / releasing key direction (vertical direction) above the key 14K (p0 to p2). Then, the sound generation and mute are controlled at a predetermined timing based on the detected position and movement. That is, as shown in FIG. 3 and FIG. 4, the tone (second tone) having the first tone color (for example, a string tone) at the timing t1 that can be regarded as (a) when the finger substantially touches the key 14K. When the tone generators 8 and 9 are instructed to start sound generation and there is a key release operation [(b), (c)], a second tone (for example, a piano tone) having a tone characteristic different from that of the first tone is selected. Musical tone (first musical tone) is generated (ta, tb), and at the timing t2 when it is confirmed that the finger has completely moved away from the key 14K (d), the first musical tone (second musical tone) Mute. Thereby, it is possible to realize a layer timbre that allows a plurality of timbres to be simultaneously generated by one performance operation with a simple configuration.

  In this electronic music apparatus, a tone generation intention area As is set above the keyboard surface of the keyboard 14K for tone control. The pronunciation intention area As is an area that is considered to have a pronunciation intention when a finger passes through this area. As shown in FIG. 3, the lower limit position p1 and the upper limit position p2 above the keyboard surface position p0 by a predetermined distance. The space between. For example, the distance between the positions p0 to p1 is selected to be about 1 cm, and the distance between the positions p1 to p2 is selected to be about 2 cm. In the case of musical tone control, simultaneous shooting by a plurality of imaging cameras 18C is advanced for a predetermined area (referred to as a keyboard upper area) above the keyboard surface position p0 of the keyboard 14K including the pronunciation intention area As. The position analysis device (image analysis device) 10 sequentially analyzes both image data of the keyboard upper area, which are simultaneously photographed, and recognizes the position of each finger of the performer in the keyboard upper area. The data processing units 1 to 3 are configured to generate a predetermined tone based on the finger position information indicating the position of each finger recognized by the finger position analysis device (image analysis device) 10. 9 is instructed.

  In the first embodiment illustrated in FIGS. 3 and 4, as shown in FIG. 4 (1), a musical tone is generated based on a normal press / release key operation (ta, tb) with respect to the keyboard 14 K by each finger of the performer. The musical sounds generated from the sections 8 and 9 are called “first musical sounds”, and for example, a timbre having a characteristic in which the sound rises / falls quickly is given to the first musical sound like a piano-type timbre. Further, when each performer's finger passes through the pronunciation intention area As, as shown in FIG. 4 (2), a “second musical tone” having different characteristics from the first musical tone is separated from the musical tone generating units 8 and 9. For example, a timbre having a characteristic that the rise / fall of the sound is slow, such as a string-type timbre, is given to the second musical tone. The rising / falling fast / slow is based on the relative relationship between the first musical sound and the second musical sound. Further, it is sufficient that at least the rising speed is fast / slow, and the falling speed may be reversed.

  Specifically, as shown in FIG. 3A, the data processing units 1 to 3 set the lower limit position p1 of the pronunciation intention area As when the player's finger passes the pronunciation intention area As from the top to the bottom. When the keyboard is over or when it reaches (touches) the keyboard (for example, the keyboard surface position p0 for the white key), the musical tone is generated so that the second musical tone is generated from this time t1, as shown in FIG. 4 (2). Sections 8 and 9 are instructed. That is, when the finger passes through the pronunciation intention area As from the top to the bottom, it is considered that there is a pronunciation intention, and at the time t1 when the finger passes the lower end p1 of the pronunciation intention area As or contacts the keyboard surface p0, Generation of a second musical tone different from the first musical tone is started. Here, the pitch of the second musical tone is the pitch of the keyboard 14K corresponding to the horizontal position of the finger passing through the region As from top to bottom.

  As shown in FIG. 4, when the first musical sound starts to be generated at the key pressing timing ta as shown in FIG. 4, the second time point sufficient to cause the second musical sound to rise at the rise of the first musical sound. This corresponds to the tone generation start timing of the musical sound, corresponding to an appropriate finger position from when the finger passes the region As until it touches the keyboard 14K. In this example, as described above, the finger passes through the lower limit position p1 and the key. The time point at which the key 14K is sufficiently approached or touched or the point at which the key 14K is touched is employed. In addition, according to the movement of the finger within the pronunciation intention area As, for example, the descent speed and the up / down / left / right fluctuations, the tone characteristic parameters such as the attack, volume, tone color, and pitch when the second tone is pronounced are controlled. Can do.

  Now, after the passage of the finger in the pronunciation intention area As (up → down) is detected, as shown in FIG. 3B, when the key operation (key-on) of the keyboard 14K is detected by the performance operation detection circuit 5, the data The processing units 1 to 3 instruct the musical tone generation units 8 and 9 to generate the first musical tone at the key pressing detection time ta. Further, as shown in FIG. 3C, when there is a key release of the keyboard 14K pressed at the time tb, the first musical sound that has been generated is muted. In this case, the other musical sound (second musical sound) generated prior to key-on by the keyboard continues to sound as shown in FIG. Then, as shown in FIG. 3D, when the finger moves away from the keyboard 14K and passes through the region As from the bottom to the top, it is considered that there is an intention to mute, and the second time t2 when reaching the upper end p2 of the pronunciation intention region As. Mutes the tone.

  Thus, prior to key-on of the first musical tone by the keyboard, when another second musical tone is generated by passing through the pronunciation intention area As, for example, the first musical tone is changed to a “piano” tone with a fast attack, and the second musical tone. Is a string with a slow attack, and a musical tone having two tones such as a “piano + strings” tone can be generated by one performance operation. In this case, it is possible to substantially match the attack position of the first musical tone and the attack position of the second musical tone by generating the first musical tone after generating the second musical tone having a slow attack first.

  On the other hand, as shown in FIG. 3 (a ′) → FIG. 3 (d), when the finger passes upward in the pronunciation intention area As without any key depression of the keyboard 14K, there is no sound of the first musical tone, Only two musical tones are sounded from the downward passage time t1 of the sound generation intention area As and are muted at the upward passage time t2. That is, the performer can intentionally generate only the second musical tone as shown in FIG.

  In this case, the second tone of the second tone can be started to sound from the timing t1 when the key is touched, and the sound generation timing of the second tone having the second tone having a slow attack, such as a string-type tone, is determined by the player's operation feeling. Thus, the second musical sound generated at such a sound generation timing t1 can be muted without a sense of incongruity at the mute timing t2.

  After the second musical tone is generated, after the finger leaves the keyboard or after passing through the lower end p1 of the pronunciation intention area As, for example, the finger's ascent speed and the up / down / left / right direction of the finger The tone characteristic parameters (volume, tone color, pitch, etc.) of the second tone during sound generation or release (after a mute instruction) can be controlled in accordance with shaking.

  In addition, after the first musical sound is muted by releasing the key, if the key is pressed again without passing through the pronunciation intention area As, the first musical sound is generated again while the second musical sound is continuously generated. In this case, instead of continuing the pronunciation of the second musical sound, the second musical sound may be temporarily muted, and the second musical sound may be re-sound simultaneously with the first musical sound. Further, when a key is pressed without first passing through the pronunciation intention area As, the first musical tone and the second musical tone may be started simultaneously, or only the first musical tone may be started. It may be. As in these examples, when the second musical sound is generated by pressing the key, it is preferable to mute the second musical sound by releasing the key.

[Musical sound control processing flow example (1)]
FIG. 6 is a flowchart showing an example of the procedure of the musical tone control process of the first embodiment in the electronic music apparatus according to one embodiment of the present invention. In this electronic music apparatus, the musical tone control process is started by the operation of the musical tone control process start switch in the setting operator 15, the imaging of the upper area of the keyboard by the imaging camera 18C is started, and the processing flow of FIG. 6 is executed. The musical tone control process is terminated by an end instruction based on the operation of the musical tone control process stop switch.

  First, in the first step A1, the CPU 1 instructs the image analysis device 10 to take in the image data in the upper area of the keyboard from the imaging device 18. In the next step A2, the image analysis apparatus 10 recognizes the vertical position and left / right (pitch) position of each finger of the performer through image processing on the image data from the imaging apparatus 18, and the CPU 1 recognizes each recognized finger. The vertical position and the horizontal position information representing the vertical position and the horizontal (pitch) position are acquired from the image analysis apparatus 10.

  In the next step A3, it is determined whether or not the finger has passed through the pronunciation intention area As in the direction from top to bottom (up to down) based on the vertical position information, and a certain finger has passed from up to down. Is determined (A3 = YES), the process proceeds to step A4. In step A4, a pitch corresponding to the left / right (pitch) position recognized for the finger is obtained based on the vertical position and left / right position information of the finger, and whether the finger passes the lower end p1 of the pronunciation intention area As. Alternatively, the musical tone generators 8 and 9 are instructed to start sounding the second musical tone having the pitch at the timing (t1) when reaching the contact position with the keyboard 14K.

  When it is determined that the finger does not pass over the pronunciation intention area As (A3 = NO) or after instructing the start of the second musical tone in step A4, in step A5, based on the vertical position information, It is determined whether or not the finger has passed through the pronunciation intention area As in the direction from the bottom to the top (from bottom to top). Here, when the finger passes from the lower side to the upper side (A5 = YES), the process proceeds to step A6, and the finger reaches the upper end p2 of the pronunciation intention area As based on the vertical position and left / right position information of the finger. At the timing (t2), the musical tone generators 8 and 9 are instructed to mute the second musical tone having the pitch recognized for the finger.

  When it is determined that the finger does not pass through the pronunciation intention area As down (up) (A5 = NO) or after the instruction to mute the second musical tone in step A6, is there a key pressing operation on the keyboard 14K in step A7? Determine whether or not. Here, when a key pressing operation is performed (A7 = YES), the process proceeds to step A8, and at the timing (ta) when the key pressing operation is performed, the first musical tone corresponding to the operation key is started to be generated. The musical sound generation units 8 and 9 are instructed.

  When it is determined that there is no key pressing operation (A7 = NO) or after the start of sound generation of the first musical tone is instructed in step A8, it is determined in step A9 whether or not a key release operation has been performed on the keyboard 14K. If there is a key release operation (A9 = YES), the process proceeds to step A10, and at the timing (tb) at which the key release operation is performed, the musical sound is muted so that the first musical sound corresponding to the operation key is muted. The generation units 8 and 9 are instructed. Then, when it is determined that there is no key release operation (A9 = NO) or after the instruction to mute the first musical tone is given in Step A10, the process returns to Step A1, and the operations in Steps A1 to A10 are repeated until an end instruction is given. .

[Second Embodiment]
In the electronic music apparatus according to one embodiment of the present invention, the movement of the performer's finger is detected above the keyboard, and a musical sound expressing a playing method different from the musical sound based on the keyboard performance is generated according to the detected movement of the finger. be able to. FIG. 5 is a diagram for explaining a second embodiment of the musical tone control in the electronic music apparatus according to one embodiment of the present invention. In the second embodiment, the vertical movement of the finger obtained from the finger position information above the keyboard. On the basis of the above, another musical tone having a different playing style from the musical tone of the keyboard performance is generated by an operation of releasing the finger after touching the key.

  According to the characteristics related to the musical tone generation corresponding to the playing method by the electronic music apparatus, as shown in FIG. 5, the position of the performer's finger is detected above p0 to p2 above the key 14K, and based on the detected position and movement. Then, another musical tone (second musical tone) that expresses a different playing method from the musical tone (first musical tone) produced by the keyboard performance is generated. In other words, if the performer's finger passes through the predetermined spaces p2 to p1 above the key from the top to the bottom, it is considered that there is an intention to generate a musical tone, and further, when the finger leaves the key 14K after touching the key 14K ( t3) A second musical tone that expresses a light touch special performance like a harmonic tone of a guitar tone is generated (1b, 2b). If the performer's finger is pressed before the key 14K is released (ta), the first musical tone, which is different from the second musical tone like the normal tone of the guitar tone and expresses the standard playing style, is generated. After that, even if the finger is removed from the key, the second musical tone is not generated (1a, 2a).

  In the second embodiment illustrated in FIG. 5, as shown in (2a), from the tone generation units 8 and 9 based on the normal key release operation (ta, tb) on the keyboard 14K by each finger of the performer. The generated musical tone is called a “first musical tone”, and expresses a standard playing method, for example, a normal tone of a guitar tone. Further, when there is no key pressing operation and each finger of the performer just touches the keyboard 14K, as shown in (2b), a “second musical tone” having a performance different from the first musical tone is generated as the musical tone generating unit 8, 9 is generated separately. For example, the second musical tone represents a light touch playing method, such as a harmonic tone of a guitar tone.

  More specifically, (1a) as shown in the upper part, when the key 14K is depressed (key-on), and this is detected by the performance operation detection circuit 5, the data processing units 1 to 3 are connected to the player's finger. Regardless of whether or not the sound has passed through the pronunciation intention area As, as shown in (2a), the musical tone generators 8 and 9 are configured to generate the first musical tone at the detection time ta of the key depression. Instruct. Thereafter, (1a) as shown in the lower part, when the pressed key 14K is released (key-off), the sounding first musical sound is muted at the time tb of the key release.

  On the other hand, as shown in the upper part of (1b), when the performer's finger passes the pronunciation intention area As from top to bottom, the data processing units 1 to 3 are based on the finger position information from the finger position detection units 18 and 10. Thus, it is determined that the region As has passed from the upper side to the lower side, and it is considered that the performer has the intention to pronounce the second musical tone. Further, after the finger touches the keyboard, as shown in (1b) lower part, when the player releases the key without pressing the key, as shown in (2b), when the finger is released from the key. At the time t3 when the upper end p2 of the pronunciation intention area As is reached, the musical sound generation units 8 and 9 are instructed to start the generation of the second musical sound different from the first musical sound due to the key depression of the keyboard. The generated second musical tone is automatically muted at a time t4 when a predetermined time (second musical tone pronunciation setting time) Td has elapsed since the sounding start time t3.

  For example, as described above, if the first musical sound is a normal guitar sound and the second musical sound is a harmonic sound of the guitar, the player presses the keyboard 14K to play the normal guitar sound. To do. When it is desired to play the harmonics sound of the guitar instead of the normal guitar sound, the operation is performed by passing the sound generation intention area As and once touching the keyboard, and then releasing the finger from the keyboard 14K.

  When playing a harmonic sound in a natural musical instrument guitar, pick the finger while gently touching the position of the string that becomes the knot of the string vibration, for example, a position that is 1/2 the length of the string. This operation is common. On the other hand, the operation (1b) in which the keyboard 14K is lightly touched and then released is similar to the harmonics playing method of a natural musical instrument guitar and matches the player's feeling.

  When playing the second musical sound, the finger movement after the finger has left the keyboard 14K, for example, the attack speed, volume, tone, pitch, etc., depending on the finger's ascent speed, finger's up / down, left / right shaking, etc. The musical tone characteristic parameter of the two musical tones may be controlled.

[Music control control flow example (2)]
FIG. 7 is a flowchart showing an example of the procedure of the musical tone control process of the second embodiment in the electronic music apparatus according to one embodiment of the present invention. In this electronic music apparatus, the musical tone control process is started by the operation of the musical tone control process start switch in the setting operator 15, the imaging of the upper area of the keyboard by the imaging camera 18C is started, and the processing flow of FIG. 7 is executed. The musical tone control process is terminated by an end instruction based on the operation of the musical tone control process stop switch.

  First, in the first step B1, the CPU 1 instructs the image analysis device 10 to take in the image data of the upper area of the keyboard from the imaging device 18. In the next step B2, the image analysis apparatus 10 recognizes the vertical position and left / right (pitch) position of each finger of the performer through image processing on the image data from the imaging apparatus 18, and the CPU 1 recognizes each recognized finger. The vertical position and the horizontal position information representing the vertical position and the horizontal (pitch) position are acquired from the image analysis apparatus 10.

  In the next step B3, it is determined whether or not the finger has passed through the pronunciation intention area As from the top to the bottom (upward to down) based on the vertical position information, and a certain finger has passed from top to bottom. Is determined (B3 = YES), the process proceeds to step B4. In step B4, it is considered that the performer intends to generate the second musical tone, and the second musical tone pronunciation intention flag = “1” is set on the RAM 2 for the finger, and “the second musical tone pronunciation intention is intended” is set. .

  When it is determined that the finger does not pass over the pronunciation intention area As (→ B3 = NO) or after “second musical tone intention” is set in step B4, in step B5, based on the vertical position information Then, it is determined whether or not the performer's finger is separated from the keyboard 14K. Here, when the finger is separated from the keyboard 14K (B5 = YES), the process proceeds to step B6 to check the second musical tone pronunciation intention flag of the finger, and whether or not “second musical tone intention” is set. Determine.

  Here, when “with the second musical tone pronunciation intention” is set (B6 = YES), the process proceeds to step B7 and reaches the time point when the finger leaves the keyboard 14K or the upper end p2 of the pronunciation intention area As. At time (t3), the musical tone generation units 8 and 9 are instructed to start the second musical tone having a pitch corresponding to the pitch position recognized for the finger. Note that the second musical sound that has started to be sounded is automatically silenced by the musical sound generation units 8 and 9 after a predetermined time Td (t4).

  When it is determined that the finger is not separated from the keyboard 14K (B5 = NO), when it is determined that “the second musical tone is intended to sound” is not set (B6 = NO), or at step B7, the second After instructing the start of musical tone generation, it is determined in step B8 whether or not there has been a key pressing operation on the keyboard 14K. If there is a key pressing operation (B8 = YES), the process proceeds to step B9, and the second musical tone pronunciation intention flag of the finger at the pitch position corresponding to the key 14K for which the key pressing operation has been performed = “0”. And “no intention to pronounce second music”, the process further proceeds to step B10, and at the timing (ta) of the key pressing operation, the tone generator is configured to start sounding the first music corresponding to the operation key 14K. 8 and 9 are instructed.

  When it is determined that there is no key pressing operation (B8 = NO) or after the start of sound generation of the first musical tone is instructed in step B10, it is determined in step B11 whether or not a key release operation has been performed on the keyboard 14K. Here, when there is a key release operation (B9 = YES), the process proceeds to step B10, and at the timing (tb) when the key release operation is performed, the musical sound is muted so as to mute the first musical sound corresponding to the operation key. The generation units 8 and 9 are instructed. When it is determined that there is no key release operation (B11 = NO) or after the instruction to mute the first musical tone is given in step B12, the process returns to step B1, and the operations of steps B1 to B12 are repeated until an end instruction is given. .

[Other Embodiments]
In an electronic music apparatus according to an embodiment of the present invention, as another embodiment, the position of a player's finger above the keyboard is recognized by image processing from image data in the upper area of the keyboard imaged by the camera, and the recognized press and release is performed. The key direction and pitch direction positions can be used to control the characteristics of the musical sound that is produced by pressing the key. FIG. 8 is a diagram for explaining another embodiment of the tone control in the electronic music apparatus according to one embodiment of the present invention. In another embodiment, the movement of the finger obtained from the finger position information above the keyboard is shown. Based on this, it controls the characteristics of the musical sound by keyboard operation.

  The example of FIG. 8 controls the initial pitch at the start of sound generation related to a musical sound by keyboard operation and the subsequent pitch change according to the movement of the finger above the keyboard. In FIG. 8 (1), the finger position information above the keyboard. Is applied to the control of the portamento performance, and in FIG. 8 (2), it is applied to the control of the pitch bend performance. Note that the performance motions m1 to m5 in each figure represent finger movements.

  In the example of FIG. 8 (1) in which the portamento performance is controlled with respect to the musical sound by the keyboard operation, the player's finger passes the pronunciation intention area As from the upper side to the lower side and touches the key K1 by the first performance (contact) motion m1. Then, the tone generation of the musical tone is started at the initial pitch corresponding to the pitch direction position of the finger. Next, when the finger moves from the left to the right in the pronunciation intention area As by the second performance (movement) motion m2, the pitch of the musical sound that has started to be generated is the pitch corresponding to the pitch direction position of the moving finger. The pitch of the pronunciation is changed according to the movement of the finger. Then, when the key K3 is depressed by the third performance (key depression) motion m3, the pitch is determined at the pitch corresponding to the operation key Kb with respect to the musical sound whose pitch has been sequentially changed.

  In the example of FIG. 8 (2) in which the pitch bend performance is controlled with respect to the musical tone generated by the keyboard operation, the first finger (vibration) motion m4 causes the performer's finger to move upward in the pitch direction slightly while moving upward. When the key K5 is depressed by the second performance (key depression) motion m5 after passing downward, the key depression m5 starts at a predetermined pitch and reaches the pitch of the key depression key K5 (key depression pitch). Starts the sound of a changing musical tone. In this case, the pitch change time can be changed according to the time when the finger passes the pronunciation intention area As. The start pitch may be, for example, a pitch corresponding to a pitch direction position when the finger passes the upper end p2 of the pronunciation intention area As, or a pitch change value (for example, a key pressing pitch) with respect to a predetermined key pressing pitch. It may be a semitone lower sound).

  As described above, the finger position information above the keyboard can be used not only for pitch control during sound generation, but also for tone control other than pitch control during sound generation, for example, control of sound characteristic parameters such as volume, tone, and attack. It can be used for various musical tone control after mute, that is, during release, for example, control of musical tone characteristic parameters such as pitch, volume, tone color, and release. For example, the volume and attack rate during sound generation may be controlled according to the vertical movement of the finger, or the timbre during sound generation may be controlled according to the left / right (pitch) direction movement. Also, the volume and release rate at the time of mute (release) may be controlled according to the movement of the finger in the vertical direction, and the pitch and tone at the time of mute may be controlled according to the movement in the left and right (pitch) direction. .

[Various Embodiments]
The preferred embodiment of the present invention has been described above with reference to the drawings. However, this is merely an example, and the present invention can be variously modified without departing from the spirit of the invention. For example, the finger position analysis device (10) may be configured with dedicated hardware, or the finger position may be analyzed by software processing by the CPU (1). Also, the start and end of the musical tone control process (FIGS. 6 and 7) may be based on the on / off of the power switch of the electronic music apparatus.

It is a block diagram showing the hardware constitutions of the musical tone control apparatus (electronic music apparatus) by one Example of this invention. It is a figure showing the example of installation of the imaging camera by one Example of this invention. It is a figure showing the performance action in 1st Embodiment of this invention. It is a figure showing an example of the sound generation timing in 1st Embodiment of this invention. It is a figure for demonstrating the musical tone control aspect by 2nd Embodiment of this invention. It is a flowchart showing the example of a musical tone control process by 1st Embodiment of this invention. It is a flowchart showing the example of a musical tone control process by 2nd Embodiment of this invention. It is a figure for demonstrating other embodiment of this invention.

Explanation of symbols

18 an imaging device having an imaging camera 18C,
p0 Upper surface position of keyboard 14K,
As a pronunciation intention area set between positions p1 and p2 above the keyboard 14K,
ta, tb key press and key release timing,
t1, t3 Second musical tone sounding timing,
t2, t4 The timing at which the second musical sound is muted,
Td Second musical tone pronunciation setting time,
m1-m5 Finger movement (performance motion).

Claims (3)

A plurality of imaging devices for imaging the space above the keyboard;
Position recognition means for recognizing the position in the key release direction and the position in the keyboard arrangement direction of the player's finger above the keyboard from the images captured by these imaging devices;
A musical tone control means for controlling a musical tone based on the position in the key release direction and the position in the key arrangement direction recognized by the position recognition means, and according to the position in the key release direction recognized by the position recognition means, What is claimed is: 1. A musical sound control device comprising: a device for controlling the start or mute of musical sound.   2. The musical tone control apparatus according to claim 1, wherein the musical tone control means controls a pitch of a musical tone to be generated according to a position in a key arrangement direction recognized by the position recognition means. The tone control means controls the characteristics of the tone generated based on the operation of the keyboard according to the position in the key release direction or the position in the key arrangement direction recognized by the position recognition means. The musical tone control apparatus according to claim 1 or 2 .

Images