JP2001195059A - Musical performance interface - Google Patents

Abstract

PROBLEM TO BE SOLVED: To control a musical sound variously according to the movement and state of a participant in musical performance and to enable anybody to easily enjoy playing a musical sound. SOLUTION: When the participant in musical performance moves motion sensors MS1 to MSn, this musical performance interface analyzes the movement of the operator by an information analysis part AN by using operation (gesture) information on the operation sensors. A musical performance parameter determination part PS determines musical performance parameters according to the analysis result and a musical sound reproduction part 1S generates musical sound performance data based upon the musical performance parameters. The information analysis part AN, while analyzing the movement of a motion sensor, analyzes the physical body state of the participant in musical performance by using state information (biological information and physiological information) on state sensors SS1 to SSn and generates musical performance parameters corresponding to these analysis results. A musical sound which is variously controlled according to the movement and physical body state of the participant is radiated through sound and speaker systems 3 and 4.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a musical performance interface, and more particularly, to a musical tone generator which is interposed between a musical performance participant and a musical sound generator such as an electronic musical instrument or a musical sound reproducing apparatus. The present invention relates to a performance interface capable of variously controlling a generator.

[0002]

2. Description of the Related Art In general, in an electronic musical instrument, a desired musical tone can be produced when four performance parameters of a timbre, a pitch, a volume and an effect are determined.
DAT, MIDI (Musical Instrument Digital Int
(Erface) A tone reproducing apparatus for reproducing acoustic information such as a source can reproduce a desired tone when the three performance parameters of tempo, volume and effect are determined. Therefore, if a performance interface is provided between the operator and a musical sound generator such as an electronic musical instrument or a musical sound reproducing device, and the above-described 4 or 3 performance parameters are determined using the performance interface by the operation of the operator, It is possible to output a desired musical sound according to the operation of the operator.

Conventionally, as this kind of interface,
There has already been proposed one that controls performance parameters of musical tones output from an electronic musical instrument or a musical sound reproducing device in accordance with the movement of an operator. However, since the number of operators is limited to one, and there is only one musical tone generator and its performance parameter, many people cannot participate or enjoy various musical tone outputs.

[0004]

SUMMARY OF THE INVENTION The present invention provides a music ensemble by providing various functions to a performance interface for controlling performance parameters of a musical sound generator such as an electronic musical instrument in accordance with the movement and physical condition of a performance participant. As a new musical tone controller for drama, music education, sports, events, concerts, theme parks and even music games, anyone from infants to the elderly can easily participate in musical performances and enjoy playing. The aim is to provide a new type of playing interface that can.

[0005]

According to one aspect of the present invention, there is provided a performance interface for generating performance control information for controlling a tone generated from a tone generator according to the physical condition of an operator. A motion detector that can be installed or carried by the operator to detect motion based on the motion of the operator and output a motion detection signal, and analyze the motion of the operator based on the motion detection signal received from the motion detector The present invention provides a performance interface comprising: a physical condition analyzing means for performing performance control information generating means for generating corresponding performance control information based on a plurality of motion analysis results obtained by the physical condition analyzing means. The motion detector in the performance interface is typically of a type that is gripped by an operator by hand or worn on the operator's hand or foot.

According to another feature of the present invention, there is provided a performance interface for generating performance control information for controlling a tone generated from a tone generator according to the physical condition of an operator. A plurality of motion detectors based on the plurality of motion detections based on the plurality of motion detection signals and the plurality of motion detection signals output from the motion detectors. A performance interface is provided, comprising: a physical condition analyzing means for analyzing the motion of the game; and performance control information generating means for generating performance control information corresponding to each of the plurality of motion analysis results.

According to another feature of the present invention, there is provided a performance interface for generating performance control information for controlling a tone generated from a tone generator according to the physical condition of the operator, and the performance interface is adapted to the operation of the operator. A motion detector that can be installed or possessed by an operator that outputs a motion detection signal corresponding to a motion based on the motion, and a state that can be installed or possessed by an operator that detects a physical condition of the operator and outputs a corresponding status detection signal A detector, a body condition analyzing means for analyzing the motion and the body condition of the operator based on the motion detection signal and the condition detection signal received from the motion detector and the condition detector, and a motion analysis result and a body condition analysis result. A performance interface including performance control information generating means for generating performance control information corresponding to each of the performance interfaces is provided.

In the performance interface according to the present invention, the state detector is configured to detect at least one of a pulse, a body temperature, a resistance between skins, an electroencephalogram, breathing and eye movement, and to output a corresponding state detection signal. Can be. The performance control information can be configured to control the volume, tempo, timing, tone, effect, or pitch of a musical tone. Further, the motion detector can be a one-dimensional sensor that detects a motion in a predetermined direction based on the operation of the operator and outputs a detection signal as a simplest configuration. A two-dimensional sensor or a three-dimensional sensor that detects two or three directions of movement and outputs corresponding two or three types of detection signals can be used.

According to one feature of the present invention, the performance interface comprises a motion detector that can be carried by the operator or can be installed on the operator (hands, feet, etc.), and a tone generator. And a main unit system for generating performance control information for controlling the generated musical sounds. Here, the performance control information is, for example, the volume, tempo, timing, timbre, effect, or pitch of a musical tone, and is controlled by a motion detection signal. The operation detection signal is, for example,
One or more types of motions based on the operation of the operator are detected and output by a motion (acceleration, velocity, position) detector such as a one to three-dimensional sensor. The main system is
Receive the motion detection signal from the motion detector, and analyze the motion of the operator from the motion detection signal by the body condition analysis means,
Performance control information generation means generates performance control information corresponding to a plurality of motion analysis results.

According to another feature of the present invention, a motion detector detects a plurality of types of motion based on the motion of an operator,
A plurality of corresponding operation detection signals are output. The main body system receives a plurality of motion detection signals from the motion detector, analyzes the operator's motion from the motion detection signal by the body condition analysis means, and responds to the plurality of motion analysis results by the performance control information generation means Is generated.

As described above, in the performance interface of the present invention, when the operator (performance participant) moves the motion detector variously, the operator (movement (gesture) information) of the motion detection signal of the motion detector is used. It is configured to analyze various movements of the music and generate performance control information according to the various analysis results. Can be controlled.

According to another feature of the present invention, the performance interface controls a motion detector and a state detector that can be carried by the operator or can be installed on the operator, and a tone generated by the tone generator. And a main body system for generating performance control information. The motion detector is a sensor capable of detecting, for example, acceleration, speed, position, gyro, impact, inclination, and angular velocity, detects a motion based on the motion of the operator, and outputs a corresponding motion detection signal. State detector, for example, pulse, body temperature, skin resistance,
Detects the physical condition of the operator, such as EEG, breathing, eye movement,
A corresponding state detection signal is output. The main body system receives the detection signals from the motion detector and the state detector, and analyzes the operation of the operator from the operation detection signal and the body state of the operator from the state detection signal by the body condition analysis means, Performance control information generating means generates performance control information corresponding to each analysis result.

As described above, in the performance interface of the present invention, when the operator moves the motion detector, the motion of the operator is analyzed from the contents of the motion detection signal of the motion detector, and at the same time, the state of the state detector It is configured to analyze the operator's physical condition from the content of the detection signal [state information (biological information, physiological information)] and to generate performance control information in accordance with the results of these analysis. Music performance can be variously controlled in consideration of the operator's physical condition.

[Various Modes] More specifically, in the performance interface of the present invention, the musical sound generation device is connected to the musical performance generator in accordance with physical information such as movement (gesture) information and state (biological, physiological) information of the performer. By generating the control information of the above, and having a function of controlling the performance parameters of the musical tone generator based on this control information, it is possible to output a musical tone produced according to the gestures and biological conditions of the performers, Anyone can easily participate in the production of musical sounds.

In order to obtain such physical information, for example, a motion sensor such as a one-dimensional or three-dimensional speed sensor or an acceleration sensor is used for motion (gesture) information, and a pulse or a pulse is used for state information. Using a biological information sensor that generates a measurement output such as interskin resistance, two or more performance parameters of the musical sound generator are controlled by the acquired physical information.

In one embodiment of the present invention, a system can be provided in which a plurality of performance participants simultaneously share and control a tone generator such as an electronic musical instrument and a tone generator.

More specifically, one or a plurality of performance participants install a motion sensor or a biological information sensor at a predetermined body part (for example, a hand or a foot), and wirelessly transmit data detected by the sensor to the musical sound generator. The data is transmitted wirelessly to the receiver, and the musical sound generator analyzes the detected data, and controls the performance parameters of the musical sound generator based on the analysis result.

In one embodiment of the present invention, a motion sensor such as a one-dimensional or three-dimensional sensor is used as physical information input means of a performance interface to control two or more performance parameters of a tone generator. Alternatively, biometric information is input as physical information to control some performance parameters. Further, the performance parameters can be controlled by simultaneously using the output of the motion sensor and the biological information.

In another embodiment of the present invention, a movement sensor such as a one-dimensional to three-dimensional sensor is used as the physical information input means of the performance interface to control the tempo of the output musical sound. In this case, the periodicity of the motion sensor output is used as a performance parameter. Further, it is possible to control the tempo of the musical tone by inputting the biological information, and to control the performance parameters by simultaneously using the output of the motion sensor and the biological information.

In another embodiment of the present invention, detection data from a plurality of performance participants by physical information detection sensors such as two (for example, both hands) or one (for example, one hand) motion sensors are described below. A simple average or a weighted average data value or a data value selected according to a predetermined rule is calculated for all or a plurality of arbitrarily selected detection data, and the performance parameters are controlled using the calculated data value.

The present invention can be applied to various musical performance environments as well as pure musical performances. Examples of the application environment of the present invention include the following: (1) Music performance control (conducting mode = pro mode, semi-auto mode). (2) Accompaniment sound and external sound control A plurality of persons or one person controls music performance by using various percussion instrument sounds, bells, and natural sounds or by using an external sound source. For example, the sound source of a predetermined performance track is a handbell,
Sounds such as Japanese musical instruments, gamelan, percussion (ensemble), etc., are inserted into the music (main melody playing track). (3) Performance by a plurality of persons (music ensemble) A plurality of persons each have a sensor, and the average value data obtained by simple or weighted averaging of the sensor output values from two or more persons, or the time data in a certain time range The music performance is controlled based on data selected according to a predetermined rule, such as the last or first data. (Usage example) Music performance in the field of music education = For example,
The teacher has a master sensor and controls the tempo and volume of the music. The student uses the handset sensor to insert various selection sounds into the music, such as handbell-like functions, Japanese traditional (wagaku) drums and kane (and sounds of natural wind and water at the same time). (The teacher and the student have a joyful lesson while participating in the musical performance.) (4) Accompaniment of tap dance. (5) Network music performance between remote locations (using video)
(Music game) People from different places control music performances simultaneously through a communication network. For example, a person in a distant place views a video in a music classroom or the like through a network and simultaneously produces a musical sound performance. (6) Excited state response tone control in the game. (7) Background music (BGM) control in sports such as jogging, aerobics, etc. (bio mode, health mode) For example, listening to the tempo in accordance with the heart rate, taking into account the operation of jogging, aerobics, etc. and the heart rate, etc.
When the heart rate exceeds a certain set heart rate, the music tempo, volume and the like are naturally reduced. (8) Theater. Controls the sound effects such as cutting sound and wind noise according to the movement of the sword during a swordplay dance during a play. (9) Events. In various events, for example, a participatory remote controller, a participatory controller, a participatory input device, a participatory Game, and the like. (10) Concert. At the concert hall, the performer performs the main control (tempo, dynamics, etc.) of the song, and the viewer can easily participate in the music performance like a clap according to the light emitted by the LED or the like with the sub-control unit. Perform the production. (11) Theme park. In a parade at a theme park, control the performance of songs and control the production of light by light-emitting devices.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below in detail with reference to the drawings. The following embodiment is merely an example, and various modifications can be made without departing from the spirit of the present invention.

FIG. 1 shows a schematic configuration of an entire performance system including a performance interface according to an embodiment of the present invention. In this example, the system includes a plurality of physical information detection transmitters 1T1-1.
Tn, an information reception and tone controller 1R and a tone reproduction unit 1S, a main system 1M, a host computer 2, a sound system 3 and a speaker system 4, and a physical information detection transmitter 1T1 to 1Tn and information reception and tone. A performance interface is constituted by the controller 1R.

A plurality of physical information detection transmitters 1T1 to 1Tn
Are the operation sensors MS1-MSn and the state sensors SS1-
SSn and / or either of them, and both sensors MS
a, SSa (a = 1 to n) are held by the hands of a plurality of operators participating in the performance of the performance information, or are attached to predetermined portions of the body of each performance participant. Each motion sensor MSa detects a gesture or motion of each performance participant and outputs a motion signal in accordance with the gesture.
In addition to a so-called “three-dimensional sensor” (x, y, z) such as a three-dimensional acceleration sensor and a three-dimensional velocity sensor, a two-dimensional sensor (x,
y), a distortion detector. The state sensor SSa is a so-called “biological information sensor” that outputs a state signal obtained by measuring a pulse (pulse wave) of the body of each participant, a resistance between skin, an electroencephalogram, respiration, pupil movement, and the like. .

A plurality of physical information detection transmitters 1T1 to 1Tn
Are respectively connected to physical information sensors MS1 to MSn and SS1 to SSn via a signal processing and transmitting device (not shown).
The information signal and the tone signal are sent to the tone controller 1R as detection signals. The information reception and tone controller 1R includes a reception processing unit RP, an information analysis unit A
N and a host computer 2 comprising a personal computer (PC)
And performs data processing for performance parameter control in connection with the host computer 2.

That is, the information reception and tone controller 1
R receives the detection signal from the physical information detection transmitters 1T1 to 1Tn, and in the reception processing unit RP, corresponding data is extracted under predetermined conditions, and the extracted operation data or state data is information as detection data. It is handed over to the analysis unit AN. The information analyzer AN analyzes the detected data, for example, detecting the body tempo from the repetition period of the detection signal. Then, the performance parameter determination unit PS
The performance parameters of the musical sound are determined based on the analysis result of the detection data.

The musical sound reproducing section 1S includes a performance data control section MC.
And a tone generator SB, and generates a tone signal based on performance data in, for example, a MIDI format. The performance data control unit MC changes performance data generated by the main system 1M or previously prepared performance data according to the performance parameters set by the performance parameter determination unit PS. Then, the tone generator SB generates a tone signal based on the changed performance data, transmits the tone signal to the sound system 3, and emits a corresponding tone from the speaker system 4.

In the performance interface (1T1 to 1Tn.1M) according to one embodiment of the present invention, when the operator moves the motion sensors MS1 to MSn, the information analyzer AN causes the motion sensors MS1 to MS1 to move. MS
The movement of the operator is analyzed from the n detected data. The performance parameter determination section PS determines performance parameters in accordance with the analysis result, and the tone reproduction section 1S generates tone performance data based on the performance parameters. Therefore, the desired controlled tone reflecting the motion of the motion sensor is emitted through the sound and the speaker systems 3 and 4. Information analysis unit A
N also analyzes the physical state of the operator from the state information (biological information, physiological information) from the state sensors SS1 to SSn at the same time as the movement analysis of the movement sensors MS1 to MSn, and performs according to the analysis results. Generate parameters. Therefore, the music performance can be variously controlled in consideration of not only the movement of the operator but also the physical condition of the operator.

[Configuration of Physical Information Detection Transmitter] FIG. 2 shows an example of the configuration of a physical information detection transmitter according to one embodiment of the present invention. Each physical information detection transmitter 1Ta (a = 1 to n)
Has a signal processing and transmitting device in addition to a physical information sensor such as a motion sensor MSa and a state sensor SSa, and the signal processing and transmitting device is a transmitter central processing unit (transmitter CP).
U) T0, memory T1, high-frequency transmitter T2, display unit T3, charging controller T4, transmission power amplifier T5, operation switch T6, etc. The motion sensor MSa has a structure that can be held by a performance participant in a hand or attached to an arbitrary part of the performance participant. When the motion sensor MSa is of a hand-held type, the signal processing and transmitting device can be incorporated in the sensor housing together with the motion sensor MSa. Also, the state sensor SSa
Is attached to a predetermined part of the body according to the state to be detected.

The transmitter CPU T0 operates based on the transmitter operation program recorded in the memory T1 and operates the operation sensor MS.
a, controls the operation of the state sensor SSa, the high-frequency transmitter T2, the display unit T3, and the charge controller T4. The detection signals from the physical information sensors MSa and SSa are subjected to predetermined processing such as ID number addition processing by the transmitter CPUT0, transmitted to the high-frequency transmitter T2, and further amplified by the transmission power amplifier T5.
The signal is transmitted to the main system 1M via the transmission antenna TA.

The display unit T3 includes, for example, a 7-segment type LED or LCD display, and one or more LED light emitters (none of which are shown). Displays the middle and power alarms. For example, the LED light emitter emits light continuously or transmits the light according to the operation state of the operation switch T6.
Under the control of UT0, it blinks according to the detection output of the motion sensor MSa. The operation switch T6 is used not only for controlling the blinking of the LED light emitter but also for performing various settings such as mode setting. The charging controller T4 controls charging of the battery power supply T8 when a commercial power supply is connected to the AC adapter T7, and turns on a power switch (not shown) provided in the battery power supply T8 to supply power from the battery power supply T8 to various parts of the transmitter. Supply is made.

[Configuration of Main System] FIG. 3 is a block diagram of the hardware configuration of the main system according to one embodiment of the present invention. In this example, the main body system 1M includes a main body central processing unit (main body CPU) 10, a read-only memory (ROM) 11, a random access memory (RAM) 1
2, an external storage device 13, a timer 14, first and second detection circuits 15, 16, a display circuit 17, a sound source circuit 18, an effect circuit 19, a reception processing circuit 1A, and the like.
0 to 1A are connected to each other via a bus 1B.
To the bus 1B, a communication interface (I / F) 1C for communicating with the host computer 2 is connected, and further, a MIDI interface (I / F) 1D is connected.

Main unit CP for controlling the entire main unit system 1M
The U10 performs various controls in accordance with a predetermined program under time management by a timer 14 used for generating a tempo clock, an interrupt clock, and the like. In particular, the U10 performs performance parameter determination, performance data change, and playback control according to the present invention. Perform the interface processing program centrally. The ROM 11 stores predetermined control programs for controlling the main system 1M. These control programs include a performance interface processing program relating to performance parameter determination, performance data change and playback control, and various data / tables. Etc. can be included. The RAM 12 is used as a work area for storing data and parameters necessary for these processes and for temporarily storing various data being processed.

The keyboard 1E is connected to the first detection circuit 15, the pointing device 1F such as a mouse is connected to the second detection circuit 16, and the display 1G is connected to the display circuit 17. This allows the keyboard 1 to be visually recognized while viewing various screens displayed on the display 1G.
By operating the E and the pointing device 1F, setting various modes necessary for controlling performance data in the main system 1M,
Various setting operations can be performed, such as assignment of processing and functions corresponding to the ID number and setting of a tone (sound source) to a performance track.

According to the present invention, an antenna distribution circuit 1H is connected to the reception processing circuit 1A. The antenna distribution circuit 1H is composed of, for example, a multi-channel high-frequency receiver and includes a plurality of physical information detection transmitters 1T1 to 1Tn. Are received via the receiving antenna RA. The reception processing circuit 1A converts the received signal into the main system 1
The data is converted into operation data and state data that can be processed by the M, is taken into the system, and is stored in a predetermined area of the RAM 12.

With the performance interface processing function of the main body CPU 10, operation data and state data representing the body movement and state of each performance participant are analyzed, and performance parameters are determined based on the analysis result. An effect circuit 19 composed of a DSP or the like includes a sound source circuit 18 and a main body CPU 1.
0, the function of the sound source section SB is realized, and by controlling the performance data to be performed based on the set performance parameters, the performance data which has been produced in accordance with the physical information of the performance participant. Generate The sound system 3 connected to the effect circuit 19 emits a performance musical tone via the speaker system 4 in accordance with a musical tone signal based on the performance data subjected to the effect processing.

The external storage device 13 includes a hard disk drive (HDD), a compact disk
It is composed of storage devices such as an only memory (CD-ROM) drive, a floppy disk drive (FDD), a magneto-optical (MO) disk drive, and a digital versatile disk (DVD) drive, and can store various control programs and various data. it can. Therefore, the performance interface processing program, various data, and the like necessary for determining performance parameters, changing performance data, and controlling playback are stored in a ROM.
11 as well as RA from external storage device 13.
It can be read into M12, and the processing result can be recorded in the external storage device 13 as needed. Further, the external storage device 13, particularly, a CD-ROM, an FD,
In the media such as MO and DVD, for example, various musical composition data in the MIDI format are stored as MIDI files, and by using this, desired musical composition data can be imported into the main system.

Further, such processing programs and performance music data may be taken into the main system 1M from the host computer 2 connected via the communication I / F 1C and the communication network, or sent out to the host computer 2. Can be. For example, software such as sound source software and performance music data can be distributed through a communication network. Furthermore, MID
It communicates with another MIDI device 1J connected to the II / F1D to receive and use performance data and the like, or conversely, sends out performance data that has been rendered by the performance interface function of the present invention. by this,
Sound source section of main system 1M (Fig. 1: SB, Fig. 3: 18
19) is omitted, and the function of the sound source section is changed to another MIDI device 1J.
Can be given to.

[Structural Example of Motion Sensor] FIGS.
5 shows an example of a body information detection mechanism that can be used for the performance interface of the present invention. FIG. 4A is an example of a baton-shaped hand-held type physical information detection transmitter. This physical information detection transmitter incorporates the various devices shown in FIG. 2 except for the operation unit and the display unit (does not include the state sensor SSa).
For the built-in motion sensor MSa, for example, a three-dimensional sensor such as a three-dimensional acceleration sensor or a three-dimensional speed sensor is used. And an operation signal corresponding to the size can be output.

The external structure of this transmitter is shown in FIG.
It consists of a base (left side in the figure) and an end (right side in the figure), both ends of which have a large diameter and a small diameter on the center side.
The average diameter is smaller than the end, making it easier to grasp with the hand, and functions as a grip. The LED display TD of the display unit T3 and the power switch TS of the battery power supply T8 are provided on the outer surface of the bottom (left end in the figure), the operation switch T6 is provided on the outer surface of the center, and the display unit T3 is provided near the end of the end.
Are provided.

The baton-type transmitter shown in FIG. 4A outputs an operation signal according to the operation direction and operation force from a built-in three-dimensional sensor when a performer grips and operates the baton base. Is done. For example, when a three-dimensional acceleration sensor is built in with the detection axis in the x direction aligned with the mounting direction of the operation switch T6, when the baton is shaken up and down with the mounting position of the operation switch T6 up, When a signal output representing the acceleration αx in the x direction according to the acceleration (force) is generated, and the baton is swung right and left (perpendicular to the paper surface), a signal output representing the acceleration αy in the y direction according to the swing acceleration (force) is generated. When the baton strikes and pulls back and forth (left and right on the paper), z
A signal output representing the acceleration αz in the direction is generated.

FIG. 4B shows an example of a shoe shape in which a motion sensor is embedded in the heel portion of the shoe. Motion sensor M
Sa is, for example, a strain detector (one-dimensional sensor in the vertical x-axis direction) or a secondary to three-dimensional sensor (left-right y-axis direction to toe z-axis direction), and is embedded in the heel portion of the shoe. In this example, the physical information detection transmitter 1 shown in FIG.
The devices other than the sensor portion of Ta are incorporated in, for example, a signal processing and transmitter device (not shown) attached to the waist belt, and the operation detection output of the operation sensor MSa is transmitted via a wire (not shown). It is input to a detection process and a transmitter device. Such a shoe-shaped motion sensor controls the tempo of a musical piece performed in accordance with the cycle of a detection signal or increases the percussion instrument volume in accordance with the detection timing, for example, when performing a tap dance in the performance of Latin music or the like. Or inserting a tap sound (using a specific performance track).

On the other hand, the state sensor SSa can be implemented by a hand-held type similar to the above-mentioned baton type when the state information can be acquired by holding it by hand. Attached to the body condition detection part, and outputs the state detection output of the state sensor SSa to other required parts of the performance participant (jacket, hat, glasses,
(E.g., collars, waist belts, etc.).

FIG. 5 shows another example of the structure of the body information detecting mechanism. In this example, a ring-type body information sensor IS and a signal processing and transmission device TTa constitute a body information detection transmitter 1Ta. As the physical information sensor IS, for example, a motion sensor MSa such as a two-dimensional or three-dimensional sensor or a strain detector, or a state sensor SSa such as a pulse (pulse wave) detector can be used. , 1
It can be installed not only on one finger (index finger) but also on multiple fingers. Various devices other than the sensor unit in the physical information detection transmitter 1Ta of FIG. 2 are incorporated in the detection processing and transmitter device TTa supported in a bracelet shape, and the physical information sensor IS
Is input to the signal processing and transmitting apparatus TTa via a wire (not shown).

The signal processing and transmitter device TTa has the configuration shown in FIG.
LED display TD, power switch TS as in (1)
And an operation switch T6 are provided.
L is not provided. In addition, when using the motion sensor MSa for the physical information sensor IS, the state sensor SSa is provided in another part capable of detecting the physical condition as needed, and when the state sensor SSa is used for the physical information sensor IS, If necessary, the motion sensor MSa [for example, FIG.
The motion sensor MSa] as in (2) can be provided at another portion where the motion can be detected.

[Sensor Data Format] In one embodiment of the present invention, the ID number of each sensor is assigned to the data represented by the detection signals of the motion sensor and the status sensor, and the main system 1M identifies the sensor. , Are used to perform corresponding processing. FIG. 6 shows a format example of the sensor data. As the ID number, the most significant 5 bits (bits 0 to 4) are used, and a maximum of 32 numbers can be assigned.

The next three bits (bits 5 to 7) are switch (SW) bits, which can be used to specify up to eight types of various specifications such as mode designation, start / stop, song selection, and cueing. It can be decoded according to a switch table set for each ID number on the main system 1M side.
As the SW bit, all bits can be designated by the operation switch T6, only some bits can be set, and the remaining bits can be set in advance for each sensor, or all bits can be set in advance. Normally, the operation switch T6
It is preferable that at least the first SW bit A (bit 5) can be specified to be play mode on (A = “1”) / off (A = “0”).

The next three bytes (= 8 bits) are data bytes. When a three-dimensional sensor is used, as shown, x-axis data (bits 8 to 15) and y-axis data (bits 16 to 15) are used. 23) and z-axis data (bits 24 to 3)
1) is assigned, the third data byte (bits 24 to 31) can be used as an extended data area in the case of a two-dimensional sensor, and the second and third data bytes (bits 16 to 31) in the case of a one-dimensional sensor. ) Can be used as the extension data area. For other types of information sensors, data values according to the respective detection modes can be assigned to these data bytes.

[Utilization of Motion Sensor = Utilization of a plurality of Analysis Outputs] In one embodiment of the present invention, a plurality of analysis outputs obtained by processing motion sensor outputs generated by a performance participant operating an operation body. Thus, desired control can be added to the music performance. For example, when a one-dimensional acceleration sensor that detects acceleration (force) in only one direction is used as a motion sensor, a plurality of performance parameters related to music performance can be controlled by a basic configuration as shown in FIG. . Here, the one-dimensional acceleration sensor MSa
For example, in the baton-shaped body information detection transmitter shown in FIG. 4A, an acceleration detector (x-axis detection) that detects acceleration (force) in only a predetermined one-direction (for example, up-down x-axis direction) component (Part) Sa is configured as an operating body with built-in Sa.

In FIG. 7, when the performer performs an operation such as shaking the operating body in his hand, the one-dimensional acceleration sensor MSa outputs a predetermined direction (x-axis) of the operating acceleration (force). A detection signal Ma representing the acceleration α of only the (direction) component is output to the main system 1M. Main system 1
In M, if it is confirmed that a preset ID number is added to the detection signal, a band-pass filter function for removing a noise component by low-pass and high-frequency removal and passing an effective component, and a gravity component Through the reception processing unit RP having a DC cut function for removal, effective data representing the acceleration α is passed to the information analysis unit AN.

The information analyzing unit AN analyzes the acceleration data, and indicates the peak time Tp indicating the occurrence time of the local peak of the time-lapse waveform | α | (t) of the absolute acceleration | α | and the height of the local peak. Peak value Vp, the following equation (1)
The peak Q value Q indicating the sharpness of the local peak represented by
Extract p, peak interval representing the time interval between local peaks, valley depth between local peaks, intensity of high frequency components of peaks, polarity of local peaks of acceleration α (t), etc .:

(Equation 1) Here, w is the time width of the point where the height of the local peak of the acceleration waveform α (t) is half of the peak value Vp.

The performance parameter determination unit PS determines the beat timing B in accordance with the detected outputs Tp, Vp, Qp,.
T, dynamics (velocity and volume) DY, or articulation AR, pitch (pitch),
Various performance parameters such as tone color are determined, and the tone reproduction unit 1
In the performance data control unit of S, the performance data is controlled based on the determined performance parameters, and the musical performance is output via the sound system 3. For example, the beat timing BT is controlled according to the peak occurrence time (time point) Tp,
The dynamics DY is controlled according to the peak value Vp, the articulation AR is controlled according to the peak Q value Qp, and the front beat or back beat is determined according to the polarity of the local peak, and the beat number is discriminated. Do.

FIG. 8 schematically shows an example of a trajectory of a hand movement and a waveform example of acceleration data α when a command operation is performed while holding the one-dimensional acceleration sensor MSa in a hand. (T) "represents the absolute value (no polarity) of the acceleration data α, that is, the absolute acceleration" | α | (t) ". FIG. 8A shows an example of a motion trajectory (a) and an example of an acceleration waveform (a) during a 2-beat espressivo = expressively ("expressively express") commanding operation.
In this locus example (a), the operating point P indicated by a black circle
The commanding operation is not stopped at 1 and P2, and the movement is always smooth and soft. On the other hand, FIG. 8B shows an example of a motion trajectory (b) and an acceleration waveform example (b) during a 2-beat staccato (staccato; “clearly separate sound”) commanding operation. b) operating points P3 and P4 indicated by crosses
Pause at, indicating a quick and sharp commanding action.

Therefore, in response to such a commanding operation,
For example, peak occurrence time (time point) Tp = t1, t2, t
3,...; Beat timing BT by t4, t5, t6,.
Is determined, the dynamics DY is determined by the peak value Vp, and the articulation parameter AR is determined by the Q value Qp of the local peak. That is, during the espresso operation shown in FIG. 8A and the staccato operation shown in FIG. 8B, there is not much difference in the peak value Vp, but the Q value of the local peak is sufficiently different.
The value Qp is used to control the degree of articulation between staccato and espresso. The use of the articulation parameter AR will be described in more detail below.

The MIDI music data has information indicating a tone generation start timing and a silence start (tone generation stop) timing together with pitch information and the like for a large number of musical tones.
The time from the start to the stop of the sound, that is, the length of time during which the sound is sounding is called "gate time". If the gate time GT is made shorter than the gate time value of the music data, that is, by using a coefficient Agt by which the music data gate time GT0 is multiplied, for example, Ag
By setting t = 0.5 and 0.5 times the gate time GT0, a staccato performance can be obtained. Conversely, for example, when a = 1.8, that is, 1.8 times, longer than the original gate time value of the music data, an espresso-like performance can be obtained.

Therefore, the gate time coefficient Agt is adopted as the articulation parameter AR and is changed according to the Q value Qp of the local peak. For example,
As shown in the following equation (2), the linear conversion of the Q value Qp of the local peak is performed, and the gate time GT is adjusted by using the coefficient Agt that changes according to the Q value Qp, thereby achieving the articulation. AR can be controlled: Agt = k1 × Qp + k2 (2)

For controlling the performance parameters, the peak Q value Qp
8, the depth of a valley, the strength of a high frequency component, etc. in the time waveform of the absolute acceleration | α | shown in each waveform example (a) and (b) of FIG. Good. In the illustrated trajectory example (b), the pause time is longer than in the trajectory example (a), the valley of the waveform is deep, and the value is closer to “0”. In the trajectory example (b), since the operation is sharper than in the trajectory example (a), the high-frequency component is stronger than in the trajectory example (a).

Further, for example, the timbre can be controlled by the peak Q value Qp. Generally, in a synthesizer, the envelope shape of a sound waveform is determined by an attack (rising) portion A, a decay portion D, a sustain S, and a release R. If the rising speed (incline) of the attack portion A is low, a soft tone is obtained. On the other hand, if it is high, the tone tends to be sharp. Therefore, when the performance participant wears the operating body equipped with the one-dimensional acceleration sensor MSa, the rising speed of the attack part A is controlled by the Q value of the local peak in the time waveform of the hand movement acceleration (αx). Thus, an equivalent tone can be controlled.

In this example, the sound wave envelope shape AD
Although the tone is controlled equivalently by controlling a part of the SR, the tone itself (so-called “voice”) may be switched, for example, from a double bass tone to a violin tone, This method may be used in combination with the method based on ADSR control. Also, as cause information,
Not only the Q value of the local peak but also the strength of the high frequency component of the waveform and other information may be used.

Further, it is also possible to control an effect parameter such as reverb (so-called "effect") according to the detection output. For example, the reverb time is controlled using the Q value of the local peak. A high Q value corresponds to a motion of the performance participant shaking the operating body sharply, but such a sharp and sharp motion shortens the reverb to improve the crispness of the sound. Conversely, when the Q value is low, the reverb is lengthened to produce a gentle and slow sound. Of course, this relationship may be intentionally reversed, another effect parameter (for example, a filter cutoff frequency of the sound source section SB) may be controlled, or the plurality of effect parameters may be controlled. In this case, as the cause information,
Not limited to the Q value of the local peak, the strength of the high frequency component of the waveform and other information can be used.

Further, a percussion instrument sounding mode for generating a percussion instrument sound at the time of occurrence of a local peak using a peak interval can be produced. In this percussion instrument sounding mode, when the extracted peak interval is long, for example, a low pitched percussion instrument such as a bass drum is played, and when the peak interval is short due to a quick swing motion, for example, a high pitch such as a triangle is used. Plays pitched percussion instruments. Of course, the relationship may be reversed, and only the pitch (pitch) may be changed continuously or stepwise without changing the timbre (voice). Furthermore, the tone (voice) is not two kinds,
So that many things can change, or
You may make it change gradually, performing a volume crossfade. Further, the use of the pitch interval is not limited to the percussion instrument. For example, the tone and the pitch of another instrument are changed, such as changing the tone of the stringed instrument and changing the pitch, for example, from a double bass to a violin. You may change it.

[Utilization of a plurality of motion sensor outputs] In one embodiment of the present invention, a plurality of motion sensor outputs generated by a performance participant operating an operating body are processed to obtain a desired music performance. Control can be added. Such a motion sensor includes, for example, FIG.
It is preferable to use a two-dimensional sensor having an x-axis and a y-axis detector or a three-dimensional sensor further having a z-axis detector in a baton-shaped structure as shown in (1). Holding the operating body equipped with the motion sensor, x
When the operation is performed in the direction from the axis to the y-axis, and further in the z-axis direction, the operation detection output of each axis sensor is analyzed and the operation (movement) type is determined, and a plurality of performance parameters of the music, for example, Tempo, volume, etc. are controlled. Therefore,
The performance participant can behave like a conductor to the music performance (conductor mode).

In the command mode, a plurality of designated controllable performance parameters are always controlled by the operation detection output of the sensor (pro mode), and if there is a sensor output, the performance parameters are controlled by the sensor output. When there is no sensor output, a mode in which the original MIDI data is reproduced as it is (semi-auto mode) can be set.

Here, if a two-dimensional sensor is used as a motion sensor for a commanding operation, various analyzes can be performed and various performance parameters can be controlled in accordance with the analysis, as in the case of using a one-dimensional sensor. In addition, an analysis output that more accurately reflects the swing operation can be obtained as compared with a one-dimensional sensor. For example, when the user performs a hand-holding motion in the same manner as the one-dimensional sensor of FIGS. , A signal representing the acceleration αx in the x-axis (vertical / vertical) direction and the signal αy in the y-axis (horizontal / lateral) direction are output to the main system 1M. In the main body system 1M, the acceleration data of each axis is passed to the information analysis unit via the reception processing unit, and the acceleration data of each axis is analyzed.
An absolute acceleration represented by the following equation (3), that is, an absolute value | α | of the acceleration is obtained:

(Equation 2)

FIG. 9 shows a performance bar in which a two-dimensional acceleration sensor using two acceleration detectors (electrostatic acceleration sensor: Topre “TPR70G-100”) for the x-axis and the y-axis is mounted on a command rod. An example of a trajectory and an acceleration waveform of a hand movement obtained when a user performs the commanding operation by holding the commanding bar with his right hand is shown. The command trajectory is represented as a two-dimensional motion trajectory, and as shown in FIG. 9A, for example, (a) a 2-beat espressivo commanding operation, (b) a 2-beat staccato commanding operation,
(C) 2-beat espressivo commanding action and (d) 3
The trajectory of the beat staccato command operation is obtained. In the figure, “(1)” to “(3)” represent swing motion (beating motion) classifications, and (a) and (b) represent “two swing”.
(2 beats), (c) and (d) are "three swings"
(3 beats). FIG. 9 (2) shows the detection output obtained from each axis detector of the sensor corresponding to each of the trajectories (a) to (d) by the commanding operation of the performance operator.

Here, similarly to the case of the above-described one-dimensional sensor, the detection output of each axis detector is passed through a band-pass filter that removes a frequency component unnecessary for commanding operation recognition in the reception processing unit of the main system 1M. . If the sensor is fixed to a desk or the like, the acceleration sensor outputs αx, αy, |
α | does not become zero, but since this component is unnecessary for command recognition, it is removed by a DC cut filter. The direction of the commanding operation appears as the sign or strength of the detection output of the two-dimensional acceleration sensor, and the timing at which the swinging operation (beating operation) occurs appears as a local peak of the absolute acceleration value | α | Used for timing. Therefore, two-dimensional acceleration data αx and αy having positive and negative values are used for discriminating the beat number, but only the absolute acceleration value | α | is used for beat timing detection.

The accelerations αx and αy at the time of occurrence of the beating operation actually change greatly in polarity and strength depending on the direction of the beating operation and have a complicated waveform including many pseudo peaks. It is difficult to get. Therefore, as described above, the signal passes through a 12th-order moving average filter that removes unnecessary high frequency components from the absolute acceleration value. (A) to (d) of FIG. 9 (2) show an example of the acceleration waveform after passing through the obi-cast filter which is composed of the two filters, and each of the trajectory diagrams (FIG. 9 (1)) It is a signal when carefully conducting according to a) to (d). Each waveform shown on the right side of FIG. 9B represents a vector locus for one cycle of the two-dimensional acceleration signals αx and αy. Each waveform shown on the left side represents three seconds of the time-domain waveform | α | (t) of the absolute acceleration value | α |, and each local peak corresponds to a beat operation.

When local peaks are extracted to detect a beat operation, it is necessary to avoid false peaks and miss detections such as missed detection of beat peaks. An appropriate technique such as a resolution pitch detection technique should be applied. Also, acceleration αx, α
Although y takes a positive or negative value as seen on the right side of FIG. 9 (2), the hand always keeps moving delicately and does not stop completely in human commanding operation. For this reason, there is no case where both the accelerations αx and αy become zero at the same time and stop at the origin. Therefore, the time-domain waveform | α | (t) is also commanded as shown on the left side of FIG. Inside is not zero.

[Three-dimensional sensor use mode = three-axis processing] When the number of detection axes is further increased and a three-dimensional sensor is used as the motion sensor MSa, the operation (movement) of this sensor is analyzed and its various operations are analyzed. Thus, various and various performance controls corresponding to the operation can be performed. FIG.
3 shows a functional block diagram in the case of performing a musical performance using a three-dimensional sensor. In the three-dimensional sensor use mode of FIG. 10, the three-dimensional sensor MSa is built in the baton-type detection transmitter 1Ta described in FIG. 4A, and the performance participant holds the baton-type detection transmitter 1Ta with one or both hands. , An operation signal corresponding to the operation direction and the operation force can be output.

When a three-dimensional acceleration sensor is used as the three-dimensional sensor, the x-axis, y-axis, and z-axis detection units SX, SY, and SZ of the three-dimensional sensor MSa in the baton-shaped body information detection transmitter 1Ta respectively Acceleration αx in x (vertical) direction, acceleration αy in y (horizontal) direction and z (front and rear)
Signals Mx, My, Mz representing the acceleration αz in the direction are output to the main system 1M. When the main body system 1M confirms that a preset ID number is added to these signals, the acceleration data of each axis is passed to the information analysis unit AN via the reception processing unit RP. In the information analysis unit AN, each axis acceleration data is analyzed.
The absolute value | α | of the acceleration represented by the following equation (4) is obtained:

(Equation 3)

Next, the accelerations αx and αy are compared with the acceleration αz. For example, when the relationship of αx <αz and αy <αz is satisfied, that is, when the z-direction acceleration αz is larger than the x and y-direction accelerations αx and αy, a “thrust operation” of striking the baton is performed. It is determined that there is.

On the other hand, when the z-direction acceleration αz is smaller than the x and y-direction accelerations αx and αy, it is determined that the air is cut by the baton. In this case, the values of the accelerations αx and αy in the x and y directions are compared with each other to determine whether the direction of the “cutting operation” is “vertical” (x) or “horizontal” (y). Can be.

In addition to the comparison between the components in the respective x, y, and z directions, the magnitude of each of the components αx, αy, and αz is compared with a predetermined threshold value. Can be determined as a “combination operation”. For example, αz> αx, αy and αx> “x
If the threshold value is a “component threshold value”, it is determined that “push motion while cutting in the vertical direction (x direction)”, and αz <αx, αy, αx> “x component threshold value” and αy> “y component threshold value” are satisfied. In this case, it is determined that the operation is “oblique (both x and y directions) cutting operation”. Further, by detecting a phenomenon in which the values of the y-direction accelerations αx and αy relatively change so as to draw a circular locus, it is possible to determine the “turning operation” of turning the baton around.

The performance parameter determination section PS determines various performance parameters in accordance with these discrimination outputs, and the performance data control section of the tone reproduction section 1S controls the performance data based on the determined performance parameters. The musical performance is output via the sound system 3. For example, the volume of the performance data is controlled in accordance with the absolute value of the acceleration | α | or the component having the maximum value among the directional components αx, αy, and αz. Further, other performance parameters are controlled based on the determination result in the analysis unit AN.

For example, the tempo is controlled in accordance with the cycle of the vertical (x-direction) cutting operation. Apart from this, articulation is given if the "vertical cutting action" is short and high, and the pitch (pitch) is lowered if it is long and low. In addition, a slur effect is provided by the determination of the “horizontal (y-direction) cutting operation”. When the "thrust operation" is determined, the stuttering effect is given by shortening the musical sound generation timing at that timing, or a single sound (percussion instrument sound, shouting, etc.) according to the size is inserted into the musical performance, and the "thrust operation" is performed. Is determined, the above-described control is used together. When the “turning operation” is determined, control is performed so that the reverberation effect is enhanced according to the size when the cycle is large, and a trill is generated according to the cycle when the cycle is small.

Of course, control similar to the control described when a one-dimensional sensor or a two-dimensional sensor is used can be performed. That is, in the three-dimensional acceleration sensor, when the absolute acceleration projected on the xy plane represented by the equation (3) is “xy absolute acceleration | αxy |”, x−
Time lapse waveform of y absolute acceleration | αxy | | αxy |
(T) Time of occurrence of local peak, local peak value, peak Q value indicating sharpness of local peak, peak interval indicating time interval between local peaks, depth of valley between local peaks, strength of high frequency component of peak Well,
Extracting the polarity of the local peak of the acceleration α (t) and the like, controlling the beat timing of the musical composition according to the peak occurrence time, controlling the dynamics according to the local peak value,
It controls the articulation AR according to the peak Q value Qp, and so on. When the equation (5) is satisfied and it is determined to be a “thrust operation”, a single sound (percussion instrument sound, shouting, etc.) is inserted into the musical performance in parallel with these controls, or the z-direction acceleration αz In this case, a tone change or a reverb effect is given in accordance with the magnitude of, and other performance control not controlled by the xy absolute acceleration | αxy | is performed.

A one-dimensional to three-dimensional sensor is installed in the sword, and in a swordplay accompanied by music in a theater or the like, a sound (x Axis or y axis), wind noise (y axis or x axis)
Axis), and can also be used to control the sounding of sound effects such as thrusting sounds (z axis).

[Other Examples of Use of Motion Sensor] In a one-dimensional to three-dimensional acceleration sensor, if the output of each axis of the sensor is integrated or a speed sensor is used instead of the acceleration sensor, the operation (motion) of the sensor is performed. Similarly, depending on speed,
The operation can be determined and the performance parameters can be controlled.
Further, by integrating each axis integrated output of the acceleration sensor or integrating each axis output of the speed sensor, an operation (movement) position of the sensor is estimated, and another performance parameter is controlled according to the estimated position. (For example, the pitch is controlled according to the pitch in the x direction). Also, two motion sensors such as one-dimensional to three-dimensional sensors are prepared as baton-shaped operating bodies as shown in FIG. 4A, and these operating bodies are operated by one operator with the left and right hands, respectively. In this way, the left and right separate controls can be added to the music performance. For example, a music performance track (part) is divided into two parts, and each group performance track (part) is individually controlled based on left and right motion analysis.

[Use of State Sensor = Bio Mode] According to one embodiment of the present invention, by performing processing for detecting biological information, it is possible to enjoy the music performance by reflecting the state of the performance participant in the performance musical sound. For example, when a plurality of participants perform exercise such as aerobics in a group while listening to music performance, a pulse (pulse wave) detector is used as the information sensor IS in FIG. A pulse detector is attached to detect the heart rate, and when the heart rate exceeds a predetermined heart rate, the tempo of the music being played is naturally reduced in consideration of the participant's health. As a result, music performance is realized in consideration of the operation such as aerobics and the physical state such as the heart rate. In this case, it is assumed that the performance tempo is controlled in accordance with the average value of the measurement data such as the heart rate. In calculating the average value, the larger the heart rate, the greater the weight is added to calculate the average value. Is preferred.
Further, the volume may be reduced as the tempo decreases.

In the case of the above example, when the rate of increase of the heart rate falls within a predetermined range specified in advance, the tone is emitted from the speaker 4 and the LED light emitting device of the display unit is illuminated to be in a normal state. However, if it deviates from this range, it is possible to add a performance stop function according to the state of the body, such as stopping the generation and emission of musical sounds. Further, other equivalent biological information can be used instead of the heart rate. For example, the same effect can be obtained by using the respiration rate. As a sensor of the respiratory rate, there are a method of attaching a pressure sensor to the chest and abdomen, and a method of attaching a temperature sensor for detecting air movement to a nostril or the like.

Another example of responding to the biological information is to analyze the excitement state (increase in pulse and respiratory rate, decrease in skin-to-skin resistance, increase in blood pressure and body temperature, etc.) from the biological information, and obtain the excitement state obtained as a result. For example, there is a method of performing tone control of an excitement degree response for controlling a performance parameter in a direction opposite to the above-described example in which the health is taken into consideration, such as increasing a performance tempo or a volume according to the rise of the performance. This method is used for BG of various games in which multiple people participate.
It is suitable for M performance and music performance that a plurality of participants enjoy while dancing in the hall, and the degree of excitement is calculated from an average value of the plurality of participants, for example.

[Operation / State Combination Mode] According to an embodiment of the present invention, the operation sensor and the state sensor are used together to detect the operation and biometric information of the performance participant, thereby enabling a plurality of types of performance participant to be detected. The state can be reflected in the musical performance to produce a variety of music performances. FIG.
Fig. 7 shows a functional block diagram in the case where a musical performance is performed using the motion sensor and the state sensor in combination. In this example, a two-dimensional sensor including the x-axis and y-axis detection units SX and SY described above is used as the motion sensor MSa, but a one-dimensional sensor or a three-dimensional sensor may be used as necessary. Can be. This motion sensor MSa is built in a baton-shaped structure as shown in FIG. 4A, and performs a commanding operation of a music performance by, for example, shaking it with the right hand of a performance operator. In addition, the state sensor SSa is provided with a viewpoint (line of sight).
A tracking unit (Eye Tracking System) SE and a breath detection unit (Breath Sensor) SB are used and attached to a predetermined body part of the performance participant, for example, to perform gaze tracking and breath detection of the performance operator.

The detection signals from the x-axis and y-axis detection units SX and SY of the two-dimensional motion sensor MSa and the detection signals from the viewpoint tracking unit SE and the respiration detection unit SB of the state sensor SSa are respectively given individual ID numbers. The signal is output to the main system 1M via the respective signal processing and transmission units. When the main body system 1M confirms the assignment of the preset ID number, the reception processing unit RP processes the detection signals from the two-dimensional motion sensor MSa, the viewpoint tracking unit SE, and the respiration detection unit SB, and generates a corresponding two-dimensional signal. The motion data Dm, the viewpoint position data De, and the respiration data Db are converted into the respective analysis blocks AM, AE, A of the information analysis unit AN according to the ID number.
Give to B. In the motion analysis block AM, the motion data D
Analyze m, the magnitude of data value, beat timing,
Recognizing the beat number and the articulation, the viewpoint analysis block AE analyzes the viewpoint position data De to detect a watching area, and the respiration analysis block AB analyzes the respiration data Db to exhale and inhale. Detect state.

In the next performance parameter determination section PS, the first
In the data processing block PA, the beat position on the musical score of the performance data selected according to the SW bit (bits 5 to 7: FIG. 6) is estimated from the MIDI file in the performance data medium (external storage device 13), and A beat occurrence point is predicted from the currently set performance tempo, the estimated beat position, the predicted beat occurrence point, the data value size from the motion analysis block AM, the beat timing, the beat number, and the articulation. Integration processing. Second data processing block P
In B, based on the result of the integrated processing,
In addition to determining the performance tempo and each musical sound generation timing, a specific performance part is instructed according to the gaze area detected in the viewpoint analysis block AE, and based on the expiration and inspiration state detected in the respiration analysis block AB. Determine the control by breath. Then, the musical sound reproducing unit 1S controls the performance data based on the determined performance parameters, and performs a desired musical sound performance via the sound system 3.

[Operation Mode by a Plurality of Performance Operators] According to one embodiment of the present invention, a plurality of performance information transmitters can be operated by a plurality of performance operators to control music performance. Here, each performance operator can operate one or more physical information detection transmitters, and each of the physical information detection transmitters includes the motion sensor or state described above with reference to FIGS. A configuration of a sensor (including a bio mode and an operation / state combined mode) can be adopted.

[Ensemble Mode] For example, a plurality of physical information detection transmitters are constituted by one master unit and a plurality of slave units, and certain performance parameters are controlled according to the physical information detection signal of the master unit. An ensemble mode in which other performance parameters are controlled by the physical information detection signal of the slave unit can be set. FIG. 12 is a functional block diagram in the ensemble mode according to one embodiment of the present invention. One master unit 1T1 and multiple slave units 1T2
1Tn (for example, n = 24) is, for example, the master device 1
The tempo and volume of the performance parameters are controlled in accordance with the physical information detection signal of T1, and the timbre is controlled by the physical information detection signals of the slaves 1T2 to 1Tn. In this case, each of the physical information detection transmitters 1Ta (a = 1 to n) corresponds to FIG.
The operation is detected by using the baton shape of (1) and detecting the operation signal Ma (a
= 1 to n) is preferably transmitted.

In FIG. 12, physical information detection transmitter 1T
The operation signals M1 to Mn (n = 24) from 1 to 1Tn are:
Information reception of main system 1M and tone controller 1R
The selective reception processing is performed by the reception processing unit RP in. That is, these operation signals M1 to Mn are transmitted to the reception processing unit RP
In the selector SL, the ID number added to the operation signals D1 to Dn is identified in accordance with the preset “ID number assignment” (including “group setting for each ID number”), and thereby the master number is determined. The operation signal M1 from the handset 1T1 and the operation signals M2 from the handset 1T2 to 1Tn
Mn and a master data MD corresponding to the master operation signal M1, and a slave unit operation signal M
2 to Mn, and these slave data are further divided into first to m-th groups SD1 to S
Dm.

For example, the master unit 1 having the ID number “0”
At T1, by operating the operation switch T6, the first SW bit A in FIG.
= “1”) and the second SW bit B for designating “group / individual mode” is designated for group mode (B =
“1”) or individual mode designation (B = “0”), and the third SW bit C for designating “whole / partial leading mode” is designated as whole leading mode designation (C = “1”) or partial It is assumed that the leading mode is specified (C = “0”). On the other hand, in the slave units 1T2 to 1T24 (= n) having the ID numbers “1” to “23 (= n−1)”, the first SW bit A is set to the play mode ON (A = “ 1 ”), and the second and third SW bits A have arbitrary values (B =“ X ”, C =
“X”. “X” represents an arbitrary value. ).

On the other hand, by referring to “ID number assignment”, the selector SL selects the master unit 1T1
From the master unit 1T1 from the ID number "0" added to this signal, outputs the corresponding master data MD, and outputs the operation signal M2 from the slave units 1T2 to 1Tn. About ~ Mn,
Slave units 1T2 to 1Tn according to ID numbers "1" to "23"
And selects the corresponding slave unit data.
At this time, these child device data are further stored in the first to m-th groups SD according to the “group setting for each ID number”.
The data is sorted and output every 1 to SDm. The group classification based on the “group setting for each ID number” changes according to the setting contents in the main system 1M, and one group may include a plurality of child device data, There may be only one slave unit data, or there may be only one group in total.

The master data MD and the slave unit data of the first to m-th groups SD1 to SDm are passed to the information analyzer AN. In the master data analysis block MA, the master data MD is analyzed for information, the contents of the second and third SW bits B and C are checked, and the size and cycle of the data value are determined. For example, it is determined whether the “group mode” or the “individual mode” is specified by the second SW bit B of the master data MD, and the “all leading mode” or the “partial leading mode” is determined by the third SW bit C. Check if it is specified. Further, the operation content indicated by the data, the size of each content, the cycle, and the like are determined based on the content of the data byte of the master data MD.

In the slave unit data analysis block SA,
The slave unit data included in the first to m-th groups SD1 to SDm is analyzed for information, and the size, cycle, and the like of the data value are determined according to the mode specified by the second SW bit B of the master data MD. For example, "group mode"
Is set, the average value of the size and cycle of the slave unit data belonging to each of the groups SD1 to SDm is calculated. If the “individual mode” is set, the individual size and cycle of the slave unit data are calculated. Is done.

The performance parameter determination section PS at the next stage includes a main setting block MP and a sub setting block AP corresponding to the master data analysis block MA and the slave unit data analysis block SA. The performance parameters of each performance track are determined for the performance data selected from the MIDI file recorded in the MIDI file. First, in the main setting block MP, performance parameters of a predetermined performance track are determined based on the determination result in the master data analysis block MA. For example, when the “all leading mode” is designated by the third SW bit C, the volume parameter value is determined for all the performance tracks (tr) according to the determination result of the data value size, and the cycle determination result is determined. Set the tempo parameter value according to. When the “partial leading mode” is designated, the first performance track (tr) (one or more performance tracks, for example, a melody sound track) set in advance corresponding to this mode is Similarly, the sound volume parameter value and the tempo parameter value are determined according to each determination result.

On the other hand, in the sub setting block AP, the third SW
For the performance track corresponding to the mode specified by the bit C, a preset tone color is set, and the performance parameters are determined based on the determination result in the slave unit data analysis block SA. For example, when the “entire lead mode” is designated, a predetermined tone color parameter is set for a predetermined performance track (for example, all accompaniment sound tracks or sound effect tracks) set in advance in accordance with this mode. At the same time, the performance parameters of these performance tracks are changed according to the determination result of the slave unit data in addition to the settings based on the determination result of the master data (that is, the volume parameter value is further changed according to the magnitude of the slave unit data value). Change, and further change the tempo parameter value according to the cycle of the slave unit data.) In this case, it is preferable that the volume parameter value is calculated by multiplication with a variable based on the master data determination result, and the tempo parameter is calculated by arithmetic mean with the variable based on the master data analysis output. When the “partial leading mode” is designated, each determination result is obtained for a second performance track (for example, a performance track other than the first performance track) set in advance corresponding to this mode. , The sound volume parameter value and the tempo parameter value are independently determined.

The tone reproducing section 1S adopts the performance parameters determined as described above as performance parameters of each performance track of the selected performance data from the MIDI file, and sets the performance tracks in advance. Assigned sound (sound source). As a result, it is possible to generate a musical tone having a predetermined tone color corresponding to the operation of the performance participant.

For example, in a music classroom or the like, a teacher has one master unit 1T1 to control the volume and tempo of the main melody of a performance data tune, and a plurality of students have child units 1T2 to 1Tn, respectively. The user can enjoy the performance in various modes, such as generating accompaniment sound, percussion sound, and the like having a volume and a tempo corresponding to the operation of the slave unit. In this case, the percussion instrument sound is not only set to drums, hooks, etc. by selecting the timbre, but if other types of sound sources such as natural wind and water sounds are prepared in advance, these sound sources can also be used as performance tracks. , It is possible to simultaneously produce the sound of a drum and a sound of a drum, the sound of a natural wind and water, etc. while the music is in progress. Therefore, it is possible to enjoy various performance forms that anyone can participate.

Further, the master unit 1T1 and the slave units 1T2-
In 1Tn, the LED is operated by operating the operation switch T6.
The light emitting device TL is always lit or the operation sensor MSa
Can be flickered in response to the detection output of. Thereby, the LED light swings or blinks from the baton in accordance with the progress of the music performance, so that not only the music performance but also a visual effect can be enjoyed.

[Control of Various Music Performances by Multiple Operators]
It is needless to say that the plurality of physical information detection transmitters 1T1 to 1Tn may have a configuration in which the master device 1T1 is eliminated (that is, all the slave devices are not distinguished from the master device and the slave devices). Absent. The simplest example is a case where a physical information detection transmitter is provided for each of two operators to control a music performance in a duet.
One physical information detection transmitter may be provided, or a plurality of physical information detection transmitters may be provided. For example, each operator has two baton-shaped motion sensors as shown in FIG. 4A, and each person divides the performance track (part) of the music into two parts, and further divides it into two parts by the left and right hand operations. The performance track (part) can be individually controlled by a total of four motion sensors.

Further, examples of controlling music performances by a plurality of operators include network music performances and music games between remote locations. For example, in different places such as a music classroom, participants watch music through a communication network and control music performances using physical information detection transmitters provided for each participant, and simultaneously participate in musical performances. can do. Also, in various events, a physical information detection transmitter can be provided for each of a plurality of event participants, and participation in music performance control can be realized by each physical information detection output.

As another example, at a concert hall, one or more performers perform main control of the music by controlling the tempo and dynamics of the music by the main control physical information detection transmitter, and perform a plurality of viewing and listening. By having a sub-control physical information detection transmitter and performing sub-control to insert and control a sound such as clapping according to light emitted by an LED or the like, it is possible to realize an effect of participating in music performance. . Furthermore, in the parade at the theme park, multiple parade participants must also control the performance parameters of the music with the main control, control the insertion of cheering sounds with the sub-control, and produce light with the light emitting device. Can be.

[Effects of the Invention] As described above, in the performance interface according to the present invention, when a performance participant moves the motion detector variously, the content of each motion detection signal of the motion detector [motion (gesture) information] Since it is configured to analyze various movements of the operator from and generate performance control information according to various analysis results, it is possible to variously control music performance according to various movements of the operator. it can. In another performance interface of the present invention, when the operator moves the motion detector, the motion of the operator is analyzed from the content of the motion detection signal of the motion detector, and at the same time, the state detection signal of the state detector is analyzed. Contents [Status information (biological information, physiological information)]
It is configured to analyze the physical condition of the operator from and generate performance control information according to these analysis results, so that it not only responds to the movement of the operator but also considers the physical condition of the operator In addition, the music performance can be controlled in various ways.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of an entire performance system including a performance interface according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration example of a physical information detection transmitter according to one embodiment of the present invention.

FIG. 3 is a block diagram showing a hardware configuration of a main body system according to one embodiment of the present invention.

FIG. 4 is a diagram showing an example of a body information detection mechanism usable for the performance interface of the present invention;
(1) shows an example of a baton-shaped hand-held type physical information detection transmitter, and FIG. 4 (2) shows an example of a shoe-shaped.

FIG. 5 is a diagram showing another configuration example of a physical information detection mechanism usable for the performance interface of the present invention.

FIG. 6 shows a format example of sensor data used in an embodiment of the present invention.

FIG. 7 is a functional block diagram of a system that uses a plurality of analysis outputs based on the output of one motion sensor (one-dimensional sensor) according to an embodiment of the present invention.

FIG. 8 is a diagram schematically illustrating an example of a trajectory of a hand movement and a waveform example of acceleration data at the time of a command operation of a one-dimensional acceleration sensor according to an embodiment of the present invention.

FIG. 9 is a diagram illustrating an example of a trajectory of a hand gesture and an acceleration output waveform of a sensor according to an embodiment of the present invention.

FIG. 10 is a diagram illustrating an example of a trajectory of a hand movement and an acceleration output waveform of a two-dimensional sensor according to an embodiment of the present invention.

FIG. 11 is a functional block diagram in the operation sensor and state sensor combined mode according to an embodiment of the present invention.

FIG. 12 is a functional block diagram in an ensemble mode according to an embodiment of the present invention.

[Explanation of symbols]

 TD LED or LCD display, TS power switch, TL LED emitter, IS physical information sensor, TTa signal processing and transmitter device: Mx, My, Mz; M1 to Mn operation detection signal, Se, Sb state detection signal, Dm Master machine motion detection data, De, Db status detection data, AM; MA, SA motion analysis block, AE, AB status (viewpoint, respiration) analysis block, PA, PB first and second data processing block, MP, AP Main Setting block and sub setting block.

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Masaki Sato 10-1 Nakazawa-cho, Hamamatsu-shi, Shizuoka F-term in Yamaha Corporation (reference) 5D378 KK25 KK34 MM13 MM14 MM22 MM47 MM51 MM62 MM65 MM67 MM68 MM72 MM92 MM93 QQ01 QQ08 QQ26 QQ35 SF01 SF02 SF11 SF15 TT17 TT19 TT22 TT31 UU31 UU42 XX27 XX45

Claims (8)

[Claims] 1. A performance interface for generating performance control information for controlling a tone generated by a tone generator in accordance with a physical condition of an operator. A motion detector that can be installed or possessed by an operator that outputs a detection signal; a physical condition analyzing unit that analyzes the operation of the operator based on a motion detection signal received from the motion detector; A performance control information generating means for generating performance control information corresponding to each of the plurality of operation analysis results. 2. A performance interface for generating performance control information for controlling a tone generated by a tone generator in accordance with a physical condition of an operator, wherein a plurality of types of movements based on an operation of the operator are detected. A motion detector that can be installed or carried by an operator that outputs a corresponding plurality of motion detection signals; and a body condition that analyzes the motion of the operator based on the plurality of motion detection signals received from the motion detector. A performance interface comprising: analysis means; and performance control information generation means for generating performance control information corresponding to each of the results based on the operation analysis results. 3. A performance interface for generating performance control information for controlling a tone generated from a tone generator in accordance with a physical condition of an operator, wherein a motion detection corresponding to a motion based on the motion of the operator. A motion detector that can be installed or possessed by an operator that outputs a signal, a state detector that can be installed or possessed by an operator that detects a physical condition of the operator and outputs a corresponding status detection signal, and a motion detector And body condition analysis means for analyzing the operation and body state of the operator based on the motion detection signal and the state detection signal received from the state detector, and based on the operation analysis result and the body state analysis result, respectively. A performance control information generating means for generating corresponding performance control information. 4. The apparatus according to claim 3, wherein said state detector detects at least one of pulse, body temperature, skin-to-skin resistance, electroencephalogram, respiration and eye movement, and outputs a corresponding state detection signal. The described playing interface. 5. The motion detector according to claim 1, wherein the motion detector is gripped by an operator by hand, or is attached to an operator's hand or foot. Performance interface described in. 6. The performance interface according to claim 1, wherein the performance control information controls a volume, a tempo, a timing, a timbre, an effect or a pitch of a musical tone. 7. The motion detector according to claim 1, wherein the motion detector is a one-dimensional sensor that detects a motion in a predetermined direction based on an operation of an operator and outputs a detection signal. The performance interface according to claim 1. 8. A two-dimensional sensor or three-dimensional sensor for detecting movement in two orthogonal directions or three directions based on an operation of an operator and outputting two or three types of detection signals.
The performance interface according to any one of claims 1 to 6, wherein the performance interface is a dimension sensor.