JP4665811B2 - Performance equipment - Google Patents

Description

  The present invention relates to a performance device.

  Of course, to enjoy playing a musical instrument, it is necessary to have the instrument body. To experience a plurality of types of musical instrument performance, it is necessary to prepare each type of instrument body.

  Since it is unrealistic for an individual user to prepare all musical instruments, for example, electronic musical instruments having a keyboard have various musical instrument tones. Further, for example, there is a virtual musical instrument performance system using a moving image or the like. In such a virtual musical instrument performance system, for example, a user's video shot by a video camera is combined with a virtual musical instrument video and displayed on the screen, and a user's action is detected and a musical tone is played in accordance with the detected movement. There is something to generate. (For example, see Patent Document 1)

JP 2001-041038 A

  An electronic musical instrument with ordinary multi-tone colors can enjoy the timbre of various musical instruments, but cannot experience the playing method of natural musical instruments.

  Moreover, in the conventional virtual musical instrument performance system as described above, although the virtual instrument performance can be visually enjoyed, it is only possible to experience the performance virtually on a small screen, so that the actual nature of the instrument form, touch, etc. You can't get a sense of playing an instrument.

  The objective of this invention is providing the performance apparatus which can experience the performance, even if there is no actual musical instrument.

  Another object of the present invention is to provide a performance device that allows the user to easily experience various types of natural musical instruments.

According to an aspect of the present invention, a performance device includes a storage unit that stores a plurality of musical instrument performance posture data and automatic performance data, a posture acquisition unit that acquires a posture of a user, the acquired posture, and the musical instrument performance. Posture collating means for comparing posture data, instrument type determining means for specifying a musical instrument type based on a comparison result by the posture collating means , and physical pressure on the user's body based on the automatic performance data To generate a musical tone based on the automatic performance data and tactile sensation imparting means for correcting the user's posture or movement so that the user's movement or posture is appropriate for the performance of the specified musical instrument type. And a musical sound generating means.

  ADVANTAGE OF THE INVENTION According to this invention, the performance apparatus which can experience the performance, even if there is no actual musical instrument can be provided.

  In addition, according to the present invention, it is possible to provide a performance device that can easily experience various types of natural musical instruments.

  FIG. 1 is a block diagram showing a hardware configuration of a performance apparatus 1 according to an embodiment of the present invention.

  A bus 6 of the performance device 1 includes a RAM 7, a ROM 8, a CPU 9, an external storage device 15, a detection circuit 11, a display circuit 13, a MIDI interface 16, a sound source circuit 18, an effect circuit 19, a communication interface (I / F) 21, an objective. A camera 30, a (user) viewpoint camera 35, a sensor interface 31, and a drive control circuit 41 are connected.

  The RAM 7 has a working area for the CPU 9 that stores a buffer area, flags, registers, various parameters, and the like.

  The ROM 8 can store various parameters and control programs, or a program for realizing the present embodiment. In this case, it is not necessary to store programs or the like in the external storage device 15 in an overlapping manner.

  The CPU 9 performs calculation or control according to a control program or the like stored in the ROM 8 or the external storage device 15. The timer 10 is connected to the CPU 9 and supplies a basic clock signal, interrupt processing timing, and the like to the CPU 9.

  The user can perform various inputs, settings, and selections using the setting operator 12 connected to the detection circuit 11. The setting operator 12 may be any device that can output a signal corresponding to a user input, such as a character input keyboard, mouse, switch, pad, fader, slider, rotary encoder, joystick, jog shuttle, and the like. . The setting operator 12 may be a soft switch or the like displayed on the display 14 that is operated using another operator such as a mouse.

  The display circuit 13 is connected to the display 14 and can display various information on the display 14. The user makes various inputs and settings with reference to information displayed on the display 14. The display 14 may be configured by connecting an external display device.

  The external storage device 15 stores performance data MD, a musical instrument performance posture database (DB) 151, a musical instrument performance operation database (DB) 152, and a video database (DB) 153 shown in FIG. The external storage device 15 includes an interface for an external storage device, and is connected to the bus 6 via the interface. The external storage device 15 is, for example, a flexible disk or floppy (registered trademark) disk drive (FDD), hard disk drive (HDD), magneto-optical disk (MO) drive, CD-ROM (compact disk-read only memory) drive, DVD ( Digital Versatile Disc) drive, semiconductor memory, and the like.

  When a hard disk drive (HDD) is connected as the external storage device 15, the control program or the program for realizing the present embodiment may be stored in the hard disk (HDD) in the external storage device 15. it can. By reading the control program or the like from the hard disk into the RAM 7, the CPU 9 can be operated in the same manner as when the control program or the like is stored in the ROM 8. In this way, it is possible to easily add or upgrade a control program or the like.

  When a CD-ROM drive is connected in addition to the hard disk drive, a control program or a program for realizing the present embodiment can be stored in the CD-ROM. A control program, a program for realizing the present embodiment, and the like can be copied from the CD-ROM to the hard disk. New installation and version upgrade of control programs and the like can be easily performed.

  The MIDI interface (MIDI I / F) 16 can be connected to a MIDI device 17, other musical instruments, audio devices, computers, and the like, and can transmit and receive at least MIDI signals. The MIDI interface 16 is not limited to a dedicated MIDI interface, and may be configured by using a general-purpose interface such as RS-232C, IEEE 1394 (eye triple 1394). In this case, data other than MIDI messages may be transmitted and received simultaneously. Note that the USB interface 30 may be used as the MIDI interface 16.

  The MIDI device 17 is an acoustic device, a musical instrument or the like connected to the MIDI interface 16. The form of the MIDI device 17 is not limited to a keyboard instrument, and may be a string instrument type, a wind instrument type, a percussion instrument type, or the like. In addition, the sound source device, the automatic performance device, and the like are not limited to those built in one electronic musical instrument body, but each is a separate device, and each device is connected using a communication method such as MIDI or various networks. May be.

  The tone generator circuit 18 is an accompaniment pattern data recorded in the external storage device 15, ROM 8, RAM 7, etc., performance data or performance signals supplied from the performance operator 22, MIDI equipment 17 connected to the MIDI interface 16, etc. A musical sound signal is generated according to the above and supplied to the sound system 20 via the effect circuit 19.

  The effect circuit 19 gives various musical effects to the musical sound signal supplied from the sound source circuit 18. The sound system 20 includes a D / A converter and a speaker, converts a digital musical tone signal supplied to an analog format and generates a sound.

  The communication interface 21 can be connected to a communication network 3 such as a LAN (local area network) and the Internet, and can be connected to the server 2 and other electronic musical instruments via the communication network 3.

  The communication interface 21 and the communication network 3 are not limited to wired ones and may be wireless. Moreover, you may provide both. The communication interface 21 may be a built-in one or a detachable one such as a PC card.

  The objective camera 30 is an imaging device for capturing a moving image for acquiring (capturing) the posture and motion of the user. In this specification, “posture” includes not only the posture of the user's body but also the posture of the finger.

  The viewpoint camera 35 is an imaging device for capturing a moving image for acquiring (capturing) a video viewed from the user's viewpoint. The viewpoint camera 35 acquires an image to be used together with the image by the objective camera 30 in order to align the virtual space displayed on the display (HMD) 14 and the real space.

The sensor I / F 31 is an interface for connecting a plurality of sensors 32. The sensor 32 is a sensor group that is attached to the user's body and acquires (detects) the posture and motion of the user. For example, any or all of a pressure sensor, an acceleration sensor, a position sensor, a direction sensor, and the like. Composed of a combination.
The drive control circuit 41 receives instructions from the CPU 9 and controls the plurality of drive units 42. The drive unit (tactile sensation imparting device) 42 applies a forcing force to an actuator that gives a tactile sensation by applying pressure or the like to the user's body and a movable part such as a user's joint, and moves the user's body. Or a device having a haptic function for forcing a user to perform a specific action. Moreover, you may add as the drive part 42 what gives heat (including heating and cooling) as one of tactile sensations.

  For example, when the user operates the performance operator of the virtual musical instrument and the user's finger is positioned at the spatial coordinates determined to have touched the performance operator, by applying pressure to the user's finger, When a touch is given to the musical instrument and the operator is pressed, the pressure is gradually increased and the movement of the user's finger is restricted so that the weight of the operator is experienced. Further, when the performance operator is pressed down to the limit, further movement of the user is restricted. Further, the movement of the user is forcibly stopped with respect to the housing of the virtual musical instrument.

  Note that the user's movement is restricted and forced, for example, by using two or more wires, placing one on the inside of the joint and the other on the outside of the joint, and tensioning and relaxing them.

  FIG. 2 is a schematic diagram illustrating a usage example of the posture acquisition device 33 and the tactile sensation imparting device 42 according to the embodiment of the present invention.

  As shown in the drawing, each sensor 32 and the drive unit 42 are mounted so as to cover the joint part and its periphery corresponding to the joint part of the user's body. The sensor 32 is provided with a plurality of types of sensors corresponding to one joint. Also, the drive unit 42 is arranged corresponding to one joint so as to apply pressure and to restrict / force the operation. Further, the drive unit 42 that applies pressure and heat is not limited to the joint and its surroundings, but is preferably mounted on a body part that is considered to be touched by the user on the virtual musical instrument, particularly preferably on the fingertip.

  In addition, since it is necessary to attach a large number of sensors 32 and driving units 42 to the user's hand, in this embodiment, a glove-like attachment 43 including a large number of sensors 32 and driving units 42 is provided. Wearing.

  The objective camera 30 is installed in a place where the user's upper body or whole body can be photographed. In order to easily determine the user's posture based on the video captured by the camera 30, for example, an identification device may be attached to each joint portion of the user.

  The HMD (head mounted display) 14 is a display device that is worn in the user's field of view, and displays a synthesized virtual space including the virtual musical instrument and the user's body that operates the virtual musical instrument. For example, an image of a natural musical instrument (in this example, a trumpet) as shown in FIG. 2B or an image of an electronic musical instrument as shown in FIG. The The background may also be displayed. In this case, the background may be an actual background photographed by the camera 30 or the camera 35 or an image of another place.

  Moreover, in order to switch the image displayed on the HMD 14 according to the user's line of sight, it is preferable to attach a sensor 32 (for example, a direction sensor or a position sensor) to the HMD 14. In FIG. 2A, only the trumpet image is taken as an example, but natural instruments displayed on the HMD 14 are wind instruments such as horn, trombone, clarinet, flute, saxophone, violin, cello, guitar, mandolin. String instruments such as piano, organ, harpsichord and other keyboard instruments (including percussion instruments), drum sets, percussion instruments such as Latin percussion, etc. may be used.

  Furthermore, in order to increase the accuracy of the alignment of the virtual space image displayed on the HMD 14 and the real space, it is preferable to attach a viewpoint camera 35 that captures the image from the user's viewpoint to the HMD 14.

  The headphone 20 is a sound system for generating a musical sound when a user plays a virtual musical instrument. The musical sound generated from the headphones 20 is subjected to an acoustic effect applying HRTF (head transfer function) theory to change the musical sound according to the position of the virtual instrument, the direction of the user, and the like. It is preferable to make it.

  Note that the posture acquisition device 33 and the tactile sensation imparting device 42 are not limited to the example of wearing in FIG. 2, and can acquire the user's posture and motion. Any form can be used as long as it can be used. In the example of FIG. 2, only the upper body is shown, but it may be applied to the whole body. Furthermore, you may make it mount | wear with combining some apparatuses, such as mounting | wearing only the glove-shaped attachment 43. FIG.

  FIG. 3 is a functional block diagram of the performance device 1 according to the embodiment of the present invention.

  The external storage device 15 stores a plurality of performance data MD, a musical instrument performance posture database (DB) 151, a musical instrument performance operation database (DB) 152, and a video database (DB) 153.

  The performance data MD is, for example, automatic performance data such as SMF format MIDI data.

  The musical instrument performance posture database (DB) 151 is a database that stores the posture of the performer (musical instrument performance posture data) when playing various musical instruments. The musical instrument performance posture data is data for discriminating the type of musical instrument that the user is trying to perform as compared with the video obtained by photographing the user's posture from the camera 30, and the general performance posture for each musical instrument type. , Data stored in a form comparable to the video from the camera 30. For example, general performance postures (such as a violin playing posture and a trumpet blowing posture) by players of various musical instruments are stored as comparative performance posture images. Moreover, you may make it memorize | store the sensor output value of the sensor group 32 assumed when performing various musical instruments.

  The musical instrument performance operation database (DB) 152 stores a plurality of player attitudes (musical instrument performance operation data) at the time of performance operation of each musical instrument corresponding to the musical instrument type stored in the musical instrument performance attitude database (DB) 151. Database. The musical instrument performance operation data is compared with the sensor output from the sensor group 32 and the user's performance operation image from the camera 30 to detect the performance operation intended by the user, and in the automatic performance mode described later, the drive unit 42 On the other hand, it is data for controlling the user to take an ideal performance operation posture. For example, the sensor output value of the sensor group 32 assumed when a predetermined performance operation is performed on a predetermined musical instrument is stored, and a general player's posture when the performance operation is performed is stored as an image. .

The video database (DB) 153 is a database that stores various videos to be displayed on the HMD 14. The images stored in the database include at least actual images or computer graphic images from various angles of the same musical instrument type stored in the musical instrument performance posture database (DB) 151 and the musical instrument performance operation database (DB) 152. including. It is also possible to store an image of the performer's body (particularly the fingers) to be displayed in combination with the instrument image. In addition, a background video can be stored. Note that an image related to one musical instrument may be stored for one instrument type, or an image of a plurality of musical instruments may be stored. In addition, not only the musical instruments that actually exist, but also images of old musical instruments and imaginary musical instruments that do not exist may be stored. In that case, it is necessary to store data relating to the musical instrument in the musical instrument performance posture database (DB) 151 and the musical instrument performance operation database (DB).
The posture collation unit 92 acquires the posture image of the user photographed by the camera 30, and whether or not the musical instrument performance posture data that matches or approximates the acquired video is stored in the musical instrument performance posture database (DB) 151. Determine whether.

  The instrument type determination unit 91 specifies the instrument type based on the collation (judgment) result in the posture collation unit 92. For example, if there is data that matches or approximates the acquired video in the posture collation unit 92, the posture detection unit 90 identifies the musical instrument that the user intends to play, assuming that the same musical instrument as the data is played. Instruct the instrument type. At the same time, the tone color of the musical tone generated by the musical tone generator 180 is determined, and the type of instrument video used by the video generator 130 is specified. The specification of the musical instrument type may be determined in consideration of the sensor output from the sensor group 32 in addition to the video from the camera 30. Further, it may be determined only from the sensor output from the sensor group 32. In that case, sensor output values assumed when playing various musical instruments are stored in advance as musical instrument performance attitude data.

  The posture detection unit 90 acquires the sensor output from the sensor group 32 and the user's performance operation video from the camera 30, and based on the instrument type determined by the instrument type determination unit 91, the acquired sensor from the sensor group 32 It is determined whether or not musical instrument performance operation data that matches or approximates the output and the user's performance operation video from the camera 30 is stored in the musical instrument performance operation database (DB) 152. If there is data that matches or approximates, the performance operation intended by the user is identified as performing the same performance operation as that data. The specified performance operation is sent to the drive control unit 910, the tone generation unit 180, and the video generation unit 130 as performance operation information together with the instrument type information determined by the instrument type determination unit 91. In an automatic performance mode to be described later, an ideal performance posture is calculated from the performance data MD and output as performance operation information.

  The drive control unit 910 controls the drive unit 42 based on the instrument type information determined by the instrument type determination unit 91 and the performance operation information determined by the posture detection unit 90. For example, when the user's finger is positioned at a spatial coordinate determined that the user has touched the performance operator, the pressure is applied to the user's finger to control the instrument so that it feels tactile. When the operator is pressed, the pressure is gradually increased and the movement of the user's finger is restricted to control the weight of the operator. Further, when the performance operator is pressed down to the limit, control is performed so as to restrict further user movement, and for the virtual musical instrument housing, control is performed so as to forcibly stop the user movement. . In the automatic performance mode described later, each drive unit 42 is controlled so that the user takes an ideal performance posture.

  The musical tone generation unit 180 uses the tone color specified in the musical instrument type information determined by the musical instrument type determination unit 91, and the musical tone (predetermined when the musical instrument is performed in the performance operation information determined by the posture detection unit 90). (Music tone with a predetermined effect applied) with a predetermined pitch and a predetermined pitch. In an automatic performance mode, which will be described later, a musical tone is generated based on performance data. The performance data may be reproduced at the original tempo, or automatic performance may be performed by determining the reproduction timing using a user's performance operation as a trigger. The musical sound generated here is generated from the headphones 30, for example.

  The video generation unit 130 uses a musical instrument specified in the musical instrument type information determined by the musical instrument type determination unit 91 to display a video when a performance operation specified in the performance operation information determined by the posture detection unit 90 is performed in a video database. It is generated using 153 images. At this time, using the images acquired by the objective camera 3 and the viewpoint camera 35, the generated virtual space image and the real space are aligned. The generated video is presented to the user by the HMD 14.

  FIG. 4 is a flowchart showing performance processing according to the embodiment of the present invention. It is assumed that automatic performance data is selected in advance as necessary before the start of this process. In addition, it is assumed that the user can set the instrument type specified in step SA7, which will be described later, as an initial setting as necessary.

  In step SA1, processing is started. In step SA2, various flags, registers, buffers, etc. are initialized.

  In step SA3, the posture image of the user is acquired from the camera 30 (and the camera 35) in FIG. In step SA4, the camera image acquired in step SA3 and the performance attitude data are compared and verified. That is, musical instrument performance posture data that matches or approximates the acquired video is extracted from the musical instrument performance posture database (DB) 151. If none of the images stored in the musical instrument performance posture database (DB) 151 matches or approximates, the process returns to step SA3 to acquire a camera image again.

  In step SA5, the sensor output from the sensor group 32 of FIG. 1 is acquired. In step SA6, musical instrument performance posture data that matches or approximates the sensor output acquired in step SA5 is extracted from the musical instrument performance posture database (DB) 151. If none of the sensor output values stored in the musical instrument performance posture database (DB) 151 matches or approximates, the process returns to step SA5 to acquire the sensor output value again, or returns to step SA3 and captures the camera image again. It is also possible to acquire and repeat the subsequent processing.

  In step SA7, the musical instrument that the user intends to play is specified based on the musical instrument performance posture data extracted in steps SA4 and SA6. If the musical instrument performance attitude data extracted in step SA4 and step SA6 do not match, either one of the preset results may be given priority, or the same instrument type currently set may be given priority. You may do it.

  In step SA8, it is determined whether or not the instrument specified in step SA7 is the same as the currently set instrument. If they are the same, the process proceeds to step SA10 indicated by a YES arrow. If they are not the same, that is, if it is determined that the user has newly taken a posture corresponding to the performance of a different instrument, the step indicated by the NO arrow Proceeding to SA9, a timbre is set based on the musical instrument type specified in step SA7.

  In step SA10, it is determined whether or not the current mode is the automatic mode. If it is in the automatic mode, the process proceeds to step SA13 indicated by an arrow of YES, and if it is not the automatic mode, the process proceeds to step SA11 indicated by an arrow of NO. The automatic mode is for generating a musical sound based on automatic performance data and correcting or restricting the posture and operation of the user.

  In step SA11, assuming the shape of the musical instrument type specified in step SA7, the drive unit 42 is controlled so as to restrict the user's movement and posture.

  In step SA12, a tone generation parameter such as a pitch value is determined based on the camera image acquired in step SA3 and the sensor output acquired in step SA5. For example, musical instrument performance operation data that matches or approximates the acquired sensor output from the sensor group 32 and the user's performance operation video from the camera 30 is extracted from the musical instrument performance operation database (DB) 152 and the performance represented by the data is displayed. A tone generation parameter corresponding to the operation is generated. Note that if there is no musical instrument performance operation data that matches or approximates the acquired sensor output from the sensor group 32 and the user's performance operation video from the camera 30, it is considered that there was no performance operation. Thereafter, the process proceeds to Step SA16.

In step SA13, automatic performance data stored in the external storage device 15 is read. Note that the automatic performance data is selected in advance from a plurality of automatic performance data. If there are a plurality of parts (tracks) in the automatic performance data, the part corresponding to the musical instrument type specified in step SA7 is set to be played by the user, or the part to be played is selected in advance by the user. Also, the part played by the user can be prevented from being pronounced by automatic performance or can be pronounced. Also, so-called any key play can be set. In this case, the user's performance operation is determined as determining the timing for proceeding with the automatic performance.
In step SA14, assuming the shape of the instrument type specified in step SA7, based on the automatic performance data read in step SA13, the user's posture and movement are set so that the user's movement and posture are appropriate for the musical instrument performance. The drive unit 42 is controlled to correct the above.

  In step SA15, a musical tone generation parameter such as a pitch value is determined based on the automatic performance data read in step SA13.

  In step SA16, sound generation processing is performed based on the musical tone generation parameters determined in step SA12 or step SA15.

  In step SA17, based on the instrument type determined in step SA7 and the performance operation determined in step SA12 (in the automatic mode, the performance operation determined based on the automatic performance data), the virtual instrument, the body (finger), the background image, etc. Is generated and displayed on the HMD 14. At this time, an image assuming the user's viewpoint may be generated according to the sensor outputs of the direction sensor and the position sensor provided in the HMD 14. Further, based on the video acquired by the objective camera 30 and the viewpoint camera 35, the virtual space video displayed on the HMD 14 and the real space are aligned. By this alignment, the user can put out a hand from the real space with respect to an object (virtual instrument or the like) in the virtual space.

  In step SA18, an end instruction by the user is detected. If an end instruction is detected, the process proceeds to step SA19 indicated by the YES arrow, and the performance process ends. If the end instruction is not detected, the process returns to step SA3 indicated by the NO arrow, and the subsequent processing is repeated.

  As described above, according to the embodiment of the present invention, the video of the virtual musical instrument can be displayed in the user's field of view and the tactile sensation of the virtual musical instrument can be given to the user. In addition, by restricting the user's operation with respect to the housing of the virtual musical instrument, the user can virtually experience the musical instrument performance operation.

  Further, according to the embodiment of the present invention, it is possible to acquire the user's posture, specify the musical instrument type based on the acquired posture, and specify the tone color in the tone generation. Therefore, it is possible to easily switch (switch) the virtual musical instrument and switch the timbre.

  Furthermore, according to the embodiment of the present invention, a musical sound can be generated based on the automatic performance data, and a user can perform a performance operation suitable for performance according to the automatic performance data. Therefore, the user who does not know the performance operation method can let the user experience the performance operation of the musical instrument.

  Although the present invention has been described with reference to the embodiments, the present invention is not limited thereto. It will be apparent to those skilled in the art that various modifications, improvements, combinations, and the like can be made.

It is a block diagram showing the hardware constitutions of the performance apparatus 1 by the Example of this invention. It is the schematic showing the usage example of the attitude | position acquisition apparatus 33 and the tactile sensation provision apparatus 42 by the Example of this invention. It is a functional block diagram of the performance apparatus 1 by the Example of this invention. It is a flowchart showing the performance process by the Example of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Electronic music apparatus, 2 ... Server, 3 ... Communication network, 6 ... Bus, 7 ... RAM, 8 ... ROM, 9 ... CPU, 10 ... Timer, 11 ... Detection circuit, 12 ... Setting operation element, 13 ... Display circuit , 14 ... Display (HMD), 15 ... External storage device, 16 ... MIDI I / F, 17 ... MIDI device, 18 ... Sound source circuit, 19 ... Effect circuit, 20 ... Sound system (headphone), 21 ... Communication I / F , 22 ... Performance operator, 30 ... Camera, 31 ... Sensor I / F, 32 ... Sensor, 33 ... Attitude acquisition device, 41 ... Drive control circuit, 42 ... Drive unit (actuator), 43 ... Attachment, 90 ... Attitude detection 91: Musical instrument type determination unit, 92 ... Posture collation unit, 130 ... Video generation unit, 151 ... Musical instrument performance posture DB, 152 ... Musical instrument performance operation DB, 153 ... Video DB, 180 ... Music generation , 410 ... drive control unit

Claims (2)

Storage means for storing a plurality of musical instrument performance posture data and automatic performance data;
Posture acquisition means for acquiring the posture of the user;
Posture collation means for comparing the acquired posture and the musical instrument performance posture data;
Instrument type determination means for specifying the instrument type based on the comparison result by the posture verification means;
Based on the automatic performance data, by applying physical pressure to the user's body, the user's posture or movement is made suitable for the performance of the specified musical instrument type. A tactile sensation imparting means for correcting
A musical performance device comprising musical tone generation means for generating musical sounds based on the automatic performance data . A program executed by a computer having storage means for storing a plurality of musical instrument performance posture data and automatic performance data,
A posture acquisition procedure for acquiring the user's posture;
A posture verification procedure for comparing the acquired posture and the musical instrument performance posture data;
An instrument type determination procedure for specifying an instrument type based on a comparison result by the posture verification procedure;
Based on the automatic performance data, by applying physical pressure to the user's body, the user's posture or movement is made suitable for the performance of the specified musical instrument type. Tactile sensation imparting procedure to correct ,
A program for causing a computer to execute a performance process having a musical sound generation procedure for generating a musical sound based on the automatic performance data .

Images