JP2013195582A - Performance device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a performance device capable of changing layout information such as a layout of a virtual music instrument set, according to a degree of proficiency of a player.SOLUTION: A CPU designates an area, in virtual pads, to be shot with a stick part at shot timing; detects position coordinates of the stick part at the shot timing; when the position detected coordinates belong to either of the virtual pads, instructs production of musical sound with corresponding tone color; calculates a distribution degree of the position coordinates of the stick part in the designated pad at the shot timing, and sets the layout of the designated virtual pad according to the calculated distribution degree.

Description

  The present invention relates to a performance device and a program.

2. Description of the Related Art Conventionally, there has been proposed a performance device that generates an electronic sound corresponding to a performance operation when a performance performance of the performer is detected. For example, a performance device (air drum) that produces percussion instrument sounds only with members on a stick is known. In this performance device, when a performer performs a performance operation such as hitting a drum, such as holding and shaking a stick-like member containing a sensor by hand, the sensor detects the performance operation, and percussion instrument sounds are generated. Pronounced.
According to such a performance device, since the musical sound of the musical instrument can be generated without the need for an actual musical instrument, the performer can enjoy the performance without being restricted by the performance place or performance space. .

  As such a performance device, for example, in Patent Document 1, a performance action using a stick-like member of a performer is imaged, and an image of the performance action and a virtual image showing a musical instrument set are synthesized. There has been proposed a musical instrument game apparatus configured to display a composite image on a monitor and generate a predetermined musical sound in accordance with positional information between a stick-shaped member and a virtual musical instrument set.

Japanese Patent No. 3599115

  However, when the musical instrument game apparatus described in Patent Document 1 is applied as it is, layout information such as the placement of a virtual musical instrument set is predetermined, so that the layout information is changed according to the skill level of the performer. I couldn't.

  The present invention has been made in view of such a situation, and an object of the present invention is to provide a performance device capable of changing layout information such as the placement of a virtual musical instrument set according to the skill level of the performer. .

In order to achieve the above object, a performance device according to one aspect of the present invention includes:
A performance member that the performer can hold;
An imaging device that captures a captured image of the performance member as a subject and detects position coordinates of the performance member on the captured image plane;
Storage means for storing layout information defining an area arranged on the captured image plane;
Specific operation position detecting means for detecting position coordinates of the performance member on the captured image plane at the timing when the specific performance operation is performed;
Discriminating means for discriminating whether or not the position coordinates of the performance member detected by the specific operation position detecting means belong to an area arranged based on the layout information;
A sounding instruction means for instructing the sound generation of the musical sound corresponding to the area when the determination means determines that the sound belongs to the area;
Distribution calculating means for calculating a distribution degree in the region of the position coordinates of the detected performance member;
Updating means for updating layout information stored in the storage means in order to change the region according to the distribution degree calculated by the distribution calculating means;
It is provided with.

  According to the present invention, layout information such as the placement of a virtual musical instrument set can be changed according to the skill level of the performer.

It is a figure which shows the outline | summary of one Embodiment of the performance apparatus of this invention. It is a block diagram which shows the hardware constitutions of the stick part which comprises the said performance apparatus. It is a perspective view of the said stick part. It is a block diagram which shows the hardware constitutions of the camera unit part which comprises the said performance apparatus. It is a block diagram which shows the hardware constitutions of the center unit part which comprises the said performance apparatus. It is a figure which shows the set layout information which concerns on one Embodiment of the performance apparatus of this invention. It is the figure which visualized on the virtual plane the concept which the said set layout information shows. It is a flowchart which shows the flow of a process of the said stick part. It is a flowchart which shows the flow of a process of the said camera unit part. It is a flowchart which shows the flow of a process of the said center unit part. It is a flowchart which shows the flow of the level determination process of the said center unit part. It is a figure which shows the example of a setting of a virtual pad. It is a figure which shows the example of a setting of a virtual pad. It is a figure which shows the set layout information which concerns on the said embodiment. It is a figure which shows the specific example of a level alert | report.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Outline of the performance device 1]
First, with reference to FIG. 1, the outline | summary of the performance apparatus 1 as one Embodiment of this invention is demonstrated.
As shown in FIG. 1 (1), the performance device 1 of the present embodiment includes stick units 10R and 10L, a camera unit unit 20, and a center unit unit 30. In the present embodiment, the stick portion 10R is hereinafter appropriately referred to as a “right hand stick portion 10R”, and the stick portion 10L is hereinafter appropriately referred to as a “left hand stick portion 10L”. The performance device 1 of the present embodiment is provided with two stick portions 10R and 10L in order to realize a virtual drum performance using two sticks, but the number of stick portions is not limited to this. There may be one, or three or more. In the following, when it is not necessary to distinguish the right-hand stick portion 10R and the left-hand stick portion 10L from each other, both are collectively referred to as the “stick portion 10”.

The stick portion 10 is a stick-like performance member extending in the longitudinal direction. The performer performs an operation of holding up one end (the base side) of the stick unit 10 and swinging it up or down around the wrist or the like as a performance operation. Various sensors such as an acceleration sensor and an angular velocity sensor are provided at the other end (front end side) of the stick unit 10 in order to detect such performance performance of the performer (motion sensor unit 14 described later). The stick unit 10 transmits a note-on event to the center unit unit 30 based on the performance operation detected by these various sensors.
A marker unit 15 (see FIG. 2), which will be described later, is provided on the distal end side of the stick unit 10, and the camera unit unit 20 is configured to be able to determine the distal end of the stick unit 10 during imaging.

  The camera unit 20 is configured as an optical imaging device, and a space (hereinafter referred to as “imaging space”) including a player who performs a performance operation while holding the stick unit 10 as a subject (hereinafter referred to as an “imaging space”) at a predetermined frame rate. The image is captured and output as moving image data. The camera unit unit 20 identifies the position coordinates of the marker unit 15 that is emitting light in the imaging space, and transmits data indicating the position coordinates (hereinafter referred to as “position coordinate data”) to the center unit unit 30.

  When the center unit unit 30 receives a note-on event from the stick unit 10, the center unit unit 30 generates a predetermined tone according to the position coordinate data of the marker unit 15 at the time of reception. Specifically, the center unit 30 stores the position coordinate data of the virtual drum set D shown in FIG. 1 (2) in association with the imaging space of the camera unit 20, and the virtual drum set D Based on the position coordinate data and the position coordinate data of the marker unit 15 at the time of receiving the note-on event, the instrument that is virtually hit by the stick unit 10 is specified, and a musical tone corresponding to the instrument is generated.

  Next, the configuration of the performance device 1 of the present embodiment will be specifically described.

[Configuration of the performance device 1]
First, with reference to FIG. 2 to FIG. 5, each component of the performance device 1 of the present embodiment, specifically, the configuration of the stick unit 10, the camera unit unit 20, and the center unit unit 30 will be described.

[Configuration of Stick Unit 10]
FIG. 2 is a block diagram illustrating a hardware configuration of the stick unit 10.
As shown in FIG. 2, the stick unit 10 includes a CPU 11 (Central Processing Unit), a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a motion sensor unit 14, a marker unit 15, and data. A communication unit 16 and a switch operation detection circuit 17 are included.

  The CPU 11 controls the stick unit 10 as a whole. For example, based on the sensor value output from the motion sensor unit 14, in addition to detecting the posture of the stick unit 10, shot detection, and action detection, the marker unit 15 emits light.・ Execute control such as turning off lights. At this time, the CPU 11 reads the marker feature information from the ROM 12, and executes light emission control of the marker unit 15 according to the marker feature information. Further, the CPU 11 executes communication control with the center unit unit 30 via the data communication unit 16.

The ROM 12 stores a processing program for executing various processes by the CPU 11. The ROM 12 stores marker feature information used for light emission control of the marker unit 15. Here, the camera unit 20 is appropriately referred to as a marker portion 15 (hereinafter referred to as “right marker” as appropriate) of the right hand stick portion 10R and a marker portion 15 (hereinafter referred to as “left marker” as appropriate) of the left hand stick portion 10L. ) Must be distinguished. The marker feature information is information for the camera unit 20 to distinguish between the right marker and the left marker. For example, in addition to the shape, size, hue, saturation, or luminance at the time of light emission, blinking at the time of light emission. Speed can be used.
The CPU 11 of the right-hand stick unit 10R and the CPU 11 of the left-hand stick unit 10L read different marker characteristic information and execute light emission control of each marker.

  The RAM 13 stores values acquired or generated in the process, such as various sensor values output by the motion sensor unit 14.

  The motion sensor unit 14 is a variety of sensors for detecting the state of the stick unit 10 and outputs a predetermined sensor value. Here, as a sensor which comprises the motion sensor part 14, an acceleration sensor, an angular velocity sensor, a magnetic sensor, etc. can be used, for example.

FIG. 3 is a perspective view of the stick unit 10, and a switch unit 171 and a marker unit 15 are disposed outside.
The performer holds the one end (base side) of the stick unit 10 and performs a swing-down operation centering on the wrist or the like, thereby causing the stick unit 10 to move. At this time, a sensor value corresponding to this motion is output from the motion sensor unit 14.

  CPU11 which received the sensor value from the motion sensor part 14 detects the state of the stick part 10 which a player has. As an example, the CPU 11 detects the timing of hitting a virtual musical instrument by the stick unit 10 (hereinafter also referred to as “shot timing”). The shot timing is a timing immediately after the stick unit 10 is swung down and immediately before it is stopped, and is a timing at which the magnitude of acceleration in the direction opposite to the swing-down direction applied to the stick unit 10 exceeds a certain threshold.

  Returning to FIG. 2, the marker unit 15 is a light emitting body provided on the tip side of the stick unit 10, and is composed of, for example, an LED, and emits light and extinguishes according to control from the CPU 11. Specifically, the marker unit 15 emits light based on the marker feature information read from the ROM 12 by the CPU 11. At this time, since the marker feature information of the right-hand stick unit 10R and the marker feature information of the left-hand stick unit 10L are different, the camera unit 20 uses the position coordinates of the marker unit (right marker) of the right-hand stick unit 10R. The position coordinates of the marker portion (left marker) of the left hand stick portion 10L can be individually distinguished and acquired.

  The data communication unit 16 performs predetermined wireless communication with at least the center unit unit 30. The predetermined wireless communication may be performed by any method, and in the present embodiment, wireless communication with the center unit unit 30 is performed by infrared communication. The data communication unit 16 may perform wireless communication with the camera unit unit 20, or may perform wireless communication with the right hand stick unit 10R and the left hand stick unit 10L. .

  The switch operation detection circuit 17 is connected to the switch 171 and receives input information via the switch 171.

[Configuration of Camera Unit 20]
This completes the description of the configuration of the stick unit 10. Next, the configuration of the camera unit unit 20 will be described with reference to FIG.
FIG. 4 is a block diagram illustrating a hardware configuration of the camera unit unit 20.
The camera unit unit 20 includes a CPU 21, a ROM 22, a RAM 23, an image sensor unit 24, and a data communication unit 25.

  The CPU 21 controls the entire camera unit 20 and, for example, based on the position coordinate data and marker feature information of the marker unit 15 detected by the image sensor unit 24, the right-hand stick unit 10R and the left-hand stick unit 10L. Control is performed to calculate each position coordinate of the marker unit 15 (right marker and left marker) and output position coordinate data indicating each calculation result. Further, the CPU 21 executes communication control for transmitting the calculated position coordinate data and the like to the center unit unit 30 via the data communication unit 25.

  The ROM 22 stores a processing program for executing various processes by the CPU 21. The RAM 23 stores values acquired or generated in the process, such as the position coordinate data of the marker unit 15 detected by the image sensor unit 24. The RAM 23 also stores the marker feature information of each of the right-hand stick unit 10R and the left-hand stick unit 10L received from the center unit 30.

  The image sensor unit 24 is, for example, an optical camera, and captures a moving image of a performer who performs a performance operation with the stick unit 10 at a predetermined frame rate. Further, the image sensor unit 24 outputs imaging data for each frame to the CPU 21. Note that the specification of the position coordinates of the marker unit 15 of the stick unit 10 in the captured image may be performed by the image sensor unit 24 or the CPU 21. Similarly, the image sensor unit 24 may specify the marker characteristic information of the imaged marker unit 15 or may be specified by the CPU 21.

  The data communication unit 25 performs predetermined wireless communication (for example, infrared communication) with at least the center unit unit 30. Note that the data communication unit 16 may perform wireless communication with the stick unit 10.

[Configuration of Center Unit 30]
This completes the description of the configuration of the camera unit section 20. Then, with reference to FIG. 5, the structure of the center unit part 30 is demonstrated.
FIG. 5 is a block diagram illustrating a hardware configuration of the center unit 30.
The center unit unit 30 includes a CPU 31, a ROM 32, a RAM 33, a switch operation detection circuit 34, a display circuit 35, a sound source device 36, and a data communication unit 37.

  The CPU 31 executes control of the entire center unit 30, for example, control for generating predetermined musical sounds based on the shot detection received from the stick unit 10 and the position coordinates of the marker unit 15 received from the camera unit 20. Execute. Further, the CPU 31 executes communication control between the stick unit 10 and the camera unit unit 20 via the data communication unit 37.

  The ROM 32 stores processing programs for various processes executed by the CPU 31. The ROM 32 stores waveform data of various tones, for example, wind instruments such as flutes, saxophones, and trumpets, keyboard instruments such as pianos, stringed instruments such as guitars, bass drums, hi-hats, snares, cymbals, and tomographs. Timbre data) is stored in association with position coordinates and the like.

  For example, as shown in FIG. 6 as set layout information, the set layout information includes n pieces of pad information from the first pad to the nth pad, and each pad is further stored. Corresponds to the presence / absence of a pad (presence / absence of a virtual pad in a virtual plane described later), position (position coordinates in a virtual plane described later), size (virtual pad shape and diameter, etc.), tone color (waveform data), etc. Is stored.

Here, a specific set layout will be described with reference to FIG. FIG. 7 is a diagram in which the concept indicated by the set layout information (see FIG. 6) stored in the ROM 32 of the center unit 30 is visualized on a virtual plane.
FIG. 7 shows a state in which four virtual pads 81, 82, 83, and 84 are arranged on a virtual plane. The virtual pads 81, 82, 83, and 84 include the first pad to the nth pad. Among them, the pad presence / absence data corresponding to “with pad” is supported. For example, four pads of the second pad, the third pad, the fifth pad, and the sixth pad correspond to each other. Further, virtual pads 81, 82, 83, 84 are arranged based on the position data and the size data. Furthermore, timbre data is associated with each virtual pad 81. Therefore, when the position coordinates of the marker unit 15 at the time of shot detection belong to the area corresponding to the virtual pads 81, 82, 83, 84, the timbre corresponding to the virtual pads 81, 82, 83, 84 is generated.
The CPU 31 may display the virtual plane on the display device 351 described later together with the virtual pad 81 arrangement.
In the present embodiment, the position coordinates on the virtual plane coincide with the position coordinates on the captured image of the camera unit unit 20.

Returning to FIG. 5, the RAM 33 receives the state of the stick unit 10 received from the stick unit 10 (shot detection, etc.), the position coordinates of the marker unit 15 received from the camera unit unit 20, and the set layout read from the ROM 32. Stores values obtained or generated in the process, such as information.
The CPU 31 reads the timbre data (waveform data) corresponding to the virtual pad 81 in the area to which the position coordinates of the marker unit 15 belong at the time of shot detection (that is, when a note-on event is received) from the set layout information stored in the RAM 33. Thus, a musical sound corresponding to the performance performance of the performer is generated.

The switch operation detection circuit 34 is connected to the switch 341 and receives input information via the switch 341. The input information includes, for example, a change in the volume of the tone to be generated and a tone color of the tone to be generated, setting and changing the set layout number, and switching the display on the display device 351.
The display circuit 35 is connected to the display device 351 and executes display control of the display device 351.

The tone generator 36 reads waveform data from the ROM 32 in accordance with an instruction from the CPU 31 to generate musical tone data, converts the musical tone data into an analog signal, and generates a musical tone from a speaker (not shown).
The data communication unit 37 performs predetermined wireless communication (for example, infrared communication) between the stick unit 10 and the camera unit unit 20.

[Processing of the performance device 1]
In the above, the structure of the stick part 10, the camera unit part 20, and the center unit part 30 which comprises the performance apparatus 1 was demonstrated. Next, processing of the performance device 1 will be described with reference to FIGS.

[Processing of the stick unit 10]
FIG. 8 is a flowchart showing a flow of processing executed by the stick unit 10 (hereinafter referred to as “stick unit processing”).
Referring to FIG. 8, CPU 11 of stick unit 10 reads motion sensor information from motion sensor unit 14, that is, sensor values output by various sensors, and stores them in RAM 13 (step S1). Thereafter, the CPU 11 executes a posture detection process of the stick unit 10 based on the read motion sensor information (step S2). In the posture detection process, the CPU 11 calculates the posture of the stick unit 10, for example, the roll angle and pitch angle of the stick unit 10 based on the motion sensor information.

  Subsequently, the CPU 11 executes shot detection processing based on the motion sensor information (step S3). Here, when a performer performs using the stick unit 10, generally, a performance operation similar to the operation of hitting an actual musical instrument (for example, a drum) is performed. In such a performance operation, the performer first raises the stick unit 10 and then swings it down toward the virtual instrument. Then, a force to stop the operation of the stick unit 10 is applied just before the stick unit 10 is hit by a virtual musical instrument. At this time, since it is assumed that the performer generates a musical sound at the moment of hitting the stick unit 10 on a virtual musical instrument, it is desirable that the performer can generate a musical sound at a timing assumed by the performer. Therefore, in the present embodiment, the musical sound is generated at the moment when the player strikes the stick unit 10 on the surface of the virtual musical instrument or slightly before that.

In the present embodiment, the shot detection timing is a timing immediately after the stick unit 10 is swung down and immediately before it is stopped, and a threshold value having a magnitude of acceleration opposite to the swing-down direction applied to the stick unit 10. The timing is over.
The shot detection timing is set as the sound generation timing. When it is determined that the sound generation timing has arrived, the CPU 11 of the stick unit 10 generates a note-on event and transmits it to the center unit unit 30. As a result, the center unit 30 executes a sound generation process to generate a musical sound.
In the shot detection process shown in step S3, a note-on event is generated based on motion sensor information (for example, a sensor composite value of an acceleration sensor). At this time, the generated note-on event may include the volume of the musical sound to be generated. The volume of the musical sound can be obtained from the maximum value of the sensor composite value, for example.

Subsequently, the CPU 11 transmits the information detected in the processes of steps S1 to S3, that is, motion sensor information, posture information, and shot information, to the center unit unit 30 via the data communication unit 16 (step S4). At this time, the CPU 11 transmits motion sensor information, posture information, and shot information to the center unit 30 in association with the stick identification information.
As a result, the process returns to step S1, and the subsequent processes are repeated.

[Processing of Camera Unit 20]
FIG. 9 is a flowchart showing a flow of processing (hereinafter referred to as “camera unit section processing”) executed by the camera unit section 20.
Referring to FIG. 9, the CPU 21 of the camera unit 20 executes an image data acquisition process (step S11). In this process, the CPU 21 acquires image data from the image sensor unit 24.

  Subsequently, the CPU 21 executes a right marker detection process (step S12) and a left marker detection process (step S13). In these processes, the CPU 21 detects the position coordinates, size, and angle of the marker unit 15 (right marker) of the right hand stick unit 10R and the marker unit 15 (left marker) of the left hand stick unit 10L detected by the image sensor unit 24. Marker detection information such as is acquired and stored in the RAM 23. At this time, the image sensor unit 24 detects marker detection information for the marker unit 15 that is emitting light.

  Then, CPU21 transmits the marker detection information acquired by step S12 and step S13 to the center unit part 30 via the data communication part 25 (step S14), and transfers a process to step S11.

[Processing of Center Unit 30]
FIG. 10 is a flowchart showing a flow of processing (hereinafter referred to as “center unit section processing”) executed by the center unit section 30.
Referring to FIG. 10, the CPU 31 of the center unit 30 notifies the player of the virtual pad to be shot (step S21). In this process, the CPU 31 selects a virtual pad to be shot from among the virtual pads 81, 82, 83, and 84, and notifies the player. The virtual pad may be selected randomly or may be selected based on the operation of the switch 341 by the performer. As a notification method, the CPU 31 displays, for example, an image that causes the selected virtual pad to blink at a constant cycle on the display device 351 via the display circuit 35. At this time, the CPU 31 may notify the number of shots that the performer should shot the virtual pad, or may notify the predetermined time to be shot.

  Subsequently, the CPU 31 receives marker detection information for each of the right marker and the left marker from the camera unit 20 and stores them in the RAM 33 (step S22). Further, the CPU 31 receives the motion sensor information, the posture information, and the shot information associated with the stick identification information from each of the right hand stick unit 10R and the left hand stick unit 10L, and stores them in the RAM 33 (step S23). Further, the CPU 31 obtains information input by operating the switch 341 (step S24).

Subsequently, the CPU 31 determines whether or not there is a shot (step S25). In this process, the CPU 31 determines whether or not there is a shot depending on whether or not a note-on event has been received from the stick unit 10. At this time, if it is determined that there is a shot, the CPU 31 executes shot information processing (step S26). When determining that there is no shot, the CPU 31 shifts the processing to step S22.
In the shot information processing, the CPU 31 selects any one of the virtual pads 81, 82, 83, and 84 in the region to which the position coordinates included in the marker detection information of the right marker and the left marker belong from the set layout information read out to the RAM 33. Timbre data (waveform data) corresponding to is output to the sound source device 36 together with the volume data included in the note-on event. Then, the tone generator 36 generates a corresponding musical sound based on the received waveform data.

  Subsequently, the CPU 31 stores the shot position (step S27). In this process, the CPU 31 stores in the RAM 33 the position coordinates included in the marker detection information for each of the right marker and the left marker.

  Subsequently, the CPU 31 determines whether or not the performance is finished (step S28). In this process, the CPU 31 determines whether or not the number of shots to be shot on the virtual pad has reached a predetermined number. Note that the CPU 31 may determine whether or not a predetermined time has elapsed since the notification in step S21. If it is determined that the performance is not finished, the CPU 31 shifts the process to step S22.

  When it is determined that the performance is finished, the CPU 31 executes a level determination process described later with reference to FIG. 11 (step S29).

  When the process of step S29 ends, the CPU 31 ends the center unit part process.

[Level determination process of center unit 30]
FIG. 11 is a flowchart showing a detailed flow of the level determination process of step S29 in the center unit section process of FIG.
Referring to FIG. 11, CPU 31 calculates the degree of variation in shot position (step S31). Specifically, the CPU 31 belongs to the virtual pad area selected in step S21 in FIG. 10 among the position coordinates included in the marker detection information for each of the right marker and the left marker acquired in step S27 in FIG. After calculating the average position coordinates of all the position coordinates, calculating the distances to all the position coordinates belonging to the selected virtual pad area from the average position coordinates, and adding all the squares of the calculated distances, The average deviation is calculated by dividing by the number of shots. The CPU 31 stores this average deviation in the RAM 33 as the degree of variation for both hands. Further, the CPU 31 separately calculates the variation degree for the position coordinates included in the marker detection information for the right marker and the position coordinates included in the marker detection information for the left marker in the same manner as described above, and the variation degree for the right hand is calculated. And stored in the RAM 33 as the degree of variation for the left hand. In this embodiment, the average deviation is used as the degree of variation, but a standard deviation that is the square root of the degree of variation may be used as the degree of variation (distribution degree).

  Subsequently, the CPU 31 performs level determination (step S32). In this process, the CPU 31 determines a level indicating the skill level of the performer according to the degree of variation for both hands stored in the RAM 33 in step S31. At this time, the CPU 31 determines that the level is a beginner when the degree of variation for both hands is greater than a predetermined threshold value, and determines that the level is an advanced person when the level is equal to or less than the predetermined threshold value. A plurality of threshold values may be provided, and the level may be divided into a plurality of levels.

  Further, the CPU 31 may determine a level indicating the skill level of the performer according to each of the right hand variation degree and the left hand variation degree stored in the RAM 33. For example, when the degree of variation for the right hand is less than or equal to a predetermined threshold value and the degree of variation for the left hand is larger than the predetermined threshold value, the level is determined by a method of determining a beginner. A plurality of threshold values may be provided, and the level may be divided into a plurality of levels.

  Subsequently, the CPU 31 performs virtual pad setting (step S33). In this process, the CPU 31 sets the position and size of the virtual pad on the virtual plane based on the degree of variation calculated in step S31. For example, when the variation degree for both hands when the shot target is the virtual pad 81 is larger than a predetermined threshold, the CPU 31 increases the size of the virtual pad 81 and increases the size as shown in FIG. Is written in the size information of the layout information stored in the RAM 33. Further, the CPU 31 may change the position on the virtual plane. Note that when the degree of variation for both hands is equal to or less than a predetermined threshold, the position and size of the virtual pad 81 are not changed. Further, when a plurality of threshold values are provided, the position and size corresponding to each level determined based on the plurality of threshold values are determined.

  Further, based on each of the right hand variation degree and the left hand variation degree, the regions of the virtual pad 81 may be set separately for the right hand stick portion 10R and the left hand stick portion 10L. For example, when the degree of variation for the right hand is equal to or smaller than a predetermined threshold and the degree of variation for the left hand is larger than the predetermined threshold, the CPU 31 uses the virtual pad 81 for the right hand stick unit 10R as shown in FIG. The right hand region 91 is set without changing the position and size, and the left hand region 92 for the left hand stick unit 10L is set with a larger size. At this time, the position may be changed. When the position coordinates included in the marker detection information of the right marker at the shot timing belong to the right hand region 91, the position is sounded with a tone corresponding to the virtual pad 81 and is included in the marker detection information of the left marker at the shot timing. Even when the coordinates belong to the left-hand region 92, the sound is generated with a tone corresponding to the virtual pad 81.

  In this case, as shown in FIG. 14, in each piece of pad information of the set layout information stored in the RAM 33, the CPU 31 stores the position and size of the right-hand area 91 as position (right) and size (right), respectively. The area is written, and the position and size of the left-hand area 92 are written in the storage area of position (left) and size (left), respectively.

  Subsequently, the CPU 31 performs level notification (step S34). In this processing, the CPU 31 notifies the level determined in step S32 by displaying it on the display device 351 via the display circuit 35. A specific display example of the notification is shown in FIG. When the process of step S34 ends, the CPU 31 ends the level determination process.

The configuration and processing of the performance device 1 according to the present embodiment have been described above.
In the present embodiment, the CPU 31 designates an area in which the stick unit 10 should be shot at the shot timing among the virtual pads 81 to 84, detects the position coordinates of the stick unit 10 at the shot timing, and is detected. When the position coordinates belong to any of the virtual pads 81 to 84, the tone generation of the corresponding timbre is instructed, and the distribution degree of the position coordinates of the stick unit 10 at the shot timing is calculated and calculated. The arrangement of the designated virtual pad is set according to the distribution degree specified.
Therefore, the layout of the virtual pads in the layout information can be changed according to the degree of variation when the designated virtual pad is shot, that is, the skill level of the performer.

Further, in the present embodiment, the stick unit 10 is a pair of performance members including a right hand stick unit 10R and a left hand stick unit 10L, and the set layout information includes the right hand for each of the virtual pads 81 to 84. An area for the stick part 10R for use and an area for the stick part 10L for the left hand are provided. Further, the CPU 31 separately detects the position coordinates of the right-hand stick unit 10R and the position coordinates of the left-hand stick unit 10L, the degree of distribution of the position coordinates of the right-hand stick unit 10R, and the position of the left-hand stick unit 10L. The coordinate distribution degree is calculated separately, and the arrangement of the area of the designated virtual pad with respect to the right hand stick part 10R is set according to the calculated distribution degree of the right hand stick part 10R, and the calculated left hand is calculated. The arrangement of the area of the designated virtual pad with respect to the left-hand stick unit 10L is set according to the distribution degree of the stick unit 10L for use.
Therefore, the right hand region and the left hand region can be set separately for one virtual pad according to the skill level of the left and right stick operations of the performer.

In the present embodiment, the CPU 31 determines the performance technique level of the performer according to the calculated distribution degree, and notifies the determined performance technique level.
Therefore, the performer can objectively know his / her performance skill level by viewing the notified performance skill level.

In the present embodiment, the CPU 31 determines the performance skill level of the performer according to the calculated distribution degree of the position coordinates of the right hand stick unit 10R and the distribution degree of the position coordinates of the left hand stick unit 10L. The determination is made and the determined performance skill level is notified.
Therefore, the performer can objectively know his / her performance skill level by viewing the notified performance skill level.

  As mentioned above, although embodiment of this invention was described, embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalents thereof.

  In the above embodiment, the virtual drum set D (see FIG. 1) has been described as an example of a virtual percussion instrument. However, the present invention is not limited to this, and the present invention generates a musical tone by swinging down the stick unit 10. It can be applied to other instruments such as xylophone.

  In the above embodiment, the region is set only for the virtual pad designated as the shot target. However, the present invention is not limited to this, and the region may be set including the virtual pad other than the shot target.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
A performance member that the performer can hold;
An imaging device that captures a captured image of the performance member as a subject and detects position coordinates of the performance member on the captured image plane;
Storage means for storing layout information defining an area arranged on the captured image plane;
Specific operation position detecting means for detecting position coordinates of the performance member on the captured image plane at the timing when the specific performance operation is performed;
Discriminating means for discriminating whether or not the position coordinates of the performance member detected by the specific operation position detecting means belong to an area arranged based on the layout information;
A sounding instruction means for instructing the sound generation of the musical sound corresponding to the area when the determination means determines that the sound belongs to the area;
Distribution calculating means for calculating a distribution degree in the region of the position coordinates of the detected performance member;
Updating means for updating layout information stored in the storage means in order to change the region according to the distribution degree calculated by the distribution calculating means;
A performance apparatus characterized by comprising:
[Appendix 2]
Level determining means for determining a performance technique level of the performer according to the distribution degree calculated by the distribution calculating means;
The performance apparatus according to appendix 1, further comprising notification means for notifying the performance technique level of the performer determined by the level determination means.
[Appendix 3]
The performance member comprises a pair of a right-hand performance member and a left-hand performance member,
The layout information defines an area for the right-hand performance member and an area for the left-hand performance member;
The specific operation position detecting means separately detects the position coordinates of the right-hand performance member and the position coordinates of the left-hand performance member,
The distribution calculating means separately calculates the distribution degree of the position coordinates of the right-hand performance member and the distribution degree of the position coordinates of the left-hand performance member,
The updating means changes a region for the right-hand performance member according to the calculated distribution degree of the right-hand performance member, and performs the left-hand performance according to the calculated distribution degree of the left-hand performance member. The performance device according to claim 1, wherein the layout information stored in the storage means is updated in order to change a region for the member.
[Appendix 4]
Level determining means for determining the performance skill level of the performer according to the calculated distribution degree of the position coordinates of the right-hand performance member and the distribution degree of the calculated position coordinates of the left-hand performance member. ,
An informing means for informing the performance technique level of the performer determined by the level determining means;
The performance device according to Supplementary Note 3, wherein
[Appendix 5]
A performance member that can be held by a performer, an imaging device that captures a captured image of the performance member as a subject, and that detects a position coordinate of the performance member on the captured image plane, and is disposed on the captured image plane A computer used as a performance device having storage means for storing layout information for defining a region,
A specific operation position detection step of detecting a position coordinate of the performance member on the captured image plane at a timing when the specific performance operation is performed;
A determination step of determining whether or not the detected position coordinates of the performance member belong to an area arranged based on the layout information;
A sound generation instruction step for instructing sound generation of a musical sound corresponding to the area, when determined to belong to the area;
A distribution calculating step of calculating a distribution degree of the position coordinates of the detected performance member in the region;
An update step of updating layout information stored in the storage means in order to change the region in accordance with the calculated distribution degree;
A program characterized by having executed.

  DESCRIPTION OF SYMBOLS 1 ... Performance apparatus, 10R ... Stick part for right hand, 10L ... Stick part for left hand, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Motion sensor part , 15 ... Marker part, 16 ... Data communication part, 17 ... Switch operation detection circuit, 171 ... Switch, 20 ... Camera unit part, 21 ... CPU, 22 ... ROM , 23 ... RAM, 24 ... image sensor unit, 25 ... data communication unit, 30 ... center unit, 31 ... CPU, 32 ... ROM, 33 ... RAM, 34 ... Switch operation detection circuit, 341 ... switch, 35 ... display circuit, 351 ... display device, 36 ... sound source device, 37 ... data communication unit, 81-84 ... virtual pad, 91 ... Right-hand area , 92 ... left hand area

Claims (5)

A performance member that the performer can hold;
An imaging device that captures a captured image of the performance member as a subject and detects position coordinates of the performance member on the captured image plane;
Storage means for storing layout information defining an area arranged on the captured image plane;
Specific operation position detecting means for detecting position coordinates of the performance member on the captured image plane at the timing when the specific performance operation is performed;
Discriminating means for discriminating whether or not the position coordinates of the performance member detected by the specific operation position detecting means belong to an area arranged based on the layout information;
A sounding instruction means for instructing the sound generation of the musical sound corresponding to the area when the determination means determines that the sound belongs to the area;
Distribution calculating means for calculating a distribution degree in the region of the position coordinates of the detected performance member;
Updating means for updating layout information stored in the storage means in order to change the region according to the distribution degree calculated by the distribution calculating means;
A performance apparatus characterized by comprising: Level determining means for determining a performance technique level of the performer according to the distribution degree calculated by the distribution calculating means;
The performance apparatus according to claim 1, further comprising notification means for notifying the performance technique level of the performer determined by the level determination means. The performance member comprises a pair of a right-hand performance member and a left-hand performance member,
The layout information defines an area for the right-hand performance member and an area for the left-hand performance member;
The specific operation position detecting means separately detects the position coordinates of the right-hand performance member and the position coordinates of the left-hand performance member,
The distribution calculating means separately calculates the distribution degree of the position coordinates of the right-hand performance member and the distribution degree of the position coordinates of the left-hand performance member,
The updating means changes a region for the right-hand performance member according to the calculated distribution degree of the right-hand performance member, and performs the left-hand performance according to the calculated distribution degree of the left-hand performance member. 2. The performance apparatus according to claim 1, wherein the layout information stored in the storage means is updated in order to change a region for the member. Level determining means for determining the performance skill level of the performer according to the calculated distribution degree of the position coordinates of the right-hand performance member and the distribution degree of the calculated position coordinates of the left-hand performance member. ,
An informing means for informing the performance technique level of the performer determined by the level determining means;
The performance device according to claim 3. A performance member that can be held by a performer, an imaging device that captures a captured image of the performance member as a subject, and that detects a position coordinate of the performance member on the captured image plane, and is disposed on the captured image plane A computer used as a performance device having storage means for storing layout information for defining a region,
A specific operation position detection step of detecting a position coordinate of the performance member on the captured image plane at a timing when the specific performance operation is performed;
A determination step of determining whether or not the detected position coordinates of the performance member belong to an area arranged based on the layout information;
A sound generation instruction step for instructing sound generation of a musical sound corresponding to the area, when determined to belong to the area;
A distribution calculating step of calculating a distribution degree of the position coordinates of the detected performance member in the region;
An update step of updating layout information stored in the storage means in order to change the region in accordance with the calculated distribution degree;
A program characterized by having executed.

Images