JP4232582B2 - Data processing apparatus having sound generation function and program thereof - Google Patents

Description

  The present invention relates to a data processing apparatus having a function of generating sound and a program thereof.

The technology for electronically generating sound is not limited to musical instruments, and is used in products in many fields such as stationery and toys. As one of the stationery that uses electronic sound generation technology, handwriting that simulates the rubbing sound between the writing instrument and paper according to the handwriting input operation using the stylus pen on the user's touch panel An input device has been proposed (see, for example, Patent Document 1).
JP-A-8-190450

  Since the above-described conventional handwriting input device is configured to generate a sound imitating a rubbing sound between a writing instrument and paper based only on speed information of a handwriting input operation by a user, variations in generated sound There was a problem that it was scarce.

  In view of the above situation, an object of the present invention is to provide a data processing device and a program that generate rich variations of sounds in response to a user's operation on a panel.

In order to achieve the above object, the present invention provides a storage means for storing a plurality of model performance control data for instructing the tone generation control in time series, and an acquisition means for acquiring position data indicating the indicated position on a plane. And output means for reading the model performance control data from the storage means and outputting the read model performance control data. The output means outputs the first model performance control data read from the storage means. Position data acquired by the acquisition means during
And calculating a cumulative movement amount obtained by accumulating the movement amount by which the designated position moves , reading out the second template performance control data from the storage unit according to the cumulative movement amount , Provided is a data processing device characterized in that the output of the second template performance control data is started simultaneously with the completion of the output.

In the data processing apparatus, the storage means stores a plurality of template performance control data corresponding to one measure constituting the musical piece, and the output means stores the plurality of template performance control data respectively corresponding to one measure. The first and second template performance control data may be read and output from the inside .

In the data processing apparatus, the acquisition means acquires pressure data indicating the pressure intensity at the indicated position together with position data indicating the indicated position on the plane, and the output means controls the model performance control read from the storage means. to the data, after modified to velocity該雛type performance data in accordance with pressure data acquired by the acquisition unit after the reading the model performance control data, outputs the stationery performance control data added with the change It is good also as composition to do.

In the data processing device, the storage unit associates each of a plurality of template performance control data corresponding to phrases constituting a musical piece with a basic note length which is a basic note length included in the corresponding phrase. And storing the output means in association with a basic note length corresponding to the amount of movement of the position indicated by the position data acquired by the acquisition means while outputting the first model performance control data. It is also possible to read out the stored model performance control data from the storage means as the second model performance control data.
In the data processing apparatus, the storage unit stores each of the plurality of template performance control data corresponding to the phrases constituting the music in association with the degree of pitch change of the musical sound included in the corresponding phrase. The output means is stored in association with the degree of pitch change according to the amount of movement of the position indicated by the position data acquired by the acquisition means while outputting the first template performance control data. The model performance control data that is stored may be read from the storage means as the second model performance control data.

The data processing apparatus may include a drawing unit that performs drawing based on the position data and the pressure data.

According to this data processing apparatus, the user's operation is drawn as an image, so that the user can visually confirm the operation.

  According to this data processing device, the user can quantitatively check the similarity between the writing operation as a model and his own writing operation.

  In the data processing apparatus, the data generation means may generate performance control data for instructing control of sound generation as data used for sound generation.

  According to this data processing device, the amount of data used for sound generation is reduced, and at the same time, it is easy to change the timbre at the time of sound generation.

  In the data processing device, the output unit may be a transmission unit that transmits data to another device.

  Further, the data processing device may include a transmission unit that transmits the position data and the pressure data to another device.

  According to these data processing devices, the generated data can be easily used by other devices.

  In the data processing apparatus, the obtaining unit may be a receiving unit that receives data from another device.

  According to this data processing device, data instructing the generation of various sounds is generated using the data indicating the position and pressure intensity generated by another device.

Furthermore, the present invention stores a plurality of template performance control data for time-sequentially instructing the tone generation control, and reads out the first template performance control data from the stored plurality of template performance control data. A process of outputting the first model performance control data; a process of acquiring position data indicating a designated position on a plane while outputting the first model performance control data;
Based on the position data acquired while outputting the first model performance control data, an accumulated movement amount is calculated by accumulating the movement amount by which the designated position moves , and the stored movement amount is stored according to the accumulated movement amount . A computer that reads out the second model performance control data from the plurality of model performance control data and starts outputting the second model performance control data simultaneously with the completion of the output of the first model performance control data. Provide a program to be executed.

  According to the data processing device and the program according to the present invention, the user can generate various sounds desired by the user by a handwriting operation on the panel or the like.

[1. First Embodiment]
[1.1. Configuration of data processing apparatus]
FIG. 1 is a perspective view showing an appearance of a panel type musical instrument 1 which is a first embodiment of a data processing apparatus according to the present invention. The panel-type musical instrument 1 includes a panel-type musical instrument main body 10 and a stick 20.

The stick 20 is used as a performance operator. In the present embodiment, the performance is performed by the user performing an operation of rubbing the surface of the touch panel 102 (described later) of the panel-type musical instrument main body 10 using the stick 20. Further, in the present embodiment, the pressure intensity against the touch panel 102 of the stick 20 is used as one of the influences the performance parameter. In order to obtain information regarding the pressing strength, the head unit 200 at the tip of the stick 20 includes a pressure sensor, a wireless communication unit, and an antenna (all not shown). When pressure is applied to the head unit 200, the applied pressure intensity is measured by the pressure sensor, and the measurement result is transmitted from the antenna as a radio wave signal. Instead of the configuration in which information indicating the pressing strength is transmitted from the stick 20 to the panel-type musical instrument main body 10 by radio wave signals, the stick 20 and the panel-type musical instrument main body 10 are connected by a cable, and the cable is connected via the cable. The information indicating the pressure strength may be transmitted.

  The panel-type musical instrument main body 10 includes a central display unit 100, a right display unit 101, a touch panel 102, an operation unit 103, an antenna 104, two speakers 105, and a wired communication unit 106.

  The central display unit 100 and the right display unit 101 are, for example, liquid crystal displays. The central display unit 100 displays the locus of the contact position between the stick 20 and the touch panel 102 and the like. Hereinafter, as an example, the central display unit 100 is assumed to be a liquid crystal display having 980 pixels in the X direction and 600 pixels in the Y direction. In addition, each position on the central display unit 100 is expressed using coordinates in which the upper left corner of the central display unit 100 is the origin, the right direction is the X positive direction, and the lower direction is the Y positive direction. Therefore, for example, the position coordinates of the pixel at the lower right corner of the central display unit 100 are (979,599). In addition, the right display unit 101 displays information regarding operation contents and various controls using the operation unit 103 of the user.

  The touch panel 102 is disposed so as to cover the front surface of the central display unit 100, and outputs a signal indicating the contact position when the stick 20 comes into contact. The operation unit 103 includes a numeric keypad and receives operations such as changing various parameters by the user. The antenna 104 receives a radio wave signal transmitted from the stick 20.

  One speaker 105 is arranged on each of the left and right sides to generate various sounds. In the following description, the expression “speech” is used as an expression indicating general sounds that are not limited to human voices. The wired communication unit 106 transmits and receives data to and from other devices via a wired communication line. The panel-type musical instrument main body 10 may include a communication unit that performs wireless communication with another apparatus instead of the wired communication unit 106.

  FIG. 2 is a functional block diagram in which the configuration of the panel-type musical instrument 1 is divided by function. As described above, the panel type musical instrument 1 includes the panel type musical instrument main body 10 and the stick 20. The panel-type musical instrument main body 10 includes a control unit 107, a position data generation unit 108, a wireless communication unit 109, a pressure data generation unit 110, an audio data generation unit 111, and a template MIDI data storage unit in addition to the components described above. 112, a MIDI data generation unit 113, a D / A (Digital-to-Analog) converter 114, and an amplifier 115. The stick 20 includes a pressure sensor 201, a wireless communication unit 202, and an antenna 203. The arrows in FIG. 2 indicate the flow of data between the components, but the arrows indicating the flow of data between the control unit 107 and the operation unit 103 and other components are omitted for simplification of the drawing. Has been.

  The audio data generation unit 111 includes a PCM (Pulse Code Modulation) sound source unit 1111 and an FM (Frequency Modulation) sound source unit 1112. The PCM sound source unit 1111 stores waveform data for generating musical sounds of a plurality of types of musical instruments. The PCM tone generator 1111 receives performance control data for instructing sound generation, selects waveform data corresponding to the tone color specified by the received performance control data, and based on this, selects the pitch and the pitch specified by the performance control data. The sound data of a musical tone having strength is generated. On the other hand, the FM sound source unit 1112 receives performance control data for instructing sound generation, and information relating to tone color and sound intensity with respect to sound data indicating a basic waveform of a frequency corresponding to a pitch included in the received performance control data. Audio data is generated by performing volume adjustment processing, filtering processing, and the like according to the above.

  The template MIDI data storage unit 112 stores performance control data, which is a template indicating the performance of a song, for a plurality of songs. In the present embodiment, when the user touches the touch panel 102 with the stick 20, the panel-type musical instrument main body 10 determines the template MIDI depending on which position on the touch panel 102 the stick 20 first touches. The performance control data of one piece of music is selected from the plurality of performance control data stored in the data storage unit 112, and the reproduction is started. Thereafter, when the user slides the stick 20 on the touch panel 102, the panel-type musical instrument main body 10 determines the playback speed of the selected performance control data according to the moving speed of the stick 20 and the pressing strength of the stick 20 against the touch panel 102. Accordingly, the reproduction of the performance control data is continued while adjusting the intensity of the musical tone indicated in the selected performance control data.

  In the present embodiment, data according to the MIDI (Musical Instrument Digital Interface) standard (hereinafter referred to as “MIDI data”) is used as performance control data. MIDI data originally refers to individual event data for performing control instructions such as tone generation in real time. As a format for storing MIDI data, SMF (Standard) in which time information about execution timing is added to each event data. MIDI File) is widely used. The MIDI data stored in the template MIDI data storage unit 112 is data in the SMF format. Hereinafter, the data stored in the template MIDI data storage unit 112 is referred to as “model MIDI data”. Note that performance control data in a format other than MIDI data may be used.

  FIG. 3 is a diagram showing a configuration of template MIDI data stored in the template MIDI data storage unit 112. The template MIDI data is associated with the song title and range data. The range data is data for designating the range of the contact position of the stick 20 from which the template MIDI data corresponding to the song name is selected. The position data indicating the position of the upper left vertex of the range on the touch panel 102, the lower right The position data indicating the position of the vertex is included.

  The MIDI data generating unit 113 is a component that generates MIDI data for performance by processing the template MIDI data. Details of the operation of the MIDI data generation unit 113 will be described in the following operation description. The MIDI data generation unit 113 has a RAM (Random Access Memory) 1131 for temporarily storing template MIDI data and the like.

[1.2. Operation of data processing apparatus]
When performing a performance using the panel-type musical instrument 1, the user uses the stick 20 like a stylus pen and performs the same operation as the one-stroke writing on the touch panel 102.

  The touch panel 102 does not output a position signal to the position data generation unit 108 when the stick 20 is not touching. When the user starts a performance operation, the stick 20 comes into contact with the touch panel 102, and the touch panel 102 outputs a position signal indicating the contact position to the position data generation unit 108. When receiving the position signal, the position data generating unit 108 starts generating time-series digital data obtained by sampling the received position signal, that is, position data. For example, when the position data generation unit 108 receives a position signal corresponding to the position of the coordinates (10, 15) on the touch panel 102, the position data generation unit 108 generates position data (10, 15). Thereafter, while the performance operation of the user is performed, the output of the position signal by the touch panel 102 and the generation of the position data by the position data generation unit 108 are continued. When the position data generation unit 108 generates position data, the position data generation unit 108 sequentially transmits the generated position data to the MIDI data generation unit 113 and the control unit 107.

  The pressure sensor 201 of the stick 20 does not output a pressure signal to the wireless communication unit 202 when the head unit 200 of the stick 20 is not in contact with the touch panel 102. When the user starts a performance operation, the stick 20 comes into contact with the touch panel 102. Therefore, the pressure sensor 201 transmits a pressure signal indicating the pressure intensity generated between the stick 20 and the touch panel 102 by the user's performance operation to the wireless communication unit 202. Output to. The pressure signal output from the pressure sensor 201 is transmitted as a radio signal from the antenna 203 via the wireless communication unit 202 and is received by the antenna 104 of the panel type musical instrument main body unit 10. The received radio wave signal is output from the wireless communication unit 109 to the pressure data generation unit 110 as a pressure signal. Upon receiving the pressure signal, the pressure data generation unit 110 starts generating time-series digital data (hereinafter referred to as “pressure data”) obtained by sampling the received pressure signal. After that, while the performance operation of the user is performed, the output of the pressure signal by the pressure sensor 201 and the generation of the pressure data by the pressure data generation unit 110 are continued. When the pressure data generation unit 110 generates the pressure data, the pressure data generation unit 110 sequentially sends the generated pressure data to the MIDI data generation unit 113 and the control unit 107.

  The MIDI data generation unit 113 does not receive position data and pressure data from the position data generation unit 108 and the pressure data generation unit 110 in a state where the user separates the stick 20 from the touch panel 102. When the MIDI data generation unit 113 receives position data and pressure data for the first time after receiving no position data and pressure data for a predetermined time or longer, for example, 10 seconds or longer, the MIDI data generation unit 113 first receives the position data and pressure data from the template MIDI data storage unit 112. The template MIDI data corresponding to the received position data is read and stored in the RAM 1131. For example, when the position data received first by the MIDI data generation unit 113 is (10, 15), according to the example shown in FIG. 3, the MIDI data generation unit 113 stores the position data (10, 15). Template MIDI data associated with the range data (0, 0) to (19, 19) indicating the range to be included is read.

  The MIDI data generation unit 113 sequentially stores the position data and pressure data received from the position data generation unit 108 and the pressure data generation unit 110 in the RAM 1131 in time series. In parallel with the storage processing, the MIDI data generation unit 113 generates performance MIDI data by performing adjustment processing on the template MIDI data read from the template MIDI data storage unit 112, and generates the generated performance MIDI data. Processing for sending data to the audio data generation unit 111 is started. The process will be specifically described below.

  First, while the position data is sequentially stored in the RAM 1131, the MIDI data generation unit 113 always calculates a speed envelope indicating the time series change of the speed of the stick 20 based on the position data stored in time series. ing. On the other hand, the MIDI data generation unit 113 counts periodically generated tempo clocks, grasps the elapsed time based on the count value, and is designated by time information associated with each event data included in the template MIDI data. And has a function of sending event data to the audio data generation unit 111. In the present embodiment, the MIDI data generation unit 113 changes the tempo clock for determining the transmission timing of the event data every moment according to the calculated speed envelope. As a result, event data is transmitted from the MIDI data generation unit 113 to the audio data generation unit 111 at a tempo corresponding to the moving speed of the stick 20.

  Further, while the pressure data is sequentially stored in the RAM 1131, the MIDI data generation unit 113 relates to event data including velocity as a strength parameter according to the pressure envelope indicated by the pressure data stored in time series. , Adjust the velocity. The MIDI data generation unit 113 directly converts the event data that does not include velocity, and the event data that includes velocity after adjustment of the velocity at the timing after adjusting the tempo according to the velocity envelope as described above. The data is sequentially sent to the data generation unit 111.

  When receiving the event data, the audio data generation unit 111 generates audio data by the PCM sound source unit 1111 or the FM sound source unit 1112 according to the received event data, and sequentially sends the generated audio data to the D / A converter 114. . The audio data sent to the D / A converter 114 is converted into an analog audio signal by the D / A converter 114, and after the volume is adjusted by the amplifier 115, the sound is produced as sound from the speaker 105. The sound generated from the speaker 105 in this way is a sound that indicates the performance of the music that has been selected by the user's performance operation and that has been subjected to tempo adjustment and sound intensity adjustment.

  As described above, while the sound is generated according to the performance performance of the user, the control unit 107 determines the position where the stick 20 contacts the touch panel 102 on the central display unit 100 based on the position data and the pressure data. The image shown is displayed. For example, when position data (10, 15) is generated by the position data generation unit 108 and pressure data (30) is generated by the pressure data generation unit 110 at the same time, the control unit 107 is positioned at the coordinates (10, 15). The pixel is colored, for example, dark blue, and the pixel located within a certain distance around the pixel located at the coordinates (10, 15) is changed from dark blue to light blue according to the distance from the coordinates (10, 15). Drawing data shown to be colored is generated, and the generated drawing data is sent to the central display unit 100. The central display unit 100 displays an image according to the drawing data received from the control unit 107. The image displayed by the central display unit 100 in this way is an image showing a trajectory that the stick 20 has moved on the touch panel 102 due to a performance operation by the user.

  As described above, by using the stick 20 as a stylus pen, the user can cause the panel-type musical instrument main body 10 to play a desired music piece by performing the same operation as the writing operation on the touch panel 102. At that time, the user can adjust the tempo and strength in real time according to the speed of the performance operation and the strength of the pressing.

[1.3. Performance system with multiple data processing devices]
As described above, the user can perform not only a performance operation alone by using the panel-type musical instrument 1, but also a dual performance or a multiple performance with other users in different places. FIG. 4 is a diagram showing a configuration of a performance system in a case where a user S and a user T who are in different places perform a duo using the panel-type musical instrument 1, respectively. Hereinafter, the panel type musical instrument main body 10 used by the user S is referred to as a panel type musical instrument main body 10S, and the panel type musical instrument main body 10 used by the user T is referred to as a panel type musical instrument main body 10T.

  The panel-type musical instrument main unit 10S and the panel-type musical instrument main unit 10T are connected to each other via the Internet 70 by the wired communication unit 106 included in each. The Internet 70 is a group of communication networks connected to each other by the Internet protocol. The wired communication unit 106 packetizes and transmits various data such as voice data to the Internet 70, receives various data packetized from the Internet 70, reconstructs the packetized data, and stores voice data and the like. Is generated. Hereinafter, as an example, it is assumed that the wired communication unit 106 transmits and receives data to and from other devices using RTP (Realtime Transport Protocol) which is a kind of Internet protocol. RTP is a communication protocol for transmitting and receiving a plurality of types of data such as audio data, image data, and text data in real time via the Internet.

  In the performance system of this embodiment, the panel-type musical instrument main body 10S and the panel-type musical instrument main body 10T may be connected by a data communication network of another form instead of the Internet 70. For example, the panel-type musical instrument main body 10S and the panel-type musical instrument main body 10T may be connected via a LAN (Local Area Network) or a dedicated line. Further, a part or all of the communication path between the panel-type instrument main body 10S and the panel-type instrument main body 10T may be wirelessly connected.

  Prior to starting the duo, the user S first designates the instrument part that each of the panel-type instrument body 10S and the panel-type instrument body 10T is responsible for. That is, the control unit 107 causes the right display unit 101 to display a list of instrument parts in response to an instruction operation for displaying a list of instrument parts using the operation unit 103 of the user S. The user S selects one or more desired instrument parts from the list displayed on the right display unit 101. Data indicating the instrument part selected by the user S is sent from the control unit 107 to the MIDI data generation unit 113 and stored in the RAM 1131. On the other hand, data indicating the musical instrument part selected by the user S is also sent from the control unit 107 to the wired communication unit 106, and transmitted from the wired communication unit 106 to the panel-type musical instrument main body unit 10T. The wired communication unit 106 of the panel-type instrument main body 10T sends the received data indicating the instrument part to the MIDI data generation unit 113 of the panel-type instrument main body 10T. When the MIDI data generation unit 113 receives data indicating the instrument part selected by the user S, the MIDI data generation unit 113 stores in the RAM 1131 data indicating an instrument part not indicated by the data, that is, an instrument part not selected by the user S.

  Subsequently, the user S brings the stick 20 into contact with the touch panel 102 in order to instruct the start of the performance process. As a result, the position data generation unit 108 and the pressure data generation unit 110 generate position data and pressure data, respectively, and send the generated data to the MIDI data generation unit 113, the control unit 107, and the wired communication unit 106.

  When receiving the position data and the pressure data, the MIDI data generating unit 113 reads the template MIDI data and stores it in the RAM 1131 based on the first received position data. Subsequently, the MIDI data generation unit 113 sends a note-on message for instructing the pronunciation of a metronome sound for one measure to the audio data generation unit 111 in accordance with the reference tempo and time signature indicated by the read template MIDI data. As a result, the metronome sound starts to be generated from the speaker 105.

  When the control unit 107 receives position data and pressure data from the position data generation unit 108 and the pressure data generation unit 110, the control unit 107 sends drawing data corresponding to the received position data and pressure data to the central display unit 100. As a result, an image showing the locus of movement of the stick 20 is displayed on the central display unit 100.

  When receiving the position data and the pressure data, the wired communication unit 106 sequentially transmits the received data to the panel type musical instrument main body unit 10T. When the wired communication unit 106 of the panel-type musical instrument main body 10T receives position data and pressure data from the panel-type musical instrument main body 10S, the wired communication unit 106 sends the received data to the MIDI data generation unit 113 and the control unit 107.

  When receiving the first position data and pressure data from the wired communication unit 106, the MIDI data generation unit 113 of the panel type musical instrument main body unit 10T, based on the position data and pressure data, similarly to the panel type musical instrument main unit unit 10S, Reading out MIDI data and storing it in the RAM 1131. Further, the MIDI data generation unit 113 sends a note-on message that instructs the pronunciation of the metronome sound to the audio data generation unit 111. As a result, the metronome sound starts to be generated from the speaker 105 of the panel-type musical instrument main body 10T.

  At the timing when the metronome sound for one measure ends, the user S and the user T each start a performance operation on the touch panel 102 using the stick 20. Position data and pressure data indicating the performance operation of the user S are transmitted from the panel-type musical instrument main body 10S to the panel-type musical instrument main body 10T. In the panel-type musical instrument main body 10T, the wired communication unit 106 transmits the MIDI data generation unit 113 and It is sent to the control unit 107. Similarly, position data and pressure data indicating the performance operation of the user T are transmitted from the panel-type musical instrument main body 10T to the panel-type musical instrument main body 10S, and the panel-type musical instrument main body 10S generates MIDI data from the wired communication unit 106. Sent to the unit 113 and the control unit 107. Hereinafter, since the operation of the panel-type instrument main body 10S and the panel-type instrument main body 10T is the same, only the operation of the panel-type instrument main body 10S will be described.

  The MIDI data generation unit 113 and the control unit 107 of the panel-type musical instrument main body unit 10S are arranged by the position data and pressure data (hereinafter, “ "Own machine position data" and "Own machine pressure data"), position data and pressure data received from the panel-type musical instrument main body 10T (hereinafter referred to as "other machine position data" and "other machine pressure data", respectively) Call) at the same time.

  The MIDI data generation unit 113 calculates the speed envelope based on the position data of the other apparatus simultaneously with the calculation of the speed envelope based on the position data of the own apparatus. Subsequently, the MIDI data generation unit 113 calculates an average value of the speeds indicated by the two generated speed envelopes, and performs a tempo adjustment process of the template MIDI data based on the calculated average value of the speeds. Further, the MIDI data generation unit 113 performs velocity adjustment based on its own pressure data for the event data included in the template MIDI data related to the instrument part selected by the user S, and for other event data, Velocity adjustment based on the pressure data of other aircraft. The MIDI data generation unit 113 sequentially sends the template MIDI data thus adjusted to the audio data generation unit 111 as performance MIDI data.

  On the other hand, the control unit 107 generates drawing data indicating the locus of movement of the stick 20 by the performance operation of the user S, for example, in blue, based on the own device position data and the own device pressure data. Further, the control unit 107 generates drawing data indicating the movement trajectory of the stick 20 by the performance operation of the user T in a color different from blue, for example, red, based on the other device position data and the other device pressure data. The control unit 107 combines the generated two drawing data to simultaneously generate drawing data indicating the performance actions of the user S and the user T as images, and sends the generated drawing data to the central display unit 100. Instructs the display of the image.

  Through the above-described processing of the MIDI data generating unit 113, the performance sound of the music selected by the performance operation of the user S is generated from the speaker 105 with the tempo adjusted in real time by the performance operation of the user S and the user T. The At that time, the intensity of the musical instrument part selected by the user S is adjusted in real time by the performance operation of the user S, and the musical sound of the musical instrument part not selected by the user S is adjusted by the user T in real time. It will be given. In addition, by the processing of the control unit 107, the locus of movement of the stick 20 by the performance operation of the user S and the user T is displayed on the central display unit 100 in different colors in real time.

  Therefore, the user S and the user T can enjoy the duo even in a place away from each other by the same operation as the writing operation using the stick 20 as a stylus pen. At that time, the user S and the user T can adjust the tempo and the strength in real time according to the speed of the writing operation and the strength of the pressing. Furthermore, the user S and the user T can perform a performance operation while visually confirming the performance of the other party by an image in addition to their performance.

  The panel-type musical instrument main body 10S and the panel-type musical instrument main body 10T may be configured to transmit and receive MIDI data or audio data in addition to position data and pressure data.

  For example, when MIDI data is transmitted and received between the panel-type instrument main body 10S and the panel-type instrument main body 10T, the MIDI data generation unit 113 of the panel-type instrument main body 10S only relates to the instrument part selected by the user S. Then, performance MIDI data is generated by adjusting the template MIDI data. The MIDI data generation unit 113 transmits the generated MIDI data to the audio data generation unit 111 and simultaneously transmits it to the panel-type musical instrument main body unit 10T via the wired communication unit 106. On the other hand, for the musical instrument part not selected by the user S, the wired communication unit 106 receives MIDI data from the panel-type musical instrument main unit 10T and sends the received MIDI data to the audio data generation unit 111. As a result, sound corresponding to the performance operation of the user S and the user T is simultaneously generated from the speaker 105.

  When audio data is transmitted / received between the panel-type instrument main body 10S and the panel-type instrument main body 10T, the MIDI data generation unit 113 of the panel-type instrument main body 10S only relates to the instrument part selected by the user S. Then, performance MIDI data is generated by adjusting the template MIDI data. The MIDI data generation unit 113 sends the generated MIDI data to the audio data generation unit 111. When the audio data generation unit 111 generates audio data according to the MIDI data received from the MIDI data generation unit 113, the audio data generation unit 111 transmits the generated audio data to the panel-type musical instrument main body unit 10T via the wired communication unit 106. On the other hand, for the musical instrument part that is not selected by the user S, the wired communication unit 106 receives audio data from the panel-type musical instrument main unit 10T and sends the received audio data to the D / A converter 114. As a result, sound corresponding to the performance operation of the user S and the user T is simultaneously generated from the speaker 105.

  In the above description, the case where the tempo of the music is adjusted according to the velocity envelope and the intensity of the musical sound is adjusted according to the pressure data is exemplified, but the music is adjusted using the position data and the pressure data. Any parameter can be adjusted based on any data.

  In the above description, the case where the position data is generated from the position signal output from the touch panel 102 and the pressure data is generated from the pressure signal output from the stick 20 is exemplified. However, a third embodiment to be described later. As exemplified in FIG. 4, the position data and the pressure data may be generated by a signal output from a pressure sensor panel capable of measuring the pressure intensity at each position.

  The panel-type musical instrument main unit 10 connects devices corresponding to the touch panel 102, the antenna 104, and the wireless communication unit 109 to a general-purpose computer capable of audio data processing and communication processing. It can also be realized by causing a general-purpose computer to execute a program for instructing 10 processes. In that case, the control unit 107, the position data generation unit 108, the pressure data generation unit 110, the audio data generation unit 111, and the MIDI data generation unit 113 are a CPU (central processing unit), a DSP (digital signal processor), or the like that a general-purpose computer has. This is realized by performing processing according to the program. The central display unit 100 and the right display unit 101 are realized by causing the display unit to display an image divided into a plurality of areas on the display unit.

[2. Second Embodiment]
[2.1. Configuration of data processing apparatus]
The data processing apparatus according to the second embodiment sequentially selects, for example, model MIDI data indicating a performance phrase for one measure in response to an operation similar to a writing operation using a user's stick as a stylus pen, and selects the selected model MIDI. By performing adjustment processing on the data, the performance sound arranged by the user is pronounced.

  Since the second embodiment has many points in common with the first embodiment described above, only the points of the second embodiment different from the first embodiment will be described. Hereinafter, the data processing apparatus in the second embodiment is referred to as a panel type musical instrument 2.

  The appearance and configuration of the panel-type musical instrument 2 are exactly the same as those of the panel-type musical instrument 1 in the first embodiment except for the configuration of the template MIDI data stored in the template MIDI data storage unit 112. Accordingly, the operation of the panel type musical instrument 2 will be described below with reference to FIGS. 1 and 2.

  FIG. 5 is a diagram showing a configuration of data stored in the template MIDI data storage unit 112. The template MIDI data stored in the template MIDI data storage unit 112 of the panel-type musical instrument 2 is different from the template MIDI data stored in the template MIDI data storage unit 112 of the panel-type musical instrument 1, and is related to each measure included in the musical composition. This is performance control data indicating a plurality of performance phrases. Template MIDI data indicating each performance phrase is arranged hierarchically in the order of “song title”, “tune tone”, “instrument part”, “basic note length”, “pitch change”, and “measure”, and stored as template MIDI data. Stored in the unit 112.

  The “basic note length” indicates a division based on a basic note length included in the performance phrase indicated by the template MIDI data, and is distinguished by a name such as “whole note” or “half note”. For example, a performance phrase classified into “whole notes” is a performance phrase that gives a quiet impression because the sound generation time of each musical tone is longer on average than a performance phrase classified into “16th notes”. On the other hand, the performance phrase divided into “16th notes” is a performance phrase that gives a lively impression with the average pronunciation time of each musical tone being shorter than that of the performance phrase classified as “all notes”.

  “Pitch change” indicates a classification according to the tendency of the change in pitch of the performance phrase, and is distinguished by the names “rapid rise”, “slow rise”, “no change”, “slow fall”, and “sudden fall”. For example, a performance phrase classified as “rapid rise” is a performance phrase whose pitch rises rapidly from the start to the end of the performance phrase.

  According to the above classification, for example, it is indicated by the classification of music “ABC” —musical tone “rock” —instrument part “guitar” —basic note length “all notes” —pitch change “rapid rise” —measure “first measure”. One template MIDI data is specified. Hereinafter, the template MIDI data specified as such is shown as “ABC / Rock / Guitar / Full note / Rapid rise / First measure”. Note that the template MIDI data divided into the first bar includes data indicating the tempo, time signature, and reference basic note length (hereinafter referred to as “reference basic note length”) of the music.

[2.2. Operation of data processing apparatus]
Prior to starting the performance operation using the panel-type musical instrument 2, the user designates the name and tone of the music to be arranged and the musical instrument part to be arranged by the user. That is, the user operates the operation unit 103 to display the candidate list on the right display unit 101 in the order of the song name, the tone, and the instrument part, and selects a desired one from the displayed candidate list. At that time, the user can also select a plurality of instrument parts.

  In response to a user operation, the operation unit 103 sends data indicating the selected song title, tone, and instrument part to the MIDI data generation unit 113. The MIDI data generation unit 113 stores the received song name, tone, and data indicating the instrument part in the RAM 1131. Hereinafter, as an example, it is assumed that the user selects the song name “ABC”, the song tone “Rock”, and the musical instrument parts “Guitar” and “Sax”. The template MIDI data divided into the song title “ABC” and the measure “first measure” includes the tempo “120”, the time signature “4/4”, and the reference basic note length “quarter note”. And

  Subsequently, the user operates the operation unit 103 to instruct the panel-type musical instrument main body unit 10 to start playing. In response to a user operation, the operation unit 103 sends data for instructing the start of performance to the MIDI data generation unit 113. The MIDI data generation unit 113, based on the song name “ABC” and the song tone “Rock” stored in the RAM 1131 according to the data for instructing the start of the performance, out of the template MIDI data belonging to these categories, the measure “first measure”. ”Is read out.

  The MIDI data generation unit 113 generates a one-measure note-on message instructing pronunciation by a metronome tone according to the tempo “120” and the time signature “4/4” included in the read template MIDI data, and the generated note On-messages are sequentially sent to the audio data generation unit 111. As a result, a metronome sound is generated from the speaker 105.

  While the metronome sound is sounded as described above, the user can specify the basic note length and the pitch change in the first measure of the performance in the same manner as the writing operation on the touch panel 102 using the stick 20 as a stylus pen. Perform the operation.

  In generating musical performance MIDI data corresponding to each measure, the panel-type musical instrument main body 10 determines the basic note length according to the accumulated movement amount of the stick 20 in the Y direction at the time corresponding to the previous measure. Then, the template MIDI data to be used is selected by determining the pitch change according to the movement amount of the stick 20 in the X direction. Thus, for example, when the user wants to start playing with a lively performance phrase, the user touches the stick 20 on the touch panel during one measure before the first measure starts, that is, while the metronome sound is being generated. An operation of quickly reciprocating in the Y positive direction and the Y negative direction on 102 is performed. For example, when the user wants to start playing with a performance phrase whose pitch changes with an upward trend, the user slides the stick 20 in the X positive direction on the touch panel 102 while the metronome sound is being generated. I do.

  The position data generation unit 108 and the pressure data generation unit 110 generate position data and pressure data, respectively, according to the operation using the user's stick 20, and send the generated data to the MIDI data generation unit 113. When receiving the pressure data and the position data, the MIDI data generating unit 113 sequentially stores them in the RAM 1131. The MIDI data generation unit 113 receives data instructing the start of performance from the operation unit 103, and immediately before the time of one measure has elapsed, based on the position data stored in the RAM 1131 for the time of the past one measure. The template MIDI data selection process as shown below is performed.

First, the MIDI data generation unit 113 calculates the cumulative movement amount in the Y direction of the position data for the past one measure stored in the RAM 1131. That is, _assuming that the position data is arranged in time series like (X ₁ , Y ₁ ), (X ₂ , Y ₂ ),... (X _n , Y _n ), the MIDI data generation unit 113 For example, the value of equation (1) is calculated.

  Formula (1) is an example of a calculation formula for calculating the cumulative amount of movement in the Y direction. For example, various calculation formulas such as calculating the sum of absolute values of adjacent Y coordinates can be used. .

  The MIDI data generation unit 113 performs the cumulative movement in the Y direction calculated as described above based on the reference basic note length “4/4” included in the template MIDI data divided into the previously read measure “first measure”. The basic note length in the first measure is determined according to the amount. For example, when the cumulative amount of movement in the Y direction is large, the MIDI data generation unit 113 sets the basic note length of the first measure to “16th note” indicating a lively phrase segment that is more lively than the reference basic note length “quarter note”. To decide.

Subsequently, the MIDI data generation unit 113 calculates the difference between the X coordinates of the first position data and the last position data for the past one bar stored in the RAM 1131. That is, _assuming that the position data is arranged in time series like (X ₁ , Y ₁ ), (X ₂ , Y ₂ ),... (X _n , Y _n ), the MIDI data generation unit 113 (X _n −X ₁ ) is calculated. This value is a value indicating the amount of movement of the stick 20 in the X direction during the past one measure. There are various methods for calculating the amount of movement in the X direction, such as calculating the difference between the average value of the X coordinate of the first half of the time for one measure and the average value of the X coordinate of the second half. It is done. The MIDI data generation unit 113 determines a pitch change in the first measure according to the value calculated as described above. For example, if the amount of movement in the X direction is a very large positive value, the MIDI data generation unit 113 determines the pitch change of the first measure as “rapidly rising”.

  When the MIDI data generating unit 113 determines the basic note length and the pitch change in the first measure as described above, the MIDI data generating unit 113 reads out the template MIDI data corresponding to the basic note length from the template MIDI data storage unit 112 and stores it in the RAM 1131. In this case, the MIDI data generation unit 113 selects “ABC / Rock / Guitar / 16th note / rapid rise / first rise based on the data of the instrument parts“ Guitar ”and“ Sax ”selected by the user stored in the RAM 1131. The template MIDI data of “bar” and “ABC / rock / sax / 16th note / steep rise / first bar” are read out. The MIDI data generation unit 113 also relates to instrument parts not stored in the RAM 1131, such as “bass”, “drum”, etc., “ABC / rock / bass / quarter note / no change / first measure”, “ABC”. Template MIDI data such as “/ rock / drum / quarter note / no change / first measure” is read out. With respect to the template MIDI data of the instrument parts not selected by these users, the basic note length “4/4” which is the same as the reference basic note length and the pitch change “no change” are always read out. The above is the template MIDI data selection processing in the MIDI data generation unit 113.

  When the MIDI data generation unit 113 reads the template MIDI data as described above, the MIDI data generation unit 113 starts sending the event data included in the read template MIDI data to the audio data generation unit 111. At that time, the MIDI data generation unit 113 performs velocity adjustment on the event data according to the pressure envelope. However, in the second embodiment, the MIDI data generation unit 113 does not perform tempo adjustment processing according to the speed envelope.

  The audio data generation unit 111 generates audio data such as guitar, saxophone, bass, drum, etc. according to the event data received from the MIDI data generation unit 113 and sends it to the D / A converter 114. As a result, the performance sound of those musical instruments is produced from the speaker 105.

  As described above, the MIDI data generation unit 113 continuously stores the position data while performing the process of sending the note-on message related to the first measure, and at the timing immediately before the start of the second measure, Based on the position data stored in the time of one measure, the above-described template MIDI data selection processing is performed. Subsequently, the MIDI data generation unit 113 performs transmission processing of the event data included in the selected template MIDI data to the audio data generation unit 111. At this time, the instrument parts “guitar” and “sax” selected by the user are transmitted. With respect to the note-on message included in the template MIDI data, a pitch adjustment process is performed in addition to the above-described velocity adjustment process.

  For example, the template MIDI data of a certain instrument part selected for generating MIDI data for performance of the first measure starts with a pitch of “C3” and ends with a pitch of “C4”. It is assumed that the template MIDI data of the same instrument part selected for generating the MIDI data for the performance of the measure starts with the pitch “B3” and ends with the pitch “B4”. In such a case, the MIDI data generation unit 113 adds 12 to the note number that is a parameter indicating the pitch of the note-on message included in the template data of the second measure, thereby generating a musical sound that the note-on message instructs to pronounce. Shift the pitch one octave to the high pitch side. With such a pitch adjustment process, the performance sound produced from the speaker 105 has a natural connection between bars.

  As a result of the processing of the MIDI data generation unit 113 described above, the sound produced from the panel-type musical instrument main body 10 has a liveliness, a changing tendency of the pitch, and a tone intensity according to the performance operation using the stick 20 by the user. The performance sound is arranged in various ways. As a result, the user can easily perform the music arrangement performance by the same operation as the writing operation.

  In the second embodiment as well, an image showing the performance performance of the user is displayed on the central display unit 100 as in the first embodiment. Also in the second embodiment, similarly to the first embodiment, by using a plurality of panel-type musical instruments 2 connected by a communication network or the like, a plurality of users can perform a dual performance or a multiple performance. Since these operations in the panel-type musical instrument 2 are the same as those in the panel-type musical instrument 1, description thereof will be omitted.

  Further, in the above description, the case where the cumulative movement amount of the stick 20 in the Y direction is associated with the basic note length, the movement amount in the X direction is changed in pitch, and the pressure intensity on the touch panel 102 is associated with the intensity of the musical sound. Which index generated based on the position data and the pressure data is used, and which parameter of the music is adjusted according to the index can be arbitrarily changed.

  In the above description, the template MIDI data is selected in units of one measure. However, the length of the performance phrase indicated by the template MIDI data can be arbitrarily changed. For example, if the MIDI data generation unit 113 is configured to select and process a performance phrase in units of 1/4 bars, the performance operation of the user is reflected more quickly and finely in the performance.

[3. Third Embodiment]
[3.1. Configuration of data processing apparatus]
The data processing device according to the third embodiment generates time-series position data and pressure data indicating a writing operation as a model at the same time as generating a sound corresponding to the writing operation according to the user's writing operation, and the user The degree of similarity with time-series position data and pressure data generated by the writing operation is evaluated, and the evaluation result is displayed.

  Since the third embodiment has many points in common with the first embodiment described above, only the points of the third embodiment different from the first embodiment will be described. Hereinafter, the data processing apparatus in the third embodiment is referred to as a panel-type musical instrument 3. In addition, among the components included in the panel-type musical instrument 3, the same reference numerals are used hereinafter for the components common to the components included in the panel-type musical instrument 1.

  FIG. 6 is a perspective view showing the appearance of the panel-type musical instrument 3. The panel-type musical instrument 3 is composed only of the panel-type musical instrument main body 30. That is, the component corresponding to the stick 20 in the first embodiment is not provided. The panel-type musical instrument main body 30 does not include a component corresponding to the antenna 104 of the panel-type musical instrument main body 10. This is because the panel-type musical instrument 3 does not include a component corresponding to the stick 20, so that the panel-type musical instrument main body 30 does not need to receive a radio wave signal transmitted from the stick 20. The panel-type musical instrument main body 30 includes a pressure sensor panel 316 instead of the touch panel 102 of the panel-type musical instrument main body 30. In other respects, the external appearance of the panel-type musical instrument main body 30 and the panel-type musical instrument main body 30 is the same.

  The pressure sensor panel 316 includes, for example, a rectangular transparent acrylic board and high-precision pressure sensors respectively disposed at four corners of the transparent acrylic board. Each pressure sensor has, for example, a strain gauge or the like, and outputs the magnitude of the pressure applied in the direction perpendicular to the surface of the pressure sensor panel 316 at the attached position as a pressure signal. Hereinafter, the pressure signals output from the pressure sensors arranged at the four corners of the pressure sensor panel 316 are referred to as “pressure signal 1”, “pressure signal 2”, “pressure signal 3”, and “pressure signal 4”, respectively. . Note that the number of pressure sensors provided in the pressure sensor panel 316 and the arrangement method thereof can be variously changed.

  FIG. 7 is a functional block diagram in which the configuration of the panel-type musical instrument main body 30 is divided by function. Unlike the panel-type musical instrument main body 10, the panel-type musical instrument main body 30 does not include the antenna 104 as described above, and includes a pressure sensor panel 316 instead of the touch panel 102. Furthermore, instead of the position data generation unit 108 and the pressure data generation unit 110, a position pressure data generation unit 317 having these functions is provided. Further, the panel-type musical instrument main body 30 does not include the template MIDI data storage unit 112 and the MIDI data generation unit 113 provided in the panel-type musical instrument main unit 10, but the template position pressure data storage unit 318 and the index calculation unit. 319 is provided.

  The position pressure data generation unit 317 is a set of position data indicating the position of pressure applied on the pressure sensor panel 316 and pressure data indicating the pressure strength of the pressure based on the pressure signal output from the pressure sensor panel 316. Generate some position pressure data. The position pressure data generation unit 317 stores a reference pressure data group 3171 as information for converting the pressure signal into position pressure data. FIG. 8 is a diagram showing the configuration of the reference pressure data group 3171. The reference pressure data group 3171 includes reference pressure data 1 to 4 indicating values of pressure signals 1 to 4 when a pressure of a predetermined pressing strength, for example, a pressure of 1 kPa is applied to each position on the pressure sensor panel 316. Contains. For example, according to the example of FIG. 8, when a pressure of 1 kPa is applied to the position (0, 0), the pressure sensors arranged at the four corners of the pressure sensor panel 316 are 0.385 kPa, 0.107 kPa, It can be seen that pressure signals indicating pressure intensities of 0.256 kPa and 0.052 kPa are respectively output.

  The position pressure data generation unit 317 generates time-series data obtained by sampling the pressure signals 1 to 4 input from the pressure sensor panel 316, that is, pressure data 1 to 4. Subsequently, the position pressure data generation unit 317 selects, from the reference pressure data group 3171, reference pressure data in which the ratio of the reference pressure data 1 to 4 is closest to the ratio of the generated pressure data 1 to 4.

For example, the pressure data 1 to 4 generated based on the pressure signals 1 to 4 received from the pressure sensor panel 316 are h, i, j, and k, and the reference pressure data 1 corresponding to the position data (x, y). ˜4 is H _{(x, y)} , I _{(x, y)} , J _{(x, y)} and K _{(x, y)} . In that case, the position pressure data generation unit 317 calculates, for example, the value of the following equation (2) for all (x, y).

The value of Equation (2) approaches 0 as the ratio of h to k and the ratio of H _{(x, y) to} K _{(x, y)} are closer. Therefore, the position pressure data generation unit 317 selects (x, y) that minimizes the value of Expression (2). Note that equation (2) is an example of an index calculation formula indicating the degree of approximation between the ratio of the generated pressure data and the ratio of the reference pressure data, and other calculation formulas may be used.

(X, y) selected by the above processing is position data indicating the center point of the pressure applied to the pressure sensor panel 316. Subsequently, the position pressure data generation unit 317 calculates how many times the pressure signals 1 to 4 received from the pressure sensor panel 316 are larger than the reference pressure data corresponding to the selected (x, y). . That is, the position pressure data generation unit 317 calculates, for example, p = (h / H _{(x, y)} ) for the selected _{(x, y)} . The value p calculated in this way is a numerical value indicating the magnitude of pressure at the point indicated by (x, y) in kPa. The position pressure data generation unit 317 generates data obtained by combining the numerical values obtained as described above as (x, y, p), that is, position pressure data.

  The template position pressure data storage unit 318 is a component that stores data indicating a writing operation of a character serving as a model. FIG. 9 is a diagram illustrating a configuration of data stored in the template position pressure data storage unit 318. The template position pressure data storage unit 318 stores a template position pressure data group, which is data in which position pressure data is arranged in time series with respect to a combination of a character and a font, with respect to a combination of a plurality of characters and a font. For example, the template position pressure data group corresponding to the character “So” and the typeface “Line typeface” is, for example, an excellent calligrapher using a brush that does not make a black ink, and the character “So” is a line typeface pressure sensor panel. The position pressure data generated by the position pressure data generation unit 317 at the time of writing to 316 is sequentially stored in time series.

  The index calculation unit 319 displays an index indicating the degree of similarity between the position pressure data group received in time series from the position pressure data generation unit 317 (hereinafter referred to as “user position pressure data group”) and the template position pressure data group. It is a component to generate. The index calculation unit 319 includes a RAM 3191 for temporarily storing a user position pressure data group, a template position pressure data group, and the like. The processing performed by the index calculation unit 319 will be described together in the following operation description.

[3.2. Operation of data processing apparatus]
Prior to starting a writing operation using the panel-type musical instrument main body 30, the user operates the operation unit 103 to display a candidate list of characters and typefaces on the right display unit 101, and requests from the displayed candidate list. Select characters and typeface. In the following description, it is assumed that the user has selected the character “Creation” and the typeface “Line typeface”.

  The operation unit 103 sends data indicating the character and typeface selected by the user to the index calculation unit 319. The index calculation unit 319 receives data indicating characters and typefaces, reads a template position pressure data group corresponding to the data from the template position pressure data storage unit 318, and stores it in the RAM 3191. Subsequently, the index calculation unit 319 sequentially sends the position pressure data included in the template position pressure data group stored in the RAM 3191 to the voice data generation unit 111 and the control unit 107.

  Upon receiving the position pressure data from the index calculation unit 319, the sound data generation unit 111 generates sound data corresponding to the received position pressure data. For example, the FM sound source unit 1112 of the audio data generation unit 111 changes the frequency of the basic waveform according to the X coordinate of the received position pressure data, and changes the shape of the basic waveform from a sine wave to a rectangular wave according to the Y coordinate. To change continuously. Further, the FM sound source unit 1112 performs tone adjustment, filter processing, and the like according to the pressure data included in the position pressure data. The sound data generation unit 111 sends the sound data generated as described above to the D / A converter 114. As a result, the speaker 105 generates various pitches according to the position pressure data included in the template position pressure data group. Sounds with varying timbre and volume are produced.

  On the other hand, when the control unit 107 receives the position pressure data from the index calculation unit 319, the control unit 107 generates drawing data indicating a line or the like according to the received position pressure data, and sends the generated drawing data to the central display unit 100. The central display unit 100 is instructed to display an image. As a result, the character “So” as a model is drawn as a moving image on the central display unit 100.

  Subsequently, the user uses a brush that is not inked, referring to a model already displayed on the central display unit 100, and in any place of the central display unit 100, for example, the right half of the central display unit 100. Write the word “So”. When the user brings the writing brush into contact with the pressure sensor panel 316, the position pressure data generation unit 317 sends the position pressure data to the control unit 107 and the index calculation unit 319. When receiving the position pressure data from the index calculation unit 319, the control unit 107 instructs the central display unit 100 to display lines and the like, as in the case of receiving the position pressure data included in the template position pressure data group described above. .

  On the other hand, when the index calculation unit 319 receives the first position pressure data from the position pressure data generation unit 317, the index position shifts between the character position indicated by the template position pressure data group and the character position written by the user's writing operation. In order not to affect the generation of audio data, the position adjustment processing is performed. More specifically, first, the index calculation unit 319 is included in the X and Y coordinates included in the position pressure data first received from the position pressure data generation unit 317 and the template position pressure data group read out earlier. The difference between the X coordinate and the Y coordinate of the first position pressure data is calculated. For example, the first position data included in the “position” template position pressure data group is (250, 320), and the position data included in the position pressure data generated first by the user's writing operation is (730, 300). ), The index calculation unit 319 calculates (−480, +20) as the difference between the X coordinate and the Y coordinate. Thereafter, the index calculation unit 319 performs position data adjustment processing on the position pressure data received from the position pressure data generation unit 317 using the calculated coordinate difference. For example, when the index calculation unit 319 receives the position pressure data (725, 270, 25) from the position pressure data generation unit 317, the index calculation unit 319 subtracts 480 from the X coordinate and adds 20 to the Y coordinate. (245, 290, 25) is generated as the position pressure data. The index calculation unit 319 sequentially stores the generated position pressure data after the adjustment of the X coordinate and the Y coordinate in the RAM 3191 and sends it to the audio data generation unit 111.

  Upon receiving the position pressure data from the index calculation unit 319, the sound data generation unit 111 generates sound data in the same manner as when the position pressure data included in the template position pressure data group described above is received, and the generated sound data is The data is sent to the D / A converter 114. As a result, the speaker 105 emits sound whose pitch, sound quality, and volume vary depending on the user's writing operation. Therefore, the user compares the voice of the template position pressure data group previously pronounced from the speaker 105 with the voice pronounced by his / her own writing operation to determine which part of the writing operation is different. The writing operation can be performed while aurally checking.

  The index calculation unit 319 stores the position pressure data corresponding to the user's writing operation in the RAM 3191 by the same number as the number of position pressure data included in the template position pressure data group previously stored in the RAM 3191. A process of calculating an index indicating the degree of similarity between the group and the position pressure data stored in time series by the user's writing operation, that is, the user position pressure data group is performed.

For example, the template position pressure data group is (X ₁ , Y ₁ , P ₁ ), (X ₂ , Y ₂ , P ₂ ),..., (X _m , Y _m , P _m ), and the user position pressure data groups _{(x 1, y 1, p} 1), and is _{(x 2, y 2, p} 2), ···, (x m, y m, p m). In this case, for example, the index calculation unit 319 calculates the value of the following formula (3).

  The value of Equation (3) approaches 0 as the value of data included in the template position pressure data group and the value of data included in the user position pressure data group are closer. Therefore, the value of the expression (3) is an index indicating how close the writing operation of the user is to the writing operation of a calligrapher as a model. The calculation formula shown in Formula (3) is an example of a calculation formula for obtaining an index indicating the degree of similarity between the template position pressure data group and the user position pressure data group, and various other calculation formulas can be used. It is.

  The index calculation unit 319 sends the index calculated as described above to the control unit 107. The control unit 107 generates drawing data indicating the index received from the index calculation unit 319 and sends it to the right display unit 101. The right display unit 101 displays an index according to the drawing data received from the control unit 107. As a result, the user can quantitatively confirm how much his writing operation is close to the writing operation of the model. In addition, the control unit 107 is configured to cause the speaker 105 to generate a sound for notifying the user of the index by transmitting sound data to the sound data generating unit 111 instead of transmitting drawing data to the right display unit 101, for example. It is good.

[3.3. Performance system with multiple data processing devices]
In the third embodiment, by using a plurality of panel-type musical instrument main bodies 30 connected by a communication network or the like, for example, a calligraphy teacher can perform distance education for students. For example, it is assumed that a panel-type musical instrument main body 30S possessed by a user S who is a calligraphy student and a panel-type musical instrument main body 30T possessed by a user T who is a calligraphy teacher are connected via the Internet 70. First, a writing operation as a model is performed by the user T who is a teacher. The panel-type musical instrument main body 30T transmits a user position pressure data group indicating the writing operation of the user T to the panel-type musical instrument main body 30S. Hereinafter, the user position pressure data group generated by the writing operation of the user T is referred to as “example position pressure data group”, and is distinguished from the user position pressure data group generated by the writing operation of the user S.

  The wired communication unit 106 of the panel-type musical instrument main body 30S sends the sample position pressure data group received from the panel-type musical instrument main body 30T to the control unit 107 and the index calculation unit 319. When the control unit 107 receives the model position pressure data group, the control unit 107 sends drawing data such as lines corresponding to the received data to the central display unit 100. In addition, when the index calculation unit 319 receives the model position pressure data group, the index calculation unit 319 sends the received data to the audio data generation unit 111. As a result, the central display unit 100 displays an image showing the locus of the writing brush by the writing operation of the user T. From the speaker 105, the sound whose tone pitch, sound quality, and volume vary in accordance with the writing operation of the user T is displayed. Pronounced.

  When the writing operation by the user T is completed, the user S performs a writing operation on the panel-type musical instrument main body 30S. In accordance with the writing operation of the user S, an image showing the locus of the brush stroke by the writing operation of the user S is displayed on the central display unit 100 of the panel type musical instrument main body 30S, and from the speaker 105 according to the writing operation of the user S. Voices with varying pitch, quality and volume are produced.

  Further, the wired communication unit 106 of the panel-type instrument main body 30S transmits the user position pressure data group generated by the drawing operation of the user S to the panel-type instrument main body 30T. The panel-type musical instrument main body 30T displays an image showing the trajectory of the brush strokes by the writing operation of the user S on the central display unit 100 based on the user position pressure data group transmitted from the panel-type musical instrument main body 30S. To generate a sound whose pitch, sound quality and volume change according to the writing operation of the user S. As a result, the user T can easily grasp improvements and the like in the writing operation by the user S using visual and auditory senses.

  In the above description, the case where the user performs a writing operation using a brush that does not make ink is described as an example. However, the user places paper on the pressure sensor panel 316 and applies ink from above. The writing operation may be performed using a brush. The brush is an example of a writing tool, and various writing tools such as a pencil, a pen, and a crayon can be used.

[4. Fourth Embodiment]
[4.1. Configuration of data processing apparatus]
The data processing apparatus according to the fourth embodiment generates a musical tone having a pitch, tone, and volume according to a position where the stick and the touch panel are in contact with each other and a pressing strength in accordance with a performance operation such as rubbing or hitting the touch panel using the user's stick. Pronounce.

  Since the fourth embodiment shares many points with the first embodiment described above, only the differences of the fourth embodiment from the first embodiment will be described. Hereinafter, the data processing apparatus in the fourth embodiment is referred to as a panel-type musical instrument 4. In addition, among the components included in the panel-type musical instrument 4, the same reference numerals are used hereinafter for the components common to the components included in the panel-type musical instrument 1.

  FIG. 10 is a perspective view showing the appearance of the panel-type musical instrument 4. The panel type musical instrument 4 includes a panel type musical instrument main body 50 and a stick 20. The panel-type musical instrument main body 50 includes an upper display 520 and a left display 521 as components that the panel-type musical instrument main body 10 does not have. The upper display unit 520 and the left display unit 521 are, for example, liquid crystal displays, and the upper display unit 520 displays, for example, a keyboard image indicating a pitch as information related to the X direction of the touch panel 102, while the left display unit 521 As the information regarding the Y direction of the touch panel 102, for example, an instrument name indicating a timbre is displayed.

  The panel-type musical instrument main body 50 is different from the panel-type musical instrument main body 10 in that it does not include the template MIDI data storage unit 112 in its internal configuration. In addition, the MIDI data generation unit 113 provided in the panel-type musical instrument main body unit 50 does not perform adjustment processing on the template MIDI data as in the first and second embodiments, but applies position data and pressure data. In response, the event data itself is generated.

[4.2. Operation of data processing apparatus]
Prior to performing a performance operation using the panel-type musical instrument 4, the user operates the operation unit 103 to set the tone range and tone color for the panel-type musical instrument main body unit 50. As an example, the user sets the tone range corresponding to a 49-key keyboard in the X direction and the tone of “Piano”, “Synth Pad”, “Guitar”, “Sax”, “Bass” and “Drum Set” in the Y direction. It shall be. In accordance with the user's setting operation, the control unit 107 displays an image indicating a 49-key keyboard on the upper display unit 520, characters indicating a tone such as “piano” on the left display unit 521, and X direction on the central display unit 100. And grid lines indicating positions in the Y direction are displayed.

  Data indicating the range and tone color set by the user is also sent from the operation unit 103 to the MIDI data generation unit 113. The MIDI data generation unit 113 stores data indicating the received tone range and timbre in the RAM 1131 and, at the same time, generates a program change message that is MIDI data for designating a timbre based on the data, and the generated program change message is voiced. The data is sent to the data generation unit 111. In response to the received program change message, the sound data generation unit 111 sets the tone settings of the first to sixth instrument parts to “piano”, “synth pad”, “guitar”, “sax”, “bass”, and “drum”. Change each to “set”.

  Subsequently, the user rubs the surface of the touch panel 102 using the stick 20 like a stylus pen, or hits the surface of the touch panel 102 using the stick 20 like a drum stick. In accordance with such user performance, the position data generation unit 108 and the pressure data generation unit 110 generate position data and pressure data, respectively, and send the generated data to the MIDI data generation unit 113.

  When the MIDI data generating unit 113 receives the position data and the pressure data, based on the X coordinate included in the received position data, the MIDI data generating unit 113 determines a note number that is a parameter indicating a pitch based on the Y coordinate included in the received position data. The channel number, which is a parameter for designating the instrument part, and the velocity, which is a parameter for designating the sound intensity, are determined based on the received pressure data. The MIDI data generation unit 113 generates a note-on message using the note number, channel number, and velocity determined as described above.

  The MIDI data generation unit 113 sends the generated note-on message to the audio data generation unit 111. When the audio data generation unit 111 receives the note-on message from the MIDI data generation unit 113, for example, the waveform data of the tone of the instrument part specified by the channel number included in the received note-on message and the pitch specified by the note number On the other hand, the audio data is generated by adjusting the volume according to the velocity. The audio data generation unit 111 sends the generated audio data to the D / A converter 114, and as a result, a musical tone having a pitch, tone color and volume corresponding to the performance operation using the user's stick 20 is generated from the speaker 105. The

  In the fourth embodiment, as in the first embodiment, an image showing the performance performance of the user is displayed on the central display unit 100. Also in the fourth embodiment, similarly to the first embodiment, by using a plurality of panel-type musical instruments 4 connected by a communication network or the like, a plurality of users can perform a dual performance or a multiple performance. Since these operations in the panel-type musical instrument 4 are the same as those in the panel-type musical instrument 1, description thereof will be omitted.

  In the above description, the case where the pitch is assigned in the X direction and the tone color is assigned in the Y direction is exemplified, but which parameter is assigned in which direction can be arbitrarily changed.

  In the above description, the voice data generation unit 111 exemplifies the case of generating voice data indicating a voice that changes in semitone units such as a piano. However, the voice data generation unit 111 is arbitrary such as a violin according to position data. It may be configured to generate audio data of the pitch. In the above description, the case where different discontinuous timbres are assigned in the Y direction has been exemplified. For example, a violin timbre is assigned to the upper part in the Y direction, and a flute timbre is assigned to the lower part. A configuration may be adopted in which the timbre changes continuously from the upper part to the lower part, for example, by crossfading the data. By freely assigning pitches and timbres as described above, the user can use the panel-type musical instrument 4 to perform music expression that has been difficult with a conventional music keyboard or the like.

  In the above description, the case where only one stick 20 is used has been exemplified. However, for example, the panel-type musical instrument 4 may be configured to include a plurality of sticks 20. In that case, for example, different frequency bands are assigned to the respective sticks 20 so that radio signals transmitted from the respective sticks 20 can be distinguished, and for example, a plurality of antennas 104 and wireless communication units 109 are provided. The position of each stick 20 may be specified from the received radio wave intensity.

[5. Aspect]
The present invention can be applied to various electronic devices such as an electronic stationery such as a PDA, a toy, a musical instrument, and the like. Examples of embodiments when the present invention is applied to a musical instrument are shown below.

[5.1. First aspect]
Acquisition means for acquiring position data indicating the indicated position on the plane and pressure data indicating the pressure intensity at the indicated position; and control of tone data indicating a tone waveform or tone generation based on the position data and the pressure data. A panel-type musical instrument comprising: data generation means for generating at least one of performance control data to be instructed; and output means for outputting data generated by the data generation means.

[5.2. Second aspect]
Acquisition means for acquiring position data indicating the indicated position on the plane and pressure data indicating the pressure intensity at the indicated position; and control of tone data indicating a tone waveform or tone generation based on the position data and the pressure data. Data generation means for generating at least one of performance control data to be instructed, output means for outputting data generated by the data generation means, and drawing means for drawing based on the position data and the pressure data A panel-type instrument characterized by

[5.3. Third aspect]
Acquisition means for acquiring position data indicating the indicated position on the plane and pressure data indicating the pressure intensity at the indicated position; storage means for storing model performance control data for instructing control of sound generation; and the position data; Data generating means for generating at least one of musical tone data indicating a waveform of musical tone or performance control data for instructing control of pronunciation of musical tone using the pressure data and the model performance control data; and generated by the data generating means A panel-type musical instrument comprising output means for outputting data.

[5.4. Fourth aspect]
Acquisition means for acquiring position data indicating the indicated position on the plane and pressure data indicating the pressure intensity at the indicated position; storage means for storing a plurality of model performance data indicating performance phrases; the position data and the pressure data; On the basis of the selection means for selecting one or more model performance data from a plurality of model performance data stored by the storage means, the position data, the pressure data, and the model performance data selected by the selection means are used to generate musical sounds. Data generation means for generating at least one of musical sound data indicating a waveform or performance control data for instructing control of sound generation, and output means for outputting the data generated by the data generation means Panel type instrument.

[5.5. Fifth aspect]
Acquisition means for acquiring position data indicating the indicated position on the plane and pressure data indicating the pressure intensity at the indicated position; and control of tone data indicating a tone waveform or tone generation based on the position data and the pressure data. Data generating means for generating at least one of performance control data to be instructed, output means for outputting data generated by the data generating means, model position data indicating a locus of a position on a plane, and pressure intensity on the locus Storage means for storing model data indicating a model writing operation including model pressure data indicating changes in the position, and the writing operation indicated by the position data and pressure data acquired by the acquisition unit and the model data. Index calculating means for calculating an index indicating the degree of similarity between the writing operation and a notifier for notifying the index Panel-type instrument, characterized in that it comprises and.

[5.6. Sixth aspect]
Acquisition means for acquiring position data indicating the indicated position on the plane and pressure data indicating the pressure intensity at the indicated position; and control of tone data indicating a tone waveform or tone generation based on the position data and the pressure data. Data generation means for generating at least one of the performance control data to be instructed, and transmission means for transmitting at least one of the position data, the pressure data, and the data generated by the data generation means to another device. A panel-type instrument characterized by

[5.7. Seventh Aspect]
Receiving means for receiving the position data indicating the indicated position on the plane and the pressure data indicating the pressure intensity at the indicated position from another device; and the tone data or tone data indicating the waveform of the tone based on the position data and the pressure data A panel-type musical instrument comprising: data generation means for generating at least one of performance control data for instructing sound generation control; and output means for outputting data generated by the data generation means.

[5.8. Eighth aspect]
An acquisition means for acquiring coordinate data indicating the indicated position on the plane by coordinates and pressure data indicating the pressure intensity at the indicated position; and a tone color according to any one of the X component and the Y component included in the coordinate data A timbre determining means for determining, a pitch determining means for determining a pitch according to a component that the timbre determining means does not use for determining a timbre among the X component and the Y component included in the coordinate data, and the pressure data Sound intensity determining means for determining the sound intensity in response, tone color determined by the tone color determining means, pitch determined by the pitch determining means, and sound intensity determined by the sound intensity determining means Data generating means for generating at least one of musical tone data indicating a waveform of a musical tone corresponding to the sound or performance control data for instructing control of pronunciation of the musical sound, and the data generating means Panel-type instrument according to an outputting means for outputting made data.

1 is a perspective view showing an appearance of a panel-type musical instrument according to a first embodiment of the present invention. It is the functional block diagram which divided the structure of the panel type musical instrument which concerns on 1st Embodiment of this invention according to the function. It is a figure which shows the structure of the model MIDI data based on 1st Embodiment of this invention. It is a figure showing composition of a performance system using a plurality of panel type musical instruments concerning a 1st embodiment of the present invention. It is a figure which shows the structure of the data which the model MIDI data storage part concerning 2nd Embodiment of this invention has memorize | stored. It is a perspective view which shows the external appearance of the panel type musical instrument which concerns on 3rd Embodiment of this invention. It is the functional block diagram which divided the structure of the panel type musical instrument which concerns on 3rd Embodiment of this invention according to the function. It is a figure which shows the structure of the reference pressure data group which concerns on 3rd Embodiment of this invention. It is a figure which shows the structure of the data memorize | stored in the model position pressure data storage part which concerns on 3rd Embodiment of this invention. It is a perspective view which shows the external appearance of the panel type musical instrument which concerns on 4th Embodiment of this invention.

Explanation of symbols

1, 2, 3, 4... Panel type musical instrument, 10. Operation unit 104 · 203 ... antenna 105 · speaker · 106 · wired communication unit · 107 · control unit · 108 · position data generation unit · 109 · 202 · wireless communication unit · 110 · pressure data generation unit · 111 · audio data Generation unit 112 ... Template MIDI data storage unit 113 ... MIDI data generation unit 114 ... D / A converter 115 ... Amplifier 200 ... Head unit 201 ... Pressure sensor 316 ... Pressure sensor panel 317 ... Position pressure data Generation unit, 318 ... template position pressure data storage unit, 319 ... index calculation unit, 520 ... upper display unit, 521 ... left display unit, 111 ... PCM sound generator section, 1112 ... FM sound source unit, 1131 · 3191 ... RAM, 3171 ... the reference pressure data group.

Claims (7)

Storage means for storing a plurality of model performance control data for instructing control over the pronunciation of musical sounds in time series;
Obtaining means for obtaining position data indicating a designated position on a plane;
Read out the model performance control data from the storage means, and output means for outputting the read model performance control data,
The output means accumulates a movement amount by which the indicated position moves based on the position data acquired by the acquisition means while outputting the first model performance control data read from the storage means. The second model performance control data is read from the storage means according to the accumulated movement amount , and the output of the second model performance control data is output simultaneously with the completion of the output of the first model performance control data. A data processing apparatus characterized by starting. The storage means stores a plurality of model performance control data respectively corresponding to one measure constituting the music,
The data processing according to claim 1, wherein the output means reads out and outputs the first and second model performance control data from the plurality of model performance control data respectively corresponding to one measure. apparatus. The acquisition means acquires pressure data indicating the pressing strength at the indicated position together with position data indicating the indicated position on the plane,
Said output means to read model performance control data from said storage means, make changes to the velocity of該雛type performance data in accordance with pressure data acquired by the acquisition unit after the reading the model performance control data The data processing apparatus according to claim 1, wherein the modified performance data is output after the change. Based on prior Symbol position data and the pressure data, the data processing apparatus according to claim 3, characterized in that it comprises a drawing means for drawing. Before term memory means, respectively each of the corresponding plurality of model performance control data, and stored in association with the basic note length is a note length of the underlying included in the corresponding phrases to the phrase that constitutes the music,
The output means is stored in association with the basic note length corresponding to the amount of movement of the position indicated by the position data acquired by the acquisition means while outputting the first model performance control data. the data processing apparatus according to any one of claims 1 to 4, characterized in that read from said storage means stationery performance control data as said second model performance control data. Before SL storage means stores each of the plurality of model performance control data respectively corresponding to the phrase that constitutes music, in association with the degree of pitch change of the tone contained in the corresponding phrases,
The output means is stored in association with the degree of pitch change corresponding to the amount of movement of the position indicated by the position data acquired by the acquisition means while outputting the first template performance control data. The data processing apparatus according to any one of claims 1 to 4 , wherein the model performance control data is read from the storage means as the second model performance control data. A process for storing a plurality of template performance control data for chronologically instructing the control of the sound generation;
A process of reading out the first model performance control data from the stored plurality of model performance control data and outputting the first model performance control data;
A process of acquiring position data indicating a designated position on a plane while outputting the first model performance control data;
Based on the position data acquired while outputting the first model performance control data, an accumulated movement amount is calculated by accumulating the movement amount by which the designated position moves , and the stored movement amount is stored according to the accumulated movement amount . A process of reading the second model performance control data from the plurality of model performance control data and starting outputting the second model performance control data simultaneously with the completion of the output of the first model performance control data. A program to be executed.

Images