JP5821170B2 - Electronic music apparatus and program - Google Patents

Description

  The present invention relates to an electronic music apparatus and a program for generating a musical sound in response to a touch operation on a virtual performance operator drawn on a display screen capable of multipoint touch detection. In particular, the present invention relates to a technique for preventing occurrence of sound interruption and drawing delay due to an increase in the number of simultaneous pronunciations when a plurality of virtual performance operators drawn on a screen are simultaneously touch-operated.

  Recently, for example, even on a small and light portable terminal device such as a smartphone or a tablet computer, any application software selected by a user can be appropriately operated by a CPU (Central Processing Unit) or the like. ing. One example of such application software is an image simulating a performance instrument of a natural instrument such as a piano keyboard or a guitar string or frame on a touch-detectable display screen (hereinafter referred to as a virtual performance instrument). The virtual performance operation element that has been drawn is touched by the user with a finger, a touch pen, or the like, and is associated with the virtual performance operation element that has been touched in advance. In addition, there are music software programs for generating a predetermined musical sound waveform and generating a musical sound. By operating this, the user can enjoy the portable terminal device as an electronic music device. As a device related to such an apparatus, for example, there is a technique described in Patent Document 1 shown below.

Special table 2007-527061 gazette

  By the way, in an electronic music apparatus that realizes drawing of virtual performance operators and generation of musical sound waveforms by software processing, software processing is executed as the number of virtual performance operators simultaneously drawn and the number of generated musical sound waveforms increases. Naturally, the processing load of the CPU to be increased. In particular, in an electronic music apparatus having a display capable of multi-point touch detection, a large number of drawn virtual performance operators are simultaneously touch-operated to increase the number of simultaneous pronunciations, that is, sound generation processing (musical sound waveform generation processing) in parallel. ) Increases the number of musical tones that need to be performed, a waiting time will occur before the processing exceeds the processing capacity of the CPU, and as a result, the musical sounds are interrupted (sound interruptions) and depending on the touch operation. There has been a problem that inconveniences such as the fact that the virtual performance operator cannot be updated (drawing delay) cannot be avoided.

  The present invention has been made in view of the above points, and is capable of preventing the occurrence of sound interruption and drawing delay caused by simultaneous touch operation of a plurality of virtual performance operators to increase the number of simultaneous pronunciations. An object is to provide an electronic music apparatus and a program.

An electronic music apparatus according to the present invention is an electronic music apparatus that generates a musical sound based on a musical sound waveform generated in accordance with execution of predetermined musical sound generation software including a plurality of processing steps by a processor. A drawing means for drawing one or more virtual performance operators on the screen of the display device capable of simultaneously detecting touch operations at a plurality of positions, and one for the virtual performance operator drawn on the screen. In response to detection of a plurality of touch operations, the operated virtual performance operator is identified, a predetermined musical sound waveform associated with the identified virtual performance operator is generated, and the drawing means a music generation means for instructing a drawing control to vary the display mode of the identified virtual performance operator, processing according to the musical tone waveform generation in the tone generating means upon detection of said touch operation A load measuring unit that measures a load, and a limit value of the number of simultaneous touches that are valid according to the measured processing load are increased and decreased, and the increased or decreased number of simultaneous touches among the detected one or more touch operations. the up limit value after the generation and the drawing Esei control instructions of the musical tone waveform, also with respect to the musical tone generating means so as not to perform the instruction for the drawing control for touch operation exceeds the limit value Load control means for instructing.

According to the present invention, when one or a plurality of touch operations to the virtual performance operator drawn on the screen is detected, the operated virtual performance operator is identified and associated with the identified virtual performance operator. A predetermined musical tone waveform is generated, and the processing load related to the musical tone waveform generation of the musical sound generation means that instructs the drawing means to change the display mode of the specified virtual performance operator is measured. . Then, the limit value of the number of simultaneous touches that is effective according to the measured processing load is increased or decreased, and the musical tone waveform from the detected one or more touch operations to the limit value of the increased or decreased number of simultaneous touches. Is instructed to the tone generation means to control generation and drawing. In this way, when touch operations are performed simultaneously on a large number of virtual performance operators, the processing load related to the musical sound waveform generation at that time is measured, and the number of simultaneous touches that are effective according to the measured processing load is determined. perform increase or decrease of the limit value, until said detected the first to the increased or decreased by simultaneous touch number of the limit values of the plurality of touch operation performed to generate and the drawing Esei control instructions of the musical tone waveform, also, the By not performing the drawing control instruction for touch operations exceeding the limit value, it is possible to perform tone waveform generation and drawing control without imposing excessive processing load on the tone generation means. For this reason, even if touch operations are performed simultaneously on a large number of virtual performance operators, the number of simultaneous pronunciations does not increase until an excessive processing load is applied. Cutting and drawing delays cannot occur. In addition, the display mode of the virtual performance operator is changed for a touch operation within the limit value range, and the display mode of the virtual performance operator is not changed for a touch operation exceeding the limit value. Therefore, the user can easily grasp which virtual performance operator is invalid.

  The present invention can be constructed and implemented not only as a device invention but also as a method invention. Further, the present invention can be implemented in the form of a program of a processor such as a computer or a DSP, or can be implemented in the form of a storage medium storing such a program.

  According to the present invention, even when touch operations are simultaneously performed on a large number of virtual performance operators, the sound waveform is generated according to all touch operations and the processing load related to the sound waveform generation is controlled without controlling the drawing. Since the sound waveform is generated and drawn according to only a part of the touch operations, the number of simultaneous pronunciations increases when multiple virtual performance operators drawn on the screen are touched simultaneously. There is an effect that it is possible to prevent the occurrence of sound interruption, drawing delay, and the like.

1 is a block diagram of a hardware configuration showing an example of the overall configuration of an electronic music apparatus according to the present invention. It is a functional block diagram for demonstrating a drawing function and a musical sound production | generation function. It is a flowchart which shows one Example of the process which implement | achieves the function of a load control part. It is a flowchart which shows one Example of a touch start detection process. It is a conceptual diagram which shows one Example of a coordinate list. It is a flowchart which shows one Example of a touch movement detection process. It is a flowchart which shows one Example of a touch completion | finish detection process. It is a flowchart which shows one Example of the process which implement | achieves the function of a software sound source part.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a hardware configuration block diagram showing an embodiment of the overall configuration of an electronic music apparatus according to the present invention. The electronic music apparatus shown in this embodiment is a tablet computer, for example, and is controlled by a microcomputer comprising a CPU 1 (Central Processing Unit), a read only memory (ROM) 2 and a random access memory (RAM) 3. The CPU 1 controls the operation of the entire electronic music apparatus. ROM 2, RAM 3, detection circuits 4 and 5, display circuit 6, sound system 7, storage device 8, and communication interface (I / F) 9 are connected to the CPU 1 via a data and address bus 1D. .

  The ROM 2 stores various control programs executed by or referred to by the CPU 1, various data, and the like. The RAM 3 is used as a working memory for temporarily storing various data generated when the CPU 1 executes a predetermined program, or as a memory for temporarily storing a currently executing program and related data. The A predetermined address area of the RAM 3 is assigned to each function, and is used as a register, a flag, a table, a temporary memory, or the like.

  The setting operation element (switch or the like) 4A is arranged around the display 6A, for example, and can be realized according to display contents displayed at predetermined corresponding positions on various screens (not shown) displayed on the display 6A. 1 to a plurality of general-purpose switches whose control functions are changed. More specifically, for example, switches for performing various settings (musical tone control parameters, etc.) relating to musical tone control such as pitch, tone, and effect, and automatic accompaniment It can function as various operators such as buttons for instructing start / stop. Of course, the setting operator 4A may include various operators such as a numeric keypad for inputting numeric data and a keyboard for inputting character data instead of or in addition to the above. The detection circuit 4 detects the operation state of the setting operation element 4A, and outputs switch information and the like corresponding to the operation state to the CPU 1 via the data and address bus 1D.

  The detection circuit 5 outputs a plurality of positions (coordinates) on the display 6A touched by the user detected by the multipoint detection type touch panel 5A to the CPU 1 via the data and address bus 1D. The display circuit 6 is provided on a display 6A composed of, for example, a liquid crystal display panel (LCD) or the like, for example, a musical instrument performance operator such as a piano keyboard, a guitar string and frame, or a drum set, or Tenorion (trademark). A virtual performance operator imitating a matrix type performance operator is displayed. That is, the touch panel 5A and the detection circuit 5, the display 6A, and the display circuit 6 described above constitute a display device with a multi-touch function that has a detection function for simultaneously detecting (recognizing) a plurality of user touch operations performed on the screen. doing. Needless to say, the setting status of various settings (musical sound control parameters and the like) related to the musical tone control, various data stored in the ROM 2 and the storage device 8, or the control state of the CPU 1 may be displayed on the display 6A. .

  The sound system 7A includes a D / A converter, an amplifier, a speaker, and the like, and generates a musical sound based on musical sound waveform data generated by execution of software processing by the CPU 1. Note that generation of musical sound waveform data by the CPU 1 may employ any of various musical sound synthesis methods such as FM, PCM, physical model, and formant synthesis. The musical sound waveform data may be generated by executing software processing by a dedicated DSP (Digital Signal Processor).

  The storage device 8 stores various control programs executed by the CPU 1 in addition to various data such as image information related to the virtual performance operator and waveform data (not shown) of various timbres. When the control program is not stored in the ROM 2 described above, the control program is stored in the storage device 8 (for example, a hard disk) and is read into the RAM 3 to store the control program in the ROM 2. It is possible to cause the CPU 1 to execute the same operation as in FIG. In this way, control programs can be easily added and upgraded. The storage device 8 is not limited to a hard disk (HD), but may be a flexible disk (FD), a compact disk (CD-ROM / CD-RAM), a magneto-optical disk (MO), or a DVD (Digital Versatile Disk). Any storage device may be used as long as it uses a storage medium in the form. Alternatively, a semiconductor memory such as a flash memory may be used.

  A communication interface (I / F) 9 is an interface for transmitting and receiving a control program and various data between the apparatus and an external device (not shown). The communication interface 9 is, for example, a dedicated wired interface for music such as MIDI, a general-purpose short-distance wired interface such as USB or IEEE1394, a general-purpose network interface such as Ethernet (registered trademark), or a general-purpose short-range wireless such as wireless LAN or Bluetooth (trademark). One or more interfaces may be provided.

  The electronic music apparatus according to the present invention is not limited to a tablet computer, and may be in the form of any device / equipment such as an electronic musical instrument, a personal computer, a mobile communication terminal such as a mobile phone, or a game device. In the case of a mobile communication terminal, not only the case where a predetermined function is completed with only the terminal, but also a part of the function is provided on the server side so that the predetermined function is realized as a whole system including the terminal and the server. May be. In the case of an electronic musical instrument, it may physically have an operator dedicated to performance such as a keyboard provided with a plurality of keys for selecting the pitch of a musical tone. Of course, such an operator is not limited to a keyboard instrument, and may be any type of instrument such as a stringed instrument, a wind instrument, or a percussion instrument.

  In the electronic music apparatus according to the present invention, with the execution of software processing by the CPU 1, a virtual performance operator is mainly drawn on a display device with a multi-touch function, and a user touches the drawn virtual performance operator. According to the operation, a predetermined musical sound waveform is generated and a musical sound is generated. Therefore, the drawing function and the tone generation function among various functions realized by the execution of software processing in the CPU 1 will be described with reference to FIG. FIG. 2 is a functional block diagram for explaining the drawing function and the tone generation function. In FIG. 2, the arrows in the figure represent the flow of signals.

  The performance operator drawing unit A displays virtual performance operators such as images simulating performance operators of natural instruments such as keyboards, percussion instruments, and string instruments, and images of matrix-type performance operators such as Tenorion (trademark). Drawing is performed on the display device B with a touch detection function (hereinafter simply referred to as a multi-touch display device). Of course, not only drawing the virtual performance controls, but also setting images such as sliders (for example, sliders, etc.) and setting values for setting operation modes, musical tone control parameters, and the like. You may make it display the character etc. to represent.

  As described above, the multi-touch display device B on which virtual performance operators, virtual setting operators, and the like are drawn includes the display 6A and the multi-point detection type touch panel 5A that can simultaneously detect a plurality of positions on the display 6A. Prepare. As the touch detection method of the touch panel 5A, any known method can be adopted, and for example, a capacitance method can be adopted. When the touch start on the display 6A by the user, touch movement (moving on the display 6A while touching), or touch end on the display 6A is detected by the touch panel 5A, the current coordinates (Xn, Yn) at that time are detected. The old coordinates (Xn, Yn) at the previous detection are notified to the load control unit E as coordinate information. Here, the symbol “n” in the coordinate (Xn, Yn) is an integer (for example, 10) indicating the maximum limit of the number of touches that can be simultaneously detected of 1 or more. The display 6A is not limited to liquid crystal, and may be any system such as electronic paper.

  The load control unit E receives notification of a touch operation from the multi-touch display device B (more specifically, each event of touch start, touch movement, and touch end and coordinate information corresponding to each event), and the coordinates are displayed on the display. The performance operator drawing unit A is inquired as to whether the virtual performance operator or virtual setting operator drawn on 6A corresponds to the drawing position. Based on the inquiry result, it is determined whether the touch operation by the user corresponds to the on / off operation of the virtual performance operator or the operation of the virtual setting operator, and if it is determined to correspond to these operations, the instrument core unit Notify D to that effect. For example, if “touch start” is on the virtual performance controller, the “virtual performance controller is on” operation. If the “touch start” is off the virtual performance controller by “touch movement”, the “virtual performance controller is off” operation. When moving to a virtual performance control from a location other than the virtual performance control by “move”, “virtual performance control on” is operated. When “touch end” is on the virtual performance control, “virtual performance control is off”. Judged to be applicable to the operation. However, depending on the load state of the CPU 1 notified from the load measuring unit H, the notification to the musical instrument core unit D is not performed as if the notification of the touch operation from the multi-touch display device B is ignored. (Details will be described later).

  The musical instrument core part D (corresponding to a musical tone generating means) is touch-operated in accordance with each operation information such as an on / off operation event of a virtual performance operator and an operation event of a virtual setting operator notified from the load control unit E. In addition, each of the operators is specified, and according to the specified operator, instructions for sound generation / mute (occurrence of note-on / note-off event) and instructions for setting a sound control parameter are given to the software tone generator G. The performance operator drawing section A is notified of the operation information of each operator. In response to the notification of the operation information, the performance operator drawing unit A performs drawing for changing the display mode of the virtual performance operator and the virtual setting operator (for example, as if the virtual performance operator is a keyboard type, the keyboard By changing the setting value of the musical tone control parameter or the like, the operation state and the setting state are notified to the user.

  The automatic performance section C can record and / or reproduce automatic performance (song) data, and can record and / or reproduce automatic accompaniment and automatic rhythm patterns. Along with the reproduction of the automatic performance data, automatic accompaniment, and automatic rhythm pattern data, the musical instrument core unit D is notified of a sounding on event, an off event, etc. separately from the touch operation by the user. In this case, the musical instrument core unit D instructs the software sound source unit G to generate / mute music according to an on / off event or the like notified from the automatic performance unit C. Further, an on / off event notified from the automatic performance unit C is notified to the performance operator drawing unit A as necessary, and an image drawing based on the automatic performance or the like is performed by the performance operator drawing unit A. Also good.

  The software tone generator G (corresponding to the musical sound generating means together with the musical instrument core D) is configured to generate musical sounds based on various events such as musical sound generation / mute instructions and musical sound control parameter setting instructions notified from the musical instrument core D. Generate waveform data. This musical sound waveform data generation process is performed periodically. For example, if the operating frequency is 44100 Hz and the frame size (number of musical sound waveform samples generated in one process) is 1024, the musical sound waveform generation cycle is “about 23.2 ms (= 1024 × 1000/44100)”, which is about 23.2. Generation processing of musical sound waveform data for one frame is performed every ms.

  The time required for generating musical sound waveform data for one frame and generating a musical sound based on this data varies depending on the number of simultaneous sound generations, the setting state of musical sound control parameters, and the like. For example, if it is assumed that the number of simultaneous pronunciations is 16 and the musical tone control parameter is in the first predetermined state, it can be generated or pronounced in 12 ms. If the control parameter is assumed to be in a second predetermined state (for example, a state in which the number of parameters to be controlled is larger than that of the first predetermined state and complicated waveform processing is required), the control parameter is generated in 20 ms. Or can be pronounced (however, the numerical values described here are merely examples).

The load measuring unit H measures the time taken to generate a musical sound based on the generation of the musical sound waveform data described above or based on this, and the processing load related to the musical sound waveform generation in the CPU 1 (hereinafter simply referred to as CPU) based on the following equation (1). Called load).
(Equation 1)
CPU load = time required to generate or sound musical tone waveform data for one frame
/ Period for generating musical sound waveform data for one frame Specifically, for example, the CPU load when generating or sounding in 12 ms is “about 52% (= 12 × 100 / 23.2)”, and generating or sounding in 17 ms. If it is possible, the CPU load is “approximately 73% (= 17 × 100 / 23.2)”. The CPU load thus obtained is notified to the load control unit E.

  By the way, if the CPU load is high, there is a possibility that further musical tone waveform generation according to a new touch operation or drawing of a virtual performance operator may not be able to catch up, and in the worst case, sound interruption or drawing delay may occur. . In particular, the electronic music device is not a dedicated device that operates only the dedicated music software program, but a general-purpose device capable of operating various application software in parallel. Other application software is available in addition to generating sound waveforms and drawing virtual performance controls. When the system is operated and other processes are executed at the same time, the sound is cut off and the drawing is more prominent. Therefore, in the electronic music apparatus according to the present invention, in order to prevent the occurrence of sound interruption and drawing delay as described above, control is performed so as to reduce the load on the generation of the musical sound waveform and the drawing of the virtual performance operator in accordance with the CPU load. I have to.

  Specifically, the load control unit E increases or decreases the number of simultaneous touches notified from the multi-touch display device B according to the touch number limit value in accordance with the CPU load notified from the load measurement unit H, and increases the number of simultaneous touches to the musical instrument core unit D. Notify operation information. For example, when the CPU load is high, up to 5 touches are valid when the CPU load is 50%, up to 3 touches when the CPU load is 60%, and up to 2 touches when the CPU load is 70%. Decreases the number of effective simultaneous touches, and increases the number of effective simultaneous touches when the CPU load is low. Such a limit on the number of simultaneous touches depending on the CPU load may be set using a table (not shown) prepared in advance, or obtained by a calculation formula (not shown) representing the characteristics of such a table. You may do it.

  Here, the increase / decrease in the number of simultaneous touches according to the CPU load will be described using a specific example. For example, it is assumed that automatic performance data is being reproduced in the automatic performance section C, and at the same time, musical sound waveform data for 10 sounds has already been reproduced. Assuming that the CPU load at this time is 40%, the touch number limit value sets the number of effective simultaneous touches up to five. If the user touches five different virtual performance operators (for example, a keyboard) using five fingers, the instrument core unit D is notified of all five note-ons, and the software tone generator unit G Then, musical tone waveform data of five tones is newly generated and a tone is generated, and the display mode of the virtual performance operator operated by the performance operator drawing unit A is changed. When musical tone waveform data for a total of 15 tones including 5 new tones is thus generated, the CPU load at this time is 60%. When five fingers are released from the operated virtual performance operator, five note-offs are notified to the musical instrument core part D, and the corresponding five musical tones are muted, and the virtual performance operator is displayed. Is restored. Then, the CPU load is reduced to 40% when the musical sound waveform data for 10 sounds is being reproduced.

  It is assumed that the number of musical tones reproduced by the automatic performance section C is further increased, for example, musical sound waveform data for 20 sounds is being reproduced at the same time, and the CPU load at this time is 60%. In this case, for example, the number of simultaneous touches is limited to 3 based on the touch number limit value, so that even if the user touches 5 different virtual performance operators using 5 fingers, Only touch operations on the virtual performance controls are valid. Therefore, only three note-ons are notified to the musical instrument core part D, and the musical sound waveform data for a total of 23 sounds including the three newly added sounds by the software sound source part G is generated and the performance operation is performed. Only the three virtual performance operators operated in the child drawing section A are changed in display mode. The CPU load is 65%. In this way, when the CPU load is high and the number of simultaneous touches is reduced, for example, the display mode of the virtual performance operators is changed only for three, even though five virtual performance operators are touch-operated. As a result, the user can easily grasp which virtual performance operator is invalid.

It should be noted that the processing corresponding to which virtual performance operator is invalidated depends on first-come-first-served, random, last-arrival priority, importance of the performance operator (for example, white keys are more important than black keys, Arbitrary methods such as judging from the current performance content may be adopted.
Further, as described above, a valid limit value of the number of simultaneous touches (touch number limit value) may be determined only from the current value of the CPU load, but it should be processed based on past CPU load trends and the next processing timing. A future CPU load may be predicted by referring to the number of events (that is, the number of on / off events accumulated in the processing waiting buffer), and the limit value of the number of simultaneous touches may be determined from the predicted value. Such a measurement of the CPU load and the limitation on the number of simultaneous touches according to the measurement value may be performed every time a touch operation is detected, or may be performed every musical sound waveform generation cycle of one frame unit.

  Digital musical sound waveform data generated by the software tone generator G is sent to the sound system 7, converted into an analog musical sound waveform signal by the D / A converter F, and emitted from the speaker I.

  Next, a specific process for realizing the function of the load control unit E as described above will be described. FIG. 3 is a flowchart illustrating an example of a process for realizing the function of the load control unit E. The processing is started every predetermined time (for example, several ms), and these touch operations are performed in accordance with acquisition (notification) of touch start, touch movement, and touch end events and coordinate information from the multi-touch display device B. The processing corresponding to the type of touch and the touch position coordinates is executed. That is, as shown in FIG. 3, in step S1, a touch start detection process (see FIG. 4) corresponding to the touch start operation is executed. In step S2, a touch movement detection process (see FIG. 6) corresponding to the touch movement operation is executed. In step S3, a touch end detection process (see FIG. 7) corresponding to the touch end operation is executed.

  The “touch start detection process” (see step S1 in FIG. 3) will be described with reference to FIG. FIG. 4 is a flowchart illustrating an example of “touch start detection processing”.

  In step S11, it is determined whether or not a touch start operation is detected. That is, it is determined whether or not a touch start event is notified from the multi-touch display device B. If it is determined that the touch start operation has not been detected (NO in step S11), the process ends. On the other hand, if it is determined that the touch start operation has been detected (YES in step S11), coordinate information (referred to as touch start coordinates) notified from the multi-touch display device B together with the touch start event is added to the coordinate list ( Step S12).

  An example of the coordinate list is shown in FIG. As shown in FIG. 5, the coordinate list is provided for each touch operation that can be detected simultaneously (however, as an example, the maximum limit (n) of one or more simultaneously detectable touches is 10). The start coordinates, previous coordinates, current coordinates, and valid / invalid of touch operations can be registered. The registered contents of the start coordinate, previous coordinate, and current coordinate can change depending on the touch operation event notified from the multi-touch display device B, that is, the touch type. When the touch start event is notified, all of the start coordinate, previous coordinate, and current coordinate are touched. The start coordinates are registered (that is, the same coordinates are registered for all). The start coordinate indicates a coordinate at which a touch operation is performed first, and is not updated except for being deleted when a touch end event is notified. On the other hand, the previous coordinates and the current coordinates are not only deleted when a touch end event is notified, but are also updated to appropriate coordinates when a touch movement event is notified (details will be described later).

  Returning to the description of FIG. 4, the step S13 inquires the performance operator drawing unit A about the touch start operation corresponding to the notified touch start coordinates. In step S14, it is determined whether or not the inquiry result is a touch start operation corresponding to the “virtual performance operator ON” operation. Here, when the touch start position on the display 6A based on the touch start coordinates is on the virtual performance operator drawn on the multi-touch display device B, the touch corresponding to the “virtual performance operator on” operation is performed. It is determined that the operation is a start operation. If it is determined that the touch start operation does not correspond to the “virtual performance operator ON” operation (NO in step S14), processing corresponding to the touch start position on the display 6A based on the touch start coordinates is executed ( Step S15), the process jumps to step S20. As a process executed at this time, for example, an instruction to set a musical sound control parameter when the touch start position on the display 6A based on the touch start coordinates is on a virtual setting operator drawn on the multi-touch display device B. (For example, parameter on setting for starting reflection of parameters in musical tone).

  When it is determined that the touch start operation corresponds to the “virtual performance operator ON” operation (YES in step S14), the CPU load is acquired from the load measuring unit H, and the touch count limit value (in accordance with the load) ( n, for example, 10) is determined (step S16). Then, it is determined whether or not the total number of valid touches registered as “valid” (valid touch count) in the coordinate list shown in FIG. 5 is less than the upper limit of the determined touch count limit value (n). Determination is made (step S17). If it is determined that the current valid touch count is not less than the upper limit of the touch count limit value, that is, the touch operation exceeds the limit (NO in step S17), the detected new touch start operation is invalidated and the corresponding coordinate list is displayed. The fact (invalid) is registered in the place (step S18). In this case, an on / off operation event of the virtual performance operator is not notified to the musical instrument core unit D, and the generation of musical sound waveform data by the software tone generator G or the virtual performance operator by the performance operator drawing unit A is not performed. Drawing that changes the display mode is not performed.

  On the other hand, when it is determined that the current number of valid touches is less than the upper limit of the touch number limit value (n) (YES in step S17), the detected new touch start operation is validated and the corresponding position in the coordinate list is set. The fact (valid) is registered, and a “virtual performance operator ON” operation event is notified to the musical instrument core part D (step S19). In this case, the musical instrument core unit D instructs the software sound source unit G to generate a musical tone or notifies the performance operator drawing unit A of the operation information according to the notification. The drawing that changes the display mode of the virtual performance operator by the generation and performance operator drawing unit A is performed.

  In step S20, it is determined whether or not a further touch start operation is detected at the same time, that is, whether or not another touch start event is notified based on a plurality of touch operations performed simultaneously. At the same time, if it is determined that a further touch start operation is detected (YES in step S20), the process returns to step S12, and the processes from step S12 to step S19 are repeatedly executed. At the same time, if it is determined that no further touch start operation has been detected (NO in step S20), the process ends.

  Next, the “touch movement detection process” (see step S2 in FIG. 3) will be described with reference to FIG. FIG. 6 is a flowchart illustrating an example of the “touch movement detection process”.

  A step S31 decides whether or not a touch movement operation has been detected. That is, it is determined whether or not a touch movement event is notified from the multi-touch display device B. If it is determined that the touch movement operation has not been detected (NO in step S31), the process ends. On the other hand, if it is determined that a touch movement operation has been detected (YES in step S31), the previous and current coordinates in the coordinate list are updated based on the coordinate information notified from the multi-touch display device B together with the touch movement event ( Step S32). That is, when a touch movement operation is detected, the coordinates before movement (old coordinates) and the current coordinates (current coordinates) are received from the multi-touch display device B as coordinate information, so that a value matching the previous coordinates in the coordinate list is present. A record registered as coordinates is searched, and the previous and current coordinates of the corresponding record are updated with the received value. A step S33 inquires the performance operator drawing unit A for a touch movement operation based on the notified pre-movement coordinates and current coordinates.

  Here, in the processing of step S34 to step S43 shown below, the case of the touch movement operation is divided based on the combination of the pre-movement coordinates and the current coordinates, and in each case (here, the following four cases). The processing according to is performed. The first is a case where a touch movement operation is performed from the already-on virtual performance operator to a new (other) virtual performance operator. In this case, the virtual performance operation before the touch movement operation is once performed. The musical sound that is generated based on the operation of the child is muted, and the musical sound based on the operation of the new virtual performance operator after the touch movement operation is newly started to be generated. The second is a case where a touch movement operation is performed on a new (not operated at that time) virtual performance operator other than the virtual performance operation element. In this case, a new virtual performance operation after the touch movement operation is performed. A new tone based on the child's operation is started. The third is a case where a touch movement operation is performed from a virtual performance operator that is already turned on to all virtual performance operators including the virtual performance operation element. In this case, the touch movement operation before the touch movement operation is performed. The musical sound that is sounded based on the operation of the virtual performance operator is muted, and if necessary (if any virtual setting operator etc. is drawn at the current coordinates after the touch movement operation), the corresponding operator processing is performed. Run. The fourth case is a touch movement operation within the range of the virtual performance operator that is already turned on, or a touch movement operation that has nothing to do with the virtual performance operation element. If a virtual setting operator or the like is drawn on the current coordinates after the operation, the corresponding operator processing is executed.

  In step S34, as a case corresponding to the first case, the musical instrument core unit D is notified of the “virtual performance operator ON” operation event from the “ON” virtual performance operator to the new virtual performance operator. It is determined whether or not this is a touch movement operation. In the first case, that is, when it is determined that the touch movement operation is performed from the already-on virtual performance operator to the new virtual performance operator (YES in step S34), the on-virtual performance operator before the touch movement operation is determined. Based on the operation, the musical instrument core section D is notified of the “virtual performance control element off” operation event in order to mute the musical sound being generated (step S35). Then, the CPU load is acquired from the load measuring unit H, the touch number limit value (n, for example, 10) corresponding to the load is determined (step S36), and “valid” is registered in the coordinate list shown in FIG. It is determined whether or not the total number of effective touches (number of effective touches) is less than the upper limit of the determined touch number limit value (n) (step S37). If it is determined that the current number of valid touches is not less than the upper limit of the touch number limit value (NO in step S37), the corresponding touch operation is invalidated after the touch movement operation, and this is indicated (invalid) in the corresponding position of the coordinate list. Registration is performed (step S38). In this case, only the virtual performance operator off operation event is notified to the musical instrument core part D, the generation of the corresponding musical sound waveform data by the software sound source part G is stopped, and the musical sound is silenced. Drawing that changes the display mode of the virtual performance operator before the touch movement operation by the performance operator drawing unit A is performed.

  On the other hand, when it is determined that the current number of valid touches is less than the upper limit of the touch number limit value (YES in step S37), the detected new touch start operation is validated and the corresponding point in the coordinate list is notified to that effect ( (Valid) is registered, and a “virtual performance operator ON” operation event is notified to the musical instrument core D (step S39). In this case, the musical instrument core unit D instructs the software sound source unit G to generate a musical tone or notifies the performance operator drawing unit A of the operation information in accordance with the notification. Drawing is performed to change the display mode of the virtual performance operator after the generation of data and the touch movement operation by the performance operator drawing unit A. After step S38 or step S39 is completed, the process goes to step S44.

  If it is determined in step S34 that the movement is not from the already-on performance operator to the new performance operator position (NO in step S34), a case other than the virtual performance operator (NO in step S34) ( It is determined whether or not the movement is from a non-performance operator to a virtual performance operator (step S40). In the second case, that is, when it is determined that the touch movement operation is performed from the non-performance controller to the new virtual performance controller (YES in step S40), the process proceeds to step S36, and the processes in steps S36 to S39 described above. I do. In this case, a musical tone based on the operation of the new virtual performance operator after the touch movement operation is newly started (provided that the current effective touch count is less than the upper limit of the touch count limit value).

  When it is determined that the movement is not from the non-performance controller to the performance operator position (NO in step S40), the non-performance operation is performed from the virtual performance operator that has been turned on as the case corresponding to the third or fourth case. It is determined whether or not the operation is a touch movement operation to a child (step S41). In the third case, that is, when it is determined that the movement is from the turned-on virtual performance operator to the non-playing performance element (YES in step S41), based on the operation of the turned-on virtual performance operator before the touch movement operation. In order to mute the musical sound that is being generated, a “virtual performance operator OFF” operation event is notified to the musical instrument core D (step S42). On the other hand, when it is determined that the touch movement operation is not performed from the on-virtual performance operator to the non-performance operator (NO in step S41), the touch movement within the fourth case, that is, the range of the turned-on virtual performance operator is performed. This is an operation or a touch movement operation from the non-performance operator to the non-performance operator, and the process jumps to step S43. A step S43 executes processing corresponding to the position after the touch movement operation on the display 6A based on the coordinates before movement and the current coordinates. In this way, if there is a musical sound that is sounded based on the operation of the virtual performance operator before the touch movement operation, it is muted and the corresponding operation processing after the touch movement operation is executed as necessary. .

  In step S44, it is determined whether or not a further touch movement operation is detected at the same time, that is, whether or not another touch movement event is notified based on a plurality of touch operations performed simultaneously. At the same time, if it is determined that a further touch movement operation is detected (YES in step S44), the process returns to step S32, and the processes from step S32 to step S43 are repeatedly executed. At the same time, if it is determined that no further touch movement operation has been detected (NO in step S44), the process ends.

  Next, the “touch end detection process” (see step S3 in FIG. 3) will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of “touch end detection processing”.

  A step S51 decides whether or not a touch end operation has been detected. That is, it is determined whether or not a touch end event is notified from the multi-touch display device B. If it is determined that the touch end operation has not been detected (NO in step S51), the process ends. On the other hand, if it is determined that a touch end operation has been detected (YES in step S51), the coordinate list corresponds to the coordinate information (referred to as touch end coordinates) notified from the multi-touch display device B together with the touch end event. The record is deleted (step S52). That is, when a touch end operation is detected, a record in which the touch end coordinates received from the multi-touch display device B are registered as the current coordinates in the coordinate list is searched for and deleted.

  In step S53, the performance operator drawing unit A is inquired about the touch end operation corresponding to the notified touch end coordinates. In step S54, it is determined whether or not the inquiry result is a touch end operation corresponding to an operation of “turning off an already-on virtual performance operator”. Here, when the touch end position on the display 6A based on the touch end coordinates is on the “on” virtual performance operator drawn on the multi-touch display device B, the “on virtual performance operator” It is determined that the touch end operation corresponds to the “ON” operation. If it is determined that the touch end operation does not correspond to the “turn off of the virtual player already turned on” operation (NO in step S54), processing corresponding to the touch end position on the display 6A based on the touch end coordinates is performed. Execute (step S56), and go to step S57. As a process executed at this time, for example, an instruction to set a musical sound control parameter when the touch start position on the display 6A based on the touch start coordinates is on a virtual setting operator drawn on the multi-touch display device B. (For example, parameter off setting for ending reflection of parameters to musical tone).

  On the other hand, when it is determined that the touch end operation corresponds to the “turn off of the virtual player already turned on” operation (YES in step S54), the sound is generated based on the operation of the virtual player already turned on before the touch end operation. In order to mute the musical sound in the middle, the instrument core unit D is notified of a “virtual performance operator OFF” operation event (step S55). In this case, the musical instrument core unit D instructs the software sound source unit G to mute the sound and notifies the performance operator drawing unit A of the operation information according to the notification. Drawing to change the display mode of the virtual performance operator by the generation stop or performance operator drawing unit A is performed. Even if the touch end operation position is on the virtual performance operator, if the touch start operation to the virtual performance operation is “invalid” (that is, it is not an on-virtual performance operator). ), It goes without saying that the generation of the musical sound waveform data is not stopped and the drawing for changing the display mode of the virtual performance operator is not performed.

  In step S57, it is determined whether or not a further touch end operation is detected, that is, whether or not another touch end event is notified based on a plurality of touch operations performed simultaneously. At the same time, if it is determined that a further touch end operation has been detected (YES in step S57), the process returns to step S52, and the processes from step S52 to step S56 are repeated. At the same time, if it is determined that no further touch end operation has been detected (NO in step S57), the process ends.

  Next, specific processing for realizing the function of the software tone generator G for generating musical sound waveform data will be described with reference to FIG. FIG. 8 is a flowchart showing an embodiment of a process for realizing the function of the software tone generator G.

  A step S61 decides whether or not a note-on event has been received from the musical instrument core part D. Note that various events received from the musical instrument core D include a note on / off event for each virtual performance operator based on a touch operation on the multi-touch display device B, an on / off event for each virtual setting operator, and a setting value change. Events include a note on / off event and a setting parameter event received from the automatic performance unit C. If it is determined that a note-on event has been received from the musical instrument core D (YES in step S61), the well-known musical sound waveform generation engine (not shown) is instructed to start sound generation corresponding to the note-on event (step S62). . A step S63 decides whether or not a note-off event has been received from the musical instrument core part D. If it is determined that a note-off event has been received from the musical instrument core D (YES in step S63), the musical sound waveform generation engine is instructed to start mute corresponding to the note-off event (step S64). A step S65 decides whether or not another event has been received from the musical instrument core part D. If it is determined that the other event has been received from the musical instrument core D (YES in step S65), the musical sound waveform generation engine is instructed to execute processing corresponding to the other event (step S66).

  In step S67, it is determined whether or not the generation timing of the musical sound waveform data is present. This generation timing is set for each time required to reproduce the musical sound waveform for 1024 frames. For example, as described above, when the operating frequency is 44100 hertz, it is set about every 23.2 ms. Of course, the present invention is not limited to this, and if the number of frames to be generated at one time and the sampling (operation) frequency are varied, the cycle of the generation timing of the musical sound waveform data also varies. If it is determined that it is the tone generation timing (YES in step S67), the load measurement unit H is instructed to start measurement (step S68). Further, in response to the instruction, a musical sound waveform generation process (not shown) for 1024 frames corresponding to the event instructed by the musical sound waveform generation engine is executed (step S69). In step S70, the load measuring unit H is instructed to end the measurement. In this way, by instructing the load measuring unit H to start and end timing at the start and end of the musical sound waveform data generation, the load measuring unit H can generate a waveform generation actual time (for example, about 23.2 frames) for 1024 frames. ms), and based on this, the CPU load can be calculated as described above.

  As described above, in the electronic music apparatus according to the present invention, the predetermined musical sound waveform associated with the virtual performance operator is detected when one or more touch operations are detected on the virtual performance operator drawn on the screen. The load measuring unit H measures the processing load related to the musical sound waveform generation of the software sound source unit G that generates. Then, the musical instrument core part D specifies a virtual performance operator according to only a part of the detected one or more touch operations in accordance with the measured processing load, and the specified virtual performance The software tone generator unit G is instructed to generate a predetermined musical sound waveform associated with the operator, and the performance operator drawing unit A is instructed to perform drawing control of the specified virtual performance operator. . In this way, when touch operations are performed simultaneously on a large number of virtual performance operators, the processing load related to the musical sound waveform generation at that time is measured, and musical sounds corresponding to all the operations are measured according to the measured processing load. By generating the tone waveform corresponding to only a part of the operation without controlling the waveform generation and drawing, it is possible to control the tone waveform especially without applying excessive processing load to the software tone generator G. Generation can be performed. For this reason, even if touch operations are performed simultaneously on a large number of virtual performance operators, the number of simultaneous pronunciations does not increase until an excessive processing load is applied. Cutting and drawing delays cannot occur.

  As mentioned above, although an example of embodiment was demonstrated based on drawing, this invention is not limited to this, It cannot be overemphasized that various embodiment is possible. For example, in the above-described embodiments, the number of effective touches is limited at the time of the touch start operation. However, the present invention is not limited to this, and a desired touch operation is performed in the middle according to the CPU load from the already effective touch operations. May be invalidated. For example, when the performance operator is touched with five fingers, but only three of them are valid and two are invalid, the number of pronunciations by automatic performance data increases and the CPU load is 70%. Suppose that If the number of effective touches is limited to two by increasing the CPU load, one sound is reduced from the three effective touches (it is considered to be turned off). At this time, the virtual performance operator to be turned off may be determined by an appropriate method.

In addition, when the CPU load decreases from the state where the number of effective touches is reduced and the limit on the number of effective touches is increased, the touch that has been invalidated (however, the touched state is maintained) is switched to valid. May be. For example, it is assumed that the number of simultaneous sounds by the automatic performance data is reduced from the state where the number of effective touches is limited to two and the CPU load is 50%. If the CPU load is reduced and the limit on the number of effective touches is increased to 5, note-on events corresponding to virtual performance operators that have been treated as off until then are generated in the software tone generator G and the performance operator drawing unit A. In response to the notification, the generation of the musical sound waveform data and the drawing for changing the display mode of the virtual performance operator are performed.
In addition to invalidating the touch operation, the simultaneous sound number may be limited based on the reproduction of the automatic performance data. This has the advantage of reducing the number of touch operations by the user as much as possible, but there is a risk that the performance content of the automatic performance may become thin or unnatural. It is possible to take advantage of the above advantages.

  In addition, the validity / invalidity of the touch related to the start of the new sound generation is controlled according to the CPU load, but in addition to this, the validity of the touch related to the effect imparting to the musical sound already sounded according to the CPU load is controlled. / Invalidity may be controlled. For example, in the case of controlling the characteristics of musical sound when touch movement is detected within the range of the virtual performance operator that has been turned on (for example, after touching is performed), the presence or absence of control may be controlled according to the CPU load. . At this time, if there is control, the display mode of the virtual performance operator is also controlled, but if there is no control, the display mode is not controlled. Alternatively, in the case where the musical sound characteristics are controlled according to a touch operation (an operation of drawing a circle so as to turn the operation element) on a virtual performance operator imitating a rotary encoder, the presence or absence of control according to the CPU load ( Display mode control of the virtual performance operator may also be controlled.

DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... ROM, 3 ... RAM, 4,5 ... Detection circuit, 4A ... Setting operation element, 5A ... Touch panel, 6 ... Display circuit, 6A ... Display, 7 ... Sound system, 8 ... Storage device, 9 ... Communication interface, 1D ... Data and address bus, A ... Performance operator drawing unit, B ... Display device with multi-touch detection function, C ... Automatic performance unit, D ... Musical instrument core unit, E ... Load control unit, F ... D / A conversion unit, G ... software sound source unit, H ... load measurement unit, I ... speaker

Claims (3)

An electronic music apparatus for generating a musical sound based on a musical sound waveform generated in accordance with execution of predetermined musical sound generation software including a plurality of processing steps by a processor,
Drawing means for drawing one or more virtual performance operators on the screen of the display device capable of simultaneously detecting touch operations on a plurality of positions on the screen;
In response to detection of one or more touch operations on the virtual performance operator drawn on the screen, the operated virtual performance operator is identified, and a predetermined associated with the identified virtual performance operator Musical tone generation means for instructing drawing control to change the display mode of the specified virtual performance operator with respect to the drawing means,
Load measuring means for measuring a processing load related to musical sound waveform generation in the musical sound generating means when detecting the touch operation;
The limit value of the number of simultaneous touches that are valid is increased or decreased according to the measured processing load, and the musical tone waveform is increased up to the limit value of the increased or decreased number of simultaneous touches among the detected one or more touch operations . performs generation and the drawing Esei control instruction, also, electrons above relative to the limiting value touched more than and a load control means for instructing to the musical tone generating means so as not to perform the instruction of the drawing control Music device.   The load control unit invalidates a part of the detected one or more touch operations according to a restriction of a touch operation determined in advance according to the measured processing load, and the invalidated part The electronic music apparatus according to claim 1, wherein the musical sound generation unit is instructed to generate a musical sound waveform only in accordance with an effective touch operation except for the music sound. On the computer,
Drawing one or more virtual performance operators on the screen of the display device capable of simultaneously detecting touch operations to a plurality of positions on the screen;
In response to detection of one or more touch operations on the virtual performance operator drawn on the screen, the operated virtual performance operator is identified, and a predetermined associated with the identified virtual performance operator Generating a musical sound waveform and instructing drawing control to change the display mode of the specified virtual performance operator;
A procedure for measuring a processing load related to the musical sound waveform generation upon detection of the touch operation;
The limit value of the number of simultaneous touches that are valid is increased or decreased according to the measured processing load, and the musical tone waveform is reduced to the increased or decreased number of simultaneous touches among the detected one or more touch operations . generating and performs control drawing Esei changing the display mode, also, the program to be executed to the procedure for controlling so as not to perform the drawing control to change the display mode for the touch operation exceeds the limit value.

Images