JP2013007988A - Data processing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract sound materials having various features on the basis of one musical sound waveform signal.SOLUTION: A music sound processing device extracts a sound material from a musical sound waveform signal acquired by processing audio data output on the basis of one track and stores it in a database. Processing modes includes a mode of combining the audio data with other audio data, a mode of giving sound effect, etc.. With this sound processing device, as a musical sound waveform signal indicated by the audio data of extraction source can be changed by changing modes, various sound materials can be extracted.

Description

  The present invention relates to a technique for extracting sound material from a musical sound waveform signal.

  There is a technology for storing pieces of sound material in a database and generating musical sounds by combining these sound materials. The sound material used when generating a musical sound is selected from a large number of sound materials registered in the database. Patent Document 1 discloses that a sound material to be registered in this database is extracted from a musical sound waveform signal of music according to a predetermined algorithm.

JP 2010-191337 A

  According to the technique disclosed in Patent Document 1, since sound material can be extracted from musical tone waveform signals of music and classified according to the characteristics of the sound material, it is easy to register the sound material in the database. . With the preparation of the sound material in the database, we may want to register more sound materials with various features in the database. In this case, it is necessary to prepare musical sound waveform signals of various musical pieces so that only similar sound materials are not registered.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to extract sound materials having various characteristics based on one musical sound waveform signal.

  In order to solve the above-described problems, the present invention acquires pronunciation data indicating the pronunciation content and processing data for changing the pronunciation content, and performs processing based on the processing data for the pronunciation data Means for analyzing a musical sound waveform signal indicating the sound content of the processed sound data, specifying a period of the musical sound waveform signal having a predetermined characteristic, and a musical sound waveform for the specified period A calculation unit that analyzes the signal to calculate a feature value; and a data output unit that outputs specific data indicating the tone waveform signal of the specified period and feature value data indicating the calculated feature value. A data processing apparatus is provided.

  In another preferred embodiment, the processed data includes setting data that defines a content of an acoustic effect to be imparted to the musical sound waveform signal, and the processing means adds the acquired pronunciation data to the acquired pronunciation data based on the setting data. The processing is performed so as to give an acoustic effect to the musical sound waveform signal.

  In another preferred embodiment, the processed data includes pronunciation data that defines the content of the pronunciation, and the processing means generates pronunciation data included in the processed data with respect to a musical sound waveform signal related to the acquired pronunciation data. The processing is performed by synthesizing the musical sound waveform signal according to the above.

  In another preferred embodiment, the specific data is a musical tone of the specified period based on a musical sound waveform signal related to the processed sound generation data and time information indicating the specified period of the musical sound waveform signal. In the case where a waveform signal is shown and the specifying means specifies a plurality of periods, the specifying data includes a musical sound waveform signal related to the processed sound generation data and the specified plurality of periods of the musical sound waveform signal. The musical sound waveform signal of the plurality of specified periods is indicated by the time information shown, and the calculation means calculates the feature amount for each of the specified periods.

  Moreover, in another preferable aspect, the specific data indicates a musical sound waveform signal obtained by extracting a musical sound waveform signal of the specified period.

  Further, the present invention provides a computer with processing means for acquiring pronunciation data indicating the pronunciation content and processing data for changing the pronunciation content, and processing the pronunciation data based on the processing data; Analyzing the musical sound waveform signal indicating the sound content of the processed sound data, identifying means for identifying the period of the musical sound waveform signal having a predetermined characteristic, and analyzing the musical sound waveform signal of the specified period A program for functioning as a data output means for outputting a calculation means for calculating the feature quantity, and specific data indicating the musical sound waveform signal of the specified period and feature quantity data indicating the calculated feature quantity provide.

  According to the present invention, a sound material having various characteristics can be extracted based on one musical sound waveform signal.

It is a block diagram explaining the structure of the musical tone processing apparatus in embodiment of this invention. It is a figure explaining the example of sound material DB in the embodiment of the present invention. It is a figure explaining the content of the sound material represented by the specific data in the embodiment of the present invention. It is a figure explaining the example of the sound material track | truck in embodiment of this invention. It is a block diagram explaining the structure of the reproduction | regeneration function in embodiment of this invention. It is a block diagram explaining the structure of the sound material extraction function and correction function in embodiment of this invention. It is a figure explaining an example of the display during the reproduction | regeneration program execution in embodiment of this invention. It is a figure explaining an example (branch position designation | designated display) during the extraction program execution in embodiment of this invention. It is a figure explaining an example (extraction completion display) of the display during extraction program execution in the embodiment of the present invention. It is a figure explaining an example of the display during execution of the correction program in embodiment of this invention. It is a figure explaining the example of the sound material timing data in the modification 5 of this invention. It is a figure explaining the example of the sound material track | truck in the modification 6 of this invention. It is a figure explaining another example of the sound material track in modification 6 of the present invention. It is a block diagram explaining the structure of the sound generation control system in the modification 7 of this invention. It is a figure explaining the structure of the server apparatus in the modification 7 of this invention. It is a figure explaining the example of the sound material track | truck in the modification 7 of this invention. It is a functional block diagram explaining the function of the musical tone processing apparatus and server apparatus in the modification 7 of this invention. It is a figure explaining the example of a display of the display screen at the time of the sound material track creation program execution in the modification 7 of this invention. It is a figure explaining operation | movement of the sound generation control system at the time of the sound material track creation program execution in the modification 7 of this invention. It is a figure explaining the example of a display of the display screen at the time of the sound material track creation program execution in the modification 8 of this invention. It is a figure explaining the example of a display of the display screen at the time of the sound material track creation program execution in the modification 17 of this invention. It is a figure explaining the example of a display of a display screen at the time of sound material track creation program execution in modification 18 of the present invention.

<Embodiment>
[Overview]
The musical tone processing apparatus according to the embodiment of the present invention is an information processing apparatus such as a personal computer, a mobile phone, a PDA (Personal Digital Assistant), and a tablet terminal, and executes a specific application program on an OS (Operating System). , A device that realizes a function called DAW (Digital Audio Workstation). In the DAW realized in this musical tone processing apparatus, a function of performing control for generating musical tone using a sound material extracted as a part of a musical tone waveform signal is also realized. Moreover, each function demonstrated below, such as the function to extract a sound material from a musical sound waveform signal, is also realized. Each of these functions is realized by executing a subroutine program during the execution of the application program for realizing the DAW.

[Configuration of Music Processing Device 10]
FIG. 1 is a block diagram illustrating the configuration of a musical tone processing apparatus 10 according to an embodiment of the present invention. The musical sound processing apparatus 10 includes a control unit 11, an operation unit 12, a display unit 13, an interface 14, a storage unit 15, and an acoustic processing unit 16. Each of these components is connected via a bus. In addition, the musical sound processing apparatus 10 includes a speaker 161 and a microphone 162 connected to the sound processing unit 16.

  The control unit 11 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. The control unit 11 implements various functions by executing various programs stored in the ROM or the storage unit 15. In this example, execution of the program by the control unit 11 includes execution of an application program that realizes DAW and execution of the above-described subroutine program. The subroutine program includes a reproduction program, an extraction program, and a correction program stored in the storage unit 15.

  The reproduction program is a program for realizing a reproduction function that reproduces sequence data that defines the sound content of a musical sound in the DAW, and performs a process of generating the musical sound. The extraction program is a program for realizing a sound material extraction function for extracting a sound material from a musical sound waveform signal synthesized by a reproduction function. The correction program is a program for realizing a correction function for correcting the extracted sound material data. Details of these functions will be described in the description of the configuration of each function described later.

The operation unit 12 has operation means such as an operation button that accepts an operation by a user, a keyboard, a mouse, and a touch panel, and outputs operation data indicating the contents of the accepted operation to the control unit 11. Thereby, an instruction from the user is input to the musical sound processing apparatus 10.
The display unit 13 is a display device such as a liquid crystal display, and displays content on the display screen 131 according to the control of the control unit 11. The contents displayed on the display screen 131 are various contents depending on the executed program in addition to the menu screen, the setting screen, and the like (see FIGS. 7, 8, 9, and 10).

  The interface 14 has a function of transmitting and receiving various data by connecting to an external device and communicating by wire or wireless. The interface 14 is also provided with an AUX (Auxiliary) terminal to which audio data is input from an external device. The interface 14 outputs various data input from the external device to the control unit 11, and outputs various data to the external device under the control of the control unit 11. When an analog signal is input to the AUX terminal, A / D conversion (analog / digital conversion) is performed.

The microphone 162 outputs a musical sound waveform signal indicating the content of the input sound to the acoustic processing unit 16.
The acoustic processing unit 16 includes a signal processing circuit such as a DSP (Digital Signal Processor). In this example, the acoustic processing unit 16 performs A / D conversion on the musical sound waveform signal input from the microphone 162 and outputs it as audio data to the control unit 11. The acoustic processing unit 16 performs signal processing such as acoustic processing, D / A conversion (digital / analog conversion) processing, amplification processing, and the like set on the audio data output from the control unit 11. And output to the speaker 161 as a musical sound waveform signal.
The speaker 161 outputs the sound indicated by the musical sound waveform signal input from the acoustic processing unit 16.

  The storage unit 15 is a non-volatile memory such as a hard disk or a flash memory, and has a storage area for storing the various programs. The storage unit 15 also stores storage areas for storing sequence data, a sound material database (hereinafter referred to as a sound material DB), and a waveform database (hereinafter referred to as a waveform DB) used when various programs are executed. Have

  In the waveform DB, waveform data W1, W2,... Indicating musical tone waveform signals are registered. The musical sound waveform signals indicated by these waveform data are any of various contents such as music, specific instrument sounds, sound materials, etc., and their lengths vary from less than one second to several minutes or more. There may also be one used in a loop. Even waveform data used in a loop may be used as waveform data that does not loop using a part of the section. Note that the waveform data in this example includes data of a plurality of channels (for example, Lch and Rch). In the following description, each data such as waveform data indicating a musical sound waveform signal and audio data is composed of two channels of Lch and Rch, but may be composed of a larger number of channels. , May be composed of monaural (1ch).

  FIG. 2 is a diagram illustrating an example of the sound material DB in the embodiment of the present invention. In the sound material DB, information for specifying the contents of the sound material is registered. As shown in FIG. 2, the information for specifying the content of the sound material includes specific data for specifying the content of the sound waveform signal of the sound material, feature data indicating the characteristics of the sound waveform signal of the sound material, Includes categories categorized by features.

The specific data is composed of a combination of waveform designation information for designating one of the waveform data registered in the waveform DB and time designation information for designating a data range in the waveform data by time. In this example, the time designated by the time designation information is determined as the time from the beginning of the waveform data. Here, when the waveform data indicated by the waveform designation information is used in a loop, the time at the start position in the data range may be indicated as a time after the time at the end position. In this case, the content of the sound material is obtained by connecting the musical sound waveform signal of the section from the start position to the end position after the musical sound waveform signal of the section from the start position to the end of the data. About the sound material specified by specific data, the identifier (in this example, sn1, sn2, ...) for specifying each is attached | subjected. Hereinafter, when a specific sound material is indicated, it is referred to as a sound material sn1.
For specific data for which time designation information is not defined, the waveform data itself indicated by the waveform designation information represents the content of the musical sound of the sound material. For example, the musical sound indicated by the sound material sn4 is represented by the entire musical sound waveform signal indicated by the waveform data W5.

  FIG. 3 is a diagram for explaining the contents of the sound material represented by the specific data in the embodiment of the present invention. FIG. 3A is a diagram for explaining musical tone waveform signals of musical sounds indicated by the sound materials sn1, sn2, and sn3, and FIG. 3B is a diagram for explaining musical tone waveform signals of musical sounds indicated by the sound material sn4. is there. As shown in FIG. 2, the content of the sound material sn <b> 1 is specified such that the waveform designation information is the waveform data W <b> 1 and the time designation information is ts <b> 1 to te <b> 1. Therefore, as shown in FIG. 3A, the musical sound waveform signal corresponding to the sound material sn1 is a musical sound waveform signal at times ts1 to te1 among the musical sound waveform signals indicated by the waveform data W1. Similarly, the musical sound waveform signals respectively corresponding to the sound materials sn2 and sn3 are specified as a partial range of the musical sound waveform signal indicated by the waveform data W1. On the other hand, as shown in FIG. 3B, the musical sound waveform signal indicated by the sound material sn4 is specified as the whole musical sound waveform signal indicated by the waveform data W5 because the time designation information is not defined. Hereinafter, data representing musical sound waveform signals indicated by the sound materials sn1, sn2,... Are described as sound material data sn1, sn2,.

  Returning to FIG. 2, the description will be continued. The feature amount data indicates a plurality of types of feature amounts p1, p2,... Possessed by the tone waveform signal of each sound material. For example, for sound material, the intensity of each frequency (high range, mid range, low range), time of peak amplitude (time relative to the beginning of the sound material data), peak amplitude of amplitude, degree of cooperation, It is a feature quantity for complexity and the like, and is a value obtained by analyzing sound material data. For example, the feature quantity p1 indicates the intensity of the high frequency range of the sound material by its value. Hereinafter, the feature amount data is indicated as Pa, Pb,... By a combination of values of the feature amounts p1, p2,. For example, for the sound material sn3, the feature amount data is indicated as Pc determined from the combination of the feature amounts.

The categories classified by the characteristics of the sound material, that is, the content of the feature data, are classified for each sound material having similar auditory characteristics. For example, the sound is classified as a sound with a clear attack and a strong edge. Category “classification A (for example, edge sound)”, category “classification B (for example, texture sound)” classified as sound that sounds like noise, and the like.
The feature amount and category are used, for example, when a user searches a sound material DB for a desired sound material when determining a sound material to be used in a sound material track described later. For example, when the user inputs feature amount data, the control unit 11 searches for sound material similar to the feature amount data and presents it to the user, or when the user selects a category, the control unit 11 What is necessary is just to show a user the sound material classified into the category. Further, when the user designates a specific sound material, the control unit 11 may search for a sound material having feature data similar to that sound material and present it to the user.

  The sequence data has a plurality of tracks that define pronunciation contents in time series. In this example, each type of track is assigned to any one of an audio track, a MIDI (Musical Instrument Digital Interface) track, and a sound material track. Each of these tracks is an aspect of the pronunciation data of the present invention.

  MIDI tracks are MIDI events such as note-on, note-off, note number, and velocity, and the processing timing of these events (for example, expressed by the number of bars, beats, ticks, etc. from the start of the track data) It is a track that defines the relationship. As described above, in this example, the MIDI track is defined in a commonly used MIDI format. However, the MIDI track is used to control a sound source that generates a musical sound waveform signal having a pronunciation content corresponding to an event. As long as the information track is defined, it may be defined in another format.

  The audio track is a track in which audio data and playback start timing of the data are defined. The audio data may be waveform data stored in the waveform DB, or may be data indicating a musical sound waveform signal input separately. The reproduction start timing is represented by the number of bars, the number of beats, and the number of ticks from the start of the track data, similarly to the above processing timing. Other information such as information indicating the playback volume of the audio data may be included.

FIG. 4 is a diagram illustrating an example of a sound material track in the embodiment of the present invention. As shown in FIG. 4, the sound material track is a track in which the sound material data and the reproduction start timing of the data are defined. The sound material data is specified by the identifier of each sound material in the sound material DB. The reproduction start timing is represented by the number of bars, the number of beats, and the number of ticks from the start of the track data, similarly to the above processing timing. In this example, the volume and time (represented by the number of ticks in this example) when reproducing the sound material data are also defined, but the volume and time may not be defined.
The specified time may be determined as the time from the beginning of the sound material data (the sound material data shorter than the specified time may be loop-played), or by time stretch processing It may be determined to expand and contract at this time. If the time is not specified, the sound material data may be reproduced from the start to the end of the data.
In the example shown in FIG. 4, for example, at the reproduction start timing “0002: 01: 000” (corresponding to the first beat of the second measure), it is determined to reproduce the corresponding sound material data Sn3. .

Returning to FIG. 1, the description will be continued. In addition to the above tracks, the sequence data includes a tempo (a value indicating the time of one beat) that specifies the playback speed of each track, the contents of the acoustic processing applied to each track, and a synthesis mode of musical sound waveform signals obtained from each track And setting data for defining the contents of each signal processing such as acoustic processing after the musical sound waveform signal is synthesized. As a synthesis form of the musical sound waveform signals, a musical sound waveform signal based on each track until a musical sound waveform signal based on each track is synthesized (mixed) and output to the acoustic processing unit 16 is synthesized. Signal level ratio (between tracks, between channels (Lch, Rch)).
The contents of each track and the contents of the setting data in the sequence data may be created by the user or may be acquired via the interface 14.
The above is the description of the hardware configuration of the musical tone processing apparatus 10.

[Playback function configuration]
Next, a reproduction function realized by the control unit 11 of the musical sound processing apparatus 10 executing a reproduction program will be described. Note that some or all of the components for realizing the playback function described below may be realized by hardware.

  FIG. 5 is a block diagram illustrating the configuration of the playback function in the embodiment of the present invention. When the control unit 11 executes the reproduction program, the data reading units 110-1, 110-2,... (Hereinafter referred to as the data reading unit 110 if not distinguished from each other), the sound source unit 120, and sound material data synthesis. Unit 130, acoustic effect applying units 140-1, 140-2 (hereinafter referred to as acoustic effect applying unit 140 when not distinguished from each other), audio data synthesizing unit 150, synthetic acoustic effect applying unit 160, The playback function unit 100 including the channel conversion unit 170 and the bus unit 180 is configured to realize the playback function.

  The data reading unit 110-1 reads audio data from the reproduction start timing determined by the tempo specified by the setting data and the timing specified by the audio track, and outputs the audio data to the sound effect applying unit 140-1. The speed at which each part (for example, sample unit) of audio data is read may be changed according to the tempo. In this case, the pitch of the musical sound waveform signal indicated by the audio data is not changed by the time stretch process. The audio effect applying unit 140-1 performs signal processing on the musical sound waveform signal indicated by the input audio data so as to apply the sound effect having the content defined by the setting data, and the audio data subjected to the signal processing. Are output to the audio data synthesis unit 150.

The signal line from the audio track to the audio data synthesis unit 150 is branched by the branch S1, which is a signal line for extracting the audio data read and output by the data reading unit 110-1, and the sound effect applying unit 140-1. A branch S5 is provided for extracting audio data indicating a musical sound waveform signal to which a sound effect has been applied.
Note that the audio track has a set of a data reading unit 110-1 and a sound effect applying unit 140-1 for each track. Therefore, when there are a plurality of audio tracks, a set of the data reading unit 110-1 and the sound effect applying unit 140-1 is provided by the number of audio tracks. The branches S1 and S5 are also provided for each track.

  The data reading unit 110-2 reads the event at a processing timing determined by the tempo specified by the setting data and the timing specified by the MIDI track, and outputs the event to the sound source unit 120. The sound source unit 120 is a software sound source that generates audio data indicating a musical sound waveform signal having a pronunciation content corresponding to an input MIDI event. The sound source unit 120 outputs the audio data generated according to the MIDI event input from the data reading unit 110-2 to the acoustic effect applying unit 140-2. The sound source unit 120 may be provided as an external device. In this case, the data reading unit 110-2 outputs a MIDI event to the external device via the interface 14, and the sound effect applying unit 140- 2 may acquire the audio data generated in the external device via the interface 14.

  The audio effect applying unit 140-2 performs signal processing on the musical sound waveform signal indicated by the input audio data so as to give the sound effect having the content defined by the setting data, and the audio data subjected to the signal processing. Are output to the audio data synthesis unit 150.

The signal line from the MIDI track to the audio data synthesizing unit 150 has a branch S2, which is a signal line for extracting the audio data output from the sound source unit 120, and the musical sound to which the acoustic effect is imparted by the acoustic effect imparting unit 140-2. A branch S6 is provided for extracting audio data indicating the waveform signal.
The MIDI track has a set of a data reading unit 110-2, a sound source unit 120, and a sound effect applying unit 140-2 for each track. Therefore, when there are a plurality of MIDI tracks, a set of the data reading unit 110-2, the sound source unit 120, and the sound effect applying unit 140-2 is provided by the number of MIDI tracks. Branches S2 and S6 are also provided for each track. It should be noted that the sound source unit 120 does not need to exist in plural as long as it is configured to receive input for each track and output audio data for each track.

  The data reading unit 110-3 reads out the corresponding sound material data at the reproduction start timing determined by the tempo specified by the setting data and the timing specified by the sound material track, and outputs it to the sound material data synthesizing unit 130. The sound material data synthesizing unit 130 synthesizes the musical sound waveform signal indicated by the sound material data input from the data reading unit 110-3 in time series according to the input timing, and the sound effect applying unit 140 as audio data. To -3.

Here, when two pieces of sound material data are input in order, if the subsequent sound material data is input during the reproduction of the previous sound material data, the last sound waveform signal indicated by the previous sound material data is input. There are cases where the portion and the first portion of the musical sound waveform signal indicated by the subsequent sound material data overlap during the same period. In such a period, audio data indicating the synthesized contents of the respective musical sound waveform signals is output from the sound material data synthesis unit 130.
The sound effect applying unit 140-3 performs signal processing on the musical sound waveform signal indicated by the input audio data so as to give the sound effect having the content defined by the setting data, and the audio data subjected to the signal processing. Are output to the audio data synthesis unit 150.

The signal line from the sound material track to the audio data synthesizing unit 150 has an acoustic effect by the branch S3, which is a signal line for extracting the audio data output from the sound material data synthesizing unit 130, and the acoustic effect applying unit 140-3. A branch S7 is provided for extracting audio data indicating the assigned musical sound waveform signal.
Note that the sound material track has a set of a data reading unit 110-3, a sound material data synthesizing unit 130, and an acoustic effect applying unit 140-3 for each track. Therefore, when there are a plurality of sound material tracks, a set of the data reading unit 110-3, the sound material data synthesizing unit 130, and the sound effect applying unit 140-3 is provided by the number of sound material tracks. Branches S3 and S6 are also provided for each track.

  The audio data input to the AUX terminal of the interface 14 is output to the sound effect applying unit 140-4. Since the audio data is input from the AUX terminal in time series, there is no configuration corresponding to the data reading unit. The audio effect applying unit 140-4 performs signal processing on the musical sound waveform signal indicated by the input audio data so as to give the sound effect having the content defined by the setting data, and the audio data subjected to the signal processing. Are output to the audio data synthesis unit 150.

In the signal line from the AUX terminal to the audio data synthesizing unit 150, the musical sound waveform to which the acoustic effect is given by the branch S4 which is a signal line for extracting the audio data inputted to the AUX terminal and the acoustic effect giving unit 140-4 A branch S8 is provided for extracting audio data indicating the signal.
In addition, when there are a plurality of AUX terminals in the interface 14, the sound effect imparting unit 140-4 is provided corresponding to each AUX terminal. The branch S4 is also provided for each track.

The bus unit 180 takes out the audio data (branched at B1, B2, B3, and B4) input to the sound effect applying units 140-1, 140-2, 140-3, and 140-4, synthesizes them, and generates a sound effect. It outputs to the grant part 140-5. The signal level of each audio data at the time of synthesis is determined based on the setting data. The sound effect applying unit 140-5 performs signal processing on the musical sound waveform signal indicated by the input audio data so as to give the sound effect having the content defined by the setting data, and the audio data subjected to the signal processing. Are output to the audio data synthesis unit 150.
The signal line from the bus unit 180 to the audio data synthesis unit 150 is given a sound effect by the branch S14, which is a signal line for extracting the audio data output from the bus unit 180, and the sound effect applying unit 140-5. A branch S15 is provided for extracting audio data indicating the musical sound waveform signal.

  Here, the audio data input to the acoustic effect imparting unit 140 is an aspect of the pronunciation data of the present invention. Moreover, the acoustic effect provision part 140 is an aspect of the processing means of this invention, and the setting data which prescribes | regulates the content of the acoustic effect to provide is an aspect of processing data. That is, the acoustic effect imparting unit 140 is configured to process audio data based on the setting data.

The audio data synthesis unit 150 synthesizes the input audio data based on the contents of the setting data and outputs the synthesized data. Specifically, the audio data synthesizing unit 150 determines the ratio of signal levels between tracks (including the output from the AUX terminal and the bus unit 180), and between channels of the same track (Lch) for the sound waveform signal indicated by each audio data. , Rch) signal level ratios are adjusted and synthesized for each channel. Then, the audio data synthesis unit 150 outputs audio data indicating the synthesized musical sound waveform signal to the synthesized sound effect applying unit 160. In FIG. 5, the branches S1 to S8, S14, and S15 are described as signal lines in which Lch and Rch are combined. They are shown separately.
Here, the audio data synthesizing unit 150 is an aspect of the processing means of the present invention, and any audio data before synthesis is an aspect of the pronunciation data, and other audio data is an aspect of the processed data. . That is, the audio data synthesis unit 150 is configured to process any audio data based on other audio data. Further, the synthesis mode (processing mode) is determined based on the setting data as described above. Note that the bus unit 180 can also be an aspect of the processing means of the present invention, similarly to the audio data synthesis unit 150.

The synthesized sound effect applying unit 160 performs signal processing on the input audio data so as to give the sound effect having the content defined by the setting data, and outputs the processed audio data to the channel converting unit 170. The channel conversion unit 170 converts the number of channels specified by the setting data and outputs it to the sound processing unit 16. In this example, FIG. 5 shows a case where audio data obtained by synthesizing Lch and Rch musical sound waveform signals is output to monaural, but it may be output without being synthesized. A process for increasing the number of channels may be performed and output. As described above, the channel conversion unit 170 only needs to have a function of increasing or decreasing the number of channels using matrix processing or the like based on the setting data.
Here, each of the synthesized sound effect applying unit 160 and the channel converting unit 170 is an aspect of the processing means of the present invention, and the setting data defining the content of the acoustic effect to be applied and the number of channels is an aspect of the processed data. is there.

The signal line from the audio data synthesis unit 150 to the sound processing unit 16 includes a branch S9 that is a signal line for extracting the Lch of the audio data output from the audio data synthesis unit 150, and a signal line S10 for extracting the Rch. A signal line S12 for extracting the Rch and the branch S11 that is provided and a signal line for extracting the Lch of the audio data to which the sound effect is applied by the synthesized sound effect applying unit 160 are provided. A branch S13 which is a signal line for taking out the audio data is provided.
This completes the description of the playback function.

[Sound material extraction function and correction function]
Next, the sound material extraction function realized by the control unit 11 of the musical sound processing apparatus 10 executing the extraction program will be described. A correction function realized by the control unit 11 executing the correction program will also be described. In addition, you may implement | achieve part or all of each structure which implement | achieves the sound material extraction function and correction function demonstrated below by hardware.

  FIG. 6 is a block diagram illustrating the configuration of the sound material extraction function and the correction function in the embodiment of the present invention. When the control unit 11 executes the extraction program, the sound material extraction function unit 200 including the selection unit 210, the extraction unit 220, and the data output unit 230 is configured, and the sound material extraction function is realized. Further, when the control unit 11 executes the correction program, the correction unit 300 is configured and a correction function is realized.

  The selection unit 210 selects one of the branches S1, S2,... S15, acquires audio data in the signal line of the selected branch, and outputs the audio data to the extraction unit 220. Which one to select may be selected by the user or may be determined in advance. Further, instead of selecting any one branch, a plurality of branches may be selected. In that case, the selection unit 210 may synthesize and output a musical sound waveform signal for each channel. Further, the combination of the branches S9 and S10 or the combination of the branches S11 and S12 has a relationship of Lch and Rch, and may be selected as a unit. On the other hand, when the audio data for only one channel is selected, such as selecting only the branch S9, the selection unit 210 may be silent for the other channel of the output audio data, or both channels may be silenced. Both may be the same musical sound waveform signal or may be monaural. Note that the branches S1, S2,..., S8, S14, and S15 have been described as signal lines in which Lch and Rch are combined, but the selection unit 210 sends the audio data acquired from these branches to the extraction unit 220. When outputting, audio data of only one of the channels may be used, or monaural audio data may be used. What kind of aspect the audio data is output may be determined based on the setting data.

The extraction unit 220 includes a sound material specifying unit 221 and a feature amount calculation unit 222, and extracts sound material from the input audio data by the processing of the sound material specification unit 221 and the feature amount calculation unit 222. The feature amount of the material is calculated. Then, the extraction unit 220 outputs, to the data output unit 230, information indicating a section corresponding to the extracted sound material and feature amount data indicating the calculated feature amount in the musical sound waveform signal indicated by the audio data. At this time, the audio data used for extraction (audio data input to the extraction unit 200) is also output.
Hereinafter, functions of the sound material specifying unit 221 and the feature amount calculating unit 222 will be described.

  The feature amount calculation unit 222 calculates a feature amount in a section instructed by the sound material identification unit 221 among the musical sound waveform signals indicated by the audio data input to the extraction unit 220 (hereinafter, referred to as extraction source musical sound waveform signals). The calculation result is output to the sound material specifying unit 221.

  The sound material specifying unit 221 detects an onset in which the volume change changes more than a certain level from the extracted source musical sound waveform signal, and features various sections in a predetermined time range from the onset. Instructed to 222, the feature amount is calculated. The sound material specifying unit 221 selects a section having a feature amount satisfying a predetermined specific condition from among the calculated feature amounts of each section, in the extracted source musical sound waveform signal corresponding to the sound material extracted from the audio data. As specified. The sound material specifying unit 221 extracts sound material from the entire input audio data, and specifies a section in the extraction source musical sound waveform signal corresponding to the sound material. As a method for extracting sound material from audio data as described above, any known method can be used. For example, a method disclosed in Japanese Patent Application Laid-Open No. 2010-191337 may be used.

  Then, the sound material specifying unit 221 includes, for each extracted sound material, information indicating a specified section (hereinafter referred to as a specific section) and feature amount data indicating a feature amount calculated corresponding to this section. Output audio data that has been input.

  The data output unit 230 outputs the audio data input from the extraction unit 220 to the storage unit 15 and registers it as waveform data in the waveform DB. Further, the data output unit 230 outputs the waveform specification information for identifying the registered waveform data, the specific data indicating the time specification information with the specific section as the data range, and the feature amount data to the storage unit 15 for extraction. Each sound material is registered in the sound material DB. At this time, the data output unit 230 also determines a category classified according to the registered feature data.

  In addition, when registering in the waveform DB, the data output unit 230 registers, as waveform data, audio data obtained by cutting out a musical sound waveform signal in a specific section from the audio data input from the extraction unit 220 as waveform data. In some cases. In this case, the specific data registered in the sound material DB by the data output unit 230 does not include time designation information. That is, the specific data indicates the musical sound waveform signal of the sound material by specifying the entire waveform data as the sound material.

  The user may set in advance which of the two types of waveform DB registration methods by the data output unit 230 is applied. In this example, the former registration method (when a musical sound waveform signal in a specific section is not cut out) is set as “mode 1”, and the latter registration method (when a musical sound waveform signal in a specific section is cut out) is set as “mode 2”. The registration method may be set by the data output unit 230 according to a predetermined algorithm. For example, when the number of sound materials extracted by the extraction unit 220 is equal to or greater than a predetermined number, it is set as “mode1”, and when it is less than the number, it may be set as “mode2”.

The correction unit 300 has a function of correcting the data range (time designation information) of the sound material before registration in the waveform DB and the sound material DB by the data output unit 230 according to a user instruction. As a result, the sound material extracted by the extraction unit 220 can be adjusted to be a sound material that meets the user's request. At this time, the content of the feature data may not be changed, or the content of the feature data may be recalculated by the feature data calculator 222 from the corrected sound material and updated.
Note that the correction unit 300 may correct the data range of the already registered sound material.
This completes the description of the sound material extraction function and the correction function. With the above functional configurations, the data processing apparatus of the present invention is realized in the musical sound processing apparatus 10.

[Operation example of each function]
Next, an operation example when the above-described reproduction program, extraction program, and correction program are executed will be described with reference to a display example on the display screen 131.
First, when the sequence data is reproduced on the DAW, the user inputs an instruction to execute the reproduction program to the musical sound processing apparatus 10 and also inputs the instruction of the sequence data to be reproduced. In this way, the display screen 131 displays the content shown in FIG.

  FIG. 7 is a diagram for explaining an example of display during execution of the reproduction program in the embodiment of the present invention. On the display screen 131, a data area TA indicating the contents of the sequence data and an operation image area BA for inputting a user instruction to the musical sound processing apparatus 10 using the pointer pt are displayed.

In the data area TA, an image schematically showing the contents of each track of the sequence data is displayed, and the track is defined in the vertical axis direction and the time is defined in the horizontal axis direction. The content displayed in the data area TA is a part of sequence data. The user operates the vertical axis tab vsb in the vertical axis scroll bar VBA to change the display range in the vertical axis direction, and the horizontal axis scroll bar. The display range in the horizontal axis direction is changed by operating the horizontal axis tab hsb in the HBA. In the sequence data of this example, it is assumed that the track is composed of tr1, tr2,... Tr6 and 6 tracks. Here, tr1 and tr2 are audio tracks (“audio” in the figure), tr3 and tr4 are MIDI tracks (“midi” in the figure), and tr5 and tr6 are sound material tracks (“ms” in the figure). . The number of tracks and the type of tracks can be freely set by the user.
Further, range designation arrows (start designation arrow as and end designation arrow ae) for designating the data range in the horizontal axis direction are displayed, and by moving these arrows, the start time tss and end time tse on the horizontal axis are displayed. Set.

The display screen 131 also includes a tempo control button b1 for setting the tempo of reproduction, a reproduction control button b2 for specifying reproduction start, stop, fast forward, and rewind, and sequence data in the data range designated by the range designation arrow. A conversion button b3 for starting the process of converting from sound to sound material and a setting button b4 for performing settings at the time of executing the extraction program are displayed.
When the user operates the playback control button b2 to instruct the start of playback, the data reading unit 110 in the playback function unit 100 starts reading each data based on the set tempo. As a result, audio data is output from the playback function unit 100 to the sound processing unit 16, and is output from the speaker 161 with the sound production content based on the sequence data.
When the user operates the setting button b4, the extraction program is executed, and the display shown in FIG. This display has contents for setting audio data from which sound material is extracted in the conversion.

  FIG. 8 is a diagram illustrating an example of display during execution of the extraction program (branch position designation display) in the embodiment of the present invention. As shown in FIG. 8, on the display screen 131, display corresponding to each configuration and signal line in the playback function unit 100 is performed. The track display tm indicates a portion corresponding to a configuration for outputting audio data to the sound effect applying unit 140 for each track. A bus display bm indicates a portion corresponding to the bus unit 180. The acoustic effect imparting display etm indicates a portion corresponding to the acoustic effect imparting unit 140 for each track. The synthesis unit display mm indicates a part corresponding to the audio data synthesis unit 150. The synthetic sound effect application display emm indicates a part corresponding to the synthetic sound effect application unit 160. Mono display cm indicates a part corresponding to the channel conversion unit 170.

  Further, the lines corresponding to the display of each configuration correspond to the signal lines, and the branch indications bt1, bt2,... Bt5, which are circle marks provided on the lines, are changed to the branches S1, S2,. Correspond. In this example, since the audio track is composed of two tracks tr1 and tr2, the branches S1 and S5 are provided corresponding to the respective tracks. The same applies to the MIDI track and the sound material track. When a branch display portion is selected by a user operation, the display is changed from a white circle to a black circle, and the audio data in the signal line of that portion is set as an extraction target of the sound material. In the example shown in FIG. 5, the audio data in the signal line corresponding to the branch S13 is set as a sound material extraction target. Note that the user may select a plurality of branch displays.

The display screen 131 also displays a mode selection button bm for setting a database registration mode in the data output unit 230. The user selects one of “mode1” or “mode2”, whereby the registration mode in the data output unit 230 is set. In this example, “mode 1” is set.
When the user operates the enter button b5, these settings are terminated, and the display on the display screen 131 returns to the display shown in FIG.

Subsequently, the user operates the range designation arrow to set the period of audio data to be extracted. Setting the range is not necessary for the entire range.
When the above setting is completed and the user operates the conversion button b3, processing in the sound material extraction function unit 200 is started according to the content set by the user. In this example, the sound material extraction function unit 200 causes the playback function unit 100 to generate audio data in the signal line of the branch S13 among the audio data in the period designated by the range designation arrow, and the selection unit 210 Receive at. Then, the sound material is extracted in the extraction unit 220. When the extraction is completed, extraction completion is displayed on the display screen 131 as shown in FIG.

  Note that when the selection unit 210 receives the audio data, the playback function unit 100 only has to generate the audio data. Therefore, the audio data is not necessarily output to the acoustic processing unit 16. Also, when audio data is input from the AUX terminal (the presence or absence of input may be set in advance by the user), the playback function unit 100 receives audio data for a period specified by a range specifying arrow as an acoustic processing unit. 16 and the audio data input from the AUX terminal during that period may be combined by the audio data combining unit 150.

  FIG. 9 is a diagram for explaining an example of the display during extraction program execution (extraction completion display) in the embodiment of the present invention. As shown in FIG. 9, the display screen 131 displays the source musical sound waveform signal wv indicated by the audio data received by the selection unit 210 and the period (specific interval) of the extracted sound material (the sound material sna in this figure). , Snb, snc, snd) and a display (icons ica, icb, icc, icd in this figure) showing categories classified from the feature amount data corresponding to each sound material. Further, a correction button b6 for correcting the specific section and a registration button b7 for registering the extracted sound material in the database are also displayed on the display screen 131.

When the user operates a portion corresponding to the sound material (for example, icons ica, icb, icc, icd) using the pointer pt, the sound indicated by the musical sound waveform signal of the corresponding sound material is controlled by the control unit 11 through the speaker. 161 is output.
When the user operates the registration button b7 when the user listens to the sound output from the speaker 161 and there is no need to correct a specific section of the sound material, the data output unit 230 receives the data output unit 230. While the audio data is registered in the waveform DB, the specific data and the feature data are registered in the sound material DB for each extracted sound material.
On the other hand, when the user operates the correction button b6, such as when the user wants to correct a specific section of the sound material, the correction program is executed, and the display screen 131 displays the display shown in FIG. The

FIG. 10 is a diagram for explaining an example of display during execution of the correction program in the embodiment of the present invention. As shown in FIG. 10, the portion of the sound material that was last auditioned is displayed on the display screen 131 in an enlarged manner. In this example, it is assumed that the last sound material to be auditioned is the sound material snb. In addition, range designation arrows (start designation arrow as and end designation arrow ae) for adjusting the period (specific section) of the musical sound waveform signal corresponding to the sound material are displayed.
In addition, the audition button b8 for auditioning the sound indicated by the musical sound waveform signal in the period designated by the range designation arrow, and the musical sound waveform signal in the period designated by the range designation arrow are determined as the musical sound waveform signal corresponding to the sound material. A confirmation button b9 is displayed on the display screen 131.

  The user operates the range designation arrow to adjust the period of the musical sound waveform signal, and repeats the trial listening operation by operating the audition button b8, thereby designating the period of the musical sound waveform signal so that the desired sound material is obtained. . In FIG. 10, it is assumed that the sound material snb1 corresponding to the musical sound waveform signal in the period defined as the start time tsb and the end time teb is specified. When the user operates the confirm button b9, the extracted sound material snb is corrected to the sound material snb1 designated by the user.

As described above, when the registration button b7 is operated by the user, the audio data received by the selection unit 210 is registered in the waveform DB, while the specific data and the feature amount data are extracted in the sound material DB for each extracted sound material. sign up. At this time, for the modified sound material, specific data corresponding to the modified sound material is registered in the sound material DB. The feature amount data may indicate the feature amount calculated before correction, or may indicate the feature amount re-calculated by the feature amount calculation unit 222 for the musical sound waveform signal indicated by the corrected sound material. There may be. Which one is to be determined may be determined according to a user instruction.
In this way, it is possible to adjust the feeling of rising sound by shifting the start time of the musical sound waveform signal corresponding to the sound material, or to adjust the feeling of reverberation by shifting the end time.
The above is the description of the operation example when the reproduction program, the extraction program, and the correction program are executed.

  As described above, the musical sound processing apparatus 10 according to the embodiment of the present invention not only extracts the sound material from the musical sound waveform signal indicated by the audio data output based on one track, but also obtains it by processing the audio data. The sound material is extracted from the musical sound waveform signal. Therefore, various sound materials can be extracted by changing the extraction source musical sound waveform signal by changing the processing mode. The processing of the audio data here is realized in various modes as described above. In this musical sound processing apparatus 10, a mode in which signal processing for imparting an acoustic effect to a musical sound waveform signal indicated by audio data, a mode in which other audio data is synthesized with audio data, and the number of channels of audio data are set. The sound material can be extracted from the musical sound waveform signal obtained by processing the audio data in the conversion mode.

<Modification>
As mentioned above, although embodiment of this invention was described, this invention can be implemented in various aspects as follows.
[Modification 1]
In the above-described embodiment, with respect to the mode of synthesizing other audio data with the audio data of one track as the processing mode, the musical sound waveform signal to which the acoustic effect is imparted by the acoustic effect imparting unit 140 is used for both audio data. However, it may be audio data indicating a musical sound waveform signal to which no acoustic effect is given.
That is, the audio data that can be selected as a target from which sound material is extracted includes audio data obtained from one track that has been processed in at least one of the above-described processing modes. Just do it. In this case, the musical sound processing apparatus 10 may be configured such that any processing mode can be selected according to a user instruction. The selected content may be defined by setting data or the like.

[Modification 2]
In the above-described embodiment, the data processing apparatus of the present invention is realized by the functional configurations of the reproduction function, the sound material extraction function, and the correction function, but may not have the configuration of the correction function. .

[Modification 3]
In the above-described embodiment, the data processing apparatus of the present invention is applied to the musical sound processing apparatus 10 that realizes DAW, but may be applied to a digital mixer, an analog mixer, or the like. In this case, the data processing apparatus having the sound material extraction function processes the audio data by the mixer. Then, the data processing device may acquire the audio data in each signal line of the mixer, extract the sound material from the audio data, and register it in the database. In this case, since the playback function for processing the audio data is replaced by the function of the mixer, the data processing apparatus may be configured not to realize the playback function. As described above, the data processing apparatus can be applied to any apparatus as long as it has a configuration for realizing a function of processing audio data and a configuration for realizing a sound material extraction function.

[Modification 4]
In the embodiment described above, the data output unit 230 outputs specific data or the like and registers it in the database. However, the data output unit 230 does not necessarily have to be registered in the database. For example, specific data output from the data output unit 230 may be output from the interface 14 to an external device.

[Modification 5]
In the embodiment described above, the data output unit 230 generates data (sound material timing data) corresponding to the sound material track that defines the registered sound material and the sound generation timing in accordance with the registration of the sound material in the sound material DB. You may make it output and memorize | store in the memory | storage part 15. FIG. That is, the storage unit 15 may store data in which the identifier of the sound material extracted from one waveform data extraction source musical sound waveform signal and the information indicating the start time of the specific section corresponding to the sound material are associated with each other. .

FIG. 11 is a diagram for explaining an example of sound material timing data in Modification 5 of the present invention. The sound material timing data includes the reproduction start timing and the sound material No. in the sound material track. The data corresponding to the part. The playback start timing is expressed as a relative time such as the number of measures, beats, and ticks, while the start time of a specific section is an absolute time corresponding to the elapsed time from the beginning of the musical sound waveform signal. . For this reason, the data output unit 230 converts the absolute time into the relative time by using a predetermined tempo (for example, tempo = 120 (1 beat 0.5 seconds)) as a reference. Store as material timing data. Note that the reproduction start timing may be expressed in absolute time, and in this case, the above conversion is unnecessary.
When the sound material timing data stored in this way is reproduced at the above tempo by the reproduction function, the extracted sound material portion of the extracted source musical sound waveform signal is pronounced and the other portions are muted. Such a pronunciation is output from the speaker 161.

Note that the data output unit 230 may output the data in the format of the sound material track instead of the format of the sound material timing data and store it in the storage unit 15. In this case, for sound volume and time, predetermined values may be determined, or may be determined according to the content of the sound material. For example, the volume may be determined according to the signal level of the musical sound waveform signal corresponding to the sound material. The time may be determined according to the length of the musical sound waveform signal corresponding to the sound material.
In addition, the data output unit 230 may output data stored in the storage unit 15 in association with the reproduction start timing for each sound material in a format registered in the sound material DB.

[Modification 6]
In the above-described embodiment, the sound material track defines the sound material to be sounded by its identifier, but it may be defined by feature amount data.

FIG. 12 is a diagram for explaining an example of a sound material track in the sixth modification of the present invention. As shown in FIG. 12, in the sound material track in this example, the part indicating the identifier of the sound material in the embodiment shown in FIG. 4 is the part indicating the feature data. In this case, the data reading unit 110-3 in the reproduction function reads the feature amount data from the sound material track, and searches the sound material DB for feature amount data similar to the feature amount data to specify the feature amount data. Then, the data reading unit 110-3 may output the sound material data corresponding to the specified feature amount data to the sound material data synthesizing unit 130 at the reproduction start timing specified by the sound material track.
When a plurality of types of sound material DBs are stored in the storage unit 15, the sound material DBs to be searched for the data reading unit 110-3 to specify sound material data are also recorded in time series. It may be defined in the material track. Details of this configuration will be described in Modification 7 and later.

  The sound material track can also be defined by another aspect. For example, instead of prescribing a musical sound waveform signal by an identifier of sound material, a combination of each track in the sequence data (in this example, an audio track, a MIDI track) and a data range of audio data generated based on the track Thus, a musical sound waveform signal may be defined.

FIG. 13 is a diagram for explaining another example of the sound material track in the sixth modification of the present invention. As shown in FIG. 13, in the sound material track in this example, the sound material identifier in the embodiment shown in FIG. 4 includes a portion for designating a track (designated track) and a portion for designating a data range. It has become. The designated track indicates the track number. In the example shown in FIG. 13, tr1 and tr2 are shown. However, when described using the operation example of the embodiment, both of these tracks indicate audio tracks. In addition, if it is tr3, tr4, it will become a MIDI track.
Further, the data range is defined by the time designation information in the same manner as the data range in the sound material DB. The time designation information in this case is determined as the time from the beginning of the audio data generated based on the designated track. The data range may be defined based on a case where playback is performed at a predetermined tempo (for example, tempo = 120 (1 beat 0.5 seconds)), or at a tempo defined by the setting data. It may be defined on the basis of the case where it is reproduced.

  In this sound material track, when the designated track is an audio track, the data reading unit 110-3 performs the audio data output from the data reading unit 110-1 based on the designated track (audio data output from the branch B1). A portion corresponding to the audio data indicated by the data range is output as sound material data. As described above, the data reading unit 110-3 may acquire the portion output as sound material data from the data reading unit 110-1 prior to actual sequence data reproduction. For example, before reproducing sequence data, the data reading unit 110-1 may be operated in advance to output audio data in the data range.

  In the case where the designated track is a MIDI track, the data reading unit 110-3 displays the audio indicated by the data range among the audio data output from the sound source unit 120 (audio data output from the branch B2) based on the specified track. The portion corresponding to the data is output as sound material data. As described above, the data reading unit 110-3 may acquire the portion output as the sound material data from the sound source unit 120 prior to actual sequence data reproduction. For example, before reproducing the sequence data, the data reading unit 110-2 and the sound source unit 120 may be operated in advance to output audio data in the data range.

[Modification 7]
In the case where a plurality of types of sound material DBs are stored in the storage unit 15 described in the description of the modification example 6 described above, the data reading unit 110-3 is a search target for specifying sound material data. An example in which the sound material DB may be defined in the sound material track in time series will be described in detail. Here, a case where these processes are performed not only in the musical sound processing apparatus 10 but including exchanges between a plurality of apparatuses will be described.

FIG. 14 is a block diagram illustrating the configuration of the sound generation control system 1 according to Modification 7 of the present invention. The sound generation control system 1 includes a musical sound processing device 10A and a server device 50 connected via a communication line 1000 such as the Internet.
The musical tone processing apparatus 10A has substantially the same configuration as the musical tone processing apparatus 10 shown in the above-described embodiment. The interface 14 of the musical sound processing device 10A further functions as a communication unit that exchanges various types of information with the server device 50 via the communication line 1000. In addition, the sound material DB and the waveform DB may not be stored in the storage unit 15 of the musical sound processing device 10A. In this example, these DBs are stored in the server device 50. The storage unit 15 of the musical tone processing apparatus 10A stores a sound material track creation program executed by the control unit 11. The sound material track creation program is a program for creating a sound material track in this modification.

FIG. 15 is a diagram illustrating the configuration of the server device 50 according to Modification 7 of the present invention. The server device 50 includes a control unit 51, a communication unit 54, and a storage unit 55. Each of these components is connected via a bus.
The control unit 51 includes a CPU, RAM, ROM, and the like. The control unit 51 implements various functions by executing various programs stored in the ROM or the storage unit 55. In this example, the control unit 51 executes a search program according to an instruction from the musical tone processing apparatus 10A. When the search program is executed, a function of searching the sound material DB in the storage unit 55 and transmitting the specified sound material data to the music sound processing device 10A according to an instruction from the music sound processing device 10A is realized. Details of this function will be described separately.

  The communication unit 54 is connected to the communication line 1000 under the control of the control unit 51, and exchanges information with a communication device such as the musical tone processing device 10A. The control unit 51 may update the information stored in the storage unit 55 using the information acquired via the communication unit 54. The communication unit 54 is not limited to communication via the communication line 1000, and may have an interface configured to be connectable to an external device by wire or wireless.

The storage unit 55 is a hard disk, a nonvolatile memory, or the like, and has a storage area for storing various programs such as the search program described above, in addition to a storage area for storing the sound material DB and the waveform DB. Since the waveform DB is the same as the waveform DB stored in the storage unit 15 of the musical sound processing apparatus 10 as described above, description thereof is omitted.
The sound material DB is composed of a plurality of sound material DBs (sound material DBa, DBb,...). In the following description, each of them will be referred to as a sound material DB when they are described without distinction. Although each of the sound material DBs is different from the registered sound material data, the structure of the database is the same as the sound material DB stored in the storage unit 15 of the musical sound processing apparatus 10 described above, and thus the description thereof is omitted. . The sound material DB is stored in advance in the storage unit 55, but a new type of sound material DB may be further stored by obtaining the sound material DB from an external device via the communication unit 54. .

FIG. 16 is a diagram for explaining an example of a sound material track in Modification 7 of the present invention. The sound material track in this example is composed of feature quantity designation data (FIG. 16A) and DB designation data (FIG. 16B). The feature amount designation data is associated with a reproduction time indicating a sound generation timing, feature amount data corresponding to sound material data to be generated at that timing, sound volume and time. According to the example of FIG. 16A, at the reproduction time “0002: 01: 000” (corresponding to the first bar of the second measure), the sound based on the feature amount data Pi is set to the volume “20”, the sound generation time “ 120 ”is shown to be pronounced. As will be described later, the sound based on the feature value data Pi to be generated is not limited to the sound material indicated by the sound material data of the feature value data Pi, and is a similar sound.
The DB designation data is data in which the type of sound material DB to be searched when the sound material data is specified in the server device 50 is set as the playback time range using the feature quantity data indicated by the feature quantity designation data. It is. According to the example of FIG. 16B, the database to be searched is the sound material DBa from the reproduction times “0001: 01: 000” to “0001: 03: 959”.

  Next, functions realized when the control unit 11 of the musical sound processing device 10A executes the sound material track creation program and the control unit 51 of the server device 50 executes the search program along with this will be described. Note that some or all of the components for realizing the functions described below may be realized by hardware.

FIG. 17 is a functional block diagram for explaining the functions of the musical sound processing device 10A and the server device 50 according to the seventh modification of the present invention. When the control unit 11 executes the sound material track creation program, a display control unit 310, a setting unit 320, a sound generation control unit 330, and a data output unit 340 are configured. In this example, the musical sound processing device 10A functions as a sound generation control device by including a display control unit 310, a setting unit 320, and a sound generation control unit 330. When the control unit 51 executes the search program, the specifying unit 510 is configured. In this example, the server device 50 functions as a specific device when the specifying unit 510 is configured.
The display control unit 310 controls the display content of the display screen 131 in accordance with an instruction input by the user. In this case, the display screen 131 displays the contents as shown in FIG.

  FIG. 18 is a diagram for explaining a display example of the display screen 131 when the sound material track creation program is executed according to the seventh modification of the present invention. On the display screen 131, an icon arrangement area ST and a DB arrangement area DT are displayed roughly. The icon arrangement area ST and the DB arrangement area DT are determined with the horizontal axis as a common time axis. The beat line BL is an auxiliary line indicating the position for each beat. In the icon arrangement area ST, the vertical axis is a volume axis that defines the volume of the sound. Note that the volume axis may not be provided if the volume is defined regardless of the position of the icon image.

  Icon images s1, s2,... Are images associated with feature amount data. .. Are arranged in the icon arrangement region ST, so that the sound generation timing based on the corresponding feature amount data is set according to the position in the direction along the time axis at the left end of the icon image. It is prescribed. The volume is defined according to the position in the direction along the volume axis at the lower end of the icon image. The types of patterns of the icon images s1, s2,... Are determined so as to be different for each category (classification A, B,...) Into which the associated feature amount data is classified. For example, the feature amount data corresponding to the icon image s1 and the feature amount data corresponding to the icon image s2 are classified into different categories, and the feature amount data corresponding to the icon image s2 and the feature corresponding to the icon image s4. Quantity data is classified into the same category. Note that the pattern of the icon image may not be different depending on the category. That is, all may be the same pattern, or may be controlled so as to be different according to different parameters.

  The DB images d1, d2,... Are images in which the types of sound material DBs are associated. DB images d1, d2,... Are arranged in the DB arrangement region DT, and the sound material DB corresponding to the DB image is stored in the server apparatus according to the position in the direction along the time axis from the left end to the right end of the DB image. A period to be applied as a search target in 50 is defined. For example, the sound material DBa is applied as a search target from time t0 to time t2, and the sound material DBc is applied as a search target from time t1 to time t3. That is, from time t1 to time t2, both the sound material DBa and the sound material DBc are applied as search targets.

  The display screen 131 also includes a tempo control button b11 for setting the playback tempo, and a conversion for giving an instruction to convert the sound material track into audio data based on the arrangement mode of the icon image in the icon arrangement area. An instruction button b12 and a reproduction instruction button b13 for causing the audio data generated by the conversion to sound are displayed. The tempo control button b11 has the same function as the tempo control button b1 described above. In addition, a determination button b14 for storing the created sound material track in the storage unit 15 may be displayed.

  Returning to FIG. 17, the description will be continued. The setting unit 320 sets the type of the sound material DB along the time axis according to the instruction input by the user. In this example, the setting unit 320 outputs the setting content to the display control unit 310 to display the DB image on the display screen as shown in FIG. The setting unit 320 may prevent the DB image from being displayed on the display screen 131 without outputting to the display control unit 310. In this case, the DB arrangement area DT is not necessary. That is, if the type of the sound material DB is set along the same time axis as the icon arrangement area ST, the setting content may or may not be displayed on the display screen 131.

  The display control unit 310 and the setting unit 320 generate a sound material track in accordance with the icon image arrangement mode and the sound material DB type setting mode. Here, the display control unit 310 generates feature amount designation data among the sound material tracks, and the setting unit 320 generates DB designation data among the sound material tracks. The sound material track may be determined every time an icon image is arranged or the sound material DB is set, or when the conversion instruction button b12 is operated. It may be. When the save button b14 described above is displayed on the display screen 131, when the user operates the save button b14, the sound material track generated here is saved in the storage unit 15.

  When the conversion instruction button b <b> 12 is operated by the user, the sound generation control unit 330 transmits a part or all of the generated sound material track to the server device 50 via the communication unit 14. The control unit 51 starts and executes a search program. The part of the sound material track is at least data of a part in which the feature amount data is associated with the type (type information) of the sound material DB corresponding to the feature amount data on the time axis. Then, the sound generation control unit 330 receives the sound material data from the server device 50 via the interface 14 and causes the data output unit 340 to output the audio data based on the received sound material data and the sound material track. Specifically, the sound generation control unit 330 refers to the feature value designation data of the sound material track, processes the sound material data corresponding to the icon image to change the level according to the volume, and according to the reproduction time Output as audio data at timing. The data output unit 340 outputs audio data according to the control of the sound generation control unit 330.

The specifying unit 510 receives information based on the sound material track transmitted from the sound generation control unit 330 via the communication unit 54, and among the sound material DB stored in the storage unit 55, the search target indicated by the received information A certain kind of sound material DB is searched, and sound material data having similar feature amount data is specified for each feature amount data included in the sound material track. In this example, the specifying unit 510 treats the feature amount data as a vector amount composed of a plurality of feature amounts, and refers to the type of sound material DB to be searched for feature amount data having the shortest Euclidean distance. Identify sound material data.
Note that other known algorithms may be used as the algorithm for determining the similarity. Further, the Euclidean distance may not be the shortest, and the second and third closest ones may be specified. Information necessary to perform such identification may be set in advance by a user or the like. Further, the specified sound material data may not have a similarity relationship between the feature amount data and the feature amount data included in the sound material track, and may have a predetermined specific relationship. The search target may be narrowed down by category as well as the type of sound material DB to be searched. In this case, the category to be searched may be specified by the user, for example, or may be the same category as the feature amount data included in the sound material track or a related category. Here, the related categories may be determined according to a preset algorithm, or related categories may be set in advance.

Then, the specifying unit 510 transmits the specified material data to the sound generation control unit 330 via the communication unit 54. As described above, in this example, the communication unit 54 functions as an acquisition unit that acquires information by receiving information based on a sound material track and an output unit that outputs data by transmitting specified material data. .
The above is the description of the functional configuration. Next, the operation of the sound generation control system 1 when the sound material track creation program is executed will be described with reference to FIG.

  FIG. 19 is a diagram for explaining the operation of the sound generation control system 1 when the sound material track creation program is executed according to the seventh modification of the present invention. Here, the musical sound processing apparatus 10 </ b> A will be described with reference to processing after an instruction to execute the sound material track creation program is input from the user. When the sound material track creation program is executed, an icon arrangement area ST, a DB arrangement area DT, and the like are displayed on the display screen 131 as shown in FIG. It is assumed that the icon image and the DB image are not displayed at this stage.

  The user inputs an instruction to determine the contents of the feature amount data, an instruction to place an icon image of a pattern determined by the determined contents in the icon arrangement area ST, and an instruction to place the DB image in the DB arrangement area DT, thereby A sequence for generating is generated (step S110). As a result, it is assumed that the display screen 131 is displayed with the contents shown in FIG. The sound material track generated in this state has the contents shown in FIG. 16, for example. In this case, the feature amount data corresponding to the icon images s3, s4, and s5 are Ph, Pi, and Pj.

  When the user inputs a conversion instruction (step S120) by operating the conversion instruction button b12, the musical sound processing device 10A transmits a sound material track to the server device 50 by the sound generation control unit 330 (step S130). The sound material track to be transmitted does not have to be all information as described above, and the feature amount data is associated with the type of the sound material DB having a correspondence relationship with the feature amount data on the time axis. It is sufficient if the data has the same part.

  Upon receiving this data, the server device 50 executes a search program, searches the sound material DB by the specifying unit 510, and specifies the sound material data (step S140). For example, for the feature value data Pi corresponding to the icon image s3, a sound material DBc that is a type of the sound material DB having a correspondence relationship on the time axis is searched, and a sound having feature value data similar to the feature value data Pi is searched. The material data is transmitted to the musical tone processing apparatus 10A (step S150). At this time, the server device 50 transmits the identified sound material data so as to be able to identify which feature data corresponds to the specified sound material data.

When the reception of these data is completed, the musical sound processing device 10A notifies the user to that effect. When the user inputs a reproduction instruction by operating the reproduction instruction button b13 (step S160), the sound output control unit 330 controls the data output unit 340. The sound generation control unit 330 refers to the feature value designation data in the sound material track, adjusts the volume of the received sound material data, and outputs from the data output unit 340 as audio data according to the playback time of the corresponding feature value data. The output is made (step S170) and the sound is generated from the speaker 161.
The user listens to the pronunciation from the speaker 161 and, if the sound is desired, operates the enter button b14 to store the sound material track in the storage unit 15.

In this way, audio data is output from the musical sound processing apparatus 10A according to the sequence created by the user. Here, the type of the sound material DB to be searched is defined by the DB designation data. Accordingly, since the user changes the type of the sound material DB to be searched only by changing the DB designation data, even if the content of the feature quantity designation data has not been changed, Material data also changes. Therefore, the content that is pronounced in accordance with the user's reproduction instruction also changes. At this time, even if the sound material data changes, the feature value data does not change, so the sound of the sound material data does not change completely, and in most cases it is specified from those classified in the same category Is done. Therefore, when the type of the sound material DB corresponds to, for example, the genre of music (jazz, rock, etc.), even if the user only changes the DB designation data, the same pronunciation content (musical sound) The impression can be changed, such as jazz or rock, while maintaining the pattern.
Note that the data reading unit 110-3 and the sound material data synthesis unit 130 (see FIG. 5) in the embodiment receive audio data in the same manner as the function of the sound generation control unit 330 when the configuration of the modification example 7 is applied. What is necessary is just to make it output. Further, the data reading unit 110-3 and the sound material data synthesizing unit 130 may store the audio data output from the data output unit 340 in the storage unit 15 and handle and process the audio data in the same manner as the audio track. .

[Modification 8]
In the modified example 7 described above, the icon image displayed in the icon arrangement area ST may be configured to be expandable and contractable in the time axis direction in accordance with an instruction input by the user.

FIG. 20 is a diagram for explaining a display example of a display screen when the sound material track creation program is executed according to Modification 8 of the present invention. The display control unit 310 changes the length of the icon image displayed on the display screen 131 in the direction along the time axis according to the instruction input by the user. For example, the icon image s4 shown in FIG. 18 is expanded in the direction along the time axis like the icon image s41 shown in FIG. In this case, when the sound generation control unit 330 outputs the sound material data corresponding to the icon image s41 from the output unit 340, the sound generation control unit 330 performs time stretching according to the length of the icon image s41 along the time axis. Processing may be performed by performing processing (processing for extending the waveform of sound material data), loop processing (processing for repeatedly outputting sound material data), and the like, and the data output unit 340 may output the data as audio data. In this case, the time determined corresponding to the icon image in the sound material track is changed. It should be noted that necessary information such as sound generation end timing and loop playback flag may be added as sequence information.
Note that the icon images shown in FIG. 18 are all the same size and have been expanded and contracted in the direction along the time axis by the processing in this modified example. May be displayed with a length corresponding to the time determined corresponding to the icon image.

[Modification 9]
In the modified example 7 described above, the vertical axis of the icon arrangement region ST is a coordinate axis (volume axis) indicating volume, but may be a coordinate axis (hereinafter referred to as a designated coordinate axis) indicating pitch, sound length, and the like. That is, the icon arrangement area ST may have a designated coordinate axis indicating a designated value for designating other processing contents other than the volume of the sound material data. When the designated coordinate axis indicating the pitch is used, the sound generation control unit 330 may change the pitch of the sound material data according to the position of the icon image on the designated coordinate axis and output the audio data from the data output unit 340 as audio data. . When the designated coordinate axis indicating the sound length is used, the sound generation control unit 330 performs time stretching processing (processing for extending the waveform of the sound material data), loop processing (sound material) according to the position of the icon image on the designated coordinate axis. The data output unit 340 may output the data as audio data.
In addition, the processing content specified by the specified coordinate axis may be in multiple types such as volume and pitch. In this case, the icon image can be arranged by switching the type of the specified coordinate axis according to the instruction input by the user. It may be good. The sound material data may be variously processed by arranging the icon image in the icon arrangement area ST having the designated coordinate axis that can be switched in this way.

[Modification 10]
In the modified example 7 described above, the vertical axis of the icon arrangement area ST is a designated coordinate axis for designating the processing content of the sound material data, but the coordinate axis (a specific value for identifying the sound material data in the identifying unit 510 ( Hereinafter, it may be referred to as a specific coordinate axis). In this case, the specifying unit 510 may specify the sound material data according to the position of the icon image on the specific coordinate axis. For example, if the specific value is related to the similarity, the sound material data specified by the specifying unit 510 has a lower similarity as the icon image is positioned above the icon arrangement area ST (Euclidean distance between the feature data). (Long) can be specified.
Note that the degree of similarity is not specified by the user. For example, when the user performs a predetermined operation (such as an operation of a random button), the degree of similarity corresponding to all icon images or previously specified icon images can be obtained. It may be specified in accordance with a predetermined algorithm (for example, random).

  Further, the content of the specific value may include a plurality of types. In this case, the icon image is switched by switching between the specific coordinate axis in the icon arrangement area ST and the designated coordinate axis in the modification 9 in accordance with an instruction input by the user. It is good also as what can be arranged. As another example, for example, a category to be classified by changing feature data corresponding to an icon image may be changed.

[Modification 11]
In the modification example 7 described above, the sound generation control unit 330 outputs the audio data output from the data output unit 340 during the processing of step S170 shown in FIG. 19, that is, the execution of the sound material track creation program, on the time axis of the icon image. This was realized by outputting the sound material data at a timing according to the position, but it is generated by the sound generation control unit 330 as data indicating the sound generation contents of the entire output period and then output from the data output unit 340. May be. The sound generation control unit 330 may output the generated audio data to the storage unit 15 for storage. Further, the present invention is not limited to storing audio data, and a plurality of types of data necessary for generating audio data such as a combination of a sound material track and sound material data may be stored. The instruction for storing various data may be input by the user.

  In addition, the sound generation based on the audio data is output from the speaker 161 of the musical sound processing device 10A, but may be output from an external speaker device connected to the musical sound processing device 10A. You may make it output from the apparatus 50. FIG. That is, the sound generation control unit 330 may control not only the structure of the musical sound processing apparatus 10A but also each connected structure so that audio data is output so that sound is generated from any device that outputs sound.

[Modification 12]
In the modified example 7 described above, the icon image, DB image, and the like displayed on the display screen 131 when the sound material track creation program is executed are generated on the program, but these images are stored in advance in the storage units 15, 55, and the like. May be stored.

[Modification 13]
In the modified example 7 described above, the content of the feature amount data corresponding to the icon image displayed in the icon arrangement area ST is determined according to the instruction input by the user. Alternatively, a predetermined representative value of the category corresponding to the pattern may be determined as the feature amount data by selecting (or category).

[Modification 14]
In the modified example 7 described above, the content of the DB designation data is determined by the user placing a DB image in the DB placement area DT, but according to a predetermined algorithm, the reproduction time range and the sound material DB The control unit 11 may automatically determine the relationship with the type.

[Modification 15]
In the above-described modification example 7, when there is a playback time range in which the type of the sound material DB is not specified as the contents of the DB specification data, a predetermined sound material is determined in the playback time range not specified. The DB type (all types, some specific types, etc.) may be specified as a search target, or the type specified immediately before the range may be specified.

[Modification 16]
In the embodiment described above, the sound material data is acquired from the timely waveform DB according to the data range of the sound material DB, but only the sound material data of the section that has been cut out in advance may be stored. In that case, data range information is not required in the sound material DB.

[Modification 17]
In the modified example 7 described above, the user may be able to adjust the data range of the sound material data (specific data).

  FIG. 21 is a diagram for explaining a display example of the display screen when the sound material track creation program is executed according to the seventeenth modification of the present invention. FIG. 21 shows the display contents when the user performs a predetermined operation (such as a double click operation) on the icon image s3 in FIG. 18 described in the modified example 7. The pop-up window PW shown in FIG. 21 is displayed according to the user's operation, and its contents are enlarged waveforms corresponding to the icon image s3. Here, the waveform corresponding to the icon image s3 is a waveform of a sound that is pronounced corresponding to the icon image s3 during reproduction. The user can adjust the length in the time axis direction by changing the waveform range in the pop-up window PW.

[Modification 18]
In the modified example 7 described above, the DB image in which the type of the sound material DB is associated is displayed in the DB arrangement region DT. However, the type of the sound material DB and the time range to be searched are different. May be displayed.

  FIG. 22 is a diagram for explaining a display example of the display screen when executing the sound material track creation program according to the modification 18 of the present invention. In this example, the specified sound material DB is divided into lines and displayed in the DB arrangement area DT. In each row, DB period designation images d1a, d2a,... Indicating the time range in which the designated sound material DB is a search target are arranged. On the left side of the DB placement area DT, a DB type designation area DM for designating the type of the sound material DB corresponding to each row is displayed. The user can change the type of the sound material DB corresponding to each row by a pop-up menu or the like.

In the example shown in FIG. 22, the sound material DBa, DBb, and DBc are designated. For example, the sound material DBa is a search target in the time range designated by the DB period designation image d1a. The display in FIG. 22 is a display different from that in FIG. 18, but the type of sound material DB and the time range to be searched for are the same.
Note that the check box CB displayed on the left side of the DB type specification area DM may be used to specify whether the search target specified in the corresponding row is valid or invalid. This check box CB may be used in the display shown in FIG.

[Modification 19]
In the modified example 7 described above, the icon image in the display shown in FIG. 18 is arranged according to the feature amount data determined according to the user's instruction. When the user instructs the feature data, the desired feature data may be searched from the sound material DB. At this time, for example, a method disclosed in Japanese Patent Application Laid-Open No. 2011-163171 may be used. Further, the type of sound material DB to be searched at this time may be the sound material DB specified in the check box CB shown in FIG. When the user instructs the feature amount data corresponding to the icon image in this way, a display for instructing the feature amount data may be performed on a part of the display shown in FIG. Further, when a predetermined operation (such as a double click operation) is performed on the icon image, a pop-up window or the like may be displayed so that the feature amount data corresponding to the icon image can be designated.

[Modification 20]
In the display shown in FIG. 18 in the modified example 7 described above, the user has specified the type of the sound material DB and the time range to be searched for. For example, the user may perform a predetermined operation (random button). And the like may be designated according to a predetermined algorithm (for example, random). Also, as shown in FIG. 8, when the type of the sound material DB and the time range to be searched are determined in advance, any one of the DB images is designated and the specified DB image is selected. The type of sound material DB and the time range may be changed in accordance with a predetermined algorithm (for example, random). Furthermore, the random selection of the database enables the DB arrangement region DT itself to be active (the function for selecting the sound material DB is alive) to enable selection of the sound material DB, or to be inactive (select the sound material DB). It may also be possible to randomly apply only a specific sound material DB in a state where the function is not alive. Furthermore, the sound material DB selected in the DB arrangement area DT is made active (a state in which the selected sound material DB is a search target) or inactive (the selected sound material DB is not a search target). Or the state) may be performed at random. At this time, the DB image corresponding to the active sound material DB remains colored, and the DB image corresponding to the inactive sound material DB is grayed out. For example, whether the sound material DB whose display is changed is active or inactive. It is good to be able to recognize.

[Modification 21]
In the display shown in FIG. 18 in the modified example 7 described above, when there is no sound corresponding to the icon image, the icon image may be grayed out. If there is no corresponding sound, for example, if a sound material DB (sound material DB to be searched) corresponding to the icon image is not determined, it is based on the feature amount data corresponding to the icon image. In some cases, the sound material data specified by the specifying unit 510 does not exist. The sound material data specified by the specifying unit 510 does not exist, for example, the sound material data most similar to the feature amount data corresponding to the icon image is more similar than the similarity determined in advance as a threshold value. In some cases, the category is not included in the search target sound material DB when the search target is narrowed down by category.

[Modification 22]
In the modified example 7 described above, the sound generation control system 1 is configured by the musical sound processing device 10A and the server device 50 connected via the communication line 1000. However, the sound generation control system 1 is configured as an integrated device, and the communication line 1000 is connected. It is good also as a structure which is not interposed. Moreover, even if it is a separate apparatus, a part of each structure in 10 A of musical tone processing apparatuses may be included in the structure of the server apparatus 50, and conversely, a part of each structure of the server apparatus 50 is a musical sound. It may be included in the configuration of the processing apparatus 10A. Various information stored in the storage unit 15 of the musical sound processing device 10A and various information stored in the storage unit 55 of the server device 50 are stored in any storage unit as the pronunciation control system 1 as a whole. In addition to the musical sound processing apparatus 10 </ b> A and the server apparatus 50, a storage device that stores all or part of the information may be connected to the communication line 1000. Further, these pieces of information may be used by other users by sharing them with other musical sound processing apparatuses 10A that can be connected to the communication line 1000.
As a combination, for example, while the sound material DB is stored in the storage unit 55 of the server device 50, the waveform DB is stored in the storage unit 15 and the function of the specifying unit 510 is realized in the musical sound processing device 10A. Good. Furthermore, the execution of the search program and the extraction program may be executed by the musical sound processing device 10A, or may be executed by the server device 50 based on information obtained from the musical sound processing device 10A.

[Modification 23]
Each program in the above-described embodiment is provided in a state stored in a computer-readable recording medium such as a magnetic recording medium (magnetic tape, magnetic disk, etc.), an optical recording medium (optical disk, etc.), a magneto-optical recording medium, or a semiconductor memory. Can do. In addition, the musical sound processing apparatus 10 may download each program via a network.

DESCRIPTION OF SYMBOLS 1 ... Sound generation control system 10, 10A ... Music processing apparatus, 11 ... Control part, 12 ... Operation part, 13 ... Display part, 131 ... Display screen, 14 ... Interface, 15 ... Memory | storage part, 16 ... Sound processing part, 161 ... Speaker, 162 ... Microphone, 100 ... Playback function unit, 110-1,110-2,110-3 ... Data reading unit, 120 ... Sound source unit, 130 ... Sound material data synthesis unit, 140-1,140-2, 140-3, 140-4 ... sound effect applying unit, 150 ... audio data synthesizing unit, 160 ... synthesized sound effect applying unit, 170 ... channel converting unit, 180 ... bus unit, 200 ... sound material extraction function unit, 210 ... selection , 220 ... extraction part, 221 ... sound material specifying part, 222 ... feature quantity calculation part, 230 ... data output part, 300 ... modification part, 310 ... display control part, 320 ... setting part, 3 0 ... sound control unit, 340 ... data output unit, 510 ... particular unit, 1000 ... communication line

Claims (6)

Processing means for obtaining pronunciation data indicating pronunciation content and processing data for changing the pronunciation content, and processing the pronunciation data based on the processing data;
Analyzing the musical sound waveform signal indicating the sound content of the processed sound data, and specifying the period of the musical sound waveform signal having a predetermined characteristic;
A calculation means for analyzing the musical sound waveform signal of the specified period and calculating a feature amount;
A data processing apparatus comprising: data output means for outputting specific data indicating the musical tone waveform signal of the specified period and feature amount data indicating the calculated feature amount. The processed data includes setting data that defines the content of the acoustic effect to be given to the musical sound waveform signal,
The data processing apparatus according to claim 1, wherein the processing unit performs the processing based on the setting data so as to give an acoustic effect to the musical sound waveform signal related to the acquired sound generation data. The processed data includes pronunciation data defining the pronunciation content,
The processing means performs the processing by synthesizing a musical sound waveform signal related to the sound generation data included in the processed data with a musical sound waveform signal related to the acquired sound generation data. The data processing apparatus according to claim 2. The specific data indicates a musical sound waveform signal of the specified period by a musical sound waveform signal related to the processed sound generation data and time information indicating the specified period of the musical sound waveform signal;
When the specifying means specifies a plurality of periods,
The specific data indicates a musical sound waveform signal of the plurality of specified periods by a musical sound waveform signal related to the processed sound generation data and time information indicating the plurality of specified periods of the musical sound waveform signal. ,
The data processing apparatus according to any one of claims 1 to 3, wherein the calculation unit calculates the feature amount for each specified period. The data processing apparatus according to any one of claims 1 to 3, wherein the specific data indicates a musical sound waveform signal obtained by extracting a musical sound waveform signal of the specified period. Computer
Processing means for obtaining pronunciation data indicating pronunciation content and processing data for changing the pronunciation content, and processing the pronunciation data based on the processing data;
Analyzing the musical sound waveform signal indicating the sound content of the processed sound data, and specifying the period of the musical sound waveform signal having a predetermined characteristic;
A calculation means for analyzing the musical sound waveform signal of the specified period and calculating a feature amount;
A program for functioning as data output means for outputting specific data indicating a musical sound waveform signal of the specified period and feature amount data indicating the calculated feature amount.

Images