JP3755471B2 - Sound material processing apparatus and sound material processing program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a sound material processing apparatus and a sound material processing program for processing sound material data to be added to the video data in real time while reproducing the video data and making it possible to audition the data.
[0002]
[Prior art]
Conventionally, a sound material processing apparatus capable of processing sound material data, for example, a MIDI sequencer capable of processing MIDI (Musical Instrument Digital Interface) data is known.
[0003]
Conventionally, there has been known a video editing apparatus capable of adding sound material data as background music (hereinafter referred to as “BGM”) of video data.
[0004]
[Problems to be solved by the invention]
However, in the above-described conventional video editing apparatus, when the sound material data is added as the BGM of the video data, the sound material data and the video data are not after performing an editing operation of adding the sound material data to the video data. Cannot be played simultaneously. Therefore, the user cannot confirm how the sound material data (BGM) is added to the video data unless the editing operation is performed.
[0005]
In particular, when a user wants to process sound material data and add it to video data as BGM, the user once processes the sound material data with a sound material processing apparatus and saves it in a file, and then performs editing with the video editing apparatus. Since the video data to which the sound material data (BGM) is added can be confirmed for the first time, it is very inefficient when processing the sound material data again and again.
[0006]
The present invention has been made in view of this point, and an object thereof is to provide a sound material processing apparatus and a sound material processing program capable of improving the work efficiency of video editing.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, the sound material processing apparatus according to claim 1 reproduces video data and converts the video data into the video data. Has playback parameters to control playback mode Video editing means for executing editing processing for adding sound material data; In a state where the video editing means has not started playback of the video data, first processing means for changing the setting of the playback parameter in advance according to an instruction from a user, and the video editing means Reproduction control means for starting the reproduction of the sound material data based on the reproduction parameter set and changed by the first processing means in response to starting the reproduction of The video editing means is playing back video data In the state, the second processing means for changing the setting of the reproduction parameter in response to an instruction from the user, and the reproduction of the sound material data is stopped in response to the video editing means stopping the reproduction of the video data. A reproduction stop control means; a recording means for recording the reproduction parameter set and changed by the second processing means in the sound material data in response to the reproduction of the sound material data being stopped; and the reproduction control. A period from when the reproduction is started by the means until the reproduction is stopped by the reproduction stop control means, Sound material data reproducing means for reproducing the sound material data; The video editing unit adds audio data based on the sound material data in which the reproduction parameter is stored by the recording unit to the video data. It is characterized by that.
[0010]
Claim 2 The program for sound material processing described is ,in front Claim 1 An operation similar to that of the described sound material processing apparatus is executed by a computer.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
[0012]
FIG. 1 is a block diagram showing a schematic hardware configuration of a sound material processing apparatus according to an embodiment of the present invention.
[0013]
The sound material processing apparatus according to the embodiment of the present invention is constituted by, for example, a personal computer (PC) 1.
[0014]
The PC 1 includes a CPU 2 that controls the operation of the entire PC 1. The CPU 2 includes a RAM 3, a ROM 4, a MIDI interface (I / F) 5, an operation circuit 6, a display circuit 7, a tone generator circuit 8, via a communication bus 17. An external storage device 9, a communication interface (I / F) 10, and a timer 11 are connected to each other.
[0015]
The RAM 3 temporarily stores various information related to video data or sound material data and various data generated when the CPU 2 executes various programs. The ROM 4 stores various programs and various data executed or referred to by the CPU 2 (for example, video data, video time data and title data which are characteristic data of the video data, video editing program, sound material processing / playback program, and data. Format conversion processing program, etc.). The MIDI interface (I / F) 5 is an interface for connecting the electronic musical instrument 12 to the communication bus 17 and is used for communicating MIDI data between the electronic musical instrument 12 and the PC 1. The operation circuit 6 connects the operation element 13 to the communication bus 17, and outputs input data from the operation element 13, for example, a mouse or a keyboard, to the CPU 2 via the communication bus 17. The display circuit 7 connects the display device 14 to the communication bus 17 and outputs various data such as sound material data composed of video data and MIDI data or image data corresponding to a playback player for reproducing the data to the display device 14. The display device 14 displays the various data output from the display circuit 7. The sound source circuit 8 inputs sound material data via the communication bus 17 and outputs a musical sound signal to the sound system 15 based on the sound material data. The sound system 15 generates the input musical sound signal. The timer 11 is used in various processes executed by the CPU 2.
[0016]
The external storage device 9 stores various data such as video data and sample sound material data and various programs executed or referred to by the CPU 2, and performs the same function as the ROM 4. The CPU 2 reads various data and various programs stored in the external storage device 9 into the RAM 3 and executes processing, thereby executing the same operation as when the ROM 4 stores various data and various programs. be able to. Accordingly, various programs can be easily added or upgraded by changing the external storage device 9.
[0017]
The communication I / F 10 is wired or wirelessly connected to a network 16 such as the Internet or a commercial network. The PC 1 is connected to the server computer 18 that functions as a sound material data providing site via this network 16 and other computers. It is connected to the PC 19. As a result, the sound material data can be downloaded from the server computer 18 and various data and various programs can be downloaded from the other PC 19.
[0018]
The MIDI I / F 5 is not limited to a dedicated MIDI I / F, and may be configured using a general-purpose I / F such as RS232-C, USB or IEEE1394. . In this case, data other than MIDI data may be transmitted and received simultaneously.
[0019]
The hardware configuration of the server computer 18 that provides sound material data to the PC 1 is the same as the hardware configuration of the PC 1 described above. However, since the server computer 18 is mainly intended to provide sound material data to each user terminal such as the PC 1, the MIDI / I / F 5, the sound source circuit 8, the electronic musical instrument 12, and the sound system 15 may not be provided. Good.
[0020]
FIG. 2 is a block diagram showing functional configurations of the PC 1 and the server computer 18 in FIG.
[0021]
The PC 1 includes a video editing unit 21 that edits and plays back video data, a sound material processing / playback unit 22 that processes and plays back sound material data, and a software sound source unit 23 that converts the data format of the sound material data. The server computer 18 includes a sound material database 25 having a large number of sound material data, and a search engine 24 that searches and extracts sound material data desired by the user from the sound material database 25 in accordance with instructions from the PC 1.
[0022]
The video editing unit 21, the sound material processing / playback unit 22, and the software sound source unit 23 are functions realized by executing various programs included in the PC 1 by the hardware of the PC 1, and the sound material database 25 is a collection of sound material data. The system is structured systematically (for example, for each music category) so that these data can be searched using the search engine 24. The search engine 24 is for sound material data provided in the server computer 18. This is a function realized by executing the search program by the hardware of the server computer 18.
[0023]
Specifically, in the video editing unit 21, the CPU 2 reads out the video editing program stored in the ROM 4 or the external storage device 9 into the RAM 3, and follows the instructions of the video editing program read out into the RAM 3. This is realized by editing the video data stored in the storage device 9 and displaying the pre-edited or edited video data received by the display device 14 via the display circuit 7.
[0024]
Specifically, the sound material processing / playback unit 22 reads the sound material processing / playback program stored in the ROM 4 or the external storage device 9 into the RAM 3 by the CPU 2, and processes the sound material processing / playback data read into the RAM 3. The sound material data received from the sound material database 25 is processed or reproduced in accordance with the instruction of the reproduction program, and the display device 14 visually displays the processed or processed sound material data received via the display circuit 7. At the same time, the sound system 15 is realized by generating sound material data before or after processing received via the sound source circuit 8.
[0025]
Specifically, the software tone generator unit 23 reads the musical tone waveform generation / data format conversion processing program stored in the ROM 4 or the external storage device 9 into the RAM 3 by the CPU 2, and generates the musical sound waveform generation / readout data read into the RAM 3. This is realized by generating a musical sound waveform based on the sound material data received from the sound material database 25 or converting the data format in accordance with the instructions of the data format conversion processing program.
[0026]
Specifically, in the search engine 24, the CPU 2 receives the search condition from the PC 1 via the communication I / F 10, reads the search program stored in the ROM 4 or the external storage device 9 into the RAM 3, and receives the received A search condition is incorporated in the search program and search processing is executed, sound material data corresponding to the search condition is extracted from the sound material database 25, and the extracted sound material data is transmitted to the PC 1 via the communication I / F 10. It is realized by sending to.
[0027]
The function of the system composed of the PC 1 and the server computer 18 configured as described above will be described below.
[0028]
FIG. 3 is a flowchart showing a main process executed by the video editing unit 21 and the sound material processing / playback unit 22 provided in the PC 1 and the server computer 18, that is, the sound material providing site. The processing of the sound material providing site is mainly executed by the search engine 24.
[0029]
First, when the user selects video data to which sound material data is to be added via the operator 13, the video editing unit 21 determines video data to be edited (step S301), and video time that is characteristic data of the video data. Data and title data are acquired, supplied to the sound material processing / playback unit 22 (step S302), and a trial listening process is executed (step S307).
[0030]
On the other hand, the sound material processing / playback unit 22 receives the feature data from the video editing unit 21, that is, the video time data and the title data, and uses the video time data and the title data as search conditions to the sound material providing site. Access is made (step S303).
[0031]
The sound material providing site receives the video time data and the title data from the sound material processing / playback unit 22 as search conditions, searches the sound material database 25 based on the search conditions, and the sound material data that matches the search conditions. Is extracted (step S304). A known technique is used as a method of extracting sound material data.
[0032]
As shown in steps S303 and S304, the sound material processing / playback unit 22 automatically acquires video feature data including the playback time of the video data and supplies it to the sound material providing site. The user does not need to know the playback time of the video data, and the sound material data to be added to the video data can be efficiently acquired from the sound material providing site, thereby improving the work efficiency of the video editing. Can do. Note that the sound material processing / playback unit 22 may supply one of the video data playback time and the title to the sound material providing site.
[0033]
Next, the sound material providing site creates a list of the sound material data extracted in step S304 and transmits the created sound material data list to the sound material processing / playback unit 22 (step S305). Is executed (step S307).
[0034]
The sound material processing / playback unit 22 displays a list of sound material data received from the sound material providing site (step S306), and executes a trial listening process (step S307).
[0035]
In the trial listening process in step S307, the sound material processing / playback unit 22 instructs the sound material providing site to distribute the sound material data stream, and the sound material providing site receives the sound material data stream distribution. The sound material processing / playback unit 22 performs stream playback of the sound material data distributed from the sound material providing site, and the video editing unit 21 plays back the video data in synchronization with the stream playback. Then, the user listens to the sound material data while checking the video data. A detailed description of the trial listening process will be described later with reference to FIG. Here, the stream distribution is a technique for distributing data so that the sound material providing site can perform the stream reproduction without downloading the sound material data file in the sound material processing / reproducing unit 22. This is a technique in which the sound material processing / reproducing unit 22 reproduces the sound material data distributed from the sound material providing site in real time without downloading.
[0036]
When the user confirms the desired sound material data in the trial listening process in step S307, the sound material data is designated via the operator 13, and the sound material processing / playback unit 22 receives this designation, and the sound material providing site is designated. Is instructed to download the sound material data (step S308).
[0037]
The sound material providing site receives a sound material data download instruction from the sound material processing / reproducing unit 22 and distributes the sound material data to the sound material processing / reproducing unit 22 (step S309). A series of processing ends.
[0038]
The sound material processing / playback unit 22 receives the sound material data distributed from the sound material providing site, assigns an arbitrary file name to the received sound material data, and stores it by specifying an arbitrary storage location ( Step S310), processing / preview processing is executed (step S311).
[0039]
The video editing unit 21 executes the preview / listening process in step S307, and then executes the processing / preview process (step S311).
[0040]
In the processing / preview process in step S311, the video editing unit 21 plays back the video data in the preview mode, and the sound material processing / playback unit 22 before or during playback of the video data, if necessary, in step S310. Process the saved sound material data, play back the sound material data processed as needed in synchronization with the playback of the video data, and record the processing parameters used for processing the sound material data in the sound material data To do. A detailed description of the processing / preview process will be described later with reference to FIG.
[0041]
After the processing / preview processing in step S311, the sound material processing / playback unit 22 executes MIDI to audio conversion processing (step S312).
[0042]
In the MIDI to audio conversion process, the sound material data (that is, MIDI data) in which the processing parameters are recorded by the sound material processing / playback unit 22 and the software sound source unit 23 is converted into an audio format that can be added to the video data by the video editing unit 21. A detailed description of the MIDI to audio conversion processing will be described later with reference to FIG.
[0043]
The sound material processing / playback unit 22 saves the sound material data converted into the audio format in an arbitrary file and notifies the video editing unit 21 to that effect (step S313). A series of processing ends.
[0044]
The video editing unit 21 reads the file saved in step S313, adds the read file, that is, the sound material data converted into the audio format, to the video data (step S314), and the series in the video editing unit 21. Terminate the process.
[0045]
In the main processing described above, the sound material processing / playback unit 22 directly executes a display or download instruction for a list of sound material data, but a processing unit for executing a display function or an instruction output function like a web browser is provided separately. In this case, a list of sound material data and a download instruction may be executed.
[0046]
The sound material processing / playback unit 22 stores the sound material data converted into the audio format in an arbitrary file (step S313), and the video editing unit 21 reads the stored file. After the playback unit 22 converts the audio material data into the audio format of the audio material data by the MIDI to audio conversion process (step S312), the sound material data converted into the audio format is directly stored in the video editing unit 21 without saving the file. You may make it transmit.
[0047]
FIG. 4 is a flowchart showing the trial listening process in step S307 in FIG.
[0048]
First, the user designates the position to which sound material data is to be added in the video data via the operation element 13, whereby the video editing unit 21 sets the position to which the sound material data is to be added (step S401). (Step S402).
[0049]
Next, the video editing unit 21 receives a playback instruction for video data input via the operator 13 and notifies the sound material processing / playback unit 22 that the instruction has been received (step S403).
In response to the instruction from the video editing unit 21 in step S403, the sound material processing / playback unit 22 receives the instruction from the list of sound material data displayed in step S306 in FIG. Stream distribution is instructed to the sound material providing site for the sound material data designated in advance (step S408).
[0050]
The sound material providing site receives the stream distribution instruction regarding the sound material data from the sound material processing / playback unit 22 in step S408, and executes the stream distribution of the sound material data corresponding to the instruction (step S415).
[0051]
When the stream distribution is executed by the sound material providing site, the sound material processing / reproducing unit 22 receives the sound material data from the sound material providing site, and whether or not the stream reproduction is possible, that is, the sound material for a predetermined time. It is determined whether or not data has been accumulated (step S409). This determination is performed in consideration of stream reproduction, that is, for the purpose of allowing the sound material processing / reproducing unit 22 to accumulate sound material data that does not interrupt the reproduced sound. The predetermined time is, for example, 5 seconds. However, since the stream reproduction is affected by the reproduction processing speed of the sound material data in the sound material processing / reproduction unit 22 and the communication speed of the network such as the Internet, the predetermined time is The sound material processing / playback unit 22 sets the sound material. In addition, since the stream reproduction is performed while receiving subsequent data, the accumulation of the sound material data for the predetermined time is not an accumulation of all the sound material data.
[0052]
If the sound material data for a predetermined time is accumulated as a result of the determination in step S409, the sound material processing / reproducing unit 22 instructs the video editing unit 21 to start reproducing the video data (step S410) and accumulates it. The sound material data for a predetermined time and the sound material data continuously distributed from the sound material providing site are stream-reproduced so that the sound is not interrupted (step S411).
[0053]
On the other hand, the video editing unit 21 receives the playback start instruction from the sound material processing / playback unit 22 in step S410 and executes playback of the video data (step S404), and the video data input via the operator 13 Whether or not to continue the reproduction of the video data is determined based on the reproduction stop instruction or the finite time length of the video data (step S405).
[0054]
As a result of the determination in step S409, when the video editing unit 21 continues to reproduce the video data, the process returns to step S404, and the video editing unit 21 continues to reproduce the video data, while the video editing unit 21 does not continue to reproduce the video data. Stops the reproduction of the video data (step S406).
[0055]
Next, the video editing unit 21 instructs the sound material processing / playback unit 22 to stop playback of the sound material data (step S407), ends a series of processing in the video editing unit 21, and proceeds to the main processing of FIG. Return.
[0056]
After starting the sound material data stream reproduction, the sound material processing / reproducing unit 22 determines whether or not the sound material data reproduction stop instruction transmitted from the video editing unit 21 is received in step S407 (step S412).
[0057]
As a result of the determination in step S412, if the sound material data reproduction stop instruction has not been received, the process returns to step S411, and the sound material data stream reproduction is executed, while the sound material data reproduction stop instruction is received. If so, the stream reproduction of the sound material data is stopped (step S413).
[0058]
Next, the sound material processing / playback unit 22 instructs the sound material providing site to stop the stream distribution of the sound material data (step S414), and the series of processing in the sound material processing / playback unit 22 is terminated. Returning to 3 main processing.
[0059]
The sound material providing site receives the stream distribution stop instruction transmitted from the sound material processing / playback unit 22 after starting the sound material data stream distribution, and stops the sound material data stream distribution (step S416). The series of processes at the sound material providing site is terminated, and the process returns to the main process in FIG. In step S412, it is determined whether or not the sound material processing / playback unit 22 has received a sound material data playback stop instruction from the video editing unit 21, but the sound material processing / playback unit 22 performs the video editing unit 21. To determine whether or not video data playback stop has been detected, and as a result, if video data playback stop has not been detected, the process returns to step S411 to detect video data playback stop. If so, the process may proceed to step S413.
[0060]
In the trial listening process described above, the sound material processing / playback unit 22 instructs the sound material providing site to stream the sound material data, and the sound material providing site receives the stream distribution of the sound material data. The sound material processing / reproducing unit 22 performs stream reproduction of the sound material data distributed from the sound material providing site, and the video editing unit 21 reproduces the video data in synchronization with the stream reproduction. Therefore, the user confirms how sound material data (for example, BGM) is added to the video data by stream reproduction before downloading the sound material data to add to the video data. Therefore, it is possible to improve the work efficiency of video editing. (Files that are not required for stream playback when the file name required for download and the storage destination are specified are not necessary for stream playback, and the result of the audition does not matter.) (Erasing work is also unnecessary.) In particular, since the sound material data stream playback is synchronized with the playback of the video data, not the stream playback of the sound material data, the video data and the added sound material data The relationship is very easy to understand.
[0061]
The sound material processing / playback unit 22 receives a video data playback instruction from the video editing unit 21, instructs the sound material providing site to stream predetermined sound material data, and streams from the sound material providing site. When the distributed sound material data can be played back as a stream, the video data playback unit 21 transmits permission to start playback of the video data and stream playback of the sound material data, so that the video data starts playback first. Therefore, the sound material data and the video data can be reliably reproduced in synchronism with each other.
[0062]
Further, the sound material processing / reproducing unit 22 receives the sound material data reproduction stop instruction, stops the sound material data stream reproduction, and further instructs the sound material providing site to stop the sound material data stream distribution. Or the sound material processing / playback unit 22 accesses the video editing unit 21 to detect the stop of playback of the video data, and stops the streaming of the sound material data to the sound material providing site according to the detection result Therefore, there is an effect that the user does not need to issue each stop instruction to the video editing unit 21, the sound material processing / playback unit 22, and the sound material providing site.
[0063]
In the trial listening process described above, when the sound material processing / playback unit 22 receives the sound material data playback stop instruction transmitted from the video editing unit 21, the sound material data stream playback is stopped. When the time length is shorter than the playback time of the video data in the video editing unit 21, the sound material providing site ends the stream distribution from the sound material processing / playback unit 22 when the stream distribution of the sound material data is completed. The stream distribution of the sound material data may be automatically stopped without receiving the stop instruction. Also, in the case of data with a small file size such as MIDI data, playback is started after all data is received and stored in the cache memory, and pseudo stream playback that is erased from the cache memory when playback ends is played back. In this case, since the file download is unnecessary, the same effect as described above can be obtained.
[0064]
FIG. 5 is a flowchart showing the processing / preview process in step S311 in FIG.
[0065]
First, the video editing unit 21 sets a preview mode in the video editing program in accordance with a user instruction input via the operator 13 (step S501).
[0066]
Next, the video editing unit 21 receives an instruction to reproduce video data input via the operator 13, and transmits to the sound material processing / reproducing unit 22 that the video data reproduction instruction has been accepted (step S502). .
[0067]
The video editing unit 21 reproduces the video data (step S503), and continues the reproduction of the video data based on the instruction to stop the reproduction of the video data input via the operation element 13 or the limited time length of the video data. Whether or not (step S504).
[0068]
As a result of the determination in step S504, when the video editing unit 21 continues to reproduce the video data, the process returns to step S503, and the video editing unit 21 continues to reproduce the video data while the video editing unit 21 does not continue to reproduce the video data. Stops the reproduction of the video data (step S505).
[0069]
Next, the video editing unit 21 instructs the sound material processing / playback unit 22 to stop playback of the sound material data (step S506), ends a series of processing in the video editing unit 21, and proceeds to the main processing of FIG. Return.
[0070]
On the other hand, the sound material processing / reproducing unit 22 first processes the sound material data stored in the file in step S310 of FIG. 3 in accordance with the sound material data processing instruction input via the operator 13 (step S310 in FIG. 3). Step S507). Specifically, this processing process adjusts parameters of sound material data. These parameters (hereinafter referred to as “processing parameters”) are, for example, part on / off, tone change, mixer, effect addition, tempo. It is composed of parameters that can easily modify the musical content of the sound material data, such as a change or a pitch change. In this processing, unlike a MIDI sequencer that performs advanced corrections such as correcting MIDI events one by one, it is possible for beginners to easily process sound material data. If no processing instruction is input via the operator 13, the process in step S507 is cancelled.
[0071]
Next, the sound material processing / playback unit 22 receives that the playback instruction of the video data transmitted from the video editing unit 21 has been received, and plays back the sound material data that has been processed as necessary ( Step S508). Thereafter, during the reproduction of the sound material data, the processing of the sound material data being reproduced is performed as necessary, similarly to step S507 (step S509).
[0072]
Next, the sound material processing / playback unit 22 determines whether or not a sound material data playback stop instruction has been received from the video editing unit 21 (step S510), and has not received the sound material data playback stop instruction. In this case, the process returns to step S508, and the sound material data is continuously reproduced. On the other hand, when the sound material data reproduction stop instruction is received, the reproduction of the sound material data is stopped (step S511). In step S510, the sound material processing / playback unit 22 determines whether or not the sound material data playback stop instruction has been received from the video editing unit 21, but the sound material processing / playback unit 22 performs the video editing unit 21. To determine whether or not the video data playback stop has been detected. If the video data playback stop has not been detected, the process returns to step S508 while the video data playback stop has been detected. In that case, the process may proceed to step S511.
[0073]
Finally, the sound material processing / playback unit 22 records the processing parameters adjusted as necessary in step S507 or step S509 in the sound material data (step S512), and ends the series of processes. Returning to the main process of FIG.
[0074]
In the processing / preview process described above, the video editing unit 21 reproduces the video data in the preview mode, and the sound material processing / reproducing unit 22 outputs the sound material data as necessary before or during the reproduction of the video data. The sound material data that has been processed and processed as needed is reproduced in synchronization with the reproduction of the video data, and the processing parameters used for the processing of the sound material data are recorded in the sound material data. Therefore, the user can process the sound material data in real time while reproducing the video data, and can process the processed sound material data without waiting for completion of the creation of the video data to which the sound material data is added. And synchronized playback of video data can be confirmed. Therefore, since the user can confirm how sound material data (for example, BGM) is added to the video data, the work efficiency of video editing can be improved. Further, since the sound material processing / playback unit 22 can perform the processing during the playback of the sound material data, it is possible to improve the work efficiency of the video editing. Further, it may be possible to compare and listen to the sound material data before and after the processing.
[0075]
In the above-described processing / preview process, when the sound material processing / playback unit 22 receives the sound material data playback stop instruction transmitted from the video editing unit 21, the sound material data playback is stopped. Is shorter than the playback time of the video data in the video editing unit 21, the sound material processing / playback unit 22 before receiving the sound material data playback stop instruction transmitted by the video editing unit 21. The reproduction of the sound material data may be automatically stopped.
[0076]
FIG. 6 is a flowchart showing the MIDI to audio conversion process in step S312 in FIG.
[0077]
First, the sound material processing / playback unit 22 reads the sound material data in which the processing parameters are recorded, that is, reads out the MIDI data while maintaining the relationship of the relative time of each event at high speed, and outputs the MIDI data for each event as the software sound source unit 23. (Step S601).
[0078]
The software tone generator 23 generates audio waveform data based on the received MIDI data for each event, temporarily stores the generated audio waveform data in a buffer (RAM 3), and stores the generated audio waveform data for a predetermined time (for example, 0.5 seconds). Min) is transmitted to the sound material processing / playback unit 22 (step S602). In the buffer for temporarily storing the audio waveform data, the audio waveform data is sequentially generated in an empty area by transmitting the audio waveform data to the sound material processing / reproducing unit 22.
[0079]
The sound material processing / reproducing unit 22 sequentially records the audio waveform data for a predetermined time received from the software sound source unit 23 sequentially (step S603). Here, “internal recording” means that the audio waveform data received from the software sound source unit 23 is sounded, and the generated sound is not recorded but is recorded without producing sound.
[0080]
Next, the sound material processing / playback unit 22 determines whether or not all of the MIDI data to be converted has been read (step S604), and if all of the MIDI data has not been read, step S604 is performed. Returning to S601, while the reading of the MIDI data is continued, if all the reading of the MIDI data has been completed, the audio editing unit 21 can handle the audio waveform data recorded internally in step S603. It can be converted into a preset audio format (step S605), the series of processing is terminated, and the processing returns to the main processing in FIG.
[0081]
In the MIDI to audio conversion process described above, the sound material data (ie, MIDI data) in which the processing parameters are recorded by the sound material processing / playback unit 22 and the software sound source unit 23 is converted into an audio format that can be added to the video data by the video editing unit 21. Therefore, in the sound material processing / playback unit 22 and the software sound source unit 23, from the processing of the sound material data to the conversion of the sound material data into the audio format can be executed at once, and the work efficiency can be improved. it can. In addition to the PC 1, there is an effect that it is not necessary to separately provide a conversion device for converting the audio format of the sound material data.
[0082]
The sound material data provided at the above-described sound material providing site is not limited to MIDI data, but may be audio format data. In that case, the MIDI to audio conversion process is unnecessary. As for the processing of audio format data, although part on / off or tone color change processing or mixer processing cannot be performed, effect addition, tempo change or pitch change processing is possible.
[0083]
Audition of the above-mentioned sound material data (when the sound material data is stream-reproduced or when the sound material data is processed and the video data is previewed), the video reproduction from the video editing unit 21 is started or stopped. Although it was performed according to the instruction, it may be performed according to an instruction from the sound material processing / reproducing unit 22. That is, when the sound material data is stream-reproduced, when the sound material processing / reproduction unit 22 instructs stream reproduction, the sound material providing site is instructed to stream the sound material, and the sound material data stream-distributed When the video can be reproduced, the video editing unit 21 is instructed to reproduce the video data. When the sound material processing / playback unit 22 is instructed to stop stream reproduction, the sound material providing site is instructed to stop stream distribution and the video editing unit 21 is instructed to stop reproduction of video data. When the video data is previewed after the sound material data is processed, when the preview processing is instructed by the sound material processing / playback unit 22, the playback of the sound material data is started and the video editing unit 21 is started. When the playback of the video data is instructed and the sound material processing / playback unit 22 is instructed to stop the preview process, the playback of the sound material data is stopped and the video editing unit 21 is instructed to stop the playback of the video data. The
[0084]
In the above processing of sound material data, the parameters for part on / off of sound material data, tone change, mixer, effect application, tempo change and pitch change are adjusted. It does not have to be provided, and at least one is sufficient. There may also be other types of processing. As for “tempo change” which is one of the parameters, the tempo may be automatically changed so that the playback time of the video matches the time length of the sound material data. The MIDI to audio conversion process is realized by the cooperation of the sound material processing / playback unit 22 and the software sound source unit 23, but this function may be realized by the software sound source unit 23 alone.
[0085]
The video editing unit 21 and the sound material processing / playback unit 22 are realized by executing different programs by the CPU 2, but may be realized by executing one program by the CPU 2.
[0086]
In the sound material site, the charging process may be executed when the sound material data is downloaded, and the charging process may not be executed when streaming the sound material data.
[0087]
It is needless to say that the functions of the above-described embodiments can also be realized in the form of a program that is software, thereby achieving the object of the present invention.
[0088]
Furthermore, a storage medium in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus, and the computer (or CPU 2) of the system or apparatus stores the program code in the storage medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.
[0089]
In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium storing the program code constitutes the present invention.
[0090]
As a storage medium for supplying the program code, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, semiconductor memory, magnetic tape, nonvolatile A memory card, ROM, or the like can be used. Further, the program code may be supplied from another electronic musical instrument 12 or another PC 19 via the network 16.
[0091]
【The invention's effect】
As explained in detail above, the sound material processing apparatus according to claim 1 and the claim 2 According to the program for sound material processing described, In a state where the video editing means has not started playing the video data, the first processing means changes the setting of the playback parameter in advance according to the instruction from the user, and the video editing means starts playing the video data. Accordingly, the reproduction control means starts reproduction of the sound material data based on the reproduction parameter set and changed by the first processing means, and in the state where the video editing means is reproducing the video data, The processing means changes the setting of the playback parameter in response to an instruction from the user, and the playback stop control means stops the playback of the sound material data in response to the video editing means stopping the playback of the video data, and the recording means When the playback of the sound material data is stopped, the playback parameters set and changed by the second processing means are recorded in the sound material data, and the sound material data is played back. Period up stage is made the stop of reproduction by the reproduction stop control means from being made the start of reproduction by the reproduction control means, so to reproduce the sound material data, The user can confirm how sound material data (for example, BGM) is added to video data, and can improve the work efficiency of video editing.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic hardware configuration of a sound material processing apparatus according to an embodiment of the present invention.
2 is a block diagram showing functional configurations of a PC 1 and a server computer 18 in FIG. 1. FIG.
3 is a flowchart showing a main process executed by the video editing unit 21, the sound material processing / playback unit 22 and the server computer 18, that is, the sound material providing site, included in the PC 1 in FIG.
4 is a flowchart showing a trial listening process in step S307 in FIG.
FIG. 5 is a flowchart showing processing / preview processing in step S311 in FIG. 3;
6 is a flowchart showing a MIDI to audio conversion process in step S312 in FIG. 3;
[Explanation of symbols]
1 PC 2 CPU (video editing means, sound material data processing means, sound material data reproduction means) 3 RAM 4 ROM 5 MIDI I / F 6 operation circuit 7 display circuit 8 sound source circuit 9 external storage device 10 communication I / F

Claims (2)

Video editing means for performing editing processing for reproducing video data and adding sound material data having playback parameters for controlling the playback mode to the video data;
In a state where the video editing means has not started playback of the video data, first processing means for changing the setting of the playback parameter in advance according to an instruction from a user;
Playback control means for starting playback of the sound material data based on the playback parameters set and changed by the first processing means in response to the video editing means starting playback of the video data;
Second processing means for changing the setting of the playback parameter in response to an instruction from a user in a state where the video editing means is playing back video data ;
Playback stop control means for stopping playback of the sound material data in response to the video editing means stopping playback of video data;
Recording means for recording the reproduction parameter set and changed by the second processing means in the sound material data in response to the reproduction of the sound material data being stopped;
A sound material data reproducing unit that reproduces the sound material data during a period from when the reproduction is started by the reproduction control unit to when reproduction is stopped by the reproduction stop control unit ;
The sound material processing apparatus , wherein the video editing unit adds audio data based on the sound material data in which the reproduction parameter is stored by the recording unit to the video data . In a computer for storing sound material data and video data having playback parameters for controlling the playback mode ,
A video editing means for executing the sound material data you added editing process to the video data while reproducing the video data,
In a state where the video editing means has not started playback of the video data, first processing means for changing the setting of the playback parameter in advance according to an instruction from a user;
Playback control means for starting playback of the sound material data based on the playback parameters set and changed by the first processing means in response to the video editing means starting playback of the video data;
Second processing means for changing the setting of the playback parameter in response to an instruction from a user in a state where the video editing means is playing back video data ;
Playback stop control means for stopping playback of the sound material data in response to the video editing means stopping playback of video data;
Recording means for recording the reproduction parameter set and changed by the second processing means in the sound material data in response to the reproduction of the sound material data being stopped;
A period of time from the start of playback by the playback control means to the stop of playback by the playback stop control means , causing the sound material data playback means to play back the sound material data ,
The audio material processing program , wherein the video editing unit adds audio data based on the audio material data in which the reproduction parameter is stored by the recording unit to the video data .

Images