JP2003255957A - Apparatus and program for phoneme material audition - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus and a program for phoneme material audition which can improve the operation efficiency of video editing. <P>SOLUTION: A phoneme material processing and reproduction part 22 instructs a phoneme material providing site to distribute a stream of phoneme material data and the phoneme material providing server distributes the stream of the phoneme material data in response; and the phoneme material processing and reproduction part 22 reproduces the stream of phoneme material data distributed from the phoneme material providing site and a video editing part 21 reproduces video data in synchronism with the stream reproduction. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound material audition apparatus and a sound material audition program capable of previewing sound material data candidates to be added to the video data while reproducing the video data.

[0002]

2. Description of the Related Art Conventionally, sound material data (eg MI
There is a service that provides DI (Musical Instrument Digital Interface) music data, etc., and allows a user to access the website with a personal computer or the like and download the sound material data.

Further, conventionally, there has been known a video editing apparatus capable of adding sound material data as background music (hereinafter referred to as "BGM") of video data.

[0004]

However, in the above-mentioned conventional video editing apparatus, the sound material data is converted to the video data B.
When adding as a GM, there is a problem that the sound material data and the video data cannot be reproduced at the same time unless the editing work of adding the already downloaded sound material data to the video data is executed. Therefore,
The user could not confirm how the sound material data (BGM) was added to the video data only after performing the editing work.

Further, in the conventional video editing apparatus, it is necessary to download the sound material data in advance when executing the editing work. Therefore, in order to obtain a desired BGM, a large number of sound material data are previously stored. Since it is necessary to download the sound material data or access the website many times to download the sound material data, the work efficiency of video editing is poor and the usability is poor.

The present invention has been made in view of the above points, and an object of the present invention is to provide a sound material audition apparatus and a sound material audition program capable of improving the work efficiency of video editing.

[0007]

In order to achieve the above object, the sound material listening apparatus according to claim 1 is a sound material listening apparatus connected to a sound material providing site for providing sound material data. And a stream distribution instruction for instructing the sound material providing site to stream the sound material data to be reproduced synchronously with the video data reproduced by the video editing means. Means and a stream reproducing means for reproducing the sound material data stream-distributed from the sound material providing site in a stream, for reproducing the sound material data in synchronization with the video data reproduced by the video editing means. And is provided.

A sound material listening apparatus according to a second aspect is the sound material listening apparatus according to the first aspect, wherein the stream reproducing means is capable of stream reproducing sound material data stream-distributed from the sound material providing site. And a video data reproduction start instruction means for instructing the video editing means to start reproduction of the video data when the judgment means judges that stream reproduction is possible. When the video data reproduction start instructing means instructs the video editing means to start reproducing the video data, the stream reproduction of the sound material data streamed from the sound material providing site is started. It is characterized by

A sound material listening apparatus according to a third aspect is the sound material listening apparatus according to the first or second aspect, wherein the stream playback means detects a playback stop detecting means for detecting a playback stop of the video data in the video editing means. And a stream distribution stop instruction transmitting means for transmitting an instruction to stop the stream distribution of the sound material data to the sound material providing site in response to the stop of the reproduction of the video data.

A sound material listening apparatus according to claim 4 is a sound material listening apparatus which is connected to a sound material providing site for providing sound material data and which operates in cooperation with a video reproducing means for reproducing video data. , Stream distribution instructing means for instructing the sound material providing site to stream the sound material data, and determining whether or not stream reproduction of the sound material data streamed from the sound material providing site is possible. And a video data reproduction start instruction means for instructing the video reproduction means to start reproduction of video data when the judgment means determines that stream reproduction is possible, and the video data reproduction start When the instructing means instructs the video reproducing means to start reproducing the video data, the sound material providing site streams the stream. Characterized in that it comprises a stream reproduction means for starting the stream reproduction of the sound material data that.

The sound material listening apparatus according to claim 5 is a sound material listening apparatus which is connected to a sound material providing site for providing sound material data and operates in cooperation with a video reproducing means for reproducing video data. Stream reproduction means for performing stream reproduction of the sound material data stream-distributed from the sound material providing site, reproduction stop detection means for detecting reproduction stop of the video data in the video reproduction means, and the reproduction stop means for detecting the reproduction stop of the video data. Stream distribution stop instruction transmitting means for transmitting a stream distribution stop instruction of the sound material data to the sound material providing site.

The sound material audition program according to claims 6, 7 and 8 causes a computer to execute the same operation as that of the sound material audition apparatus according to claims 1, 4 and 5, respectively.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a schematic hardware configuration of a sound material listening apparatus according to an embodiment of the present invention.

The sound material listening apparatus according to the embodiment of the present invention is composed of, for example, a personal computer (PC) 1.

The PC1 is a C which controls the operation of the entire PC1.
The CPU 2 includes a PU 2, and the CPU 2 has a RAM 3, a ROM 4, and a MIDI interface (I
/ F) 5, an operation circuit 6, a display circuit 7, a tone generator circuit 8, an external storage device 9, a communication interface (I / F) 10 and a timer 11 are connected to each.

The RAM 3 temporarily stores various information relating to video data or sound material data and various data generated when the CPU 2 executes various programs. ROM
Reference numeral 4 denotes various programs and various data executed or referred to by the CPU 2 (for example, video data, video time data or title data which is characteristic data of the video data, video editing program, sound material processing / playback program and data). Format conversion processing program, etc.) is stored. The MIDI interface (I / F) 5 is an interface for connecting the electronic musical instrument 12 to the communication bus 17, and is used for communicating MIDI data between the electronic musical instrument 12 and the PC 1. The operation circuit 6 connects the operator 13 to the communication bus 17, and outputs the input data from the operator 13, such as a mouse or a keyboard, to the CPU 2 via the communication bus 17. The display circuit 7 connects the display device 14 to the communication bus 17, for example, video data and MIDI.
Various data such as sound material data made of data or image data corresponding to a reproduction player for reproducing these are output to the display device 14, and the display device 14 displays the various data output from the display circuit 7. The tone generator circuit 8 inputs the sound material data via the communication bus 17, and outputs a tone signal to the sound system 15 based on the sound material data. The sound system 15 generates this inputted musical tone signal. The timer 11 is used in various processes executed by the CPU 2.

The external storage device 9 stores various data such as video data and sample sound material data, and various programs executed or referred to by the CPU 2, and performs the same function as the ROM 4. The CPU 2 reads various data and various programs stored in the external storage device 9 into the RAM 3 and executes processing, thereby performing the same operation as when the ROM 4 stores various data and various programs. be able to. Therefore, it is possible to easily add various programs or upgrade versions by replacing the external storage device 9.

The communication I / F 10 is wired or wirelessly connected to a network 16 such as the Internet or a commercial network.
It is connected to the server computer 18 and another PC 19 that function as a sound material data providing site via the. As a result, it becomes possible to download the sound material data from the server computer 18 and also download various data and various programs from the other PC 19.

The MIDI I / F5 is a dedicated MIDI
Not limited to those using I / F, RS232-C, USB
Or MIDI using general-purpose I / F such as IEEE1394
-The I / F may be configured. In this case, MI
Data other than DI data may be transmitted and received at the same time.

The hardware configuration of the server computer 18 that provides the sound material data to the PC 1 is the same as the hardware configuration of the PC 1 described above. However, since the server computer 18 is mainly intended to provide the sound material data to each user terminal such as the PC 1, the MIDI / I
/ F5, the tone generator circuit 8, the electronic musical instrument 12, and the sound system 15 may not be provided.

FIG. 2 is a block diagram showing the functional arrangement of the PC 1 and the server computer 18 shown in FIG.

The PC 1 includes a video editing section 21 for editing and reproducing video data, a sound material processing / reproducing section 22 for processing and reproducing sound material data, and a software sound source section 23 for converting the data format of the sound material data. The server computer 18 is provided with a sound material database 25 having a large number of sound material data, and a search engine 24 for searching and extracting user-desired sound material data from the sound material database 25 according to an instruction from the PC 1. There is.

Video editing unit 21, sound material processing / playback unit 22
The software sound source unit 23 is a function realized by executing various programs included in the PC 1 by the hardware of the PC 1, and the sound material database 25 is a set of sound material data, and these data are searched by the search engine 2
4 is systematically configured (for example, for each music category) so that a search program for sound material data included in the server computer 18 can be executed by the hardware of the server computer 18. It is a function that is executed and realized.

The video editing section 21 is, specifically, the CPU 2
Reads the video editing program stored in the ROM 4 or the external storage device 9 into the RAM 3, and executes the ROM according to the instruction of the video editing program read into the RAM 3.
4 or the video data stored in the external storage device 9 is edited, and the display device 14 displays the video data before or after editing received via the display circuit 7.

The sound material processing / playback unit 22 is specifically,
The CPU 2 reads the sound material processing / playback program stored in the ROM 4 or the external storage device 9 into the RAM 3, and receives the sound received from the sound material database 25 according to the instruction of the sound material processing / playback program read into the RAM 3. The sound system 15 processes or reproduces the material data and visually displays the sound material data before or after the processing, which is received by the display device 14 via the display circuit 7.
Is realized by producing sound material data before or after processing received via the tone generator circuit 8.

The software tone generator 23 is specifically
The CPU 2 reads the musical tone waveform generation / data format conversion processing program stored in the ROM 4 or the external storage device 9 into the RAM 3, and according to the instruction of the musical tone waveform generation / data format conversion processing program read into the RAM 3, the sound material It is realized by generating a musical tone waveform or converting the data format based on the sound material data received from the database 25.

The search engine 24 is, specifically, a CPU
2 receives the search condition from the PC 1 via the communication I / F 10, reads the search program stored in the ROM 4 or the external storage device 9 into the RAM 3, and incorporates the received search condition into the search program. This is realized by executing a search process, extracting sound material data corresponding to the search condition from the sound material database 25, and transmitting the extracted sound material data to the PC 1 via the communication I / F 10.

The function of the system composed of the PC 1 and the server computer 18 configured as described above will be described below.

FIG. 3 shows a video editing unit 21, a sound material processing / playback unit 22, and a server computer 18 provided in the PC 1.
That is, it is a flowchart showing the main processing executed at the sound material providing site. The processing of the sound material providing site is mainly executed by the search engine 24.

First, when the user selects the video data to which the sound material data is added via the operator 13, the video editing unit 21 determines the video data to be edited (step S30).
1) The video time data and the title data, which are the characteristic data of the video data, are acquired and supplied to the sound material processing / playback unit 22 (step S302), and the trial listening process is executed (step S307).

On the other hand, the sound material processing / playback unit 22 receives the characteristic data, that is, the video time data and the title data from the video editing unit 21, and uses the video time data and the title data as search conditions. Access the providing site (step S303).

The sound material providing site receives the video time data and the title data from the sound material processing / playback unit 22 as search conditions, searches the sound material database 25 based on the search conditions, and matches the search conditions. The sound material data is extracted (step S304). A known technique is used as a method for extracting the sound material data.

Steps S303 and S30 described above
As shown in FIG. 4, the sound material processing / playback unit 22 automatically acquires the video feature data including the playback time of the video data and supplies it to the sound material providing site. You don't have to know the playback time,
The sound material data to be added to the video data can be efficiently acquired from the sound material providing site, and the work efficiency of video editing can be improved. The sound material processing / reproducing unit 22 may supply one of the reproduction time of the video data and the title to the sound material providing site.

Next, at the sound material providing site, step S
A list of the sound material data extracted in 304 is created, and the created sound material data list is processed as a sound material.
It is transmitted to the reproducing unit 22 (step S305), and the trial listening process is executed (step S307).

The sound material processing / playback unit 22 displays a list of sound material data received from the sound material providing site (step S306) and executes a trial listening process (step S30).
7).

In the trial listening process in step S307, the sound material processing / playback unit 22 instructs the sound material providing site to stream the sound material data, and the sound material providing site receives the sound material data and receives the sound material data. The sound material processing / playback unit 22 executes the stream playback of the sound material data delivered from the sound material providing site,
The video editing unit 21 reproduces the video data in synchronization with the stream reproduction, and the user listens to the sound material data while confirming the video data. A detailed description of the trial listening process will be given later with reference to FIG. Here, “stream distribution” means that the sound material providing site is the sound material processing / playback unit 22.
Is a technique for distributing data so that it can be stream-reproduced without downloading the sound material data file. Stream reproduction means sound material data distributed by the sound material processing / reproducing unit 22 from the sound material providing site. It is a technique of playing in real time without downloading.

In the trial listening process of step S307, when the user confirms the desired sound material data, the user designates the sound material data via the operator 13, and the sound material processing / playback unit 22 receives this designation and outputs the sound. Instruct the material providing site to download the sound material data (step S30)
8).

The sound material providing site receives the download instruction of the sound material data from the sound material processing / playback unit 22,
The sound material data is distributed to the sound material processing / playback unit 22 (step S309), and a series of processing at the sound material providing site is ended.

The sound material processing / playback unit 22 receives the sound material data distributed from the sound material providing site, gives the received sound material data an arbitrary file name, and saves it by designating an arbitrary storage location. Then, the processing / preview processing is executed (step S310) (step S311).

After executing the trial listening process in step S307, the video editing section 21 executes the processing / preview process (step S311).

In the processing / preview processing of step S311, the video editing section 21 reproduces the video data in the preview mode, and the sound material processing / reproducing section 22 reproduces the video data before or during the reproduction, if necessary. Step S31
The sound material data saved as 0 is processed, the sound material data processed as necessary is reproduced in synchronization with the reproduction of the video data, and the processing parameters used for processing the sound material data are set in the sound material data. To record. A detailed description of the processing / preview processing will be described later with reference to FIG.

After the processing / preview processing in step S311, the sound material processing / playback unit 22 executes MIDI to audio conversion processing (step S312).

In the MIDI-to-audio conversion processing, the sound material data (that is, MI) in which the processing parameters are recorded by the sound material processing / playback unit 22 and the software sound source unit 23.
The DI data) is converted into an audio format that can be added to the video data by the video editing unit 21. A detailed description of the MIDI to audio conversion process will be given later with reference to FIG.

The sound material processing / playback unit 22 saves the sound material data converted into the audio format in an arbitrary file and notifies the video editing unit 21 of that fact (step S313), and the sound material processing / playback is performed. A series of processing in the unit 22 is completed.

The video editing unit 21 reads the file saved in step S313, adds the read file, that is, the sound material data converted into the audio format to the video data (step S314), and the video editing unit A series of processing in 21 is ended.

In the above-mentioned main processing, the sound material processing / playback unit 22 directly executes the display and download instruction of the list of sound material data, but the processing for executing the display function and instruction output function such as a web browser separately. A section,
You may make it perform the display of the list of sound material data, and the instruction | indication of download to this.

Further, the sound material processing / playback unit 22 saves the sound material data converted into the audio format in an arbitrary file (step S313), and the video editing unit 2
Although 1 has read the saved file, the sound material processing / playback unit 22 converts the sound material data into the audio format of the MIDI material by MIDI to audio conversion processing (step S312), and then converts the sound material data into the audio format. May be directly transmitted to the video editing unit 21 without being saved in a file.

FIG. 4 is a flowchart showing the trial listening process of step S307 in FIG.

First, the video editing section 21 sets the position to which the sound material data is added by the user designating the position to which the sound material data is to be added in the video data through the operation element 13 (step S401). , The listening mode is set (step S402).

Next, the video editing section 21 accepts a reproduction instruction of the video data input via the operator 13 and notifies the sound material processing / reproducing section 22 of the instruction (step S403). The processing / reproduction unit 22 is
In response to the instruction from the video editing unit 21 in step S403, the sound material data specified in advance from the sound material data list displayed in step S306 in FIG. A stream distribution is instructed to the material providing site (step S408).

The sound material providing site receives the stream distribution instruction regarding the sound material data from the sound material processing / playback unit 22 in step S408, and executes the stream distribution of the sound material data corresponding to the instruction (step S41).
5).

When the stream distribution is executed by the sound material providing site, the sound material processing / playback unit 22 receives the sound material data from the sound material providing site and determines whether the stream can be played, that is, for a predetermined time. It is determined whether or not the sound material data has been accumulated (step S409). This determination is made in consideration of stream reproduction, that is, for the purpose of the sound material processing / reproducing unit 22 accumulating sound material data that does not interrupt the reproduced sound. The predetermined time is, for example, 5 seconds, but since the stream reproduction is influenced by the reproduction processing speed of the sound material data in the sound material processing / reproducing unit 22 and the communication speed of the network such as the Internet, the predetermined time is It is set by the sound material processing / playback unit 22. Further, since the stream reproduction is performed while receiving the subsequent data, the accumulation of the sound material data for the predetermined time does not mean that all the sound material data is accumulated.

As a result of the determination in step S409, when the sound material data for a predetermined time has been accumulated, the sound material processing / playback unit 22 instructs the video editing unit 21 to start playing the video data (step S410). ), The accumulated sound material data for a predetermined time and the sound material data continuously distributed from the sound material providing site are stream-reproduced so that the sound is not interrupted (step S411).

On the other hand, the video editing section 21 determines in step S41.
Receiving a reproduction start instruction from the sound material processing / reproducing unit 22 of 0, the reproduction of the video data is executed (step S404),
Whether or not to continue the reproduction of the video data is determined based on the reproduction stop instruction of the video data input via the operator 13 or the finite time length of the video data (step S4).
05).

As a result of the determination in step S409, when the video editing unit 21 continues the reproduction of the video data, the process returns to step S404 and the reproduction of the video data is continued, while the video editing unit 21 continues the reproduction of the video data. If not, the reproduction of the video data is stopped (step S40).
6).

Next, the video editing section 21 processes the sound material.
The reproduction unit 22 is instructed to stop the reproduction of the sound material data (step S407), the series of processes in the video editing unit 21 is ended, and the process returns to the main process of FIG.

After starting the reproduction of the sound material data stream, the sound material processing / reproducing unit 22 determines whether or not the reproduction stop instruction of the sound material data transmitted from the video editing unit 21 is received in step S407 (step S407). S412).

As a result of the determination in step S412, when the reproduction stop instruction of the sound material data is not received, the process returns to step S411, the stream reproduction of the sound material data is executed, and the reproduction of the sound material data is stopped. When the instruction is received, the stream reproduction of the sound material data is stopped (step S413).

Next, the sound material processing / playback unit 22 instructs the sound material providing site to stop the stream distribution of the sound material data (step S414), and ends the series of processing in the sound material processing / playback unit 22. Then, the process returns to the main process of FIG.

After starting the stream distribution of the sound material data, the sound material providing site receives the stream distribution stop instruction transmitted from the sound material processing / playback unit 22, and stops the stream distribution of the sound material data (step (S416), the series of processes in the sound material providing site is ended, and the process returns to the main process of FIG. In step S412, it is determined whether or not the sound material processing / playback unit 22 has received a playback stop instruction of the sound material data from the video editing unit 21. To determine whether the stop of the video data reproduction is detected. As a result, if the stop of the video data reproduction is not detected, the process returns to step S411 to detect the stop of the video data reproduction. If so, the process may proceed to step S413.

In the above-described trial listening process, the sound material processing / playback unit 22 instructs the sound material providing site to stream the sound material data, and the sound material providing site receives the instruction to stream the sound material data. Stream distribution is performed, the sound material processing / playback unit 22 performs stream playback of the sound material data delivered from the sound material providing site, and the video editing unit 21 plays back the video data in synchronization with this stream playback. Run. Therefore, the user should confirm how the sound material data (for example, BGM) is added to the video data by stream reproduction before downloading the sound material data to add the video data. As a result, it is possible to improve the work efficiency of video editing (the file name specified when downloading and the destination to save are not necessary for stream playback, and the file when you do not like the result of the audition) (Erase work is not required.) Especially, since it is not the simple sound material data stream reproduction but the sound material data stream reproduction synchronized with the reproduction of the video data, the sound data The effect is that the relationship is very easy to understand.

Further, the sound material processing / reproducing unit 22 receives the reproduction instruction of the video data from the video editing unit 21, and instructs the sound material providing site to stream a predetermined sound material data to provide the sound material. When the sound material data stream-distributed from the site becomes stream-reproducible, the reproduction start permission of the video data is transmitted to the video editing unit 21 and the sound material data is stream-reproduced.
It is possible to surely reproduce the sound material data and the video data in synchronization with each other without starting the reproduction of the video data first.

Further, the sound material processing / reproducing unit 22 receives the instruction to stop the reproduction of the sound material data, stops the stream reproduction of the sound material data, and further performs the stream distribution of the sound material data to the sound material providing site. A stop is instructed, or the sound material processing / playback unit 22 accesses the video editing unit 21 to detect the stop of the playback of the video data, and the sound material data stream to the sound material providing site according to the detection result. Since the instruction to stop the distribution is given, the user does not need to issue each stop instruction to the video editing unit 21, the sound material processing / playback unit 22, and the sound material providing site.

In the trial listening process described above, when the sound material processing / playback unit 22 receives the playback stop instruction of the sound material data transmitted from the video editing unit 21, the stream playback of the sound material data is stopped, If the time length of the material data is shorter than the playback time of the video data in the video editing unit 21, the sound material providing site will output from the sound material processing / playback unit 22 when the stream distribution of the sound material data is completed. The stream distribution of the sound material data may be automatically stopped without receiving the stream distribution stop instruction. In the case of data with a small file size such as MIDI data, all the data is received and accumulated in the cache memory, then reproduction is started, and when the reproduction is finished, pseudo stream reproduction is performed which is erased from the cache memory. However, even in this case, since the file download is not necessary, the same effect as described above can be obtained.

FIG. 5 is a flowchart showing the processing / preview processing of step S311 in FIG.

First, the video editing unit 21 sets the preview mode in the video editing program in accordance with the user's instruction input via the operator 13 (step S5).
01).

Next, the video editing section 21 accepts a reproduction instruction of the video data input through the operator 13 and also indicates that the reproduction instruction of the video data has been accepted.
The data is transmitted to the reproduction unit 22 (step S502).

The video editing unit 21 reproduces the video data (step S503), and reproduces the video data based on the reproduction stop instruction of the video data input through the operator 13 or the finite time length of the video data. It is determined whether or not to continue (step S504).

As a result of the determination in step S504, if the video editing unit 21 continues the reproduction of the video data, the process returns to step S503 to continue the reproduction of the video data, while the video editing unit 21 continues the reproduction of the video data. If not, the reproduction of the video data is stopped (step S50).
5).

Next, the video editing section 21 processes the sound material
The reproduction unit 22 is instructed to stop the reproduction of the sound material data (step S506), the series of processes in the video editing unit 21 is ended, and the process returns to the main process of FIG.

On the other hand, the sound material processing / playback unit 22 first
According to the processing instruction of the sound material data input via the operator 13, the processing of the sound material data saved in the file in step S310 of FIG. 3 is performed (step S50).
7). Specifically, this processing is for adjusting the parameters of the sound material data, and these parameters (hereinafter referred to as “processing parameters”) include, for example, part on / off, tone change, mixer, effect addition, tempo. It is composed of parameters that can easily modify the musical content of the sound material data such as change or pitch change. In this processing, M like a MIDI sequencer
Unlike advanced corrections such as correcting IDI events one by one, even beginners can easily process sound material data. If no processing instruction is input via the operator 13, the process of step S507 is canceled.

Next, the sound material processing / playback unit 22 receives the fact that the playback instruction of the video data transmitted from the video editing unit 21 is accepted, and outputs the sound material data that has been processed as necessary. Play back (step S508). After that, during the reproduction of the sound material data, the processing of the sound material data being reproduced is performed, if necessary, as in step S507 (step S509).

Next, the sound material processing / playback unit 22 determines whether or not a sound material data playback stop instruction is received from the video editing unit 21 (step S510), and the sound material data playback stop instruction is received. If not, step S50
Returning to step 8, while continuing to reproduce the sound material data, when the reproduction stop instruction of the sound material data is received, the reproduction of the sound material data is stopped (step S511). In step S510, the sound material processing / playback unit 22 determines whether or not the playback stop instruction of the sound material data is received from the video editing unit 21. Access is made to determine whether or not the reproduction stop of the video data is detected. As a result, if the reproduction stop of the video data is not detected, the process returns to step S508 while the reproduction stop of the video data is detected. In this case, the process may proceed to step S511.

Finally, in step S507 or step S509, the sound material processing / playback unit 22 records the processing parameter adjusted as necessary in the sound material data (step S512), and the series of processing ends. Then, the process returns to the main process of FIG.

In the above-described processing / preview processing, the video editing section 21 reproduces the video data in the preview mode, and the sound material processing / reproducing section 22 reproduces the audio data before or during the reproduction of the video data, if necessary. The material data is processed, the sound material data processed as necessary is reproduced in synchronization with the reproduction of the video data, and the processing parameters used for processing the sound material data are recorded in the sound material data. Therefore, the user can process the sound material data in real time while reproducing the video data, and can process the processed sound material data without waiting for the completion of the creation of the video data to which the sound material data is added. You can check the synchronized playback with the video data. Therefore, the user can confirm how the sound material data (for example, BGM) is added to the video data, so that the work efficiency of video editing can be improved. Also, the sound material processing / playback unit 2
In No. 2, since the processing can be performed during the reproduction of the sound material data, the work efficiency of video editing can be improved. Further, the sound material data before and after the processing may be compared and auditioned.

In the above-described processing / preview processing, when the sound material processing / playback unit 22 receives the reproduction stop instruction of the sound material data transmitted from the video editing unit 21, the reproduction of the sound material data is stopped. When the time length of the sound material data is shorter than the reproduction time of the video data in the video editing section 21, the sound material processing / reproducing section 22 receives the reproduction stop instruction of the sound material data transmitted from the video editing section 21. Before playing, the reproduction of the sound material data may be automatically stopped.

FIG. 6 is a flow chart showing the MIDI to audio conversion process of step S312 in FIG.

First, the sound material processing / playback unit 22 reads out the sound material data in which the processing parameters are recorded, that is, this MIDI data at a high speed while maintaining the relationship of the relative time of each event, and reads the MIDI data for each event by software. It is transmitted to the sound source unit 23 (step S601).

The software tone generator 23 generates audio waveform data based on the received MIDI data for each event, temporarily stores the generated audio waveform data in the buffer (RAM3), and stores it for a predetermined time (for example, 0 Each time the audio waveform data of 0.5 seconds) is stored, it is transmitted to the sound material processing / playback unit 22 (step S602). The buffer for temporarily storing the audio waveform data stores the audio waveform data sequentially generated in the vacant area by transmitting the audio waveform data to the sound material processing / playback unit 22.

The sound material processing / playback unit 22 sequentially internally records the audio waveform data for a predetermined time received from the software sound source unit 23 (step S603). Here, recording internally means that the software tone generator 23
It means that the audio waveform data received from the device is sounded, and that the sound that is emitted is not recorded but is recorded without sound.

Next, the sound material processing / playback unit 22 determines whether or not all the MIDI data to be converted have been read (step S604). If all the MIDI data have not been read, Is step S6
Returning to 01, while reading the MIDI data is continued, if the reading of all the MIDI data is completed, the audio waveform data internally recorded in step S603 can be handled by the video editing unit 21. It is possible to convert to a preset audio format (step S605), terminate the series of processes, and return to the main process of FIG.

In the MIDI-to-audio conversion process described above, audio material data (ie MIDI data) in which processing parameters are recorded by the sound material processing / playback unit 22 and the software sound source unit 23 can be added to the video data by the video editing unit 21. As it is converted to a format, the sound material processing / playback unit 22 and software sound source unit 2
In 3, the processing of the sound material data to the conversion of the sound material data into the audio format can be executed all at once, and the work efficiency can be improved. Also, P
In addition to C1, it is not necessary to separately provide a conversion device for converting the audio format of the sound material data.

The sound material data provided by the above-mentioned sound material providing site is not limited to MIDI data, but may be audio format data. In that case, the MIDI to audio conversion process is unnecessary. Further, regarding the processing of the audio format data, it is possible to perform the effect addition, the tempo change or the pitch change processing, although the processing of the part on / off and the tone color change or the processing of the mixer cannot be performed.

When the sound material data is auditioned (when the sound material data is stream-reproduced or when the sound material data is processed and then the video data is previewed), the video reproduction section 21 starts the video reproduction. Alternatively, it may be performed in response to an instruction from the sound material processing / playback unit 22, although it is performed in response to an instruction to stop. That is, when the sound material data is stream-reproduced, when the sound material processing / reproducing unit 22 instructs the stream reproduction, the sound material providing site is instructed to stream the sound material, and the streamed sound material data is transmitted. When it becomes possible to reproduce, the video editing unit 21 is instructed to reproduce the video data. Further, when the sound material processing / playback unit 22 is instructed to stop the stream reproduction, the sound material providing site is instructed to stop the stream distribution and the video editing unit 21 is instructed to stop the reproduction of the video data. Further, when the sound material data is processed and then the video data is previewed, when the preview processing is instructed by the sound material processing / reproducing unit 22, the reproduction of the sound material data is started and the video editing unit 21 is started. Is instructed to reproduce the video data, and the sound material processing / reproducing unit 22 is instructed to stop the preview processing, the reproduction of the sound material data is stopped and the video editing unit 21 is instructed to stop the reproduction of the video data. It

In the above-described processing of the sound material data, the parameters of the sound material data part on / off, tone color change, mixer, effect addition, tempo change and pitch change are adjusted. Not all of them may be provided, and at least one may be provided. Further, there may be processing types other than these. Regarding “tempo change”, which is one of the parameters, the tempo may be automatically changed so that the video reproduction time and the time length of the sound material data match. MI above
The DIto audio conversion process is performed by the sound material processing / playback unit 2
2 and the software sound source unit 23 cooperate with each other, but this function may be realized by the software sound source unit 23 alone.

The video editing unit 21 and the sound material processing / playback unit 22 are realized by executing different programs by the CPU 2, but they are realized by executing one program by the CPU 2. Good.

At the sound material site, the charging processing may be executed when the sound material data is downloaded, and the charging processing may not be executed when the sound material data is streamed.

It is needless to say that the functions of the above-described embodiments can be realized also by the form of a program which is software, and thereby the object of the present invention can be achieved.

Furthermore, a storage medium recording a program code of software that realizes the functions of the above-described embodiments is supplied to a system or apparatus, and the computer (or CPU2) of the system or apparatus is stored in the storage medium. Needless to say, the object of the present invention can be achieved by reading and executing the program code.

In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium storing the program code constitutes the present invention.

As a storage medium for supplying the program code, for example, a floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, C
D-ROM, CD-R, DVD-ROM, semiconductor memory, magnetic tape, non-volatile memory card, ROM, etc. can be used. Further, the program code may be supplied from another PC 19 via another electronic musical instrument 12 or the network 16.

[0093]

As described in detail above, according to the sound material audition apparatus of claim 1 and the sound material audition program of claim 6, the sound material providing site is synchronized with the reproduced video data. Since the sound material data stream-distributed from is played in a stream, the user can download the sound material data before adding it to the video data.
By stream reproduction, it is possible to confirm how the sound material data (for example, BGM) is added to the video data, and the work efficiency of video editing can be improved.

According to the sound material audition apparatus of claims 2 and 4, and the sound material audition program of claim 7, the video data reproduction start instruction means instructs the video editing means (or the video reproduction means) to generate the video data. When you instruct to start playing
Since the stream reproduction means starts the stream reproduction of the sound material data stream-distributed from the sound material providing site, it is possible to reliably reproduce the sound material data and the video data in synchronization with each other.

According to the sound material audition apparatus of claims 3 and 5 and the sound material audition program of claim 8, the stop of the reproduction of the video data is detected, and the sound material data is sent to the sound material providing site. Since the stream distribution stop instruction is transmitted, the user does not need to issue a stream distribution stop instruction each time.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic hardware configuration of a sound material listening apparatus according to an embodiment of the present invention.

2 is a block diagram showing a functional configuration of a PC 1 and a server computer 18 in FIG.

FIG. 3 is a video editing unit 21 included in the PC 1 in FIG.
And sound material processing / playback unit 22 and server computer 1
8 is a flowchart showing the main processing executed at the sound material providing site.

FIG. 4 is a flowchart showing a trial listening process of step S307 in FIG.

FIG. 5 is a flowchart showing the processing / preview processing of step S311 in FIG.

FIG. 6 shows the MIDIt of step S312 in FIG.
It is a flow chart which shows o audio conversion processing.

[Explanation of symbols]

1 PC 2 CPU (video editing means, stream distribution instruction means, stream reproduction means, judgment means, video data reproduction start instruction means, reproduction stop instruction reception means, stream reproduction stop means, stream distribution stop instruction transmission means) 3
RAM 4 ROM 5 MIDI I / F 6 Operation circuit 7 Display circuit 8 Sound source circuit 9 External storage device 10 Communication I / F

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] February 20, 2003 (2003.2.2)
0)

[Procedure Amendment 1]

[Document name to be amended] Statement

[Name of item to be amended] Claims

[Correction method] Change

[Correction content]

[Claims]

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0007

[Correction method] Change

[Correction content]

[0007]

In order to achieve the above object, the sound material listening apparatus according to claim 1 is a sound material listening apparatus connected to a sound material providing site for providing sound material data. and image reproducing means you play, to the sound materials providing site, and streamed instruction means for instructing the streaming sound material data reproduced video data and synchronization to be reproduced by the image reproducing means, the sound materials Stream reproduction means for performing stream reproduction of the sound material data stream-distributed from the providing site, wherein the sound material data is stream-reproduced in synchronization with the video data reproduced by the video reproduction means. And

[Procedure 3]

[Document name to be amended] Statement

[Correction target item name] 0008

[Correction method] Change

[Correction content]

A sound material listening apparatus according to a second aspect is the sound material listening apparatus according to the first aspect, wherein the stream reproducing means is capable of stream reproducing sound material data stream-distributed from the sound material providing site. And a video data reproduction start instruction means for instructing the video reproduction means to start reproduction of the video data when it is judged that the stream reproduction is possible. When the video data reproduction start instruction means instructs the video reproduction means to start reproduction of the video data, the stream reproduction of the sound material data streamed from the sound material providing site is started. It is characterized by

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name] 0009

[Correction method] Change

[Correction content]

[0009] sound material listen device according to claim 3, in sound material listen device according to claim 1 or 2, wherein the stream playback device, the playback stop detection means for detecting a reproduction stop of the video data in the image reproducing means And a stream distribution stop instruction transmitting means for transmitting an instruction to stop the stream distribution of the sound material data to the sound material providing site in response to the stop of the reproduction of the video data.

[Procedure Amendment 5]

[Document name to be amended] Statement

[Correction target item name] 0010

[Correction method] Change

[Correction content]

[0010] sound material listen device according to claim 4, which is connected to the tone material providing site for providing the sound material data, in sound material listen apparatus connected to a video reproduction device for reproducing video data, the sound Stream distribution instruction means for instructing the material providing site to stream the sound material data, and judging means for judging whether or not the stream reproduction of the sound material data streamed from the sound material providing site is possible. And a video data reproduction start instruction means for instructing the video reproduction means to start reproduction of video data when the judgment means determines that stream reproduction is possible, and the video data reproduction start instruction means. The sound streamed from the sound material providing site when the video reproducing means is instructed to start reproducing the video data. Characterized in that it comprises a stream reproduction means for starting the stream reproduction of wood data.

[Procedure correction 6]

[Document name to be amended] Statement

[Correction target item name] 0011

[Correction method] Change

[Correction content]

[0011] sound material listen device according to claim 5 is connected to the tone material providing site for providing the sound material data, in sound material listen apparatus connected to a video reproduction device for reproducing video data, the sound materials provided Stream reproduction means for reproducing the sound material data stream-distributed from the site, reproduction stop detection means for detecting reproduction stop of the video data in the video reproduction means, and provision of the sound material in response to the reproduction stop of the video data. Stream distribution stop instruction transmitting means for transmitting an instruction to stop the stream distribution of the sound material data to the site.

[Procedure Amendment 7]

[Document name to be amended] Statement

[Correction target item name] 0094

[Correction method] Change

[Correction content]

According to the sound material audition device of claims 2 and 4, and the sound material audition program of claim 7, the video data reproduction start instruction means instructs the video reproduction means to start reproduction of the video data. At this time, the stream reproduction means starts the stream reproduction of the sound material data stream-distributed from the sound material providing site, so that the sound material data and the video data can be surely reproduced in synchronization with each other.

[Procedure Amendment 8]

[Document name to be amended] Statement

[Correction target item name] Explanation of code

[Correction method] Change

[Correction content]

[Description of Codes] 1 PC 2 CPU (video reproduction means, stream distribution instruction means, stream reproduction means, determination means, video data reproduction start instruction means, reproduction stop instruction reception means, stream reproduction stop means, stream distribution stop instruction transmission means ) 3
RAM 4 ROM 5 MIDI I / F 6 Operation circuit 7 Display circuit 8 Sound source circuit 9 External storage device 10 Communication I / F

[Procedure Amendment 9]

[Document name to be corrected] Drawing

[Name of item to be corrected] Fig. 4

[Correction method] Change

[Correction content]

[Figure 4]

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Toru Itayama             Yamaha stock, 10-1 Nakazawa-machi, Hamamatsu-shi, Shizuoka             In the company F-term (reference) 5C053 FA14 GB05 HA30 JA03 JA22                       KA01 KA24 LA11 LA14                 5D378 QQ02 QQ05 QQ30 QQ34

Claims (8)

[Claims] 1. A sound material auditioning apparatus connected to a sound material providing site for providing sound material data, comprising: video editing means for playing back video data and editing the video data; and for the sound material providing site. A stream distribution instructing means for instructing stream distribution of sound material data to be reproduced in synchronization with the video data reproduced by the video editing means; and a stream reproducing means for stream reproducing the sound material data streamed from the sound material providing site. A sound material audition apparatus, comprising: a stream material for reproducing the sound material data in synchronization with the video data reproduced by the video editing means. 2. The stream reproduction means is a judgment means for judging whether or not the sound material data stream-distributed from the sound material providing site can be reproduced in a stream, and the judgment means can reproduce the stream. And a video data reproduction start instructing means for instructing the video editing means to start reproduction of the video data, the video data reproduction start instructing means for the video editing means. 2. The sound material listening apparatus according to claim 1, wherein when the reproduction start of the video data is instructed, the stream reproduction of the sound material data stream-distributed from the sound material providing site is started. 3. The stream reproduction means detects reproduction stop of the video data in the video editing means, and the sound material for the sound material providing site according to the stop of reproduction of the video data. 3. The sound material listening apparatus according to claim 1, further comprising a stream distribution stop instruction transmitting unit that transmits an instruction to stop data stream distribution. 4. A sound material listening apparatus which is connected to a sound material providing site for providing sound material data and which operates in cooperation with a video reproducing means for reproducing video data, comprising: Stream distribution instructing means for instructing stream distribution of the sound material data, judging means for judging whether or not stream reproduction of the sound material data streamed from the sound material providing site is possible, and the judging means. When it is determined that stream reproduction is possible, a video data reproduction start instruction means for instructing the video reproduction means to start reproduction of video data, and the video data reproduction start instruction means for instructing the video reproduction means. Stream of sound material data streamed from the sound material providing site when instructing to start reproduction of the video data. Sound material listen apparatus; and a stream reproduction means for starting raw. 5. A sound material preview device, which is connected to a sound material providing site for providing sound material data and operates in cooperation with video reproducing means for reproducing video data, is streamed from the sound material providing site. Stream reproduction means for reproducing the sound material data in a stream, reproduction stop detection means for detecting reproduction stop of the video data in the video reproduction means, and the sound for the sound material providing site according to the reproduction stop of the video data. A sound material auditioning apparatus, comprising: a stream distribution stop instruction transmitting means for transmitting an instruction to stop stream distribution of material data. 6. A video editing procedure for reproducing video data and editing the video data on a computer connected to a sound material providing site for providing the sound material data, the video editing for the sound material providing site A stream distribution instruction procedure for instructing stream distribution of sound material data to be reproduced in synchronization with video data reproduced in a procedure, and a stream reproduction procedure for stream reproduction of sound material data stream-distributed from the sound material providing site, A program for previewing a sound material, characterized in that a program for streaming and reproducing the sound material data is executed in synchronization with the video data reproduced in the video editing procedure. 7. A computer, which is connected to a sound material providing site that provides sound material data and executes a video playback procedure for playing video data, instructs a stream distribution of the sound material data to the sound material providing site. A stream distribution instruction procedure, a determination procedure for determining whether or not the stream reproduction of the sound material data streamed from the sound material providing site is possible, and a case in which the stream reproduction is determined by the determination procedure When a video data reproduction start instruction procedure for instructing the video reproduction procedure to start reproduction of video data, and an instruction to start reproduction of the video data for the video reproduction procedure in the video data reproduction start instruction procedure, In addition, the stream reproduction that starts the stream reproduction of the sound material data stream-distributed from the sound material providing site. A program for listening to sound material, which is characterized by executing procedures. 8. The sound material data stream-distributed from the sound material providing site is stream-reproduced to a computer that is connected to a sound material providing site that provides sound material data and executes a video reproducing procedure for reproducing the video data. Stream reproduction procedure, reproduction stop detection procedure for detecting reproduction stop of video data in the video reproduction procedure, and stop of stream distribution of the sound material data to the sound material providing site in response to stop of reproduction of the sound material data A sound material preview program for executing a stream distribution stop instruction transmission procedure for transmitting an instruction.