JP2002073040A - Contents editing device, method for editing contents, program storing medium, and contents editing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable correct editing by correcting the time lag of contents data requiring real time characteristics among a plurality of terminals. SOLUTION: A time stamp is added to a first contents data DrPD of a packet unit, and they are sequentially outputted. A second contents data GiPD of a packet unit to which the same time stamp as this time stamp are added, correspondent to the first contents data DrPD, are inputted from the outside. Time synchronization between the first contents data DrPD of a packet unit and the second contents data GiPD of a packet unit can be reliably assured with the time stamp by adjusting so that the first contents data DrPD and the second contents data GiPD are synchronized in terms of time on the basis of the time stamp. Thus, correct editing can be performed by correcting the time lag between the first and second contents data requiring real time characteristics.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a content editing apparatus, a content editing method, a program storage medium, and a content editing system, and for example, guarantees synchronization of respective content data transmitted from each remote location via the Internet. Accordingly, the present invention is suitable for application to a content editing apparatus, a content editing method, a program storage medium, and a content editing system for editing content such as music data that requires real-time properties.

[0002]

2. Description of the Related Art Conventionally, personal computers have been designed so that editing processing of various contents data such as music data and video data can be easily executed with the improvement of processing capability. Processing requirements are fully realized.

[0003] Further, the personal computer can execute digital signal processing portions such as a mixer and a recorder required for recording, for example, by software by improving the processing capability, so that editing of content data can be easily performed. It has been made executable.

Accordingly, the personal computer fetches various audio data such as guitar audio data, bass audio data, piano audio data and drum audio data through a plurality of microphones connected to the personal computer.
These are temporarily recorded on a hard disk by a recorder constituted by built-in software, adjusted to a desired sound quality by a mixer, and then synthesized to produce 2 channels.
(R channel and L channel) recording data can be generated.

[0005]

In a personal computer having such a configuration, for example, when it is necessary to perform mixing adjustment by simultaneously using a plurality of types of mixer functions, the processing capacity of a CPU (Central Processing Unit) is limited. While the mixing adjustment is being performed, other recorders and effectors cannot be operated, and there has been a problem that recording data having desired sound quality cannot be generated.

In a personal computer,
For example, audio data is transferred between the personal computer installed in a studio in Tokyo and other personal computers installed in New York, London, and Paris abroad, and finally, a 2 channel recording is performed in a studio in Tokyo. In the case of generating and recording data, first, drum audio data recorded in advance is reproduced and transmitted from a studio in Tokyo to other personal computers installed in New York, London and Paris, respectively.

Another personal computer installed in New York reproduces the drum performance sound based on the audio data of the drum transmitted from the personal computer in Tokyo, outputs the sound from the speaker, and matches it with the performance sound of the drum. The audio data of the played guitar is captured via a microphone and sent back to a personal computer installed in a studio in Tokyo via the Internet.

Similarly, other personal computers installed in London and Paris also reproduce the drum performance sound based on the audio data of the drum transmitted from the personal computer in Tokyo and output it from the speaker, and output the sound of the drum. The piano and bass audio data played according to the performance sound are captured via microphones, respectively, and sent back to a personal computer installed in a studio in Tokyo via the Internet.

A personal computer installed in a studio in Tokyo takes in guitar, piano and bass audio data sent back via the Internet from other personal computers installed in New York, London and Paris, respectively.
At this time, these audio data have a round-trip transmission path delay and a delay corresponding to a signal processing time in another personal computer, and these delay times are in an unpredictable state.

Therefore, a personal computer in Tokyo is
There is a transmission path delay between the drum performance sound obtained by playing the drum audio data and the guitar, piano and bass performance sounds returned from other personal computers in New York, London and Paris. In this case, there is a problem that it is difficult to generate music recording data that requires real-time performance.

The present invention has been made in consideration of the above points, and has been made in consideration of the above-described problems, and has been made in consideration of the above-described circumstances, and has been made in consideration of the above-described circumstances. A content editing method, a program storage medium, and a content editing system are proposed.

[0012]

According to the present invention, in order to solve the above-mentioned problems, a time stamp is added to the first content data in packet units and the data is sequentially output, and the first content data having the same time stamp as the time stamp is added. The second content data in packet units corresponding to one content data is input from the outside, and the first content data and the second content data are adjusted to be temporally synchronized based on the time stamp. , The time synchronization between the first content data and the second content data can be reliably guaranteed by the time stamp, so that the time lag between the first and second content data requiring real-time performance can be corrected. And can be edited accurately.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the drawings.

(1) Overall Configuration of Remote Location Recording System In FIG. 1, reference numeral 1 denotes a remote location recording system as a whole. For example, a personal computer 3 located in Tokyo is installed in a remote location New York. Connected personal computers 4 are connected to each other via the Internet 2.

In this case, the personal computer 3 in Tokyo and the personal computer 4 in New York mutually exchange TCP / IP (Transmis
Data communication is performed using a protocol such as sionControl Protocol / Internet Protocol), and content data such as audio data and video data that require real-time characteristics can be transmitted and received in packet units.

The remote location recording system 1 will be described below by taking as an example a case where a recording and editing process relating to audio data is executed between a personal computer 3 in Tokyo and a personal computer 4 in New York in a music production environment.

The personal computer 3 in Tokyo stores, for example, drum audio data and piano audio data in a built-in hard disk (not shown) in advance, reads out the drum and piano audio data, and To the personal computer 4 in New York.

New York personal computer 4
Generates a drum playback sound and a piano playback sound based on the drum and piano audio data transmitted from the personal computer 3 in Tokyo via the Internet 2 and outputs them from the speaker 4C.

At this time, the personal computer 4 in New York fetches the audio signal of the guitar played by the player (not shown) in accordance with the reproduced sound of the drum and the piano through the microphones 4A and 4B, and then outputs the digital audio signal. The data is converted to data and transmitted to the personal computer 3 in Tokyo via the Internet 2.

The personal computer 3 in Tokyo automatically stores the audio data of the guitar transmitted from the personal computer 4 in New York in synchronization with the audio data of the drum and piano, and temporarily stores the audio data. CD (Comp
2ch (L channel and R channel) recording data to be recorded on an act disc).

By the way, when the drum and piano audio data actually reaches the personal computer 4 in New York, the personal computer 3 in Tokyo
The delay on the transmission path has already occurred since the transmission of the audio data of the drum and the piano, and the delay on the transmission path even when the audio data of the guitar transmitted from the personal computer 4 reaches the personal computer 3 in Tokyo. In addition, a delay corresponding to the signal processing time in the personal computer 4 has occurred.

Therefore, the personal computer 3 in Tokyo
When the audio data of the guitar is received from the personal computer 4 in New York, the transmission path delay for the round trip and the delay for the signal processing time have already occurred since the audio data of the drum and the piano have been transmitted. By correcting the timing shift due to the delay, it is possible to generate recording data of 2 ch for recording on a CD by executing a recording process simultaneously and in parallel between remote locations. Hereinafter, the remote recording system 1 including the delay timing correction will be described in detail.

(1-1) Circuit Configuration of Personal Computer As shown in FIG. 2, the personal computer 3 controls the personal computer 3 as a whole and edits audio data for each musical instrument to record data for two channels. (Central Processing Unit) 10 that generates
To the hard disk drive (HDD) 11, R
An AM (Random Access Memory) 12, a communication interface 13 for performing data communication with the personal computer 4 in New York via the Internet 2 using the TCP / IP protocol, an external input terminal 14, an external output terminal 15, and a monitor 16 are connected to a bus 17. Connected through.

In this case, for example, the microphones 3A and 3B (FIG. 1) are connected to the external input terminal 14, and the external output terminal 1
For example, a speaker 3C is connected to 5. Therefore, the personal computer 3 can also collect the vocal sound via the microphones 3A and 3B, and record the vocal sound again on the recording data.

The hard disk drive 11 stores various application programs such as an operating system program such as Windows 98 (trademark of Microsoft Corporation) and a recording processing program (to be described later) for executing the remote recording processing according to the present invention. Is stored.

The CPU 10 controls the hard disk drive 1
Various programs stored in the CPU 1 are read out as appropriate, loaded into the RAM 12 and executed to execute various processes, and the results of the processes are displayed on the monitor 16.

The personal computer 4 has the same circuit configuration as that of the personal computer 3, and the added numbers (210 to 217) are different from those of the personal computer 3. Therefore, the description is omitted here.

(2) Remote Location Recording Processing In this case, the remote location recording system 1 (FIG. 1)
In the description below, an example in which the recording process between remote locations is controlled mainly by the personal computer 3 in Tokyo will be described as an example.

Incidentally, a recording system between remote locations 1
In this system, a remote computer can be used to control the recording process between remote locations using a personal computer 4 installed in New York instead of the personal computer 3 in Tokyo.

(2-1) Remote Location Connection Processing First, the personal computer 3 in Tokyo (FIG. 2) is activated in response to the power button being turned on, and then given an instruction to perform remote location recording processing. CPU10
The control reads the recording processing program from the hard disk of the HDD 11 and expands it on the RAM 12.

Here, as shown in FIG. 3, the recording processing program 30 includes a workspace 31, an IO (In)
-Out) Manager 32, DSP (Digital Signal Proce)
ssor) plug-ins 33A-33N and GUI (Graphica
l User Interface) Four types of objects called plug-ins 34A to 34N are hierarchically linked to each other.

Accordingly, the personal computer 3 comprises a workspace 31, an IO manager 32, DSP plug-ins 33A to 33N, and GUI plug-ins 34A to 34N.
Perform predetermined processing, and display the processing results on the monitor 16.

The personal computer 4 in New York also has a recording processing program 230 in the same manner, and has four types of workspaces 231, IO managers 232, DSP plug-ins 233A to 233N, and GUI plug-ins 234A to 234N. Objects are hierarchically linked to each other.

The DSP plug-ins 33A to 33N and 233A to 233N correspond to the number of digital signal processing functions held by the personal computers 3 and 4, respectively, and the GUI plug-ins 34A to 34N and 234A to 234N DSP plug-ins 33A-33
N and objects for adjusting the parameters 233A to 233N, each of which has a one-to-one correspondence.

In this embodiment, for the sake of convenience, the following description will be made as a relationship between the objects, and the workspaces 31 and 231, the IO managers 32 and 232, and the DSP plug-ins 33A to 33A will be described.
N and 233A-233N, GUI plug-in 34A-
34N and 234A to 234N will be mainly described, but actually all the CPUs 10 and 210 execute various processes.

The workspace 31 controls the loading and unloading of the DSP plug-ins 33A to 33N, and the DSP plug-ins 33A to 33N or the DSP plug-ins 33A to 33N and the DSP plug-in held by the personal computer 4 in New York. 233A-233
N and manages the connection with GUI plug-ins 34A-34
N, which controls the loading and unloading of DSP plug-ins 33A-33N and DSP plug-ins 233A-233.
The display states of the connection states of all N DSP plug-ins are controlled.

That is, the workspace 31 stores the plug-in list relating to the currently loaded DSP plug-ins 33A to 33N and the DSP plug-in 33 created based on the result of the inquiry to the IO managers 32 and 232.
A to 33N and connection status data indicating the connection status of the entire DSP plug-ins 233A to 233N and a workspace ID (Identificati) unique to the workspace 31 itself.
on), and the display control described above is performed based on these various data.

The IO manager 32 controls the DSP being loaded.
Each of the plug-ins 33A to 33N has its own plug-in data and an IO manager ID unique to the IO manager 32. The connection management of the DSP plug-ins 33A to 33N and the packet unit processed by the DSP plug-ins 33A to 33N, respectively. Of the DSP plug-ins 33A to 33A for the workspace 31
The connection management state regarding N is notified at predetermined time intervals.

In this case, the plug-in data includes the manufacturer name of the loaded DSP plug-in,
Type identification information indicating a default GUI plug-in or the like associated with the SP plug-in, a plug-in ID allocated to the DSP plug-in itself, a hierarchical level of the DSP plug-in itself (0, 1, 2 in order from the highest level) …
…), An output destination list indicating the output destination of the DSP plug-in,
An input destination list indicating the input destination for the DSP plug-in is included.

The output destination list further includes an output ID corresponding to the output terminal of the DSP plug-in, an output terminal hierarchy level (0, 1, 2,... In order from the top), and connection destination information (hardware ID, plug ID). Input ID corresponding to the input terminal of the DSP plug-in, and an input ID corresponding to the input terminal of the DSP plug-in.
Input terminal hierarchy level (0, 1, 2,.
..) And a valid flag indicating validity or invalidity of the input.

As described above, the IO manager 32 holds the plug-in data, thereby enabling the DSP plug-in 3
3A to 33N can be recognized in detail, and the input / output destination of audio data in packet units is
The SP plug-ins 33A to 33N can be correctly instructed.

The DSP plug-ins 33A to 33N are loaded or unloaded by the workspace 31 and perform predetermined digital signal processing on audio data in packet units supplied from the connected preceding DSP plug-in. The resulting processing data is
In accordance with an instruction from the manager 32, the data is transmitted to a DSP plug-in at a later stage, which is a predetermined destination.

Here, the DSP plug-ins 33A to 33N
Are designed to operate on independent memory spaces, so that even if the DSP plug-in 33A operating on a certain memory space stops functioning, the effect is not affected by the other DSP plug-ins 33B. ３３33N.

The GUI plug-ins 34A to 34N include a list of DSP plug-ins corresponding to the one-to-one correspondence with a lower DSP plug-in, a connection state data list of the lower DSP plug-in, and parameter adjustment data relating to parameters of the DSP plug-in. And GUI
The plug-in has its own GUI plug-in ID, is loaded or unloaded by the workspace 31,
The parameters relating to various functions in the P plug-ins 33A to 33N can be adjusted.

When the DSP plug-ins 33A to 33N are constituted by a plurality of lower DSP plug-ins located in lower layers, the lower GUI plug-ins are loaded or unloaded by the upper GUI plug-ins 34A to 34N. It has been made.

In practice, the DSP plug-ins 33A to 33N
A voice input plug-in as an analog-to-digital conversion processing function for audio signals collected by the microphones 3A and 3B connected via the external input terminal 14, and a speaker 3C connected via the external output terminal 15. A sound output plug-in as a digital-to-analog conversion processing function of audio data for outputting a performance sound via an effector plug-in as an effector function for giving a predetermined sound quality effect to audio data,
There are a recorder plug-in as a recorder function for reproducing and recording audio data, a mixer plug-in as a mixer function for mixing, and the like.

The workspace 31 stores these various DSs.
The plug-in icons (described later) corresponding to the P plug-ins 33A to 33N are displayed on the monitor 16, and by connecting the respective plug-in icons on the screen of the monitor 16 according to the user's designation, By combining various functions of the plurality of DSP plug-ins 33A to 33N and 233A to 233N, recording data having desired sound quality can be edited and generated.

That is, the recording processing program 3
At 0, the workspace 31 first displays the workspace screen 20 as shown in FIG. The workspace screen 20 has an IO manager display section 21 at the substantially center and a plurality of various DSP plug-ins 33A to 33A.
Toolbox 2 displaying 33N plug-in list
2 and a transport window 23 for performing various operations such as playback, stop, fast rewind, fast forward, and pause.

Incidentally, the workspace screen 2 at this time
No. 0 indicates that nothing is displayed on the IO manager display unit 21 and none of the DSP plug-ins 33A to 33N is selected.

The user selects the select button 22X from the tool box 22 on the workspace screen 20 and, for example, the DSP plug-ins 33A and 33B shown in FIG. After clicking the plug-in buttons 22A and 22B corresponding to the “mixer plug-in 33B”, and clicking the IO manager display unit 21, the workspace 31 reads the recorder plug-in 33A and the mixer plug-in 33B from the hard disk of the HDD 11. 5 and the recorder plug-in icon 40A and the mixer plug-in icon 40B corresponding to the recorder plug-in 33A and the mixer plug-in 33B are displayed as shown in FIG.
To be displayed.

In this case, the recorder plug-in icon 4
0A is provided with an input terminal (shown by a circle) for 6 channels on the left side and an output terminal (shown by a circle) for 6 channels on the right side, and the mixer plug-in icon 40B has the output of the recorder plug-in icon 40A An input terminal (shown by a circle) for six channels corresponding to the terminal and an output terminal (shown by a circle) for two channels for outputting final two-channel recording data are provided.

Next, when the user clicks the connect button 22Y from the tool box 22 on the workspace screen 20 and drags and drops from the output terminal of the recorder plug-in icon 40A to the input terminal of the mixer plug-in icon 40B, the IO The manager 32 outputs the recorder plug-in 33A and the mixer plug-in 33A.
B input and IO manager display unit 2
A connection source output terminal (shown by a circle) of the recorder plug-in icon 40A and a connection destination input terminal (shown by a circle) of the mixer plug-in icon 40B are connected and displayed on the screen.

Here, the recorder plug-in 33A is H
It has a playback system that performs playback processing on drum and piano audio data recorded on the hard disk of the DD 11 and a recording system that performs recording processing on guitar audio data transmitted from an external personal computer 4 in New York. I have.

In this case, the recorder plug-in 33A
By inputting audio data of L-channel and R-channel guitars transmitted from the personal computer 4 in New York via the Internet 2 via the lowest 2ch input terminals of the 6ch input terminals of the recorder plug-in icon 40A. It is made to be able to process sound recording.

As shown by the mixer plug-in icon 40B, the mixer plug-in 33B inputs and mixes audio data for a total of six channels output from the recorder plug-in 33A, thereby recording the final two channels. Data can be output.

Next, when the user clicks the connection button 25 provided at the upper left corner of the workspace screen 20, the workspace 31 is connected to the IP (Internet Pro
(tocol) An IP address input screen (not shown) for inputting an address is superimposed on the workspace screen 20 and displayed.

When the user inputs the IP address of the personal computer 4 in New York desired to be connected to the IP address input screen, the workspace 31 connects the personal computer 3 in Tokyo and the personal computer 4 in New York via the Internet 2. A network connection is established, and an IO manager display 27 corresponding to the personal computer 4 in New York is displayed over the IO manager display 21 as shown in FIG.

As a result, the workspace 31 (FIG. 3)
IO manager 23 of recording processing program 230 in personal computer 4 in New York
2 is connected to the network via the Internet 2.

The workspace screen 20 in this state is displayed.
In the example, the IO manager display unit 27 is a currently active window, and the IO manager display unit 21 is an inactive window.

Subsequently, the user operates the workspace screen 20
Select button 22X from tool box 22
And the audio output plug-in 233A and the audio input plug-in 233B (FIG. 3)
After clicking on the plug-in buttons 22C and 22D corresponding to, the workspace 31 is clicked and the workspace 31 is provided with the audio output plug-in 233A and the audio input plug-in for the IO manager 232 of the personal computer 4 in New York. 23
A start instruction for starting 3B on the personal computer 4 side is transmitted via the Internet 2.

Thus, the workspace 31 activates the audio output plug-in 233A and the audio input plug-in 233B on the personal computer 4 side in New York, and outputs the audio output corresponding to the audio output plug-in 233A and the audio input plug-in 233B. Plug-in icon 41A and voice input plug-in icon 41B
Is displayed on the IO manager display section 27.

After that, the user clicks the connect button 22Y from the tool box 22, and the connection source output terminal of the audio input plug-in icon 41B (shown by a circle).
Drag and drop from the plug-in icon to the connection input terminal (shown by a circle) of 2 ch located at the lowermost position of the recorder plug-in icon 40A.
When dragging and dropping from the connection source output terminal B (shown by a circle) to the connection destination input terminal (shown by a circle) of the audio output plug-in icon 41A, the IO manager 32 makes the recorder plug-in 33A and the mixer plug-in 33B
And the audio output plug-in 233A and the audio input plug-in 233B are connected to each other via the Internet 2.

At this time, the workspace 31 stores the recorder plug-in icon 40A and the mixer plug-in icon 40B of the IO manager display section 21 as shown in FIG.
And the audio output plug-in icon 41A and the audio input plug-in icon 41B of the IO manager display unit 27 are connected and displayed on the workspace screen 20, so that the recorder plug-in 33A and the mixer The user can visually recognize that the plug-in 33B is connected to the audio output plug-in 233A and the audio input plug-in 233B of the personal computer 4 in New York.

As a result, the 2ch input terminal located at the bottom of the recorder plug-in 33A is transmitted from the voice input plug-in 233B of the personal computer 4 in New York via the Internet 2.
The audio data of the ch guitar is input, and the 2 ch input terminal of the audio output plug-in 233A is connected to the 2 ch output terminal of the mixer plug-in 33B.

As a result, the remote-to-remote recording system 1 reproduces the data by the recorder plug-in 33 A of the personal computer 3 in Tokyo, and
The audio data of the drum and the piano mixed in B are transmitted to the audio output plug-in 233A of the personal computer 4 in New York, and are output from the speakers (not shown) through the audio output plug-in 233A as drum and piano performance sounds. Can be output.

Then, the remote location recording system 1
Takes in the audio input plug-in 233B via the microphones 4A and 4B the audio signal of the guitar played by the personal computer 4 in New York in accordance with the performance sounds of the drum and the piano.
A reply is sent to the recorder plug-in 33A in the personal computer 3 in Tokyo via 33B as audio data of the guitar.

The personal computer 3 in Tokyo uses the recording system of the recorder plug-in 33A so as to synchronize with the audio data of the drum and piano read out from the hard disk of the HDD 11 and reproduced by the reproducing system of the recorder plug-in 33A. By storing the data, as a result, it is possible to execute a storage process in which a difference between the reproduction timing and the recording timing of the recorder plug-in 33A due to a transmission path delay between remote locations is corrected.

Further, a remote location recording system 1
Are the recorder plug-in 33A and the mixer plug-in 33B on the personal computer 3 in Tokyo, and the audio output plug-in 233A and the audio input plug-in 2 launched on the personal computer 4 in New York.
33B, the recorder function and the mixer function, the voice output function and the voice input function are distributed to the personal computer 3 in Tokyo and the personal computer 4 in New York when viewed as a whole system. Become.

Thus, the remote-to-remote recording system 1 can perform various operations even if only one of the personal computer 3 in Tokyo and the personal computer 4 in New York could not perform processing due to the limit of the processing capacity of the CPUs 10 and 210. The ability to distribute the functions and improve the overall processing capacity has made it possible to execute the entire system.

Note that the IO manager 32 loads or unloads the mixer plug-in 33B and the audio output plug-in 233A and the audio input plug-in 233B started by the personal computer 4 in New York even while the audio data is being reproduced by the recorder plug-in 33A. Can be dynamically executed, whereby the combination of various plug-ins can be freely changed at any time to construct the remote location recording system 1 combining the functions desired by the user. .

(2-2) Timing Correction Process Next, a timing correction process for correcting a difference between the reproduction timing and the recording timing of the recorder plug-in 33A due to a transmission line delay between remote locations in the remote location recording system 1 will be described. It will be described in detail.

(2-2-1) Timing Correction Processing for Transmission Line Delay In the above-mentioned remote location recording system 1,
When a personal computer 3 in Tokyo and a personal computer 4 in New York are connected via a network via the Internet 2, a delay between transmission paths always occurs between the networks. In order to store the data on the hard disk of the HDD 11 in synchronization with the audio data of the drum and the piano, it is necessary to correct a timing deviation due to a delay between transmission paths.

In practice, as shown in FIG. 8, the recorder plug-in 33A and the mixer plug-in 33B are supplied with the master clock MCLK1 generated by the internal real-time clock from the IO manager 32 for each packet cycle. Audio output plug-in 233A
The master clock MC generated by the internal real-time clock is also provided to the audio output plug-in 233B.
LK2 is supplied from the IO manager 232 every packet cycle.

In this case, the master clock MCLK1 supplied to the recorder plug-in 33A and the mixer plug-in 33B by the IO manager 32 and the IO manager 23
2 are set to be synchronized with the master clock MCLK2 supplied to the audio output plug-in 233A and the audio output plug-in 233B, so that the common master clocks MCLK1 and MCLK2 are simultaneously supplied between remote locations. .

That is, the recorder plug-in 33A reproduces the audio data of the drum and the piano by using the reproduction system, and obtains the 2ch (L channel and R channel) drum packet data DrP obtained as a result of the reproduction.
Time stamp (TS) based on master clock MCLK1 for D1 (L), DrPD2 (R) and piano packet data PiPD1 (L), PiPD2 (R)
“0000” is added under the control of the IO manager 32, and the next drum packet data DrPD1, Dr
PD2 and piano packet data PiPD1, PiPD
2, the time stamp value “0001” is added, and the time stamp value is sequentially incremented and added in packet units, thereby adding the time stamp-added drum packet data DrPD1, DrPD2 and the piano packet data PiPD1, PiPD2. , And sequentially outputs them to the mixer plug-in 33B.

The IO manager 32 recognizes the connection state of the recorder plug-in 33A, the mixer plug-in 33B, the audio output plug-in 233A, and the audio input plug-in 233B based on the plug-in data. For the DrPD2 and the piano packet data PiPD1 and PiPD2, connection destination information indicating a destination to be transmitted next is instructed.

As a result, the recorder plug-in 33A, the mixer plug-in 33B, the audio output plug-in 233A and the audio input plug-in 233B
The connection destination information is added based on the instruction, and the packet data can be accurately transmitted to the DSP plug-in of the predetermined connection destination according to the connection destination information.

At this time, at the same time, the IO manager 32 waits for the input of the mixer plug-in 33B connected to the output of the recorder plug-in 33A for four channels and waits for the input of various packet data of the time stamp “0000”. Set “0000”.

The mixer plug-in 33B stores the drum packet data DrPD1 and DrPD2 added with the time stamp “0000” from the recorder plug-in 33A and the piano packet data PiPD1 and PiPD2 for the four channels for which the waiting time stamp “0000” is set. When the input has been reached, the mixing process is started when the data has reached all the inputs.

Then, the mixer plug-in 33B adds a time stamp “0000” to the mixing data MIXPD1 and MIXPD2 generated by performing the mixing process under the control of the IO manager 32, and adds this to the time stamp “0000”. Is transmitted to the audio output plug-in 233A of the personal computer 4 in New York via the Internet 2 as the mixing data MIXPD1 and MIXPD2.

At this time, the work space 31 is mixed with the time stamp “0000” by the input of two channels of the audio output plug-in 233A connected to the output of the mixer plug-in 33B via the IO manager 232 of the personal computer 4. Data MIXPD1 and MI
Wait time stamp "00" to wait for input of XPD2.
00 ”is set.

Thereafter, the IO manager 32 receives the time stamp “00” by inputting the four channels of the mixer plug-in 33B.
The wait time stamp “0001” is set so as to wait for input of various packet data of “01”, and the value of the wait time stamp is sequentially incremented in accordance with the various packet data output from the recorder plug-in 33A. ing.

When the mixing data MIXPD1 and MIXPD2 having the time stamp “0000” from the mixer plug-in 33B reach the input set to the waiting time stamp “0000”, the audio output plug-in 233A outputs the time stamp “0000”. After the mixing data MIXPD1 and MIXPD2 are converted from digital to analog, they are output as drum and piano performance sounds via a speaker (not shown).

In practice, the audio output plug-in 233A
Mixing data M with the same time stamp
Buffering is performed sequentially when the inputs of IXPD1 and MIXPD2 are completed, the data is sequentially read out at a specified timing based on the master clock MCLK2 supplied from the IO manager 232, digital-to-analog converted, and then output via a speaker. .

As described above, the audio output plug-in 233A
By buffering the mixing data MIXPD1 and MIXPD2 and outputting them at a specified timing, even if there is a supply interval of the mixing data MIXPD1 and MIXPD2 supplied from the mixer plug-in 33B, it is possible to prevent the music from being interrupted halfway. It is designed to prevent it.

Thereafter, the IO manager 232 sets the waiting time stamp “0001” so as to wait for the input of various packet data of the next time stamp “0001” by the input of two channels of the audio output plug-in 233A, and sequentially sets the mixer. The value of the waiting time stamp is incremented in accordance with the mixing data MIXPD1 and MIXPD2 supplied from the plug-in 33B.

The audio input plug-in 233B converts the audio signal of the guitar played in accordance with the performance sound of the drum and piano output from the audio output plug-in 233A through the speaker into digital audio data and packetizes it. , And the time stamp "00
00 "to add guitar packet data Gi
It generates PD1 and GiPD2, and transmits them to the recorder plug-in 33A of the personal computer 3.

That is, the voice input plug-in 233B
Time stamp "000" from recorder plug-in 33A
0 "first drum packet data DrPD1, DrP
D2 and piano packet data PiPD1, PiPD2
Is output, the guitar packet data G
When the iPD1 and GiPD2 are output, the guitar packet data GiPD1 and GiPD2 are time-delayed by adding the time stamp “0000” to the guitar packet data GiPD1 and GiPD2, so that the drum packet data of the time stamp “0000” is added. DrPD1, Dr
PD2 and piano packet data PiPD1, PiPD
2 and the guitar packet data GiPD1 and GiPD2 with the time stamp “0000” are audio data having the same timing.

The IO manager 32 has a recorder plug-in 33 connected to the output of the audio input plug-in 233B.
Time stamp "0000" for recording system input in A
Is added to the guitar packet data GiPD1 and GiPD.
Wait time stamp "000" to wait for PD2 input
0 ”is set.

Thus, the recorder plug-in 33A
Guitar packet data G with time stamp "0000"
iPD1 and GiPD2 wait time stamp “000”
When it reaches the input of "0", the time stamp "0000"
The guitar packet data GiPD1 and GiPD2 of the time stamp “0000” are stored and controlled in the hard disk of the HDD 11 in synchronization with the drum packet data DrPD1 and DrPD2 and the piano packet data PiPD1 and PiPD2.

Thus, the recorder plug-in 33A corrects the timing shift between the drum and piano audio data and the guitar audio data caused by the transmission path delay, and synchronizes the drum and piano audio data with the guitar data. It is designed to store audio data.

(2-2-2) Timing correction processing when a new input is applied during the recording processing In the remote recording system 1, for example, as shown in FIG.
With the waiting time stamp "0002" set so as to wait for the drum packet data DrPD1 and DrPD2 of the time stamp "0002" at the input of 2ch in the mixer plug-in 33B, the piano packet data is newly transmitted from the recorder plug-in 33A at the preceding stage. PiP
When the inputs of D1 and PiPD2 are added, a waiting time stamp "0006" is set to the newly added input, for example, to wait for the piano packet data PiPD1 and PiPD2 of the time stamp "0006".

Here, the reason why the IO manager 32 sets the wait time stamp "0006" so as to wait for the input of the piano packet data PiPD1 and PiPD2 of the time stamp "0006" is that the piano packet data P
This means that the maximum delay time required from the input of iPD1 and PiPD2 to the arrival of the corresponding piano packet data PiPD1 and PiPD2 is expected.

That is, the IO manager 32 sends the recorder plug-in 3 in the preceding stage to the mixer plug-in 33B.
When the drum packet data DrPD1 and DrPD2 of the time stamp “0002” are input from 3A, FIG.
As shown in (B), the mixing data MIXPD1 and MIXPD2 of the time stamp “0002” are output, so that the drum packet data DrPD1 and DrPD
The value of the waiting timestamp for 2 is incremented to the timestamp “0003”.

Then, the IO manager 32 sends the mixer plug-in 33B to the recorder plug-in 33 in the preceding stage.
When the drum packet data DrPD1 and DrPD2 having the time stamp “0003” are input from A, the mixing data MIXPD1 having the time stamp “0003” is input.
And MIXPD2 are output, so that the drum packet data DrPD1 and DrPD2 are incremented to the waiting time stamp “0004” and are sequentially incremented to the time stamp “0006”.

That is, the IO manager 32 waits until the piano packet data PiPD1 and PiPD2 of the time stamp “0006” arrive at the input of the mixer plug-in 33B in which the wait time stamp “0006” is set, and waits for the time stamp “0002”. "~" 0005 "
Of the drum packet data DrPD1 and DrPD2 is continuously executed by the mixer plug-in 33B, and the drum packet data D
The mixing process for rPD1 and DrPD2 can be prevented from being delayed.

As shown in FIG. 9C, the IO manager 3
2 indicates that when the wait time stamp “0006” for the drum packet data DrPD1 and DrPD2 is to be set by the input of the mixer plug-in 33B, the time stamp “000” is input for all inputs for 4 channels.
6 "drum packet data DrPD1 and DrPD2.
And the piano packet data PiPD1 and PiPD2 of the time stamp “0006” are awaited.

The IO manager 32 sends the drum packet data DrPD1 and DrPD2 of the time stamp “0006” and the piano packet data PiPD of the time stamp “0006” from the recorder plug-in 33A at the preceding stage.
1 and PiPD2 are input to the mixer plug-in 33B, and the mixing data M of the time stamp "0006" is input.
When IXPD1 and MIXPD2 are output, FIG.
As shown in (D), the input for four channels is incremented to the waiting time stamp "0007", and thereafter, the mixing process is sequentially performed for the inputs for four channels.

As described above, during the recording process, the IO manager 32 newly adds the recorder plug-in 3 of the previous stage.
3A to Piano packet data PiPD1 and PiPD
When the input 2 is added, the maximum delay time required for the piano packet data PiPD1 and PiPD2 to actually reach the mixer plug-in 33B from the recorder plug-in 33A at the preceding stage is expected for the newly added input. By setting the waiting time stamp, after the elapse of the maximum delay time, the inputs for four channels can be surely aligned and the mixing processing for four channels can be executed.
Drum packet data DrPD until the minutes are input
1 and DrPD2 can be continuously executed without delay.

Incidentally, the IO manager 32 sets the waiting time stamp “000” set by the mixer plug-in 33B.
Even when the piano packet data PiPD1 and PiPD2 of the time stamp "0005" arrives at the input of "6", the input of the piano packet data PiPD1 and PiPD2 of the time stamp "0005" is not received, and the time stamp "0006" is used. Are waiting for the arrival of the piano packet data PiPD1 and PiPD2 of "."

That is, the IO manager 32 sets the waiting time stamp “0006” in anticipation of the maximum delay time, and waits for the piano packet data PiPD1 and PiPD2 of the time stamp “0006”. When the short wait time stamp “0003” is set, the time stamp “0”
006 "piano packet data PiPD1 and PiP1
It is possible to prevent a situation in which the mixing process cannot be executed forever because D2 arrives and the inputs for 4 channels are not aligned.

(2-2-3) Timing correction processing when input is released during recording processing On the other hand, in the remote-to-remote recording system 1,
For example, as shown in FIG.
A description will be given of the timing correction processing when the input of the piano packet data PiPD1 and PiPD2 is released due to partial disconnection with the recorder plug-in 33A of the preceding stage while the mixing data MIXPD1 and MIXPD2 of the time stamp “0016” are being output by 3B. I do.

In this case, the IO manager 32
As shown in (B), when the connection with the recorder plug-in 33A is canceled, the piano packet data PiPD1 and PiPD2 of the time stamp “0017” already arriving from the recorder plug-in 33A exist. After the mixing process, the waiting time stamp “0018” for the next input of the piano packet data PiPD1 and PiPD2 is set to “invalid”.

As a result, the mixer plug-in 33B invalidates the input of the next piano packet data PiPD1 and PiPD2 of the time stamp “0018” without accepting the input, and only the drum packet data DrPD1 and DrPD2 of the time stamp “0018”. A mixing process is performed.

Therefore, the IO manager 32 thereafter sets the time stamp “0” as shown in FIG.
018 ”mixing data MIXPD1 and MIXP
D2, and outputs the piano packet data PiPD1, Pi
While the wait time stamp for the input of PD2 is set to "invalid", the drum packet data
Only the waiting time stamp for the input of rPD2 is sequentially incremented.

As described above, during the recording process, the IO manager 32 releases the input of the piano packet data PiPD1 and PiPD2 due to partial disconnection between the recorder plug-in 33A and the mixer plug-in 33B at the preceding stage. At that time, the piano packet data PiPD1 and PiPD2 already sent from the recorder plug-in 33A to the mixer plug-in 33B are subjected to mixing processing. However, the piano packet data PiPD1 and PiPD2 sent thereafter are not input. By invalidating without receiving, the drum packet data
It is possible to smoothly shift to the mixing process for only 1 and DrPD2.

(2-2-4) Timing correction processing when packet data after the packet data of the waiting time stamp is transmitted Next, in the remote-to-remote recording system 1, for example, FIG. As shown, the mixer plug-in 33
B, while waiting for the drum packet data DrPD1 and DrPD2 of the time stamp “0021”, the timing when the drum packet data DrPD2 of the time stamp “0022” is transmitted from the preceding recorder plug-in 33A by the input of one channel The correction processing will be described.

In this case, the mixer plug-in 33B is
As shown in FIG. 1 (B), the drum packet data DrPD2 with the time stamp “0021” is received, but the waiting time stamp “002” is received with the drum packet data DrPD2 with the time stamp “0022”.
Since it does not match "1", the reception is refused and sent back to the IO manager 32.

The IO manager 32 temporarily buffers the drum packet data DrPD2 having the time stamp “0022”, and newly sends the drum packet data DrPD2 having the time stamp “0021” to the mixer plug-in 33B. Plug-in 33 with the drum packet data DrPD2
B.

At this time, the mixer plug-in 33B
Does not execute the mixing process until the drum packet data DrPD1 and DrPD2 of the time stamp “0021” are collected, and the drum packet data DrP
Timing deviation between D1 and drum packet data DrPD2 is reliably prevented.

As described above, when the drum packet data DrPD2 of the time stamp “0022” after the waiting time stamp “0021” is received by the mixer plug-in 33B and rejected and sent back, the IO manager 32 After temporarily buffering the drum packet data DrPD2 of “0022” and transmitting the drum packet data DrPD2 of the time stamp “0021” to the mixer plug-in 33B, the drum packet data DrPD2 of the time stamp “0022” is retransmitted to the mixer plug-in 33B. By performing this, it is possible to control the mixer plug-in 33B to execute the mixing process without disturbing the order of the packet data.

(2-2-5) D by GUI plug-in
For example, when the mixer plug-in icon 40B is double-clicked, the work space 31 loads the mixer GUI plug-in 34B provided corresponding to the mixer plug-in 33B, and loads the mixer GUI plug-in. FIG. 1 based on the in 34B
The mixer GUI plug-in screen 50 shown in FIG.
The information is superimposed and displayed on the manager display unit 21.

Mixer GUI plug-in screen 5 in this case
Reference numeral 0 denotes a GUI screen for adjusting the volume of audio data input to the mixer plug-in 33B by 2 ch for each musical instrument. The GUI screen is an operation target according to the setting of the volume value (“0 db”) by the user. The sound volume of a certain mixer plug-in 33B can be adjusted.

When the audio output plug-in icon 41A is double-clicked, the workspace 31 sends a request command for the audio output GUI plug-in 234A corresponding to the audio output plug-in 233A to the personal computer 4 in New York. Via the audio output GUI plug-in 234A transmitted from the personal computer 4 and received from the personal computer 4.
The plug-in screen can be displayed so as to overlap the IO manager display unit 27.

Thus, the workspace 31 stores the parameters relating to the audio output plug-in 233A running on the personal computer 4 in New York in the audio output GU.
It can be remotely operated based on the I plug-in screen.

As described above, in the remote recording system 1, the DSP plug-ins 33 A to 33 N and 233 A to 233 N and the GU
For example, the personal computer 3 receives the GUI plug-in 234A of the personal computer 4 by holding the I plug-ins 34A to 34N and 234A to 234N separately, and the GUI plug-in 23
The DSP plug-in 33A can be remotely controlled via the 4A, and thus remote control over a network can be easily performed.

By operating the transport window 23 (FIG. 7), the user can input a processing command for audio data reproduction processing and recording processing, and the like, so that the IO manager 32 The recorder plug-in 33A, the mixer plug-in 33B, the audio output plug-in 233A, and the audio input plug-in 233B are operated in conjunction with each other to execute a reproduction process and a recording process in response to a command.

(2-3) Recording Processing Procedure Next, the above-mentioned recording processing procedure performed by the personal computer 3 in accordance with the recording processing program 30 will be summarized with reference to the flowchart of FIG. 13, and the personal computer 3 starts the routine RT1. Entering from the step, the process proceeds to a subroutine SRT2 for loading the DSP plug-in 33.

(2-3-1) Procedure for Loading DSP Plug-in As shown in FIGS. 14 and 15, the subroutine SRT2
In step SP1 of the above, when the user clicks the select button 22X and the plug-in buttons 22A and 22B from the tool box 22 of the workspace screen 20 (FIG. 3), the DSP plug-in (recorder plug-in) The object of the DSP plug-ins 33A and 33B read out from the hard disk of the HDD 11 together with the load request command of the 33A and the DSP plug-in (mixer plug-in) 33B is sent to the IO manager 32, and the process proceeds to the next step SP2.

In step SP2, the IO manager 3
2 loads the objects of the DSP plug-ins 33A and 33B into the RAM 12, and proceeds to the next step SP3.

In step SP3, workspace 3
1 sends an IO manager update check command for checking the update status of the DSP plug-in 33 being loaded to the IO manager 32 at regular time intervals, and proceeds to the next step SP4.

In step SP4, the IO manager 3
2 sends an update result message indicating whether the DSP plug-in 33 has been updated to the workspace 31, and moves to the next step SP5.

In step SP5, workspace 3
1 judges whether or not the DSP plug-in 33 has been updated based on the content of the update result message. If a negative result is obtained here, this means that the newly loaded D
This indicates that the SP plug-in 33 does not exist and that the DSP plug-in 33 has not been updated. At this time, the workspace 31 proceeds to the next subroutine SRT3.

On the other hand, if a positive result is obtained in step SP5, this means that the newly loaded DSP plug-in 33 exists, and that the DSP plug-in 33 has been updated. At this time, the work space 31 proceeds to the next step SP6.

At step SP6, the work space 3
1 sends a plug-in data request command for requesting plug-in data relating to the updated new DSP plug-in 33 to the IO manager 32, and proceeds to the next step SP7.

At step SP7, the IO manager 3
2 sends the plug-in data relating to the updated new DSP plug-in 33 to the workspace 31 and proceeds to the next step SP8.

In step SP8, workspace 3
1 displays, for example, the recorder plug-in icon 40A and the mixer plug-in icon 40B corresponding to the new DSP plug-ins 33A and 33B loaded on the IO manager display unit 21 based on the plug-in data, thereby displaying the IO manager. The screen of the unit 21 is newly updated, and the process proceeds to the next subroutine SRT3.

(2-3-2) Connection management processing procedure of DSP plug-in As shown in FIGS. 16 and 17, the subroutine SRT3
In step SP11 of the work space 31, the connect button 22Y is clicked from the tool box 22 of the work space screen 20 (FIG. 3), for example, from the output terminal of the recorder plug-in icon 40A to the input terminal of the mixer plug-in icon 40B. When dragged and dropped, together with the connection request command of the DSP plug-in (recorder plug-in) 33A and the DSP plug-in (mixer plug-in) 33B, the connection source output, the connection destination input, the connection start time stamp at the start of the connection, etc. Is sent to the IO manager 32,
Move to the next step SP12.

In step SP12, the IO manager 32 executes a process of connecting the DSP plug-ins 33A and 33B, and proceeds to the next step SP13.

In step SP13, the workspace 31 sends an IO manager connection update check command for checking the connection status to the DSP plug-in 33 being loaded to the IO manager 32 at regular time intervals. Move on to

At step SP14, the IO manager 32 sends a connection update result message indicating the presence or absence of connection update to the workspace 31 with respect to the DSP plug-ins 33A and 33B, and proceeds to the next step SP15.

In step SP15, the workspace 31 stores the DS based on the contents of the connection update result message.
It is determined whether or not the connection status has been updated for the P plug-ins 33A and 33B. If a negative result is obtained here, this indicates that there has been no change in the connection status regarding the DSP plug-ins 33A and 33B,
At this time, the workspace 31 stores the next subroutine SRT.
Move to 4.

On the other hand, if an affirmative result is obtained in step SP15, this means that there has been some change in the connection status regarding the DSP plug-ins 33A and 33B. Move to SP16.

In step SP16, the workspace 31 sends a plug-in data request command for the DSP plug-in 33 whose connection status has been newly changed to the IO manager 32, and proceeds to the next step SP17.

At step SP17, the IO manager 32 sends plug-in data relating to the DSP plug-in 33 whose connection status has been newly changed to the workspace 31, and proceeds to the next step SP18.

In step SP18, based on the plug-in data, the workspace 31 displays the plug-in icons 40A and 40B whose connection status has been newly changed on the basis of the plug-in data.
By newly displaying the screen on the O manager display unit 21, the screen of the IO manager display unit 21 is newly updated, and the process proceeds to the next subroutine SRT4.

(2-3-3) DSP plug-in parameter adjustment processing procedure As shown in FIGS. 18 and 19, the subroutine SRT4
In step SP21 of the GUI plug-in 34,
When parameter adjustment is performed by the user on the GUI plug-in screen, parameter control information corresponding to the parameter adjustment is transmitted to the DSP plug-in 33, and the process proceeds to the next step SP22.

[0138] In step SP22, the DSP plug-in 33 adjusts the parameter (for example, volume) according to the parameter control information, and proceeds to the next step SP23.

In step SP23, the GUI plug-in 34 sends a parameter update check command for checking whether parameter adjustment has been updated to the DSP plug-in 33 to be operated at regular time intervals, and proceeds to the next step SP24. Move on.

In step SP24, the DSP plug-in 33 sends a parameter update result message indicating whether or not the parameter adjustment has been updated to the DSP plug-in 33 to the GUI plug-in 34, and proceeds to the next step SP25.

In step SP25, the GUI plug-in determines whether or not there has been a change in parameter adjustment for the DSP plug-in 33 based on the contents of the parameter update result message. If a negative result is obtained here, this means that the parameter adjustment for the DSP plug-in 33 has not been changed, and at this time, the GUI plug-in 34 proceeds to the next subroutine SRT5.

On the other hand, if a positive result is obtained in step SP25, this means that the parameter adjustment for the DSP plug-in 33 has been changed. At this time, the GUI plug-in 34 proceeds to the next step SP26. Move on.

At step SP26, the GUI plug-in 34 sends a parameter adjustment data request command to the DSP plug-in 33 whose parameter adjustment has been changed, and proceeds to the next step SP27.

In step SP27, the DSP plug-in 33 sends parameter adjustment data indicating the contents of the parameter adjustment to the GUI plug-in 34, and proceeds to the next step SP28.

At step SP28, the CPU 10
The GUI plug-in screen is updated by displaying a GUI plug-in screen in which parameters are newly changed based on the parameter adjustment data, and the process proceeds to the next subroutine SRT5.

(2-3-4) Timing Correction Processing Procedure As shown in FIG. 20, in step SP31 of the subroutine SRT5, the IO manager 32 reproduces the drum reproduced by the reproduction system of the recorder plug-in 33A as shown in FIG. The packet data DrPD1 and DrPD2 and the piano packet data PiPD1 and PiPD2 are output with a time stamp “0000” added thereto, and the mixer plug-in 33B and the audio output plug-in 2 at the next and subsequent stages are added.
The wait time stamp “0000” is set for the input of the recording system in the 33A and the recorder plug-in 33A (FIG. 8), and the routine goes to the next step SP32.

At step SP32, the IO manager 32 adds the time stamp “0000” to the mixing data MIXPD1 and MIXPD2 output from the mixer plug-in 33B at the next stage and the guitar packet data GiP output from the audio input plug-in 233B.
The time stamp “0000” is also added to D1 and GiPD2, and the routine goes to the next step SP33.

In step SP33, the IO manager 32 converts the guitar packet data GiPD1 and GiPD2 of the time stamp “0000” input from the audio input plug-in 233B to the recorder plug-in 33A into the drum of the time stamp “0000” reproduced in the reproduction system. By controlling the storage on the hard disk of the HDD 11 based on the time stamp "0000" so as to synchronize with the packet data DrPD1, DrPD2 and the piano packet data PiPD1, PiPD2, the timing deviation due to the transmission path delay is corrected, and the next step SP34 Move on.

In step SP34, the IO manager 32, for example, as shown in FIG.
It is determined whether a new input has been added to 3B. If a negative result is obtained here, this means that a new input has not been added. At this time, the IO manager 32 proceeds to step SP40 and ends all the recording processing procedures by the recording processing program 30.

On the other hand, if a positive result is obtained in step SP34, this indicates that a new input has been added, and at this time, the IO manager 32 proceeds to the next step SP35.

In step SP35, the IO manager 32 sets a waiting time stamp in consideration of the maximum delay time until packet data arrives from the preceding DSP plug-in for a new input, thereby adding a new input. The processing for the packet data that is already being input is prevented from being delayed due to the input, and the process proceeds to the next step SP36.

In step SP36, the IO manager 32 determines whether or not the input of the piano packet data PiPD1 and PiPD2 has been released during the mixing process in the mixer plug-in 33B as shown in FIG. 10, for example.

Here, if a negative result is obtained, this means that the input has not been canceled and there is no change, and at this time, the IO manager 32 proceeds to step SP40.
To end the recording processing procedure by the recording processing program 30.

On the other hand, if a positive result is obtained in step SP36, this means that the input has been released and the connection state has been changed, and at this time, the IO manager 32 proceeds to the next step SP37. .

In step SP37, the IO manager 32 sets the waiting time stamp “0000” of the input released by the mixer plug-in 33B to “invalid”, so that the drum packet data DrPD 1 and The process proceeds to the mixing process for only DrPD2, and proceeds to the next step SP38.

In step SP38, the IO manager 32 transmits packet data of a time stamp later than the waiting time stamp to the input of the mixer plug-in 33B, for example, as shown in FIG. It is determined whether or not the input of the packet data is rejected.

Here, if a negative result is obtained, this means that the input of packet data of a time stamp later than the waiting time stamp was not rejected, that is, the same time stamp as the waiting time stamp was added. This indicates that the received packet data has been input to the mixer plug-in 33B. At this time, the IO manager 32 proceeds to step SP40 and ends all the recording processing procedures by the recording processing program 30.

On the other hand, if a positive result is obtained in step SP38, this means that the input of the packet data of the time stamp after the waiting time stamp is rejected by the mixer plug-in 33B. At this time, the IO manager 32 proceeds to the next step SP39.

In step SP39, the IO manager 32 temporarily buffers the packet data rejected by the mixer plug-in 33B and retransmits it until it is accepted by the input of the mixer plug-in 33B. After the mixing process is executed accurately without changing the order of the data, the process proceeds to the next step SP40, and all the recording processing procedures by the recording processing program 30 are ended.

(3) Operation and Effect in Embodiment In the above configuration, the remote recording system 1 performs the recording processing between the personal computer 3 in Tokyo and the personal computer 4 in New York almost simultaneously in parallel. Is performed, the drum packet data DrP reproduced by the reproduction system of the recorder plug-in 33A is used.
D1 and DrPD2, piano packet data PiPD1
And a timestamp “0000” to PiPD2 and output.

At this time, the remote-to-remote recording system 1 sets a waiting time stamp “0000” at each input in the recording system of the mixer plug-in 33B, the audio output plug-in 233A, and the recorder plug-in 33A in the next and subsequent stages. A time stamp “0000” is added to the output of the plug-in 33B and the audio input plug-in 233B.

As a result, the remote-to-remote recording system 1 is a reproduction system of the recorder plug-in 33A, and the drum packet data DrPD of the time stamp “0000” is provided.
1 and DrPD2, and the piano packet data PiPD1 and PiPD2, the guitar packet data GiPD1 and GiPD2 with the time stamp “0000” can be stored and controlled in association with the hard disk of the HDD 11 so as to be synchronized. The recording process can be executed simultaneously and in parallel by correcting the timing shift due to the transmission path delay between the two.

Further, a remote recording system 1
Can also flexibly respond to new additions or cancellations of inputs during the recording process by changing the setting of the wait timestamp while maintaining temporal consistency. Can easily generate recording data of desired sound quality.

According to the above configuration, the remote-to-remote recording system 1 adds a time stamp to each packet to audio data requiring real-time property, and, based on the time stamp, a timing based on a transmission path delay. By controlling the storage so as to correct the deviation, it is possible to transmit and receive audio data requiring real-time between remote locations via the Internet 2 and execute the recording process simultaneously and in parallel.

(4) Other Embodiments In the above embodiment, the remote location recording system 1 as the content editing apparatus of the present invention is provided between the personal computer 3 in Tokyo and the personal computer 4 in New York. Although the description has been given of the case where the present invention is applied to the case where the recording process is executed, the present invention is not limited to this. For example, the recording is performed between the personal computer 3 and the personal computer 4 connected to each other by the transmission cable in the same studio. You may make it apply, when performing a process.

In the above embodiment, the recorder plug-in 33A, the mixer plug-in 33B, the audio output plug-in 233A as the first content data output means and the storage control means, and the audio input as the second content data output means. Although the case where the recording system is constructed by combining the plug-ins 233B has been described, the present invention is not limited to this, and the effector system using the audio input plug-in, the effector plug-in, and the audio output plug-in, Various other systems such as a music composition system using a synthesizer plug-in, a mixer plug-in, and an audio output plug-in may be constructed.

Further, in the above-described embodiment, a case has been described in which the personal computer 3 in Tokyo is connected to the personal computer 4 in New York for expansion. However, the present invention is not limited to this. It may be expanded by connecting to a digital signal processing device that has only a digital signal processing portion and does not have a display portion. In this case, the digital signal processing device is configured so that the IO manager executes various processes similarly to the personal computer 3 and has a simpler configuration than the personal computer because display control is not performed. The accompanying increase in space can be minimized.

Further, in the above embodiment, the GU
Although the I plug-in 34 has a hierarchical structure including lower-level GUI plug-ins and displays a GUI plug-in screen for each hierarchy, the present invention is not limited to this. Alternatively, a GUI plug-in screen having a single structure that can execute the operation of the lower-level GUI plug-in on one screen may be displayed.

Further, in the above-described embodiment, a case has been described in which the personal computer 3 in Tokyo and the personal computer 4 in New York are connected to each other via the Internet. However, the present invention is not limited to this, and The computers 3 and 4 may be connected to each other via a LAN (Local Area Network) such as Ethernet (registered trademark).

Further, in the above-described embodiment, the personal computer 3 in Tokyo and the personal computer 4 in New York are connected via the Internet 2, and the recording process is executed simultaneously and remotely between Tokyo and New York. However, the present invention is not limited to this, and various other remote locations such as London and Paris other than New York are also connected via the Internet 2 and the recording process is performed simultaneously and in parallel. You may do it.

Further, in the above-described embodiment, the recorder plug-in icon 40A and the mixer plug-in icon 40B and the audio output plug-in icon 41A and the audio input plug-in icon 41B are connected between the input terminal and the output terminal. Has described the case where the connection status of the DSP plug-ins is visually indicated to the user. However, the present invention is not limited to this, and the loaded DSP plug-ins are arranged in a matrix and displayed. The connection state of the DSP plug-ins may be indicated.

Further, in the above embodiment, the HD
The case has been described in which the above-described remote location recording process is executed based on the recording process program 30 started from the hard disk of D11. However, the present invention is not limited to this, and the program storing the recording process program 30 is not limited to this. By installing the storage medium in the personal computers 3 and 4,
The remote location recording process may be executed.

As described above, the program used to install the recording processing program 30 for executing the above-described series of remote-to-remote recording processing in the personal computers 3 and 4 and to make the personal computers 3 and 4 executable. As a storage medium, for example, a floppy (registered trademark) disk, C
D-ROM (Compact Disc-Read Only Memory), DV
Not only package media such as D (Digital Versatile Disc), but also a semiconductor memory or a magnetic disk in which the recording processing program 30 is temporarily or permanently stored may be realized. As a means for storing the recording processing program 30 in these program storage media, a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting may be used, and various communication interfaces such as a router and a modem may be used. Alternatively, the information may be stored via the Internet.

A recorder plug-in 33 as first content data output means for sequentially adding the time stamp “0000” to the drum packet data DrPD 1 and DrPD 2 and the piano packet data PiPD 1 and PiPD 2 as the first content data in packet units and sequentially outputting them.
A, and guitar packet data GiPD1 and GiPD2 as second content data in packet units corresponding to the first content data to which the same time stamp “0000” as the time stamp “0000” is added from an external personal computer 4. Plug-in 33A as input means for performing
Based on "00", the IO manager 32 as a timing correction means for adjusting the first content data and the second content data so as to be synchronized in time to construct a remote recording system 1 as a content editing device. However, the present invention is not limited to this, and the content editing apparatus may be constructed by various other first content data output means, input means, and timing correction means.

[0175]

As described above, according to the present invention, a time stamp is added to the first content data in a packet unit, and the packet is sequentially output, and the first content data corresponding to the first content data having the same time stamp as the time stamp is supported. The second content data in the packet unit is input from the outside, and the first content data and the second content data are adjusted so as to be temporally synchronized based on the timestamp, so that the first content data in the packet unit is obtained. Time synchronization between the first and second content data can be assured reliably by the time stamp, and thus the time synchronization between the first and second content data requiring real-time performance can be corrected and edited accurately. Content editing device, content editing method, program storage medium, and content It is possible to realize a tool editing system.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing an overall configuration of a remote location recording system according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a circuit configuration of a personal computer.

FIG. 3 is a schematic diagram illustrating a configuration of a recording processing program.

FIG. 4 is a schematic diagram illustrating a workspace screen.

FIG. 5 is a schematic diagram illustrating a workspace screen in which a DSP plug-in is loaded.

FIG. 6 is a schematic diagram showing a workspace screen when connected to another personal computer.

FIG. 7 is a schematic diagram showing a workspace screen when connected to a plug-in of another personal computer.

FIG. 8 is a schematic diagram used for describing a timing correction process.

FIG. 9 is a schematic diagram for explaining a timing correction process when a new input is added during the recording process;

FIG. 10 is a schematic diagram for explaining a timing correction process when a plug-in connection is released during the process;

FIG. 11 is a schematic diagram for explaining a timing correction process when packet data subsequent to the packet data of the waiting time stamp is sent;

FIG. 12 is a schematic diagram illustrating a mixer GUI plug-in screen.

FIG. 13 is a flowchart showing a recording processing procedure.

FIG. 14 is a flowchart illustrating a DSP plug-in load processing procedure;

FIG. 15 is a schematic diagram for explaining exchange between a workspace and an IO manager in a DSP plug-in load processing procedure;

FIG. 16 is a flowchart illustrating a connection management processing procedure of a DSP plug-in.

FIG. 17 is a schematic diagram for explaining exchange between a workspace and an IO manager in a connection management processing procedure of a DSP plug-in;

FIG. 18 is a flowchart illustrating a procedure for adjusting a parameter of a DSP plug-in;

FIG. 19 is a schematic diagram for explaining exchange between a GUI plug-in and a DSP plug-in in a DSP plug-in parameter adjustment processing procedure;

FIG. 20 is a flowchart illustrating a timing correction processing procedure.

[Explanation of symbols]

1. Remote recording system 2. Internet 3. Personal computer 10.
210: CPU, 11, 211: HDD, 12, 2
12 RAM, 30, 230 recording processing program 31, 231 workspace 32,
232 IO manager, 33, 233 DSP plug-in, 34, 234 GUI plug-in, 40A
... Recorder plug-in icon, 40B mixer mixer icon, 41A audio output plug-in icon, 41B audio input plug-in icon.

Claims (20)

[Claims] A first content data output unit for adding a time stamp to the first content data in packet units and sequentially outputting the first content data; and a first content data corresponding to the first content data having the same time stamp as the time stamp. Input means for externally inputting second content data in packet units, and timing correction means for adjusting the first content data and the second content data so as to be temporally synchronized based on the time stamp. A content editing device characterized by the following features: 2. The content editing apparatus according to claim 1, further comprising: storage control means for storing the first content data and the second content data temporally synchronized by the timing correction means in a predetermined storage means in association with each other. The content editing apparatus according to claim 1, further comprising: 3. The content according to claim 1, wherein the first content data and the second content data are audio data that require real-time performance corresponding to each of the musical instruments constituting the music. Editing device. 4. The apparatus according to claim 1, wherein said input means inputs said second content data only when a preset waiting time stamp matches said time stamp of said second content data. Content editing device as described in the above. 5. The input means sets a waiting time stamp in consideration of a maximum delay time until the arrival of the third content data when newly inputting the third content data during processing, and sets the waiting time stamp. 2. The content editing apparatus according to claim 1, wherein the third content data is input only when a waiting time stamp matches a time stamp added to the third content data. 3. 6. A first content data outputting step of sequentially adding a time stamp to the first content data in packet units and outputting the first content data, wherein the first content data corresponds to the first content data to which the same time stamp as the time stamp is added. An input step of externally inputting second content data in packet units; and a timing correction step of adjusting the first content data and the second content data so as to be temporally synchronized based on the time stamp. Content editing method characterized by the following: 7. The content editing method according to claim 1, further comprising: a storage control step of storing the first content data and the second content data temporally synchronized in the timing correction step in a predetermined storage means in association with each other. 7. The content editing method according to claim 6, comprising: 8. The content according to claim 6, wherein the first content data and the second content data are audio data that require real-time performance corresponding to each musical instrument constituting a music piece. Editing method. 9. The method according to claim 6, wherein the inputting step includes inputting the second content data only when a preset waiting time stamp matches the time stamp of the second content data. The content editing method described. 10. In the input step, when new third content data is input during processing, a waiting time stamp is set in consideration of a maximum delay time until the third content data arrives, and the set time stamp is set. 7. The content editing method according to claim 6, wherein the third content data is input only when the waiting time stamp and the time stamp added to the third content data match. 11. A first content data output step of adding a time stamp to the first content data in packet units and sequentially outputting the first content data, and the first content data corresponding to the first content data added with the same time stamp as the time stamp. An input step of externally inputting second content data in packet units; and a timing correction step of adjusting the first content data and the second content data so as to be temporally synchronized based on the time stamp. A program storage medium for causing an information processing device to execute a program characterized by the following. 12. The program storage medium according to claim 1, further comprising: a storage control step of storing the first content data and the second content data temporally synchronized in the timing correction step in a predetermined storage means in association with each other. The program storage medium according to claim 11, further comprising: 13. The first content data and the second content data.
12. The program storage medium according to claim 11, wherein the content data is audio data that requires real-time performance corresponding to each of the musical instruments constituting the music. 14. The method according to claim 11, wherein the inputting step includes inputting the second content data only when a preset waiting time stamp matches the time stamp of the second content data. The program storage medium described in the above. 15. The input step sets a waiting time stamp in consideration of a maximum delay time until the third content data arrives when newly inputting the third content data during the processing, and The program storage medium according to claim 11, wherein the third content data is input only when a waiting time stamp and a time stamp added to the third content data match. 16. A content editing system comprising a first information processing device and a second information processing device connected to each other via a network, wherein the first information processing device comprises: Time stamp adding means for adding a time stamp to data and sequentially outputting the data; and an input for externally inputting second content data in packet units corresponding to the first content data to which the same time stamp as the time stamp is added. Means, and timing correction means for adjusting the first content data and the second content data so as to be temporally synchronized based on the time stamp, wherein the second information processing apparatus comprises: The second generated above corresponding to the content data
A content editing system, comprising: second content data output means for adding the time stamp to content data and sequentially outputting the time stamp to the first information processing apparatus. 17. The first information processing apparatus, wherein the first content data and the second content data which are temporally synchronized by the timing correction means are stored in a predetermined storage means in association with each other. 17. The content editing system according to claim 16, comprising means. 18. The first content data and the second content data
17. The content editing system according to claim 16, wherein the content data is audio data that requires a real-time property corresponding to each musical instrument constituting the music. 19. The apparatus according to claim 16, wherein said input means inputs said second content data only when a preset waiting time stamp and said time stamp of said second content data match. Content editing system as described. 20. When inputting new third content data during processing, the input means sets a waiting time stamp in consideration of a maximum delay time until the third content data arrives, and sets the waiting time stamp. 17. The content editing system according to claim 16, wherein the third content data is input only when a waiting time stamp and a time stamp added to the third content data match.