JP4318013B2 - Content editing apparatus, content editing method, program storage medium, and content editing system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a content editing apparatus, a content editing method, a program storage medium, and a content editing system. For example, real-time performance is required by guaranteeing synchronization of content data transmitted from each remote location via the Internet. For example, the present invention is suitable for application to a content editing apparatus that edits content such as music data, a content editing method, a program storage medium, and a content editing system.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, personal computers have been able to easily execute editing processing for various content data such as music data and video data as processing capability has been improved, and the user's various editing processing requests have been sufficiently fulfilled. Has been realized.
[0003]
Also, the personal computer can execute digital signal processing parts such as a mixer and a recorder necessary for recording, for example, by software by improving the processing capability, and can easily edit content data. It is made like that.
[0004]
Accordingly, the personal computer takes in various audio data such as guitar audio data, bass audio data, piano audio data, and drum audio data via a plurality of microphones connected to the personal computer, and incorporates these. Recording data of 2ch (R channel and L channel) can be generated by recording once on a hard disk by a recorder constituted by software, adjusting to a desired sound quality by a mixer and then synthesizing.
[0005]
[Problems to be solved by the invention]
By the way, in a personal computer having such a configuration, for example, when it is necessary to perform mixing adjustment using a plurality of types of mixer functions at the same time, due to the limit of the processing capacity of a CPU (Central Processing Unit), There is a problem in that other recorders and effectors cannot be operated, and recording data having a desired sound quality cannot be generated.
[0006]
In the personal computer, audio data is exchanged between the personal computer installed in a studio in Tokyo, for example, and other personal computers installed in New York, London, and Paris, respectively. When 2ch recording data is generated and recorded in the studio, the recorded drum audio data is first played from the Tokyo studio to other personal computers installed in New York, London and Paris. Then transmit.
[0007]
Other personal computers installed in New York played drum sounds based on the drum audio data transmitted from Tokyo personal computers and output them from the speakers. The guitar audio data is taken in via a microphone and sent back to a personal computer installed in a studio in Tokyo via the Internet.
[0008]
Similarly, other personal computers installed in London and Paris also reproduce the drum performance sound based on the drum audio data transmitted from the Tokyo personal computer and output it from the speaker. The piano and bass audio data played together are fetched through a microphone and sent back to a personal computer installed in a studio in Tokyo via the Internet.
[0009]
The personal computer installed in the Tokyo studio captures the guitar, piano, and bass audio data sent back from the other personal computers installed in New York, London, and Paris, respectively. The audio data has a round-trip transmission path delay and a delay corresponding to the signal processing time in another personal computer, and these delay times are unpredictable.
[0010]
Therefore, the personal computer in Tokyo uses the drum performance sound obtained by playing back the audio data of the drum and the performance sound of the guitar, piano and bass sent back from other personal computers in New York, London and Paris. There is a problem in that it is difficult to generate recording data for music that requires real-time performance because there is a difference in timing between transmission line delays.
[0011]
The present invention has been made in consideration of the above points, and a content editing apparatus and a content editing method that can correct and correct a temporal timing shift of content data that requires real-time performance among a plurality of terminals. And a program storage medium and a content editing system.
[0012]
[Means for Solving the Problems]
In order to solve this problem, in the present invention, first content data output means for sequentially adding and outputting time stamps to first content data in units of packets, and a waiting time stamp having the same value as the time stamp are first When the waiting time stamp matches the time stamp added to the first content data after setting the input terminal, the first content data is input via the first input terminal, and the waiting time stamp is set to the second time stamp. When the time stamp in the second content data in a packet unit corresponding to the first content data to which the time stamp having the same value as the time stamp is added and the waiting time stamp match after setting to the input terminal, 2 input means for inputting content data from the outside via the second input terminal; Timing correction means for adjusting the first content data and the second content data to be temporally synchronized based on the time stamp, and the input means receives the third content data from outside during processing by the timing correction means. When additional input is performed, a waiting time stamp considering the maximum delay time until the third content data arrives is set in the third input terminal, and the set waiting time stamp and the time stamp added to the third content data are set. The third content data is input only when and match, and the timing adjustment means Added to the first content data, the second content data, and the third content data, respectively. By adjusting the first content data, the second content data, and the third content data to be temporally synchronized based on the time stamp, the first content data, the second content data, and the second content data after the maximum delay time has elapsed. The time stamps of the three content data can be adjusted so that they are all synchronized in time, and the first content data and the second content data are adjusted in time until the maximum delay time elapses. Can continue to do.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0014]
(1) Overall configuration of the remote location recording system
In FIG. 1, reference numeral 1 denotes a remote location recording system as a whole. For example, a personal computer 4 installed in New York, which is a remote location, is connected to each other via the Internet 2 for a personal computer 3 installed in Tokyo. ing.
[0015]
In this case, the personal computer 3 in Tokyo and the personal computer 4 in New York communicate with each other via the Internet 2 using the TCP / IP (Transmission Control Protocol / Internet Protocol) protocol. Content data such as video data requiring real-time properties can be transmitted and received with each other in packet units.
[0016]
This remote location recording system 1 will be described below by taking as an example a case where recording editing processing relating to audio data is executed between a personal computer 3 in Tokyo and a personal computer 4 in New York in a music production environment.
[0017]
The personal computer 3 in Tokyo stores, for example, drum audio data and piano audio data in a built-in hard disk (not shown) in advance, reads the drum and piano audio data, and transmits the drum and piano audio data via the Internet 2 in New York. The data is transmitted to the personal computer 4.
[0018]
The personal computer 4 in New York generates drum playback sound and piano playback sound based on the drum and piano audio data transmitted from the personal computer 3 in Tokyo via the Internet 2 and outputs it from the speaker 4C.
[0019]
At this time, the personal computer 4 in New York captures the audio signal of the guitar played by the performer (not shown) in accordance with the reproduced sound of the drum and piano through the microphones 4A and 4B, and then converts it into digital audio data. This is transmitted to the personal computer 3 in Tokyo via the Internet 2.
[0020]
The personal computer 3 in Tokyo automatically stores the audio data of the guitar transmitted from the personal computer 4 in New York in time with the audio data of the drum and piano, and then mixes them, for example, by CD (Compact). Recording data of 2ch (L channel and R channel) for recording on (Disc) is generated.
[0021]
By the way, when the drum and piano audio data actually reaches the personal computer 4 in New York, a delay on the transmission path has already occurred since the personal computer 3 in Tokyo transmitted the audio data of the drum and piano. Even when the guitar audio data transmitted from the computer 4 reaches the personal computer 3 in Tokyo, a delay occurs on the transmission path, and a delay corresponding to the signal processing time in the personal computer 4 also occurs.
[0022]
Therefore, when the personal computer 3 in Tokyo receives the audio data of the guitar from the personal computer 4 in New York, the transmission path delay for the round trip and the delay for the signal processing time from the time when the audio data for the drum and piano are already transmitted. Therefore, by correcting the timing shift due to the delay, 2ch recording data for recording on the CD can be generated by executing the recording process in parallel between remote locations. ing. Hereinafter, the remote location recording system 1 including delay timing correction will be described in detail.
[0023]
(1-1) Circuit configuration of personal computer
As shown in FIG. 2, the personal computer 3 controls the personal computer 3 as a whole and edits audio data for each musical instrument to generate 2ch recording data for a CPU (Central Processing Unit) 10. , Hard disk drive (HDD) 11, RAM (Random Access Memory) 12, communication interface 13 for performing data communication with New York personal computer 4 via Internet 2 using TCP / IP protocol, external input terminal 14, external output A terminal 15 and a monitor 16 are connected via a bus 17.
[0024]
In this case, for example, the microphones 3A and 3B (FIG. 1) are connected to the external input terminal 14, and the speaker 3C is connected to the external output terminal 15, for example. Therefore, the personal computer 3 can collect vocal voices via the microphones 3A and 3B and record the voices again on the recording data.
[0025]
The hard disk drive 11 stores various application programs such as an operating system program such as Windows 98 (Microsoft, trademark) and a recording processing program (to be described later) for executing remote location recording processing in the present invention. Yes.
[0026]
The CPU 10 reads various programs stored in the hard disk drive 11 as appropriate, develops them in the RAM 12 and executes them to execute various processes, and displays the processing results on the monitor 16.
[0027]
The personal computer 4 has the same circuit configuration as that of the personal computer 3, and the added numbers (210 to 217) are only different from those of the personal computer 3, so that the description thereof is omitted here.
[0028]
(2) Remote location recording process
In the remote location recording system 1 (FIG. 1) in this case, a case where the remote location recording process is controlled mainly with the personal computer 3 in Tokyo as an example will be described.
[0029]
Incidentally, in the remote location recording system 1, the remote location recording process can be controlled not by the personal computer 3 in Tokyo but by the personal computer 4 installed in New York, which is a master and which is a slave. Even if it becomes.
[0030]
(2-1) Remote location connection processing
First, the personal computer 3 in FIG. 2 (FIG. 2) reads a recording processing program from the hard disk of the HDD 11 under the control of the CPU 10 when given an instruction to perform remote area recording processing after being activated in response to the power button being turned on. And expanded in the RAM 12.
[0031]
Here, as shown in FIG. 3, the recording processing program 30 includes a work space 31, an IO (In-Out) manager 32, DSP (Digital Signal Processor) plug-ins 33A to 33N, and GUI (Graphical User Interface) plug-ins 34A to 34A. Four types of objects called 34N are hierarchically linked to each other.
[0032]
Therefore, in the personal computer 3, each object of the work space 31, the IO manager 32, the DSP plug-ins 33A to 33N, and the GUI plug-ins 34A to 34N executes predetermined processing, and the processing result is displayed on the monitor 16. ing.
[0033]
The New York personal computer 4 similarly has a recording processing program 230, and four types of objects called a workspace 231, an IO manager 232, DSP plug-ins 233A to 233N, and GUI plug-ins 234A to 234N are hierarchical. In general, they are configured in cooperation with each other.
[0034]
Incidentally, the DSP plug-ins 33A to 33N and 233A to 233N respectively correspond to the number of digital signal processing functions held by the personal computers 3 and 4, and the GUI plug-ins 34A to 34N and 234A to 234N are DSP plug-ins. Objects for adjusting the parameters 33A to 33N and 233A to 233N, each corresponding one to one.
[0035]
Here, in the present embodiment, in order to explain the following description as the relationship between the objects, the workspaces 31 and 231, the IO managers 32 and 232, the DSP plug-ins 33A to 33N and 233A to 233N, the GUI The plug-ins 34 </ b> A to 34 </ b> N and 234 </ b> A to 234 </ b> N will be mainly described, but in reality, the CPUs 10 and 210 are all executing various processes.
[0036]
The workspace 31 controls the loading and unloading of the DSP plug-ins 33A to 33N, and the DSP plug-ins 33A to 33N or the DSP plug-ins 233A to 233N held by the DSP plug-ins 33A to 33N and the personal computer 4 in New York. Management and control of loading and unloading of the GUI plug-ins 34A to 34N, and display control of the connection status of the DSP plug-ins 33A to 33N and the DSP plug-ins 233A to 233N as a whole. Has been made.
[0037]
That is, the work space 31 is a connection of the entire DSP plug-ins 33A to 33N and DSP plug-ins 233A to 233N created based on the plug-in list regarding the currently loaded DSP plug-ins 33A to 33N and the inquiry results to the IO managers 32 and 232. The connection state data representing the state and the work space ID (Identification) unique to the work space 31 are held, and the above-described display control is performed based on these various data.
[0038]
The IO manager 32 has individual plug-in data for the DSP plug-ins 33A to 33N being loaded and an IO manager ID unique to the IO manager 32 itself. The connection management of the DSP plug-ins 33A to 33N and the DSP plug-in 33A Data management for each packet processed by .about.33N is performed, and the connection management status regarding the DSP plug-ins 33A to 33N is notified to the work space 31 at predetermined time intervals.
[0039]
In this case, the plug-in data includes the manufacturer name of the loaded DSP plug-in, the type identification information indicating the default GUI plug-in associated with the DSP plug-in in advance, and the DSP plug-in itself. Plug-in ID, hierarchical level of the DSP plug-in itself (in order from the top, 0, 1, 2,...), Output destination list indicating the output destination of the DSP plug-in, input destination indicating the input destination for the DSP plug-in A list is included.
[0040]
Further, the output destination list includes an output ID corresponding to the output terminal of the DSP plug-in, an output terminal hierarchy level (0, 1, 2,... In order from the top) and connection destination information (hardware ID, plug-in ID, and Input ID corresponding to the input terminal), and the input destination list includes an input ID corresponding to the input terminal of the DSP plug-in, an input terminal hierarchy level (0, 1, 2,... A valid flag indicating whether the input is valid or invalid is included.
[0041]
In this way, the IO manager 32 can recognize the connection management state of the DSP plug-ins 33A to 33N in detail by holding the plug-in data, and can also input / output the audio data in units of packets to each DSP plug-in 33A. It is designed to be able to accurately indicate ~ 33N.
[0042]
The DSP plug-ins 33A to 33N are loaded or unloaded by the work space 31, and perform predetermined digital signal processing on the audio data in units of packets supplied from the connected DSP plug-in in the previous stage. The processed data is transmitted to the DSP plug-in at the subsequent stage, which is a predetermined transmission destination, in accordance with an instruction from the IO manager 32.
[0043]
Here, the DSP plug-ins 33A to 33N are configured to operate on independent memory spaces, so that even if the DSP plug-in 33A operating on a certain memory space stops functioning, The influence is not exerted on the other DSP plug-ins 33B to 33N.
[0044]
The GUI plug-ins 34 </ b> A to 34 </ b> N include a list in the lower level DSP plug-in of the DSP plug-in corresponding to each one-to-one, a connection status data list in the lower level DSP plug-in, parameter adjustment data regarding parameters of the DSP plug-in, and GUI plugs. The plug-in has its own GUI plug-in ID and is loaded or unloaded by the work space 31 so that parameters relating to various functions in the DSP plug-ins 33A to 33N can be adjusted.
[0045]
When the DSP plug-ins 33A to 33N are configured by a plurality of lower level DSP plug-ins located in the lower hierarchy, the lower level GUI plug-ins are loaded or unloaded by the higher level GUI plug-ins 34A to 34N. Has been made.
[0046]
In practice, the DSP plug-ins 33A to 33N include an audio input plug-in as an analog / digital conversion processing function for an audio signal collected by the microphones 3A and 3B connected via the external input terminal 14, and an external output terminal 15 Audio output plug-in as a digital / analog conversion processing function for audio data for outputting performance sound via a speaker 3C connected via the effector plug as an effector function for giving a predetermined sound quality effect to the audio data In, there are a recorder plug-in as a recorder function for reproducing and recording audio data, a mixer plug-in as a mixer function for mixing, and the like.
[0047]
The workspace 31 displays plug-in icons (to be described later) corresponding to these various DSP plug-ins 33A to 33N on the monitor 16, and each plug-in icon is displayed on the screen of the monitor 16. By connecting according to the user's specification, various recording plug-ins 33A to 33N and 233A to 233N can be combined to generate recording data of desired sound quality.
[0048]
That is, in the recording processing program 30, the work space 31 first displays the work space screen 20 as shown in FIG. The workspace screen 20 includes an almost central IO manager display unit 21, a tool box 22 displaying a plug-in list of a plurality of various DSP plug-ins 33A to 33N, and playback, stop, fast reverse, fast forward, pause, etc. Transport windows 23 for performing various operations are provided.
[0049]
Incidentally, nothing is displayed on the IO manager display unit 21 on the workspace screen 20 at this time, indicating that none of the DSP plug-ins 33A to 33N is selected.
[0050]
The user selects the select button 22X from the tool box 22 on the workspace screen 20 and, for example, the DSP plug-ins 33A and 33B shown in FIG. 3 (hereinafter referred to as the recorder plug-in 33A and the mixer plug-in). When the IO manager display unit 21 is clicked after clicking the plug-in buttons 22A and 22B corresponding to (33B), the workspace 31 reads and loads the recorder plug-in 33A and the mixer plug-in 33B from the hard disk of the HDD 11. At the same time, as shown in FIG. 5, the recorder plug-in icon 40A and the mixer plug-in icon 40B corresponding to the recorder plug-in 33A and the mixer plug-in 33B are displayed on the IO manager display unit 21.
[0051]
In this case, the recorder plug-in icon 40A is provided with 6ch input terminals (indicated by ◯) on the left side and 6ch output terminals (indicated by ◯) on the right side. 6ch input terminals (indicated by circles) corresponding to the output terminal of the recorder plug-in icon 40A and 2ch output terminals (indicated by circles) for outputting the final recording data of 2ch. Is provided.
[0052]
Next, when the user clicks the connect button 22Y in the tool box 22 on the workspace screen 20 and drags and drops it from the output terminal of the recorder plug-in icon 40A to the input terminal of the mixer plug-in icon 40B, the IO manager 32 is displayed. The output of the recorder plug-in 33A and the input of the mixer plug-in 33B are connected, and the connection source output terminal (indicated by a circle) of the recorder plug-in icon 40A of the IO manager display unit 21 and the connection destination of the mixer plug-in icon 40B Connect and display the input terminal (indicated by a circle) on the screen.
[0053]
Here, the recorder plug-in 33A refers to a playback system that performs playback processing on drum and piano audio data recorded on the hard disk of the HDD 11, and recording processing on guitar audio data transmitted from an external personal computer 4 in New York. And a recording system for performing.
[0054]
In this case, the recorder plug-in 33A transmits the L channel and the R channel transmitted from the personal computer 4 in New York via the Internet 2 via the 2ch located at the bottom of the 6ch input terminals of the recorder plug-in icon 40A. The guitar audio data can be input and recorded.
[0055]
Also, the mixer plug-in 33B, as indicated by the mixer plug-in icon 40B, inputs the audio data for a total of 6 channels output from the recorder plug-in 33A and mixes them, whereby final 2ch recording data is recorded. It can be output.
[0056]
Next, when the user clicks the connection button 25 provided at the upper left corner of the work space screen 20, the work space 31 enters an IP address input screen (not shown) for inputting a connection destination IP (Internet Protocol) address. ) Overlaid on the workspace screen 20.
[0057]
When the user inputs the IP address of the New York personal computer 4 desired to be connected to the IP address input screen, the work space 31 connects the Tokyo personal computer 3 and the New York personal computer 4 via the Internet 2 over the network. As shown in FIG. 6, an IO manager display unit 27 corresponding to the personal computer 4 in New York is displayed over the IO manager display unit 21.
[0058]
As a result, the work space 31 (FIG. 3) is connected to the IO manager 232 of the recording processing program 230 in the personal computer 4 in New York via the Internet 2.
[0059]
In the workspace screen 20 in this state, the IO manager display unit 27 is a currently active window, and the IO manager display unit 21 is an inactive window.
[0060]
Subsequently, the user selects the select button 22X from the tool box 22 on the workspace screen 20, and plug-in buttons 22C corresponding to, for example, the audio output plug-in 233A and the audio input plug-in 233B (FIG. 3). When the IO manager display unit 27 is clicked, the workspace 31 sends the audio output plug-in 233A and the audio input plug-in 233B to the IO manager 232 in the New York personal computer 4 side. The start command to start up is transmitted via the Internet 2.
[0061]
As a result, the workspace 31 activates the audio output plug-in 233A and the audio input plug-in 233B on the personal computer 4 side in New York, and also outputs the audio output plug-in icon corresponding to the audio output plug-in 233A and the audio input plug-in 233B. 41A and the voice input plug-in icon 41B are displayed on the IO manager display unit 27.
[0062]
Thereafter, the user clicks the connect button 22Y in the tool box 22, and the 2ch portion located at the lowest position of the recorder plug-in icon 40A from the connection source output terminal (indicated by a circle) of the audio input plug-in icon 41B. Drag and drop to the connection destination input terminal (indicated by a circle), and from the connection source output terminal (indicated by a circle) of the mixer plug-in icon 40B to the connection destination input terminal of the audio output plug-in icon 41A (indicated by a circle) When dragged and dropped, the IO manager 32 connects the recorder plug-in 33A and the mixer plug-in 33B, and the audio output plug-in 233A and the audio input plug-in 233B to each other via the Internet 2.
[0063]
At this time, the work space 31 includes a recorder plug-in icon 40A and a mixer plug-in icon 40B on the IO manager display unit 21, and an audio output plug-in icon 41A and an audio input plug-in icon on the IO manager display unit 27 as shown in FIG. 41B is connected and displayed on the workspace screen 20 to display the recorder plug-in 33A and mixer plug-in 33B of the personal computer 3 in Tokyo and the audio output plug-in 233A and audio input of the personal computer 4 in New York. The user can visually recognize that the plug-in 233B is connected to a remote place.
[0064]
As a result, the audio data of the 2ch guitar transmitted from the audio input plug-in 233B in the personal computer 4 in New York via the Internet 2 is input to the 2ch input terminal located at the lowest position of the recorder plug-in 33A. The 2ch input terminal of the audio output plug-in 233A is connected to the 2ch output terminal of the mixer plug-in 33B.
[0065]
Thereby, the remote location recording system 1 reproduces the drum and piano audio data reproduced by the recorder plug-in 33A in the personal computer 3 in Tokyo and mixed in the mixer plug-in 33B, and the audio output plug-in 233A in the personal computer 4 in New York. And output as drum and piano performance sounds from a speaker (not shown) via the audio output plug-in 233A.
[0066]
Then, the remote area recording system 1 takes in the audio signal of the guitar played by the personal computer 4 in New York in accordance with the performance sound of the drum and piano to the audio input plug-in 233B via the microphones 4A and 4B, and inputs the audio input It returns to the recorder plug-in 33A in the personal computer 3 in Tokyo as audio data of the guitar via the plug-in 233B.
[0067]
The personal computer 3 in Tokyo stores the audio data of the guitar by the recording system of the recorder plug-in 33A so as to synchronize with the drum and piano audio data read and reproduced from the hard disk of the HDD 11 by the playback system of the recorder plug-in 33A. As a result, it is possible to execute a storage process in which a deviation between the reproduction timing and the recording timing of the recorder plug-in 33A due to a transmission path delay between remote locations is corrected.
[0068]
Further, the remote location recording system 1 remotely connects the recorder plug-in 33A and the mixer plug-in 33B in the personal computer 3 in Tokyo, and the audio output plug-in 233A and the audio input plug-in 233B started up in the personal computer 4 in New York. By connecting to the ground, the recorder function, mixer function, voice output function and voice input function are distributed to the personal computer 3 in Tokyo and the personal computer 4 in New York when viewed as the whole system.
[0069]
Thus, the remote location recording system 1 distributes various functions even if only one of the personal computer 3 in Tokyo or the personal computer 4 in New York could not be processed due to the limit of the processing capacity of the CPUs 10 and 210. As a result, it has become possible to execute the system as a whole because the overall processing capability has been improved.
[0070]
The IO manager 32 dynamically loads and unloads the audio output plug-in 233A and the audio input plug-in 233B started up by the mixer plug-in 33B and the personal computer 4 in New York, even while the audio data is being reproduced by the recorder plug-in 33A. Thus, the remote location recording system 1 can be constructed by freely changing the combination of various plug-ins at any time and combining the functions desired by the user.
[0071]
(2-2) Timing correction processing
Next, the timing correction processing for correcting the deviation between the playback timing and the recording timing of the recorder plug-in 33A due to a transmission path delay between remote locations in the remote location recording system 1 will be described in detail.
[0072]
(2-2-1) Timing correction processing for transmission line delay
In the remote location recording system 1 described above, when a personal computer 3 in Tokyo and a personal computer 4 in New York are connected to the network via the Internet 2, an inter-transmission path delay always occurs between the networks. In order to store the guitar audio data sent from the personal computer 4 in the hard disk of the HDD 11 so as to be synchronized with the drum and piano audio data, it is necessary to correct the timing shift due to the delay between the transmission paths.
[0073]
In practice, as shown in FIG. 8, the recorder plug-in 33A and the mixer plug-in 33B are supplied with the master clock MCLK1 generated by the internal real-time clock from the IO manager 32 for each packet period, and are also connected to the audio output plug. The master clock MCLK2 generated by the internal real-time clock is also supplied from the IO manager 232 for each packet cycle to the IN 233A and the audio output plug-in 233B.
[0074]
In this case, the master clock MCLK1 that the IO manager 32 supplies to the recorder plug-in 33A and the mixer plug-in 33B is synchronized with the master clock MCLK2 that the IO manager 232 supplies to the audio output plug-in 233A and the audio output plug-in 233B. The master clocks MCLK1 and MCLK2 that are common to the remote locations are supplied at the same time.
[0075]
That is, the recorder plug-in 33A reproduces drum and piano audio data using the reproduction system, and 2ch (L channel and R channel) drum packet data DrPD1 (L) and DrPD2 (R) obtained as a result of the reproduction. Time stamp (TS) “0000” based on the master clock MCLK1 is added to the piano packet data PiPD1 (L) and PiPD2 (R) under the control of the IO manager 32, respectively, and the next drum packet data DrPD1, DrPD2 In addition, a time stamp “0001” is added to the piano packet data PiPD1 and PiPD2, and the drum packet data DrPD to which the time stamp is added is added by sequentially incrementing the time stamp value in units of packets. Generates DrPD2 and piano packet data PiPD1, PiPD2, and outputs it to a sequential mixer plug 33B.
[0076]
The IO manager 32 recognizes the connection state to the recorder plug-in 33A, the mixer plug-in 33B, the audio output plug-in 233A, and the audio input plug-in 233B based on the plug-in data, and the drum packet data DrPD1, DrPD2, and the piano The packet data PiPD1 and PiPD2 are instructed for connection destination information indicating a destination to be transmitted next.
[0077]
Accordingly, the recorder plug-in 33A, the mixer plug-in 33B, the audio output plug-in 233A, and the audio input plug-in 233B add connection destination information based on an instruction from the IO manager 32, and set a predetermined connection destination according to the connection destination information. Packet data can be accurately transmitted to the DSP plug-in.
[0078]
At the same time, the IO manager 32 waits for the input of various packet data of the time stamp “0000” at the input of 4ch of the mixer plug-in 33B connected to the output of the recorder plug-in 33A, and waits for the time stamp “0000”. Set.
[0079]
In the mixer plug-in 33B, the drum packet data DrPD1 and DrPD2 to which the time stamp “0000” is added from the recorder plug-in 33A and the piano packet data PiPD1 and PiPD2 reach the input for 4ch in which the wait time stamp “0000” is set. Then, mixing processing is started when data reaches all inputs.
[0080]
Then, the mixer plug-in 33B adds the time stamp “0000” to the mixing data MIXPD1 and MIXPD2 generated by performing the mixing process under the control of the IO manager 32, and mixes the time stamp “0000”. Data MIXPD1 and MIXPD2 are transmitted to the audio output plug-in 233A of the personal computer 4 in New York via the Internet 2.
[0081]
At this time, the work space 31 receives the mixing data MIXPD1 of the time stamp “0000” at the input of 2ch of the audio output plug-in 233A connected to the output of the mixer plug-in 33B via the IO manager 232 of the personal computer 4. A wait time stamp “0000” is set so as to wait for the input of MIXPD2.
[0082]
Thereafter, the IO manager 32 sets the waiting time stamp “0001” so as to wait for the input of various packet data of the time stamp “0001” with the input of 4 channels of the mixer plug-in 33B, and is sequentially output from the recorder plug-in 33A. The wait time stamp value is incremented in accordance with various packet data.
[0083]
When the mixing data MIXPD1 and MIXPD2 of the time stamp “0000” reach the input set to the waiting time stamp “0000” from the mixer plug-in 33B, the audio output plug-in 233A mixes the mixing data MIXPD1 of the time stamp “0000”. The MIXPD 2 is converted into a digital / analog signal and then output as drum and piano performance sounds via a speaker (not shown).
[0084]
Actually, the audio output plug-in 233A sequentially performs buffering when the input of the mixing data MIXPD1 and MIXPD2 to which the same time stamp is added, and designates timing based on the master clock MCLK2 supplied from the IO manager 232 Are sequentially read out and digital-analog converted, and then output through a speaker.
[0085]
As described above, the audio output plug-in 233A performs the buffering of the mixing data MIXPD1 and MIXPD2 and outputs them at a specified timing, so that even when there is a supply interval of the mixing data MIXPD1 and MIXPD2 supplied from the mixer plug-in 33B. Can be prevented from being interrupted on the way.
[0086]
Thereafter, the IO manager 232 sets the waiting time stamp “0001” so as to wait for the input of various packet data of the next time stamp “0001” at the input of the audio output plug-in 233A for two channels, and sequentially the mixer plug-in 33B. The wait time stamp value is incremented in accordance with the mixing data MIXPD1 and MIXPD2 supplied from.
[0087]
The audio input plug-in 233B converts the audio signal of the guitar played in accordance with the drum and piano performance sound output from the audio output plug-in 233A through the speaker into digital audio data and packetizes it. The guitar packet data GiPD1 and GiPD2 are generated by adding the stamp “0000” and transmitted to the recorder plug-in 33A of the personal computer 3.
[0088]
That is, the audio input plug-in 233B outputs the guitar packet data GiPD1 and GiPD2 from the time when the first drum packet data DrPD1 and DrPD2 and piano packet data PiPD1 and PiPD2 of the time stamp “0000” are output from the recorder plug-in 33A. Since the time is already delayed by the transmission path delay, adding the time stamp “0000” to the guitar packet data GiPD1 and GiPD2 allows the drum packet data DrPD1, DrPD2 and piano of the time stamp “0000” to be added. The packet data PiPD1 and PiPD2 and the guitar packet data GiPD1 and GiPD2 with the time stamp “0000” indicate audio data at the same timing. It has been made.
[0089]
The IO manager 32 waits for input of guitar packet data GiPD1 and GiPD2 in which the time stamp “0000” is added to the input of the recording system in the recorder plug-in 33A connected to the output of the audio input plug-in 233B. Set “0000”.
[0090]
Accordingly, when the guitar packet data GiPD1 and GiPD2 with the time stamp “0000” reach the input of the waiting time stamp “0000”, the recorder plug-in 33A causes the drum packet data DrPD1, DrPD2 with the time stamp “0000” and the piano packet data PiPD1. The guitar packet data GiPD1 and GiPD2 with the time stamp “0000” are controlled to be stored in the hard disk of the HDD 11 so as to be synchronized with the PiPD2.
[0091]
Thus, the recorder plug-in 33A corrects a timing shift between the drum and piano audio data and the guitar audio data caused by the transmission path delay, and the guitar audio data is synchronized with the drum and piano audio data. It is designed to be memorable.
[0092]
(2-2-2) Timing correction process when a new input is added during the recording process
By the way, in the remote location recording system 1, for example, as shown in FIG. 9A, the IO manager 32 waits for 2ch input in the mixer plug-in 33B to wait for drum packet data DrPD1 and DrPD2 of the time stamp “0002”. When the input of piano packet data PiPD1 and PiPD2 is newly added from the recorder plug-in 33A in the previous stage with the time stamp “0002” set, for example, the piano packet of the time stamp “0006” is added to the newly added input. A waiting time stamp “0006” is set so as to wait for the data PiPD1 and PiPD2.
[0093]
The IO manager 32 sets the waiting time stamp “0006” so as to wait for the piano packet data PiPD1 and PiPD2 having the time stamp “0006” to be input after the piano packet data PiPD1 and PiPD2 are input. In fact, the maximum delay time required for the piano packet data PiPD1 and PiPD2 to arrive is expected.
[0094]
That is, when the drum packet data DrPD1 and DrPD2 of the time stamp “0002” are input to the mixer plug-in 33B from the recorder plug-in 33A in the previous stage, the IO manager 32 performs time stamping as shown in FIG. 9B. Since the mixing data MIXPD1 and MIXPD2 of “0002” are output, the value of the waiting time stamp for the drum packet data DrPD1 and DrPD2 is incremented to the time stamp “0003”.
[0095]
Thereafter, when the drum packet data DrPD1 and DrPD2 of the time stamp “0003” are input to the mixer plug-in 33B from the recorder plug-in 33A of the previous stage, the IO manager 32 mixes the mixing data MIXPD1 of the time stamp “0003” and Since MIXPD2 is output, the drum packet data DrPD1 and DrPD2 are incremented to the waiting time stamp “0004” and sequentially incremented to the time stamp “0006”.
[0096]
That is, the IO manager 32 waits until the piano packet data PiPD1 and PiPD2 of the time stamp “0006” reach the input in which the wait time stamp “0006” of the mixer plug-in 33B is set. The mixing process for the drum packet data DrPD1 and DrPD2 of “0005” is continuously executed by the mixer plug-in 33B, and a new input is added to the mixing process for the drum packet data DrPD1 and DrPD2. It is made to be able to prevent stagnation.
[0097]
As shown in FIG. 9C, when the IO manager 32 eventually sets the waiting time stamp “0006” for the drum packet data DrPD1 and DrPD2 at the input of the mixer plug-in 33B, all inputs for four channels are made. Thus, the drum packet data DrPD1 and DrPD2 with the time stamp “0006” and the piano packet data PiPD1 and PiPD2 with the time stamp “0006” are awaited.
[0098]
The IO manager 32 inputs the drum packet data DrPD1 and DrPD2 with the time stamp “0006” and the piano packet data PiPD1 and PiPD2 with the time stamp “0006” from the recorder plug-in 33A in the previous stage to the mixer plug-in 33B. When the mixing data MIXPD1 and MIXPD2 of “0006” are output, the input for 4 channels is incremented to the waiting time stamp “0007” as shown in FIG. 9D, and thereafter the mixing processing is sequentially performed for the inputs for 4 channels. It is made to do.
[0099]
As described above, when the input of the piano packet data PiPD1 and PiPD2 is newly added from the recorder plug-in 33A in the previous stage during the recording process, the IO manager 32 actually performs the previous stage on the newly added input. By setting a waiting time stamp that anticipates the maximum delay time required for the piano packet data PiPD1 and PiPD2 to reach the mixer plug-in 33B from the recorder plug-in 33A, all inputs for 4ch are surely received after the maximum delay time has elapsed. The mixing process for 4ch can be executed at the same time, and the mixing process for the drum packet data DrPD1 and DrPD2 can be continuously executed without delay until the input for 4ch is completed.
[0100]
Incidentally, even when the piano packet data PiPD1 and PiPD2 of the time stamp “0005” arrive at the input of the waiting time stamp “0006” set by the mixer plug-in 33B, the IO manager 32 receives the piano packet of the time stamp “0005”. The input of data PiPD1 and PiPD2 is not accepted, but it waits for piano packet data PiPD1 and PiPD2 of time stamp “0006” to arrive.
[0101]
In other words, the IO manager 32 sets the wait time stamp “0006” that allows for the maximum delay time, and waits for the piano packet data PiPD1 and PiPD2 of the time stamp “0006”, so that the wait time shorter than the maximum delay time is assumed. When the stamp “0003” is set, the piano packet data PiPD1 and PiPD2 of the time stamp “0006” arrive and the situation where the mixing process cannot be executed forever because the input for 4ch is not complete is prevented. It is made to be able to do.
[0102]
(2-2-3) Timing correction processing when input is canceled during recording processing
On the other hand, in the remote location recording system 1, for example, as shown in FIG. 10A, the mixer plug-in 33B is outputting the mixing data MIXPD1 and MIXPD2 of the time stamp “0016” and the recorder plug-in 33A in the previous stage. The timing correction processing when the input of the piano packet data PiPD1 and PiPD2 is canceled due to the partial connection cancellation will be described.
[0103]
In this case, as shown in FIG. 10B, the IO manager 32, when the connection with the recorder plug-in 33A is canceled, the piano packet data PiPD1 with the time stamp “0017” already reached from the recorder plug-in 33A. And PiPD2 exist, after mixing them, the waiting time stamp “0018” for the input of the next piano packet data PiPD1 and PiPD2 is set to “invalid”.
[0104]
Thereby, the mixer plug-in 33B invalidates the piano packet data PiPD1 and PiPD2 of the next time stamp “0018” without accepting the input, and performs the mixing process only on the drum packet data DrPD1 and DrPD2 of the time stamp “0018”. Has been made to do.
[0105]
Therefore, the IO manager 32 thereafter outputs the mixing data MIXPD1 and MIXPD2 of the time stamp “0018” as shown in FIG. 10C, and “invalid” the waiting time stamp for the input of the piano packet data PiPD1 and PiPD2. With this setting, only the waiting time stamp for the input of the drum packet data DrPD1 and DrPD2 is sequentially incremented.
[0106]
In this way, during the recording process, when the input of the piano packet data PiPD1 and PiPD2 is canceled due to partial connection cancellation between the recorder plug-in 33A and the mixer plug-in 33B in the previous stage, the IO manager 32 at that time The piano packet data PiPD1 and PiPD2 already sent from the recorder plug-in 33A to the mixer plug-in 33B are mixed, but the piano packet data PiPD1 and PiPD2 sent thereafter are not accepted. By disabling, it is possible to smoothly shift to the mixing processing for only the drum packet data DrPD1 and DrPD2 without delay from the connection cancellation time point.
[0107]
(2-2-4) Timing correction processing when packet data after the waiting time stamp packet data is sent
Next, in the remote location recording system 1, for example, as shown in FIG. 11A, the mixer plug-in 33B is waiting for the drum packet data DrPD1 and DrPD2 of the time stamp “0021”, and inputs one channel. The timing correction process when the drum packet data DrPD2 with the time stamp “0022” is sent from the preceding recorder plug-in 33A will be described.
[0108]
In this case, as shown in FIG. 11B, the mixer plug-in 33B receives the drum packet data DrPD2 with the time stamp “0021” but receives the waiting time stamp “0021” with respect to the drum packet data DrPD2 with the time stamp “0022”. ”Is not matched, and the receipt is refused and sent back to the IO manager 32.
[0109]
The IO manager 32 once buffers the drum packet data DrPD2 with the time stamp “0022”, sends the drum packet data DrPD2 with the time stamp “0021” to the mixer plug-in 33B, and then the drum packet with the time stamp “0022”. The packet data DrPD2 is retransmitted to the mixer plug-in 33B.
[0110]
Incidentally, at this time, the mixer plug-in 33B does not execute the mixing process until the drum packet data DrPD1 and DrPD2 of the time stamp “0021” are prepared, and there is a timing shift between the drum packet data DrPD1 and the drum packet data DrPD2. Is surely prevented.
[0111]
In this way, the IO manager 32, when the drum packet data DrPD2 of the time stamp “0022” after the waiting time stamp “0021” is rejected and sent back by the mixer plug-in 33B, the time stamp “0022”. The drum packet data DrPD2 of the time stamp is temporarily buffered, the drum packet data DrPD2 of the time stamp “0021” is sent to the mixer plug-in 33B, and then the drum packet data DrPD2 of the time stamp “0022” is retransmitted to the mixer plug-in 33B. Thus, the mixer plug-in 33B can be controlled to execute the mixing process without causing the packet data to be out of order.
[0112]
(2-2-5) Parameter adjustment processing of DSP plug-in by GUI plug-in In the work space 31, for example, when the mixer plug-in icon 40B is double-clicked, the mixer GUI provided corresponding to the mixer plug-in 33B The plug-in 34B is loaded, and a mixer GUI plug-in screen 50 as shown in FIG. 12 is displayed on the IO manager display unit 21 based on the mixer GUI plug-in 34B.
[0113]
The mixer GUI plug-in screen 50 in this case is a GUI screen for adjusting the volume of audio data input to the mixer plug-in 33B for each instrument by two channels, and the volume value ("0db") by the user. The volume of the mixer plug-in 33B that is the operation target can be adjusted in accordance with the setting.
[0114]
When the audio output plug-in icon 41A is double-clicked, the workspace 31 sends a request command for the audio output GUI plug-in 234A corresponding to the audio output plug-in 233A to the personal computer 4 in New York via the Internet 2. Based on the audio output GUI plug-in 234A transmitted and received from the personal computer 4, the audio output GUI plug-in screen can be displayed on the IO manager display unit 27 in an overlapping manner.
[0115]
As a result, the work space 31 can remotely control the parameters related to the audio output plug-in 233A being activated on the personal computer 4 in New York based on the audio output GUI plug-in screen.
[0116]
As described above, in the remote location recording system 1, the DSP plug-ins 33A to 33N and 233A to 233N and the GUI plug-ins 34A to 34N and 234A to 234N are separately stored in the personal computers 3 and 4, for example, The personal computer 3 can receive the GUI plug-in 234A of the personal computer 4 and remotely control the DSP plug-in 33A via the GUI plug-in 234A. Thus, remote control across the network can be easily executed. it can.
[0117]
Further, the user can input processing instructions for audio data reproduction processing and recording processing by operating the transport window 23 (FIG. 7), whereby the IO manager 32 responds to the input processing instructions. Thus, the recorder plug-in 33A, the mixer plug-in 33B, the audio output plug-in 233A, and the audio input plug-in 233B are operated in an interlocking manner so that the reproduction process and the recording process can be executed.
[0118]
(2-3) Recording procedure
Next, when the above-described recording processing procedure performed by the personal computer 3 according to the recording processing program 30 is summarized using the flowchart of FIG. 13, the personal computer 3 enters from the start step of the routine RT1 and loads the DSP plug-in 33. Move on to subroutine SRT2.
[0119]
(2-3-1) DSP plug-in loading procedure
As shown in FIGS. 14 and 15, in step SP1 of the subroutine SRT2, the work space 31 is clicked by the user on the select button 22X and the plug-in buttons 22A and 22B from the tool box 22 on the work space screen 20 (FIG. 3). Then, the DSP plug-ins 33A and 33B read from the hard disk of the HDD 11 together with the load request commands of the DSP plug-in (recorder plug-in) 33A and DSP plug-in (mixer plug-in) 33B are sent to the IO manager 32. The process proceeds to the next step SP2.
[0120]
In step SP2, the IO manager 32 loads the objects of the DSP plug-ins 33A and 33B into the RAM 12, and proceeds to the next step SP3.
[0121]
In step SP3, the work space 31 sends an IO manager update check command for checking the update status of the DSP plug-in 33 being loaded to the IO manager 32 at regular time intervals, and proceeds to the next step SP4.
[0122]
In step SP4, the IO manager 32 sends an update result message indicating whether or not the DSP plug-in 33 has been updated to the work space 31, and proceeds to the next step SP5.
[0123]
In step SP5, the work space 31 determines whether or not the DSP plug-in 33 has been updated based on the content of the update result message. If a negative result is obtained here, this means that the newly loaded DSP plug-in 33 does not exist and the DSP plug-in 33 has not been updated. The process proceeds to the next subroutine SRT3.
[0124]
On the other hand, if a positive result is obtained in step SP5, this indicates that a newly loaded DSP plug-in 33 exists and the DSP plug-in 33 has been updated. The space 31 moves to the next step SP6.
[0125]
In step SP6, the work space 31 sends a plug-in data request command for requesting plug-in data relating to the updated new DSP plug-in 33 to the IO manager 32, and proceeds to the next step SP7.
[0126]
In step SP7, the IO manager 32 sends the updated plug-in data relating to the new DSP plug-in 33 to the work space 31, and proceeds to the next step SP8.
[0127]
In step SP8, the workspace 31 displays, for example, the recorder plug-in icon 40A and the mixer plug-in icon 40B corresponding to the loaded new DSP plug-ins 33A and 33B on the IO manager display unit 21 based on the plug-in data. Thus, the screen of the IO manager display unit 21 is newly updated, and the process proceeds to the next subroutine SRT3.
[0128]
(2-3-2) DSP plug-in connection management processing procedure
As shown in FIGS. 16 and 17, in step SP11 of the subroutine SRT3, the work space 31 is clicked on the connect button 22Y from the tool box 22 on the work space screen 20 (FIG. 3), for example, the recorder plug-in icon 40A. When dragged and dropped from the output terminal to the input terminal of the mixer plug-in icon 40B, the connection source output and the connection destination together with connection request commands for the DSP plug-in (recorder plug-in) 33A and the DSP plug-in (mixer plug-in) 33B Various connection information including input and a connection start time stamp at the start of connection is sent to the IO manager 32, and the process proceeds to the next step SP12.
[0129]
In step SP12, the IO manager 32 executes connection processing of the DSP plug-ins 33A and 33B, and proceeds to the next step SP13.
[0130]
In step SP13, the work space 31 sends an IO manager connection update check command for checking the connection status to the DSP plug-in 33 being loaded to the IO manager 32 at regular time intervals, and proceeds to the next step SP14.
[0131]
In step SP14, the IO manager 32 sends a connection update result message indicating the presence / absence of connection update for the DSP plug-ins 33A and 33B to the work space 31, and proceeds to the next step SP15.
[0132]
In step SP15, the workspace 31 determines whether or not the connection status has been updated for the DSP plug-ins 33A and 33B based on the content of the connection update result message. If a negative result is obtained here, this means that there has been no change in the connection status regarding the DSP plug-ins 33A and 33B, and at this time, the work space 31 moves to the next subroutine SRT4.
[0133]
On the other hand, if a positive result is obtained in step SP15, this indicates that there has been some change in the connection status relating to the DSP plug-ins 33A and 33B, and at this time the workspace 31 moves to the next step SP16. .
[0134]
In step SP16, the work space 31 sends a plug-in data request command related to the DSP plug-in 33 whose connection status is newly changed to the IO manager 32, and proceeds to the next step SP17.
[0135]
In step SP17, the IO manager 32 sends plug-in data related to the DSP plug-in 33 whose connection status is newly changed to the work space 31, and proceeds to the next step SP18.
[0136]
In step SP18, the workspace 31 newly displays the plug-in icons 40A and 40B whose connection status is newly changed on the IO manager display unit 21 based on the plug-in data, thereby displaying the screen of the IO manager display unit 21. Is newly updated, and the process proceeds to the next subroutine SRT4.
[0137]
(2-3-3) DSP plug-in parameter adjustment processing procedure
As shown in FIGS. 18 and 19, when the user performs parameter adjustment on the GUI plug-in screen in step SP21 of the subroutine SRT4, the GUI plug-in 34 displays parameter control information corresponding to the parameter adjustment in the DSP plug-in. 33, and proceeds to the next step SP22.
[0138]
In step SP22, the DSP plug-in 33 adjusts a parameter (for example, volume) according to the parameter control information, and proceeds to the next step SP23.
[0139]
In step SP23, the GUI plug-in 34 sends a parameter update check command for checking whether or not parameter adjustment is updated to the DSP plug-in 33 that is the operation target at regular time intervals, and proceeds to the next step SP24.
[0140]
In step SP24, the DSP plug-in 33 sends a parameter update result message indicating whether or not the parameter adjustment changed for the DSP plug-in 33 is updated to the GUI plug-in 34, and proceeds to the next step SP25.
[0141]
In step SP25, the GUI plug-in 34 determines whether there has been a parameter adjustment change for the DSP plug-in 33 based on the content of the parameter update result message. If a negative result is obtained here, this means that the parameter adjustment for the DSP plug-in 33 has not been changed, and at this time, the GUI plug-in 34 proceeds to the next subroutine SRT5.
[0142]
On the other hand, if a positive result is obtained in step SP25, this indicates that the parameter adjustment has been changed for the DSP plug-in 33. At this time, the GUI plug-in 34 proceeds to the next step SP26.
[0143]
In step SP26, the GUI plug-in 34 sends a parameter adjustment data request command to the DSP plug-in 33 whose parameter adjustment has been changed, and proceeds to the next step SP27.
[0144]
In step SP27, the DSP plug-in 33 sends parameter adjustment data representing the contents of parameter adjustment to the GUI plug-in 34, and proceeds to the next step SP28.
[0145]
In step SP28, the CPU 10 updates the GUI plug-in screen by displaying the GUI plug-in screen in which the parameters are newly changed based on the parameter adjustment data, and proceeds to the next subroutine SRT5.
[0146]
(2-3-4) Timing correction processing procedure
As shown in FIG. 20, in step SP31 of the subroutine SRT5, the IO manager 32 sets the time to the drum packet data DrPD1, DrPD2 and piano packet data PiPD1, PiPD2 reproduced by the reproduction system of the recorder plug-in 33A as shown in FIG. The stamp “0000” is added and output, and the waiting time stamp “0000” is set to the input of the recording system in the mixer plug-in 33B, the audio output plug-in 233A, and the recorder plug-in 33A (FIG. 8) after the next stage. Then, the process proceeds to the next step SP32.
[0147]
In step SP32, the IO manager 32 adds a time stamp “0000” to the mixing data MIXPD1 and MIXPD2 output from the mixer plug-in 33B in the subsequent stage, and also adds to the guitar packet data GiPD1 and GiPD2 output from the audio input plug-in 233B. Also adds a time stamp “0000”, and proceeds to the next step SP33.
[0148]
In step SP33, the IO manager 32 plays back the drum packet data DrPD1 with the time stamp “0000” reproduced from the playback system of the guitar packet data GiPD1 and GiPD2 with the time stamp “0000” input from the audio input plug-in 233B to the recorder plug-in 33A. Then, the timing shift due to the transmission path delay is corrected by controlling the storage on the hard disk of the HDD 11 based on the time stamp “0000” so as to be synchronized with the DrPD2 and the piano packet data PiPD1 and PiPD2, and the process proceeds to the next step SP34.
[0149]
In step SP34, the IO manager 32 determines whether or not a new input is added to the mixer plug-in 33B, for example, as shown in FIG. If a negative result is obtained here, this means that a new input has not been added. At this time, the IO manager 32 moves to step SP40 and ends the recording processing procedure by the recording processing program 30.
[0150]
On the other hand, if a positive result is obtained in step SP34, this indicates that a new input has been added. At this time, the IO manager 32 proceeds to the next step SP35.
[0151]
In step SP35, the IO manager 32 sets a waiting time stamp in consideration of the maximum delay time until the packet data arrives from the DSP plug-in in the previous stage with respect to the new input, so that the new input has been added. Therefore, it is prevented that the processing for the packet data that has already been input is delayed, and the process proceeds to the next step SP36.
[0152]
In step SP36, for example, as shown in FIG. 10, the IO manager 32 determines whether or not the input of the piano packet data PiPD1 and PiPD2 is canceled during the mixing process in the mixer plug-in 33B.
[0153]
If a negative result is obtained here, this means that the input has not been canceled and there is no change. At this time, the IO manager 32 moves to step SP40 and performs the recording processing procedure by the recording processing program 30. All ends.
[0154]
On the other hand, if a positive result is obtained in step SP36, this indicates that the input has been released and the connection state has changed, and at this time, the IO manager 32 proceeds to the next step SP37.
[0155]
In step SP37, the IO manager 32 sets the canceled input waiting time stamp “0000” of the mixer plug-in 33B to “invalid”, so that only the drum packet data DrPD1 and DrPD2 are promptly transmitted without delay from the input cancellation time point. The process proceeds to the mixing process, and the process proceeds to the next step SP38.
[0156]
In step SP38, for example, as shown in FIG. 11, the IO manager 32 sends packet data of a time stamp later than the waiting time stamp to the input of the mixer plug-in 33B. It is determined whether or not the input is rejected.
[0157]
If a negative result is obtained here, this means that packet data with a time stamp later than the waiting time stamp is not rejected, that is, packet data with the same time stamp as the waiting time stamp is added. Is input to the mixer plug-in 33B. At this time, the IO manager 32 moves to step SP40 and completes the recording processing procedure by the recording processing program 30.
[0158]
On the other hand, if an affirmative result is obtained in step SP38, this means that the mixer plug-in 33B rejected the input of packet data with a time stamp later than the waiting time stamp. 32 moves to the next step SP39.
[0159]
In step SP39, the IO manager 32 once buffers the packet data rejected by the mixer plug-in 33B, and resends it until it is received at the input of the mixer plug-in 33B. After the mixing process is accurately executed without being distorted, the process proceeds to the next step SP40, and all the recording process procedures by the recording process program 30 are terminated.
[0160]
(3) Operations and effects in the embodiment
In the above configuration, the remote inter-site recording system 1 reproduces with the playback system of the recorder plug-in 33A when performing recording processing almost simultaneously in parallel between the personal computer 3 in Tokyo and the personal computer 4 in New York. A time stamp “0000” is added to the drum packet data DrPD1 and DrPD2 and the piano packet data PiPD1 and PiPD2 and output.
[0161]
At this time, the remote location recording system 1 sets a waiting time stamp “0000” at each input in the recording system of the mixer plug-in 33B, the audio output plug-in 233A, and the recorder plug-in 33A in the subsequent stage, and the mixer plug-in 33B. And the time stamp “0000” is added at the output of the audio input plug-in 233B.
[0162]
As a result, the remote location recording system 1 is a playback system of the recorder plug-in 33A, and the drum packet data DrPD1 and DrPD2 with the time stamp “0000” and the guitar packet with the time stamp “0000” with respect to the piano packet data PiPD1 and PiPD2. The data GiPD1 and GiPD2 can be stored and controlled in association with the hard disk of the HDD 11 so as to synchronize. As a result, a timing shift due to a transmission path delay between remote locations is corrected and recording processing is executed in parallel. Can do.
[0163]
In addition, the remote location recording system 1 can flexibly respond to new additions and cancellations of input during the recording process by changing the waiting time stamp setting while maintaining temporal timing consistency. Thus, it is possible to easily generate recording data having a desired sound quality by combining user-desired functions.
[0164]
According to the above configuration, the remote inter-site recording system 1 adds a time stamp for each packet to audio data that requires real-time performance, and based on the time stamp, shifts in timing due to transmission path delays. By performing storage control so that correction is performed, audio data that requires real-time performance between remote locations can be exchanged via the Internet 2 and recording processing can be executed in parallel.
[0165]
(4) Other embodiments
In the above-described embodiment, the remote location recording system 1 as the content editing apparatus of the present invention is applied to the case where the recording process is executed between the personal computer 3 in Tokyo and the personal computer 4 in New York. However, the present invention is not limited to this. For example, the present invention is applied to a case where a recording process is executed between a personal computer 3 and a personal computer 4 connected to each other by a transmission cable in the same studio. Also good.
[0166]
In the above embodiment, the recorder plug-in 33A, the mixer plug-in 33B, the audio output plug-in 233A as the first content data output means and the storage control means, and the audio input plug-in 233B as the second content data output means. However, the present invention is not limited to this, and the present invention is not limited to this, and the effector system, sequencer plug-in, and synthesizer plug are not limited to this. Various other systems such as a composition system using the IN, mixer plug-in, and audio output plug-in may be constructed.
[0167]
Further, in the above-described embodiment, the case where the personal computer 4 in New York is connected to the personal computer 3 in Tokyo and expanded is described. However, the present invention is not limited to this, for example, digital signal processing of a personal computer. It may be extended by connecting to a digital signal processing apparatus having only a portion and no display portion. In this case, the digital signal processing apparatus is designed to perform various processes by the IO manager in the same way as the personal computer 3, and is expanded to a simple configuration as compared with the personal computer because display control is not performed. The accompanying increase in space can be minimized.
[0168]
Further, in the above-described embodiment, the GUI plug-in 34 is configured in a hierarchical structure including the lower GUI plug-in, and the GUI plug-in screen for each layer is displayed. However, the present invention is not limited to this, and a GUI plug-in screen having a single structure that can execute the operation of the GUI plug-in 34 and the lower GUI plug-in on one screen may be displayed.
[0169]
Furthermore, in the above-described embodiment, the case where the personal computer 3 in Tokyo and the personal computer 4 in New York are connected to each other via the Internet has been described. However, the present invention is not limited to this, and the personal computer 3 and 4 may be connected to each other via a LAN (Local Area Network) such as Ethernet.
[0170]
Further, in the above-described embodiment, the personal computer 3 in Tokyo and the personal computer 4 in New York are connected via the Internet 2 and the recording process is executed in parallel between remote locations in Tokyo and New York. Although the present invention is not limited to this, the present invention is not limited to this, and other various remote locations such as London and Paris other than New York may be connected via the Internet 2 to perform recording processing in parallel. good.
[0171]
Furthermore, in the above-described embodiment, the DSP plug plug icon 40A and the mixer plug-in icon 40B, and the audio output plug-in icon 41A and the audio input plug-in icon 41B are connected between the input terminal and the output terminal, thereby connecting the DSP plug. Although the case where the connection status of each other is visually shown to the user has been described, the present invention is not limited to this, and the DSP plug-in can be displayed by arranging and displaying the loaded DSP plug-ins in a matrix. You may make it show the connection state of a comrade.
[0172]
Furthermore, in the above-described embodiment, the case has been described where the above-mentioned remote location recording processing is executed based on the recording processing program 30 started up from the hard disk of the HDD 11, but the present invention is not limited to this, By installing the program storage medium storing the recording processing program 30 in the personal computers 3 and 4, the remote location recording processing may be executed.
[0173]
As a program storage medium used for installing the recording processing program 30 for executing the above-described series of remote location recording processes in the personal computers 3 and 4 and making the personal computers 3 and 4 executable. For example, not only package media such as a floppy disk, CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc., but also a semiconductor memory or magnetic field in which the recording processing program 30 is temporarily or permanently stored. You may implement | achieve with a disk etc. Further, as means for storing the recording processing program 30 in these program storage media, wired and wireless communication media such as a local area network, the Internet, digital sanitary broadcasting, etc. may be used, and various communication interfaces such as routers and modems may be used. You may make it store via.
[0174]
Recorder plug-in 33A as first content data output means for sequentially outputting drum packet data DrPD1 and DrPD2 and piano packet data PiPD1 and PiPD2 as first content data in units of packets and adding time stamp “0000”; As input means for inputting guitar packet data GiPD1 and GiPD2 as second content data in units of packets corresponding to the first content data to which the same time stamp “0000” as the stamp “0000” is added from the external personal computer 4 IO plug as timing correction means for adjusting the first content data and the second content data to be temporally synchronized based on the recorder plug-in 33A and the time stamp “0000” Although the remote location recording system 1 as a content editing apparatus is constructed by using the jar 32, the present invention is not limited to this, but various other first content data output means, input means, and timing A content editing apparatus may be constructed by correcting means.
[0175]
【The invention's effect】
As described above, according to the present invention, first content data output means for sequentially adding and outputting time stamps to first content data in units of packets, and a waiting time stamp having the same value as the time stamp are first input. When the waiting time stamp matches the time stamp added to the first content data after being set to the terminal, the first content data is input via the first input terminal, and the waiting time stamp is the second input. When the time stamp in the second content data in units of packets corresponding to the first content data to which the time stamp having the same value as the time stamp is added and the waiting time stamp match after setting to the terminal, the second time Input means for inputting content data from the outside via the second input terminal, and a time stamp And a timing correction means for adjusting the first content data and the second content data so as to be synchronized in time, and the input means additionally inputs the third content data from the outside during the processing by the timing correction means. In this case, a waiting time stamp taking into account the maximum delay time until the third content data arrives is set in the third input terminal, and the set waiting time stamp and the time stamp added to the third content data are The third content data is input only when they match, and the timing adjustment means Added to the first content data, the second content data, and the third content data, respectively. By adjusting the first content data, the second content data, and the third content data to be temporally synchronized based on the time stamp, the first content data, the second content data, and the second content data after the maximum delay time has elapsed. The time stamps of the three content data can be adjusted so that they are all synchronized in time, and the first content data and the second content data are adjusted in time until the maximum delay time elapses. Content editing apparatus, content editing method, and program storage medium capable of correcting the time timing of the first, second, and third content data for which real-time processing is required and correcting the temporal timing accurately In addition, a content editing system can be realized.
[Brief description of the drawings]
FIG. 1 is a schematic diagram showing the overall configuration of a remote location recording system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a circuit configuration of a personal computer.
FIG. 3 is a schematic diagram illustrating a configuration of a recording processing program.
FIG. 4 is a schematic diagram showing a workspace screen.
FIG. 5 is a schematic diagram showing a workspace screen loaded with a DSP plug-in.
FIG. 6 is a schematic diagram showing a workspace screen when connected to another personal computer.
FIG. 7 is a schematic diagram showing a workspace screen when connected to a plug-in of another personal computer.
FIG. 8 is a schematic diagram for explaining timing correction processing;
FIG. 9 is a schematic diagram for explaining timing correction processing when a new input is added during recording processing;
FIG. 10 is a schematic diagram for explaining timing correction processing when plug-in connection is released during processing;
FIG. 11 is a schematic diagram for explaining timing correction processing when packet data after the packet data of the waiting time stamp is sent.
FIG. 12 is a schematic diagram illustrating a mixer GUI plug-in screen.
FIG. 13 is a flowchart showing a recording processing procedure.
FIG. 14 is a flowchart showing a procedure for loading a DSP plug-in.
FIG. 15 is a schematic diagram for explaining an exchange between a workspace and an IO manager in a DSP plug-in load processing procedure;
FIG. 16 is a flowchart showing a DSP plug-in connection management processing procedure;
FIG. 17 is a schematic diagram for explaining an exchange between a workspace and an IO manager in a DSP plug-in connection management processing procedure;
FIG. 18 is a flowchart showing a DSP plug-in parameter adjustment processing procedure;
FIG. 19 is a schematic diagram for explaining an exchange between a GUI plug-in and a DSP plug-in in a DSP plug-in parameter adjustment processing procedure;
FIG. 20 is a flowchart illustrating a timing correction processing procedure.
[Explanation of symbols]
1 ... Remote location recording system, 2 ... Internet, 3, 4 ... Personal computer, 10, 210 ... CPU, 11, 211 ... HDD, 12, 212 ... RAM, 30, 230 ... Recording processing Program, 31, 231, ... Workspace, 32, 232 ... IO manager, 33, 233 ... DSP plug-in, 34, 234 ... GUI plug-in, 40A ... Recorder plug-in icon, 40B ... Mixer plug-in Icon, 41A: Audio output plug-in icon, 41B: Audio input plug-in icon.

Claims (7)

First content data output means for sequentially adding and outputting time stamps to the first content data in packet units;
When the waiting time stamp having the same value as the time stamp is set to the first input terminal and the waiting time stamp matches the time stamp added to the first content data, the first content Corresponds to the first content data to which the time stamp having the same value as the time stamp is added after inputting the data through the first input terminal and setting the waiting time stamp to the second input terminal. Input means for inputting the second content data from the outside through the second input terminal when the time stamp and the waiting time stamp in the second content data in packet units coincide with each other;
Timing correction means for adjusting the first content data and the second content data to be temporally synchronized based on the time stamp,
Said input means, during the treatment with the timing correction unit, when the additional input of the third content data from the external, the waiting time stamp consideration of the maximum delay time until the third content data reaches the third input Set the terminal, and input the third content data only when the set waiting time stamp matches the time stamp added to the third content data,
The timing correction means is configured to determine the first content data, the second content data, and the third content based on the time stamps added to the first content data, the second content data, and the third content data, respectively . A content editing apparatus that adjusts content data so as to be synchronized in time. The content editing device
2. A storage control unit that stores the first content data, the second content data, and the third content data synchronized in time by the timing correction unit in association with each other and stored in a predetermined storage unit. The content editing apparatus described in 1. 2. The content editing apparatus according to claim 1, wherein the first content data, the second content data, and the third content data are audio data that requires real-time performance corresponding to each music constituting the music. 2. The content editing according to claim 1, wherein when the connection state via the first input terminal, the second input terminal, or the third input terminal is canceled, the input unit invalidates the canceled input. apparatus. A first content data output step of sequentially adding and outputting a time stamp to the first content data in packet units by the first content data output means;
When the waiting time stamp having the same value as the time stamp is set in the first input terminal of the input means, and the waiting time stamp matches the time stamp added to the first content data, The first content data is input through the first input terminal, the waiting time stamp is set in the second input terminal of the input means, and a time stamp having the same value as the time stamp is added. An input step of inputting the second content data from the outside via the second input terminal when the time stamp in the second content data in a packet unit corresponding to the first content data matches the waiting time stamp. When,
A timing correction step for adjusting the first content data and the second content data in time synchronization by the timing correction means based on the time stamp;
In the input step, during processing by the timing correction unit, when the additional input of the third content data from the external, the waiting time stamp consideration of the maximum delay time until the third content data reaches the third input Set the terminal, and input the third content data only when the set waiting time stamp matches the time stamp added to the third content data,
In the timing correction step, the first content data, the second content data, and the third content are based on the time stamps added to the first content data, the second content data, and the third content data, respectively . A content editing method that adjusts content data to synchronize in time. On the computer,
A first content data output step of sequentially adding and outputting a time stamp to the first content data in packet units by the first content data output means;
When the waiting time stamp having the same value as the time stamp is set in the first input terminal of the input means, and the waiting time stamp matches the time stamp added to the first content data, The first content data is input through the first input terminal, the waiting time stamp is set in the second input terminal of the input means, and a time stamp having the same value as the time stamp is added. An input step of inputting the second content data from the outside via the second input terminal when the time stamp in the second content data in a packet unit corresponding to the first content data matches the waiting time stamp. When,
A timing correction step for adjusting the first content data and the second content data to be temporally synchronized based on the time stamp by a timing correction means;
In the input step, during processing by the timing correction unit, when the additional input of the third content data from the external, the waiting time stamp consideration of the maximum delay time until the third content data reaches the third input Set the terminal, and input the third content data only when the set waiting time stamp matches the time stamp added to the third content data,
In the timing correction step, the first content data, the second content data, and the third content are based on the time stamps added to the first content data, the second content data, and the third content data, respectively . A program storage medium in which a program for adjusting content data to be synchronized in time is recorded. In a content editing system including a first information processing apparatus and a second information processing apparatus connected to each other via a network,
The first information processing apparatus includes:
Time stamp adding means for sequentially adding and outputting time stamps to the first content data in packet units;
When the waiting time stamp having the same value as the time stamp is set in the first input terminal and the waiting time stamp matches the time stamp added to the first content data, the first content Corresponds to the first content data to which the time stamp having the same value as the time stamp is added after inputting the data through the first input terminal and setting the waiting time stamp to the second input terminal. Input means for inputting the second content data from the second information processing apparatus via the second input terminal when the time stamp and the waiting time stamp in the second content data in packet units match. ,
Timing correction means for adjusting the first content data and the second content data to be temporally synchronized based on the time stamp,
Said input means, during the treatment with the timing correction unit, when the additional input of the third content data from the external, the waiting time stamp consideration of the maximum delay time until the third content data reaches the third input Set the terminal, and input the third content data only when the set waiting time stamp matches the time stamp added to the third content data,
The timing correction means is configured to determine the first content data, the second content data, and the third content based on the time stamps added to the first content data, the second content data, and the third content data, respectively . Adjust the content data to be synchronized in time,
The second information processing apparatus
A time stamp having the same value as the time stamp of the first content data is sequentially added to the second content data in units of packets corresponding to the first content data and output to the first information processing apparatus. A content editing system comprising content data output means.

Images