JP2003208169A - Multi-media system, reproducing apparatus and reproducing/recording apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a multi-media system, a reproducing apparatus and a reproducing/recording apparatus which reproduce or record a plurality of musical composition data in synchronism with reproducing or recording of video data. <P>SOLUTION: In reproducing of CD audio data stored in a CD, recording or reproducing of musical composition data to an FD in synchronism with reproducing or recording of video data, a video camera 100 outputs a video time code to a time code converter 200 in reproducing or recording video data, etc. The time code converter 200 converts the video time code into a musical sound time code (MTC) and supplies it to a controller 3. The controller 3 adjusts a reproducing timing of the received CD audio data which is transferred from the CD to an HD on the basis of the received MTC. An FD recording apparatus 8 and an FD reproducing apparatus 8a adjust the recording and the reproducing timing of the musical composition data on the basis of the MTC supplied by the controller 3. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia system, a reproducing apparatus and a reproducing / recording apparatus for reproducing or recording music data in synchronization with reproduction of video data or recording of video data.

[0002]

In recent years, among users who prefer playing musical instruments such as a piano performance, a video tape (hereinafter referred to as a VT) on which video data corresponding to a scene such as a concert venue is recorded and the video are recorded on the video tape. An optical disc (hereinafter referred to as a CD) on which corresponding music data (for example, music data corresponding to an orchestra performance) is synchronously reproduced by a video camera, a CD player, etc., while a user is displayed by reproducing the video data. Playing an automatic piano, etc. while listening to the orchestral performance sound that is pronounced by playing the music data,
Music data corresponding to the performance (for example, MIDI (Musical
Instruments Digital Interface) -compliant data) on a floppy (registered trademark) disk (hereinafter FD)
There is a demand to enjoy the strangeness of the performance by recording it in a storage medium such as the above and reproducing all the data later in synchronization.
In addition, the user plays the automatic piano or the like while listening to the orchestra performance played by playing the CD, FD-records the music data corresponding to the performance, records the performance scene with a video camera, and later records all of these. There is also a request to enjoy the strange performance by playing the data synchronously. However, at present, a multimedia system which meets such a demand has not been provided yet.

The present invention has been made in view of the above-mentioned circumstances, and is a multimedia capable of reproducing or recording a plurality of music data in synchronization with reproduction of video data or recording of video data. An object is to provide a system, a reproducing device, and a reproducing / recording device.

[0004]

According to the present invention, recording information including video data and a series of video time codes representing the elapsed time from the start of recording the video data is read from a first storage medium, and the recording information is recorded. A master device that reproduces the video data in the information, a reproduction of the first music data stored in the storage means in synchronization with the reproduction of the video data by the master device, and a recording in the second storage medium. A multimedia system including a slave device for reproducing second music data, wherein the slave device acquires a video time code in the recorded information, and uses the video time code as a musical sound time code. Based on the time measurement result by the time code conversion means for converting into The first audio data stored in the storage means, the first audio data representing the music, and a series of time codes representing the elapsed time from the start of the music are stored at the time determined by the time code in the first music data. Audio data reproduction control means for performing control for reading out the audio data of the above and outputting it as a sound, and instructing performance control of the music recorded in the second storage medium based on the timing result by the timing means. Event output means for outputting the event at a time determined by the timing data in the music data including a plurality of events and timing data designating the execution timing of each event, and the event output by the event output means is automatically played. Automatic performance control means for supplying the means,
The read timing of the first audio data is
First timing adjusting means for adjusting based on a time difference between a time corresponding to a time code for musical sound output from the time code converting means and a time corresponding to a time measurement result of the time measuring means; and the event output as the event output. The timing supplied from the means to the automatic performance control means is adjusted based on the time difference between the time corresponding to the musical tone time code output from the time code converting means and the time corresponding to the time measurement result of the time measuring means. A second timing adjusting means is provided.

According to this structure, the read timing of the first audio data stored in the storage means is
The event read from the second storage medium while being adjusted based on the time difference between the time corresponding to the musical tone time code output from the time code conversion means and the time corresponding to the time measurement result of the time measurement means The timing used for the automatic performance is adjusted based on the time difference between the time corresponding to the musical tone time code and the time corresponding to the timing result of the timing means. By this adjustment, a time difference between the reproduction of the video data by the master device, the reproduction of the first music data by the slave device, and the reproduction of the second music data recorded in the second storage medium can be suppressed.

Further, according to the present invention, the record information including the video data and a series of video time codes representing the elapsed time from the start of recording the video data is read from the first storage medium, and the video in the record information is read. A master device for reproducing data, and a slave device for reproducing the first music data and recording the second music data stored in the storage means in synchronization with reproduction of the video data by the master device. In the multimedia system, the slave device obtains a video time code in the recorded information, converts the video time code into a musical sound time code, and counts clocks. And a music piece stored in the storage means on the basis of a time measurement result by the time measurement means. Control for reading the first audio data at a time determined by the time code in the first music data including the audio data of the first music and a series of time codes representing the elapsed time from the start of the music, and outputting the sound as sound. The audio data reproduction control means for performing the above and the read timing of the first audio data are set to a time corresponding to the musical tone time code output from the time code conversion means and a time corresponding to the time measurement result of the time measurement means. Event generating means for acquiring the operation state from the timing adjusting means for adjusting based on the time difference and the detecting means for detecting the operation state of the musical instrument, and generating an event for instructing the performance control of the music in accordance with the acquired operation state. And, when the event is generated by the event generating means, Timing data generating means for generating timing data designating the execution timing of the event based on the time corresponding to the result; time corresponding to the musical tone time code output from the time code converting means; and the time measuring means. Timing data adjusting means for adjusting the timing data based on a time difference from the time corresponding to the time measurement result,
The second music data including a plurality of events generated by the event generating means and timing data adjusted by the timing data adjusting means is second
Writing means for writing to the storage medium.

According to this structure, the timing data designating the execution timing of the event instructing the performance control written in the second storage medium is timed and clocked corresponding to the musical tone time code output from the time code converting means. It is adjusted based on the time difference from the time corresponding to the time measurement result of the means. By this adjustment, the time lag between the reproduction of the video data by the master device and the reproduction of the first music data stored in the storage means by the slave device and the recording of the second music data can be suppressed.

A master device for reading record information including video data and a series of video time codes representing the elapsed time from the start of recording the video data from the first storage medium and reproducing the video data in the record information. And a slave device for recording the first music data and the second music data stored in the storage means in synchronization with the reproduction of the video data by the master device, The slave device acquires a video time code in the recorded information, time code conversion means for converting the video time code into a musical sound time code, and a time measuring means for measuring time by counting a clock, The first audio data representing the music stored in the storage means and the first audio data corresponding to Audio data for performing control for reading the first audio data at a time determined by the time code in the first music data including a series of time codes representing the elapsed time from the start of the music and outputting it as a sound Based on the time difference between the reproduction control means and the read timing of the first audio data, the time corresponding to the musical tone time code output from the time code conversion means and the time corresponding to the timing result of the timing means. Timing adjusting means for adjusting, an event generating means for acquiring the operation state from the detecting means for detecting the operation state of the musical instrument, and an event generating means for generating an event for instructing performance control of the music in accordance with the acquired operation state; The time corresponding to the time code for musical tone output from the code converting means and the time measurement result by the time measuring means Based on the time difference from the corresponding time, the timekeeping time adjusting means for adjusting the time corresponding to the timekeeping result, and when the event is generated by the event generating means, at the time adjusted by the timekeeping time adjusting means. The second music including timing data generation means for generating timing data designating execution timing of the event based on the event, and an event generated by the event generation means and timing data generated by the timing data generation means. Writing means for writing data to the second storage medium.

According to this structure, the timing data designating the execution timing of the event instructing the performance control written in the second storage medium is timed and clocked corresponding to the time code for the musical sound output from the time code converting means. It is adjusted based on the time difference from the time corresponding to the time measurement result of the means. By this adjustment, the time lag between the reproduction of the video data by the master device and the reproduction of the first music data stored in the storage means by the slave device and the recording of the second music data can be suppressed.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION A. First Embodiment FIG. 1 is a block diagram showing the configuration of a multimedia system 600 which is a first embodiment of the present invention. The multimedia system 600 according to the present embodiment, which will be described in detail below, reproduces video data and audio data (hereinafter referred to as VT audio data) stored in the VT in a synchronized state and stores the data in a CD. This is a system capable of reproducing recorded audio data (hereinafter referred to as CD audio data) and recording MIDI data based on a user's performance on the FD.

In FIG. 1, the video camera 100 comprises a recording section 110, a reproducing section 120, and an operating section 130. The recording unit 110 records a performance scene, various performance sounds (a violin performance sound played with the user, etc.), etc. on the VT mounted on the video camera 100 according to an instruction input via the operation unit 130. Play a role. The recording unit 110 records the image scenery on the VT image track as image data.
And a recording means 112 for recording the performance sound as VT audio data on a musical sound track of the VT, and a series of video time codes (for example, RC time code or the like) representing the elapsed time from the start of recording the video data on the VT. And a time code adding means 113 for adding.

The reproducing unit 120 plays a role of reproducing recorded information such as video data and VT audio data already recorded in the VT according to an instruction input through the operation unit 130. When the reproduction unit 120 receives a reproduction instruction via the operation unit 130, the reproduction unit 120 reads out the video data recorded in the video track of the VT and displays the video data on the LCD (Liquor).
It is supplied to a monitor 121 such as an id crystal display) or a CRT (Cathode Ray Tube) to be displayed as an image, and VT audio data recorded in a VT musical sound track is read out and supplied to a controller 3 described later. Furthermore, the reproducing unit 120 reads the video time code added to the VT in response to the reading of the video data and the VT audio data, and sends this to the time code converter 200. In the present embodiment,
Video camera 100 that outputs this video time code
Functions as a master, and various devices (FD recording device 8 and the like) that receive the time code for musical sound from the video camera 100 via the time code converter 200 function as slaves.

The time code converter 200 is connected to the video camera 100 by, for example, a cable or the like, and the video time code sequentially supplied from the video camera 100 is converted into a musical time code (MIDI time code;
It is appropriately converted to MTC) and the converted MTC is supplied to the controller 3.

The CD drive device 1 is a device for reading out music data from a mounted CD and outputting it in accordance with a command from the controller 3. When the CD drive device 1 normally reproduces the loaded CD (that is, as described above, V
When not reproduced in synchronization with T), the music data read from the CD is output to the DSP 2. The DSP 2 generates an analog audio signal from the CD audio data in the music data read from the CD drive device 1, sends it to the mixer 5, and outputs it as a sound. On the other hand, the CD drive device 1
When playing the loaded CD in sync with VT,
Under the control of the controller 3, C before the reproduction.
The music data stored in D is transferred to a hard disk (hereinafter, HD) via the DSP 2 and the controller 3. The details will be described later, and the description will be continued.

Various CDs can be mounted on the CD drive device 1. Among these CDs, the one most strongly related to the present invention is the CD whose main recording data is CD audio data which is music data. As shown in FIG. 2, the music data includes a coded data string (CD audio data) of an audio signal generated in association with an orchestra performance or singing of a certain music. Further, time codes (hereinafter, appropriately called CD time codes) are discretely inserted in the music data. These cds
The time code is a code indicating the elapsed time from the start of the music. A CD in which data of a type different from such data is recorded can also be loaded in the CD drive device 1. For example, a CD in which CD audio data is recorded as the L channel data and MIDI data for automatic performance control is recorded as the R channel data can be mounted in the CD drive device 1. On the contrary, CD as R channel data
CD on which audio data is recorded and MIDI data for automatic performance control is recorded as L channel data
May be attached to the CD drive device 1.

As described above, the DSP 2 generates an analog audio signal from the CD audio data in the music data read from the CD drive device 1 for normal reproduction, sends it to the mixer 5, and outputs it as a sound. , V
When reproducing in synchronization with T or the like, the music data read from the CD drive device 1 is output to the controller 3. In addition, the DSP 2 identifies information (hereinafter referred to as “identification”) for identifying the CD read by the CD drive device 1 during reproduction.
C-ID) is acquired from the CD drive device 1 and sent to the controller 3. In this embodiment,
CD selected by the CD drive 1 when the DSP 2 is playing
The case where only the identification information (C-ID) for identifying the CD is acquired will be described. The identification information for identifying the music data for which the instruction to perform may be acquired.

On the other hand, the controller 3 controls the entire system according to an instruction given via the operation unit 4. Of the various controls performed by the controller 3, the one most closely related to the present invention is that data is transferred from the CD to the HD via the CD drive device 1, the DSP 2, and the controller 3 in synchronization with the reproduction of the video data by the video camera 100. This is reproduction / recording control in the case of reproducing music data including the reproduced CD audio data and recording a piano performance sound by the user (hereinafter referred to as synchronous reproduction recording). Each time the controller 3 receives an MTC from the time code converter 200, the controller 3 sends the time code (MTC) to the FD recording device 8 and, based on the MTC, reproduces the CD audio data to be reproduced. Controls reading (details will be described later). In addition, the controller 3
When a MIDI event or the like (described later) is received from the MIDI event generation circuit 14, the MIDI event or the like is sent to the FD.
It is sent to the recording device 8. Further, the controller 3 generates an analog audio signal based on the VT audio data supplied from the video camera 100 and sends it to the mixer 5.

Here, FIG. 3 is a block diagram for explaining the function of the controller 3 for controlling the reading of the CD audio data. The controller 3 includes a clock generation means 210, a clock means 211, an elapsed time adjustment means 212, and a CD audio data reproduction control means 213. The clock generation means 210 includes an oscillation circuit (both not shown) composed of a crystal oscillator and an amplifier. The clock generation means 210 is means for appropriately dividing the oscillation signal output from the oscillation circuit to generate various clocks for timing control (a clock particularly deeply related to the present invention is the tempo clock CT). is there. The clock generating means 210 sequentially supplies the tempo clock CT thus generated to the time measuring means 211 of the control circuit.

The clock means 211 is composed of an adder 211a and an elapsed time register 211b, and an accumulator for adding "+1" each time the tempo clock CT is given by the adder 211a and the elapsed time register 211b. To be done. The data stored in the adder 211a and the elapsed time register 211b and the fixed value "+1" are added and output. The initial value “0” is written in the elapsed time register 211b when the synchronous recording is started, and thereafter, every time the tempo clock CT is given to the elapsed time register 211b, the output data of the adder 211a, that is, the time The addition result of the data stored in the elapsed time register 211b and "+1" at is written. The output data N of the elapsed time register 211b is used as elapsed time data representing the elapsed time after the start of the synchronous reproduction recording.

The elapsed time adjusting means 212 is the time measuring means 21.
1 time and time code converter 200
It is a means for executing a process (elapsed time adjustment process) of adjusting the elapsed time data N in the elapsed time register 211b according to the deviation generated from the time indicated by the MTC supplied from. Here, FIG. 4 is a flowchart showing the elapsed time adjustment processing. When the MTC larger than "0" is given from the time code converter 200 in step S1, the elapsed time adjusting means 212 sets this as the first recording time data TCD, and proceeds to step S2. When the elapsed time adjusting means 212 proceeds to step S2, it acquires the elapsed time data N at that time point from the time measuring means 211, multiplies the acquired elapsed time data N by the period τ of the tempo clock CT, and obtains the second recording time. Data TFD (= N *
τ) is calculated. The second recording time data TFD represents the elapsed time from the start of the synchronous reproduction recording to the present. Next, the elapsed time adjusting means 212 determines whether or not the absolute value | TCD-TFD | of the difference between the first recording time data TCD and the second recording time data TFD is within a preset allowable range Δ. It is determined (step S3).

The elapsed time adjusting means 212 determines whether or not the absolute value | TCD-TFD | of the difference between the first recording time data TCD and the second recording time data TFD falls within a preset allowable range Δ. Whether or not (step S
3). When the elapsed time adjustment unit 212 determines that the absolute value of the difference between the first recording time data TCD and the second recording time data TFD is within the predetermined allowable range Δ (step S3; YES), the elapsed time The elapsed time adjustment process ends without adjusting the elapsed time data N in the register 211b.

On the other hand, the elapsed time adjusting means 212 determines that the absolute value | TCD-TFD | of the difference between the first recording time data TCD and the second recording time data TFD exceeds the predetermined allowable range Δ. , First recording time data TCD
Is larger than the second recording time data TFD (step S5). First recording time data TC
When D is larger than the second recording time data TFD, it can be said that the time measured by the time measuring means 211 is later than the time indicated by the MTC supplied from the time code converter 200. Elapsed time adjusting means 212
Judges that the first recording time data TCD is larger than the second recording time data TFD (step S5; Y
ES), the absolute value of the difference between the time measured by the time measuring means 211 and the time indicated by the MTC | TCD-TFD
| (> 0) is divided by the period τ of the tempo clock CT, the division result | TCD-TFD | / τ is added to the elapsed time data N in the elapsed time register 211b, and the elapsed time data N after addition is added. The elapsed time register 211b is written (step S6), and the elapsed time adjustment process ends.

On the other hand, the first recording time data TCD is the second
If it is smaller than the recording time data of the time, the time measured by the time measuring means 211 is the time code converter 20.
It can be said that it is ahead of the time indicated by the MTC supplied from 0. When the elapsed time adjusting means 212 determines that the first recording time data TCD is smaller than the second recording time data TFD (step S5; NO), the time measuring means 2
The absolute value | TCD-TFD | (> 0) of the difference between the time measured by 11 and the time indicated by MTC is divided by the period τ of the tempo clock CT, and the division result |
TCD-TFD | / τ is subtracted from the elapsed time data N in the elapsed time register 211b, the elapsed time data N after the subtraction is written in the elapsed time register 211b (step S7), and the elapsed time adjustment processing ends.

CD audio data reproduction control means 213
Is a means for controlling reproduction of music data including CD audio data stored in HD, and HD read control means 213a and audio signal generation means 213b.
It has and. The HD read control means 213a is set to H
The CD time code in the music data stored in D is compared with the elapsed time data N stored in the elapsed time register 211b to control the reading of the CD audio data in the music data. More specifically, the HD read control means 213a reads the corresponding CD audio data each time the elapsed time data N reaches the time indicated by each CD time code in the music data, and sequentially reads this to the audio signal generation means 213b. Supply. The audio signal generation means 213b is the HD read control means 213a
An analog audio signal is generated based on the CD audio data supplied from the device and sent to the mixer 5. As is clear from the above description, the reproduction of the CD audio data stored in the HD is controlled by the controller 3 based on the musical tone time code (MTC) output from the time code converter 200.

Returning to FIG. 1 again, the automatic piano 10 has a musical sound generating mechanism for generating a mechanical piano sound by striking a string in response to a key operation by the user, and also detects the key operation by the user, and outputs the detection result. It also has a musical tone generation mechanism that generates electronic piano sounds. Such an automatic piano 10
Is a piano 11, a key sensor 12, and a pedal sensor 1
3, MIDI event generation circuit 14, piano sound source 1
5 and an ensemble sound source 18. In FIG. 1, an automatic piano 10 (so-called mute type automatic piano) that produces an electronic musical sound by using a piano sound source 15 or the like.
Other than that, as in the automatic piano 20 (see FIG. 10) according to the second embodiment described later, a drive solenoid group that drives a key or pedal in accordance with a key or pedal operation instruction is used. Of course, it can be applied to all kinds of automatic pianos, such as an automatic piano that produces mechanical tones such as striking strings. The key sensor 12 and the pedal sensor 13 are provided corresponding to each of a plurality of keys and a plurality of pedals (sostenuto pedal, etc.) arranged on the piano 11, respectively, and the strength and depth when the key and the pedal are pressed. The detected result including the key number or pedal number that identifies the detected key or pedal, velocity information (data corresponding to the key pressing strength, etc.), etc.
It is supplied to the event generation circuit 14.

The MIDI event generation circuit 14 generates a MIDI event based on the detection results supplied from the key sensor 12 and the pedal sensor 13 under the control of the controller 3. Here, the MIDI event is composed of note-on / note-off information indicating that sounding or mute should be performed, note number information indicating the pitch of a sound to be sounded, velocity information indicating strength of sounding, and the like. . Based on the detection signal, the MIDI event generation circuit 14 outputs, for example, “do sound (note number) with an intensity of 10”.
Generate a MIDI event, which is a kind of event for instructing performance control such as “pronounce (note on) with (velocity)”, and generate such MIDI event in the controller 3
And the piano sound source 15.

The piano tone generator 15 is a device that electronically generates an analog audio signal of a piano sound instructed by an event supplied from the MIDI event generation circuit 14. The analog audio signal generated by the piano sound source 15 is output to the mixer 5. The ensemble sound source 18 is a device that receives an event from the controller 3 and generates a digital musical tone signal according to the event. The digital tone signal generated by the ensemble sound source 18 is converted into an analog tone signal by the controller 3 and sent to the mixer 5. Mixer 5
Is a device that mixes and outputs the analog signals output from the controller 3, the piano sound source 15, and the ensemble sound source 18. The output signal of the mixer 5 is amplified by the amplifier 6 and output as sound from the speaker 7.

The FD recording device 8 reproduces image data by the video camera 100 and a CD by the controller 3 or the like.
This is a device for recording the performance sound when the user plays the automatic piano 10 in the FD together with the reproduction of audio data, etc. Under the control of the controller 3, identification information (V- ID) and played C
It has a function of creating an SMF (Standard MIDI File) including identification information (C-ID) for identifying D, a MIDI event and the like. Here, the SMF is composed of a header chunk HT and a track chunk TT, as shown in FIG. The header chunk HT stores basic information of SMF (chunk type, etc.),
V reproduced at the time of synchronous recording of V-ID, C-ID, etc.
Identification information for identifying T and CD is stored. MIDI data is stored in the track chunk TT.
As shown in the figure, the MIDI data is time-series music data composed of an event instructing performance control and the like, and a delta time indicating an occurrence time interval between a preceding event and a subsequent event. Here, the events that compose the MIDI data are composed of MIDI events generated by the MIDI event generation circuit 14, SysEx events generated by the controller 3, and meta events.

Upon receipt of the synchronous recording start command from the controller 3, the FD recording device 8 for producing / recording the SMF starts time counting using the time measuring means described later. Then, each time the FD recording device 8 receives an event from the controller 3, the FD recording device 8 obtains the delta time by referring to the time measured by the time measuring means, and sequentially creates MIDI data including the event and the delta time. Here, the FD recording device 8 according to the present embodiment uses a timing means to record the performance sound of the automatic piano 10 in synchronization with the reproduction of the video data by the video camera 100 and the reproduction of the CD audio data by the controller 3 or the like. A function of appropriately adjusting the value of the delta time indicating the occurrence time interval between the preceding event and the subsequent event according to the difference between the time measured and the time indicated by the MTC supplied from the controller 3. Is equipped with. The configuration of the FD recording device 8 having such various functions will be described below with reference to FIG.
Will be described with reference to.

FIG. 6 is a block diagram for explaining the configuration of the control circuit provided in the FD recording device 8. The control circuit shown in FIG. 6 has a function of creating the SMF described above and recording the SMF in the FD, the time measured by the time measuring means 220, and the time code converter 200 to supply the time from the controller 3. It has a function of correcting a deviation generated from the time indicated by the MTC.

The clock means 220 has the same structure as the clock means 211 (see FIG. 3) of the controller 3 described above, and is "+1" each time the tempo clock CT is given by the adder 221 and the elapsed time register 222. An accumulator that performs addition of “” is configured. The adder 221 adds the data stored in the elapsed time register 222 and the fixed value “+1” and outputs the result. An initial value “0” is written in the elapsed time register 222 when the synchronous reproduction recording is started, and thereafter, the tempo clock CT is written in the elapsed time register 222.
Each time is given, the output data of the adder 221, that is, the addition result of the data stored in the elapsed time register 222 at that time point and “+1” is written. The output data N of the elapsed time register 222 is used as elapsed time data representing the elapsed time after the start of synchronous recording.

The SMF creation / writing control means 250 is
Under the control of the controller 3, it has a function of creating an SMF, a function of writing the created SMF in an FD mounted on the FD recording device 8, and the like. The SMF creation / writing control means 250 is used for the MIDI event generation circuit 14
When various events (MIDI events, etc.) generated by the above are received from the controller 3, a command to generate the delta time is sent to the delta time generation means 240.

When the delta time generation means 240 receives the command to generate the delta time, it generates the delta time (details will be described later) and returns it to the SMF creation / writing control means 250. SMF creation / writing control means 25
0 is the track chunk TT which is the MIDI data composed of the delta time received from the delta time generation means 240 and the event received from the controller 3.
(See FIG. 5), the identification information (VI) for identifying the reproduced VT supplied from the controller 3 is stored.
D) and identification information for identifying the reproduced CD (C
-ID) is stored in the header chunk HT (see FIG. 5) to create the SMF, and this is stored in the F attached to the FD recording device 2.
Write to D.

The delta time generation means 240 comprises a previous elapsed time register 241 and a correction value register 242, and generates a delta time based on a command from the SMF creation / writing control means 250. In the previous elapsed time register 241, the SMF creation / writing control means 2 is set in the previous time.
The elapsed time data (referred to as Nf for convenience) acquired from the timing means 220 when the command to generate the delta time is received from 50 is held. The initial value “0” is written in the previous elapsed time register 241 when the synchronous reproduction recording is started.

The correction value register 242 holds the correction value R of the delta time generated by the delta time adjusting means 230. As will be described later in detail, in the present embodiment, by appropriately setting the correction value R, the time measured by the time measuring unit 220 and the MTC supplied from the time code converter 200 via the controller 3 are indicated. Correct the deviation that occurs between time and time. An initial value “0” is written in the correction value register 242 when synchronous recording is started.

Upon receiving the instruction to generate the delta time from the SMF creation / writing control means 250, the delta time generation means 240 acquires the elapsed time data N at the present time stored in the elapsed time register 222 from the timing means 220. After that, the elapsed time data Nf held in the previous elapsed time register 241 (the elapsed time acquired from the time measuring means 220 when the instruction to generate the delta time from the SMF creation / writing control means 250 was received last time) is obtained. The elapsed time data Nf is subtracted from the read and elapsed time data N. Then, the delta time generating means 240 reads the correction value R of the delta time held in the correction value register 242, adds this correction value R and the subtraction result N-Nf, and adjusts the addition result N-Nf + R. The later delta time is returned to the SMF creating / writing control means 250. Then, the delta time generation means 24
0 updates the elapsed time data stored in the previous elapsed time register 241 to the elapsed time data newly acquired from the clock means 220 this time.

The delta time adjusting means 230, in order to perform the recording in synchronization with the reproduction of the VT, the time measured by the time measuring means 220 and the MTC supplied from the controller 3.
The value of the delta time, which indicates the time interval between the preceding event and the subsequent event, is adjusted appropriately according to the difference between the time indicated by and the time indicated by.

FIG. 7 is a flowchart showing the delta time adjustment processing executed by the delta time adjustment means 230. When the controller 3 gives an MTC larger than "0" in step Sa1, the delta time adjusting means 230 sets the MTC to the first recording time data T.
Set to CD and proceed to step Sa2. When the delta time adjusting means 230 proceeds to step Sa2, the time measuring means 220
Then, the elapsed time data N at that time is acquired, and the acquired elapsed time data N is multiplied by the period τ of the tempo clock CT to obtain the second recording time data TFD (= N * τ). The second recording time data TFD represents the elapsed time from the start of synchronous recording to the present.
Next, the delta time adjusting means 230 determines whether or not the absolute value | TCD-TFD | of the difference between the first recording time data TCD and the second recording time data TFD is within a preset allowable range Δ. It is determined (step Sa
3). When the delta time adjusting means 230 determines that the absolute value of the difference between the first recording time data TCD and the second recording time data TFD is within the predetermined allowable range Δ (step Sa3; YES), the delta time Correction value R
As a result, "0" is written in the correction value register 242 (step Sa4), and the delta time adjustment processing is ended.

On the other hand, the delta time adjusting means 230 includes the first recording time data TCD and the second recording time data TF.
If it is determined that the absolute value | TCD-TFD | of the difference of D exceeds the predetermined allowable range Δ, the first recording time data T
It is determined whether the CD is larger than the second recording time data TFD (step Sa5). When the first recording time data TCD is larger than the second recording time data TFD, it can be said that the time measured by the time measuring means 220 is behind the time indicated by the MTC supplied from the controller 3. Delta time adjustment means 230
Judges that the first recording time data TCD is larger than the second recording time data TFD (step Sa5;
YES), the second recording time data TFD and the first recording time data TFD are corrected in order to correct such a time difference by reducing the delta time.
Of the recording time data TCD of the
D-TCD (<0) is divided by the period τ of the tempo clock CT, and this division result (TFD-TCD) / τ is
The correction value R of the delta time is written in the correction value register 242 shown in FIG. 6 (step Sa6), and the delta time adjustment processing is ended. In this case, the delta time correction value R written in the correction value register 242 is a negative value.

On the other hand, the first recording time data TCD is the second
If it is smaller than the recording time data TFD of
It can be said that the time counted by 20 is ahead of the time indicated by the MTC supplied from the controller 3. When the delta time adjusting means 230 determines that the first recording time data TCD is smaller than the second recording time data TFD (step Sa5; NO), the time deviation is corrected by increasing the delta time.
Second recording time data TFD and first recording time data T
The difference with the CD is calculated, and the calculated difference TFD-TCD (>
0) is divided by the period τ of the tempo clock CT, and the division result (TFD-TCD) / τ is written in the correction value register 242 shown in FIG. 6 as the correction value R of the delta time (step Sa7), and the delta time adjustment is performed. The process ends. In this case, the correction value R of the delta time written in the correction value register 242 is a positive value. The above is the details of the multimedia system 600 according to the present embodiment. Hereinafter, the operation of the multimedia system 600 will be described in detail with reference to FIG.

FIG. 8 is a time chart showing the operation of the multimedia system 600 according to this embodiment. More specifically, FIG. 8 shows the reproduction of the CD audio data stored in the CD and the recording of the music data corresponding to the user's piano performance, etc., in synchronization with the reproduction of the video data stored in a certain VT. As an example of the case of performing, a time code (MTC) for musical sound output from the VT through the time code converter 200, video data read from the VT, VT audio data read from the VT and output from the controller 3, The CD audio data read from the HD and output from the controller 3, the time measured by the time measuring means 220 of the FD recording device 8, and the MIDI event ME generated by the MIDI event generating circuit 14.
-N (n = 1, 2, ...) Is shown in time series. In FIG. 8, p [k] (k = 0, 0.25, 0.
5, ~), va [k] (k = 0, 0.25, 0.5,
~), Ca [k] (k = 0, 0.25, 0.5, ~)
Is the video data of the part within the section from k seconds to k + 1 seconds, when the reproduction start time of the video data is 0 seconds,
These are VT audio data and CD audio data.
In addition, r [k] (k = 0, 0.25, 0.5,
When the reproduction start time of the video data is 0 second, (-) is the time measured by the time measuring means 220 when k seconds have elapsed. Here, the time code for the musical tone (MT
C) is originally output from the time code converter 200 at an interval of about 1 sec / 30, but in the following description, for convenience, the MTC is output from the time code converter 200 at an interval of 1 sec / 4 (= 250 msec). It is assumed that

When performing synchronous reproduction / recording, the user first operates the operation unit 4 or the like to reproduce C in synchronization with the reproduction of VT.
A command for selecting D and a command to wait for recording on the FD (hereinafter, referred to as a pause command) are input. When the controller 3 receives an instruction to select a CD to be reproduced and a pause instruction via the operation unit 4 or the like, the controller 3 issues to the CD drive device 1 an instruction to read the music data stored in the corresponding CD according to the instruction to select the CD. While sending, pause command FD
Transfer to the recording device 8. The CD drive device 1 reads the music data stored in the corresponding CD according to the instruction of the controller 3, and sends it to the controller 3 via the DSP 2. When the controller 3 receives the music data via the DSP 2, it stores it in the HD.

After that, the user attaches the VT to be reproduced to the video camera 110 and operates the operation unit 130 of the video camera 110 to input a command to start reproduction of video data and the like. When such a command is input, the reproducing unit 120 of the video camera 100 reads the video data p [0] recorded on the VT video track, supplies it to the monitor 121, and records it on the VT musical sound track. The VT audio data va [0] being read,
Supply to the controller 3. Furthermore, the reproducing unit 120
In response to the reading of the video data and the VT audio data, the video time code “0” added to the VT is read and sent to the time code converter 200.

The monitor 121 starts displaying an image according to the image data p [0] received from the video camera 100. On the other hand, the controller 3 uses the video camera 100
When the VT audio data va [0] is received from the VT audio data va [0] and the musical tone time code (MTC) “0” is received via the time code converter 200, the MTC “0” and a command to cancel the pause state (hereinafter, pause command). (A cancellation command) is sent to the FD recording device 8, an analog audio signal is generated from the VT audio data va [0] and output to the mixer 5, and control for outputting as a sound is started, and further based on the MTC. C stored in HD
Control of reading music data including D audio data is started.

More specifically, when the controller 3 receives the MTC "0" from the time code converter 200, it immediately starts the time counting using the time counting means 211. However, in the elapsed time adjusting means 212 of the controller 3,
The elapsed time is not adjusted using this MTC "0". After that, when 250 msec has passed, MTC "0.25" is output from the time code converter 200. At this time, the elapsed time adjusting means 212 displays "0".
Since the larger MTC is supplied, the adjustment of the elapsed time described above is performed. Then, thereafter, every time the MTC is supplied from the time code converter 200, the above-mentioned elapsed time is adjusted. On the other hand, the CD audio data reproduction control means 213 refers to the elapsed time data N stored in the elapsed time register 211b of the time measuring means 211 and the CD time code in the music data stored in the HD, and refers to the relevant time. The control for reading out the CD audio data in the music data and outputting this as a sound is performed.

As a result, when a reproduction instruction of the video data stored in the VT is issued, the video display corresponding to the video data and the generation of the sound corresponding to the CD audio data are immediately started. Then, when 250 msec elapses after the reproduction instruction is issued, the adjustment of the elapsed time by the elapsed time adjusting means 212 is started. Therefore, of the video data p [k], the VT audio data va [k], and the CD audio data ca [k], the one that is reproduced in a synchronized state is 250 msec after the beginning of the video or music. Subsequent data, that is, video data after p [0.25], va [0.2]
5] and subsequent VT audio data and ca [0.25]
It is the subsequent CD audio data.

On the other hand, the FD recording device 8 includes the controller 3
Immediately after receiving the MTC "0" and the pause cancel command from the device, the time measuring means 220 is used to start time measurement.
However, the delta time adjusting means 23 in the FD recording device 8
At 0, adjustment of the delta time using this MTC "0" is not performed. After that, when a further 250 msec elapses, the time code converter 200 moves the controller 3
The MTC “0.25” is output via. At this time,
In the delta time adjusting means 230, M larger than "0"
Since the TC has been supplied, the delta time adjustment described above is performed.

As described above, when the reproduction instruction of the video data stored in the VT is issued, the reproduction of the video data and the F
Recording of the performance sound by the D recording device 8 is started. Then, after a further 250 msec has elapsed, the delta time adjusting means 230 starts adjusting the delta time in accordance with the difference between the time measured by the time measuring means 220 and the time indicated by the MTC supplied from the controller 3. To do. Then, thereafter, each time the MTC is supplied from the controller 3, the delta time adjusting means 230 adjusts the delta time described above. After that, for example, as shown in FIG. 8, the clocking time r [1.0] by the clocking means 220,
When MIDI events ME-1, -2, and -3 are generated at r [1.5] and r [2.0], and delta times corresponding to the MIDI events are generated, these are SM.
It is supplied to the F creation / writing control means 250.

The SMF creation / writing control means 250 is
MIDI events ME-1, -2, -3 and each MIDI
Each MI when receiving the delta time corresponding to the event
The MIDI data including the DI event and each delta time is stored in the track chunk TT of the SMF. afterwards,
When a command to end the synchronous recording is input through the operation unit 4, the controller 3 identifies the reproduced VT (V-ID) and the reproduced CD. The information (C-ID) is supplied to the SMF creation / writing control means 250. The SMF creation / writing control means 250 stores the identification information (V-ID) and the identification information (C-ID) in the header chunk HT (see FIG. 5), and is composed of the header chunk HT and the track chunk TT. The SMF is written in the FD attached to the FD recording device 8, and the process ends. As is apparent from the above description, the feature of this embodiment is that the time management of the synchronous reproduction recording is performed based on the MTC output from the time code converter 200. The MI recorded in the FD
The configuration, operation, and the like of the multimedia system that reproduces the DI data, the CD audio data stored in the HD, and the video data stored in the corresponding VT in synchronization will be described in detail in the second embodiment. To clarify.

As described above, according to the multimedia system 600 according to the present embodiment, the reproduction of the CD audio data transferred from the CD to the HD and the recording of the MIDI data for the FD are synchronized with the reproduction of the VT. It becomes possible to do.

In the first embodiment described above, VT
The case where the reproduction of the CD audio data transferred from the CD to the HD and the recording of the MIDI data with respect to the FD are performed in synchronization with the reproduction of is explained. CD
It is also possible to reproduce the CD audio data transferred from the HD to the HD and record the MIDI data for the FD. In this case, the recording unit 110 of the video camera 100
Records the recording information including the video data, the VT audio data, and the video time code on the VT, and sequentially supplies the video time code to the time code converter 200. The time code converter 200 converts the sequentially supplied video time code into a musical sound time code (MTC) and supplies the converted MTC to the controller 3. In addition, in synchronization with the recording of the video data on the VT, the CD
CD audio data transferred from HD to HD,
The output timing of the MTC when recording the music data in the FD, the recording start timing of the music data in the FD, and the like can be described in almost the same manner as in the above-described first embodiment, and therefore will be omitted. As described above, by reproducing the CD audio data transferred from the CD to the HD and recording the music data in the FD in synchronization with the recording of the video data or the like on the VT, for example, for live recording / recording, promotion, and show. It is possible to easily create a video-attached content.

In the above-described first embodiment, the automatic piano 10 has been described as an example. However, for example, an electronic violin equipped with a detection mechanism for detecting string vibration, an operation state of an operator (piston, etc.), etc. are detected. The present invention can be applied to any electronic musical instrument such as an electronic trumpet having a detection mechanism for Further, not only the electronic musical instrument but also a natural musical instrument (for example, a violin) is provided with a mechanism for detecting the operation state of the natural musical instrument (for example, a detection mechanism for detecting a string vibration), whereby the electronic musical instrument is The present invention can be applied similarly to. In addition to these electronic musical instruments, a PC (Personal Compute) with music production software installed
It can be applied to various electronic devices that can detect the operation state of the user and can instruct the generation of MIDI events according to the detection result, such as r). That is, the "musical instrument" described in the claims, detects the operation state of the user,
It refers to various devices that can instruct the generation of MIDI events according to the detection result.

In the first embodiment described above, the CD
Although the HD is illustrated as a means for storing audio data and the like, various storage means such as a RAM may be used instead of the HD. As described above, when a RAM or the like is used instead of the HD, for example, the CD audio data corresponding to a portion slightly before the portion to be reproduced at that time is preread and stored in the storage unit such as the RAM. If so controlled, a storage means having a small memory capacity can be used as a storage means for storing the CD audio data. Further, in the above-described first embodiment, the FD is described as an example of the storage medium for storing MIDI data, but the present invention can be applied to any storage medium such as an MO (Magneto Optical disk) and a memory stick. Here, the present invention can also be implemented in a mode of manufacturing and selling an automatic piano including all the components shown in FIG. 1, such as a video camera, a CD drive device, an FD recording device, and a controller. According to such an automatic piano, the user can
Playback and FD of CD audio data transferred from CD to HD in synchronization with playback of video data recorded on T
Performance data (for example, performance data obtained by the user playing an automatic piano) can be recorded.

<Modification 1> The FD recording apparatus 8 according to the first embodiment described above adjusts the delta time based on the MTC output from the time code converter 200 via the controller 3 to measure the time. The deviation generated between the time measured by the means 220 and the time indicated by the MTC supplied from the controller 3 is suppressed. On the other hand, in the first modification, the VT to time code converter 200 is used.
By adjusting the elapsed time data N in the elapsed time register 222 based on the MTC output via the, the time measured by the time measuring means 220 and the time indicated by the MTC supplied from the controller 3 are adjusted. Suppress the generated deviation.

FIG. 9 is a block diagram for explaining the configuration of the control circuit provided in the FD recording apparatus 8 according to the first modification, and the elapsed time adjusting means 230 'shown in FIG.
It corresponds to the elapsed time adjusting means 212 of the controller 3 shown in FIG. The control circuit shown in FIG. 9 has an elapsed time adjusting means 23 instead of the delta time adjusting means 230.
6 is the same as the control circuit shown in FIG. 6 except that 0'is provided and the correction value register 242 is omitted. Therefore, corresponding parts are designated by the same reference numerals and description thereof is omitted. .

The elapsed time adjusting means 230 'is the time measuring means 2
It is means for adjusting the elapsed time data N in the elapsed time register 222 in accordance with the difference between the time measured by 20 and the time indicated by the MTC supplied from the controller 3. When the controller 3 gives the MTC larger than “0”, the elapsed time adjusting means 230 ′,
Similar to the elapsed time adjusting means 212 of the controller 3, the elapsed time adjusting process shown in FIG. 4 is executed. The details of the elapsed time adjustment process have been clarified in the above-described first embodiment, and a description thereof will be omitted. Also in the present modified example described above, the same effect as that of the first embodiment can be obtained.

B. Second Embodiment FIG. 10 is a diagram showing the configuration of a multimedia system 700 according to the second embodiment. The multimedia system 700 according to the present embodiment synchronizes with the reproduction of VT,
In addition to having the function of playing CD audio data stored in HD and recording MIDI data in FD, the MIDI data recorded in FD in this way and the CD audio data stored in HD , And has a function of synchronously reproducing the video data and the like stored in the corresponding VT. It should be noted that the multimedia system 700 shown in FIG. 10 is different from the multimedia system 600 shown in FIG. 1 except that an FD playback device 8a is provided and an automatic piano 20 is provided instead of the automatic piano 10. Is almost the same as. Therefore, the corresponding parts are designated by the same reference numerals, and the description of the parts overlapping with those of the first embodiment will be omitted.

The controller 3 has not only the function described in the first embodiment but also a function of performing control for synchronous reproduction of FD, HD and VT. When the synchronous reproduction instruction is given from the operation unit 4 or the like, the controller 3 issues the MIDI data reproduction instruction to the FD reproduction device 8a.
Send to. Here, for example, when the MIDI data stored in the FD should be reproduced from the beginning of the music, the controller 3
Is a command to start reading from MIDI data after 500 msec has passed from the beginning of the music.
Send to. Note that these means will be described later. Further, the controller 3 causes the time code (MT) for musical sound from the video camera 100 via the time code converter 200.
C) each time the MTC is supplied, the MTC is set to the FD reproducing device 8a.
Send to.

The automatic piano 20 is composed of an automatic piano, has the function as the automatic piano 10 described in the first embodiment, and also has a function of performing an automatic performance according to the MIDI data read from the FD by the FD reproducing device 8a. Is equipped with. The automatic piano 20 includes the components of the automatic piano 10 (piano 11 and the like), a piano electronic circuit 16, and a drive solenoid group 17. Here, when performing an automatic performance by the ensemble sound source 18, the controller 3 controls the FD reproducing device 8a.
The event received from is sent to the ensemble sound source 18. Further, when the piano sound source 15 or the piano 11 is used for automatic performance, the controller 3 controls the FD reproducing device 8a.
Sends the event received from the piano electronic circuit 16.
Which means to perform the automatic performance is determined according to an instruction from the operation unit 4.

As described above, the digital tone signal generated by the ensemble sound source 18 is converted into an analog tone signal by the controller 3 and sent to the mixer 5. The mixer 5 according to the present embodiment has both an analog input port for receiving an analog signal and a digital input port for receiving a digital signal, and mixes all signals input via these ports. Can be output. Therefore, the digital tone signal output from the ensemble sound source 18 may be directly supplied to the mixer 5 without passing through the controller 3.

The piano electronic circuit 16 is a circuit that receives an event from the controller 3 and performs control for automatic performance. The piano electronic circuit 16 controls the automatic performance by one of two methods. First, the first
In this method, the event received from the controller 3 is sent to the piano sound source 15. As mentioned above, piano sound source 1
Reference numeral 5 is a device for electronically generating an analog audio signal of a piano sound designated by an event. In the second method, the piano electronic circuit 16 controls the energization of the drive solenoid group 17 according to the event received from the controller 3. The drive solenoid group 17 includes a plurality of solenoids corresponding to a plurality of keys provided on the piano 11 and a plurality of solenoids corresponding to a plurality of pedals. When the piano electronic circuit 16 receives from the controller 3 an event instructing the pressing of a certain key, a driving current is passed through the solenoid corresponding to the key, and the solenoid generates the magnetic force required to press the key. The same applies when an event instructing key release is received. An instruction to handle the event by either the first method or the second method is given from the operation unit 4 to the controller 3, and the controller 3 sends this instruction to the piano electronic circuit 16. The piano electronic circuit 16 follows the received instruction.

There is a delay from the output of the event by the controller 3 to the generation of the automatic performance sound,
The delay time differs depending on which means is used to perform the automatic performance. When playing automatically with an automatic piano,
It takes, for example, 500 msec from the controller 3 outputting the event until the automatic performance sound is generated. Therefore, in the present embodiment, when an instruction for synchronous reproduction is given from the operation unit 4 and the automatic piano 20 is selected as a means for automatic performance, VT, HD and FD are set as follows. Performs synchronized playback of. First, FD
Command to wait for playback (ie pause command)
Is sent from the controller 3 to the FD reproducing device 8a. Then, when a command to reproduce the video data or the like (that is, a command to start the synchronous reproduction) is input to the video camera 100, the time code converter 2 from the video camera 100.
Supply of MTC to the controller 3 is started via 00. As described above, the controller 3 sends to the FD reproducing device 8a a command to start reading from the MIDI data after 500 msec has elapsed from this point, while starting reading and reproducing the CD audio data stored in the HD. . As described above, with respect to the music data stored in the FD, the reading is not started from the beginning of the music but is started from the MIDI data after 500 msec has elapsed, and thereafter, the video data in the VT, etc. Reproduction, reproduction of CD audio data in HD, and reproduction of MIDI data in FD are kept in sync.

The FD reproducing device 8a sequentially reads MIDI data from the FD and supplies it to the controller 3 in accordance with a command from the controller 3. This MIDI data is
For example, it is MIDI data recorded by the FD recording device 8 in synchronization with the reproduction of VT and HD (see FIG. 8). F
The D playback device 8a, after reading an event from the FD and sending it to the controller 3, waits for a time indicated by the delta time after the event and repeats the process of reading the subsequent event. This is the basic operation of the FD reproducing device 8a. The FD reproducing device 8a has a function of reading the MIDI data from the FD (that is, a function as a sequencer) as described above.
It has a timing adjusting function for synchronizing the read operation with the reproduction of the video data and the like by the video camera 100 and the reproduction of the CD audio data stored in the HD. Hereinafter, the FD reproducing device 8a having such various functions
The configuration will be described with reference to FIG.

FIG. 11 is a block diagram showing the configuration of the control circuit provided in the FD reproducing device 8a. The control circuit shown in FIG. 11 has a function as a sequencer and a timing adjusting function. Event buffer 3
Reference numeral 02 is a buffer for storing the events read from the FD. Here, one delta time may be followed by two or more consecutive events. In such a case, all consecutive events are read from the FD,
It is stored in the event buffer 302. The delta time register 303 is a register that stores the delta time read from the FD. As described above, this delta time specifies the elapsed time between two events that are adjacent to each other.

The adder 311 and the register 312 form an accumulator. This accumulator obtains the accumulated value of a series of delta times which are sequentially read from the FD and stored in the register 303. More specifically, the accumulated value of the delta time at the present time is stored in the register 312.
To be held by. Also, adder 3
11 adds the accumulated value stored in the register 312 and the delta time stored in the register 303, and outputs the addition result M. This M is a numerical value corresponding to the target time for transferring one or a plurality of events stored in the event buffer 302 to the controller 3. Therefore, in the following, this M is called target time data. When the time designated by the target time data M arrives, the event transfer control means 330 causes the registers 303 and 312 to end.
The write clock is supplied to. As a result, the adder 3
11 output data, that is, register 312 till then
A value obtained by adding the accumulated value stored in the register and the delta time stored in the register 303 is stored in the register 312 as a new accumulated value. The new delta time read from the FD at this time is stored in the register 303. The event transfer control means 330 will be described later.

The adder 321 and the register 322 constitute an accumulator that accumulates "+1" every time the tempo clock CT 'is given from the clock generating means 210 of the controller 3. The adder 321 adds the data stored in the register 322 and the fixed value “+1” and outputs the result. An initial value is written in the register 322 when reproduction of MIDI data from the FD is started. This initial value is
When the tempo clock cycle is τ ', 500 mse
c / τ '. Thereafter, each time the tempo clock CT ′ is given to the register 322, the output data of the adder 321, that is, the addition result of the data stored in the register 322 at that time and “+1” is written. The output data N ′ of the register 322 is used as elapsed time data representing the elapsed time after the start of reproduction of MIDI data.

When the elapsed time data N ′ reaches the target time data M, the event transfer control means 330 sends to the controller 3 a transfer instruction to read and capture one or more events in the event buffer 202. Further, at this time, the event transfer control means 330 supplies the write clock to the registers 303 and 312 as described above. As described above, in the present embodiment, M after 500 msec has elapsed from the beginning of the music data.
Reading is started from the IDI data. Therefore, the event transfer control unit 330 sets the target time data M to 500 m when the generation of MIDI data from the FD is started.
Event and delta time reading and delta time accumulation are repeatedly executed until sec / τ ′ is reached.

The timing adjusting means 341 is a circuit for adjusting the timing for synchronizing the output operation of the MIDI data event to the controller 3 with the reproduction of the video data or the like from the VT. More specifically, when an MTC larger than “0” is given from the controller 3, the timing adjusting means 341 sets the MTC as the first reproduction time data TCD ′, and the elapsed time data N ′ at that time is used as the tempo clock. Multiply the period τ ′ of CT ′ and subtract 500 msec from the multiplication result
Reproduction time data TFD '= N * τ'-500 msec
Ask for. The second reproduction time data TFD ′ represents how many seconds after the beginning of the music the currently-played portion is.

Then, the timing adjusting means 341 compares these reproduction time data TCD 'and TFD',
The following processing is performed based on the comparison result. a. When the difference between TCD ′ and TFD ′ falls within a predetermined allowable range In this case, the timing adjusting means 341 does nothing.

B. When TCD '>TFD' and the difference between the two exceeds an allowable range In this case, the video is displayed by the playback of the VT at the portion in the music currently played by the event read from the FD. Time difference than where there is TCD'-TF
It can be said that it is a place delayed by D '. Therefore, the timing adjusting means 341 divides the time difference TCD'-TFD 'by the period τ' of the tempo clock CT ', and subtracts this division result (TCD'-TFD') / τ 'from the delta time in the register 303. , The delta time after the subtraction is written in the register 303. As a result, F for a while
Automatic performance by reading events from D, video display by reading video data from VT, and C from HD
The performance by reading the D audio data will be kept in a synchronized state.

C. When TCD '<TFD' and the difference between the two exceeds the allowable range In this case, the video is displayed by the playback of the VT at the part in the music currently played by the event read from the FD. Time difference TFD'-TC
It can be said that this is a point advanced by D '. Therefore, the timing adjusting means 341 divides the time difference TFD'-TCD 'by the period τ' of the tempo clock CT ', and adds this division result (TFD'-TCD') / τ 'to the delta time in the register 303. , Delta time after addition is written in the register 303. The above is the details of the multimedia system 700 according to the present embodiment. Hereinafter, the operation of the multimedia system 700 will be described in detail with reference to FIG.

FIG. 12 is a time chart showing the operation of the multimedia system 700 according to this embodiment. More specifically, FIG. 12 shows a case where V and HD are synchronized with FD, HD, and FD from the beginning of a music piece.
A tone time code (MTC) output from T via the time code converter 200, video data read from VT, VT audio data read from VT, read from HD, read from HD,
CD audio data output from the controller 3,
MIDI data read from FD and piano 11
The sounds generated from are shown in chronological order. This Figure 1
2, p [k] (k = 0, 0.25, 0.5,
~), Va [k] (k = 0, 0.25, 0.5, ~),
ca [k] (k = 0, 0.25, 0.5, ...) Is k when the reproduction start time of the video data is 0 seconds.
These are video data, VT audio data, and CD audio data in the part within the section from 1 second to k + 1 seconds. Also, m
[K] (k = 0, 0.25, 0.5, ...) is MIDI data used for performance control of a portion within a section from k seconds to k + 1 seconds when the start time of the music is 0 seconds. Is.
Here, the time code (MTC) for musical tone is originally 1
It is output from the time code converter 200 at intervals of about sec / 30, but in the following description, for the sake of convenience, 1 sec /
Time code converter 2 at intervals of 4 (= 250 msec)
Assume that MTC is output from 00.

When performing an automatic performance using the drive solenoid group 17 and the piano 11, a pause command is first sent from the controller 3 to the FD reproducing device 8a. After that, when the user operates the operation unit 130 of the video camera 100 or the like to input a command to start the synchronous reproduction, the MTC supply from the video camera 100 is started via the time code converter 200. When the controller 3 receives the MTC “0” via the time code converter 200, it starts the control of reading the CD audio data stored in the HD as will be described later, and 500 m from this point.
A command to start reading from the MIDI data after the elapse of sec is sent to the FD reproducing device 8a. FD playback device 8a
Upon receiving the command from the controller 3, the
MIDI after 500 msec from the beginning of music from D
Data The operation of reading MIDI data is started. As a result, the MIDI data m [0.5], m [0.75],
Are sequentially read from the FD, and the events in the MIDI data are transferred to the controller 3. In this operation, based on the delta time in the MIDI data, MI
The transfer timing of the event in the DI data is controlled. The control of the transfer timing based on the delta time is as described above.

The controller 3 is the time code converter 20.
The MTC “0” received from 0 is sent to the FD playback device 8a. However, the timing adjusting means 341 in the FD reproducing device 8a does not adjust the delta time using this time code "0". Then 25 more
When 0 msec elapses, the controller 3 causes the MTC
“0.25” is output. At this time, the timing adjusting means 341 adjusts the above-mentioned delta time because the time code larger than "0" is supplied. Then, thereafter, each time the MTC is supplied from the controller 3, the above-mentioned delta time adjustment is performed.

On the other hand, when the controller 3 receives the MTC "0" from the time code converter 200, it immediately starts time counting using the time counting means 211. But,
In the elapsed time adjusting means 212 of the controller 3, this M
The elapsed time is not adjusted using TC “0”. After that, when 250 msec has passed, MTC "0.25" is output from the time code converter 200.
At this time, since the MTC larger than "0" is supplied to the elapsed time adjusting means 212, the above-mentioned elapsed time is adjusted. Then, thereafter, every time the MTC is supplied from the time code converter 200, the above-mentioned elapsed time is adjusted. On the other hand, the CD audio data reproduction control means 213 refers to the elapsed time data N stored in the elapsed time register 211b of the time measuring means 211 and the CD time code in the music data stored in the HD, and refers to the relevant time. The control for reading out the CD audio data in the music data and outputting this as a sound is performed.

As a result, when a VT reproduction instruction is issued,
Video of an orchestra, etc. that immediately corresponds to video data, V
A performance corresponding to the T audio data and a performance corresponding to the CD audio data are started. And 5 more
When 00 msec has elapsed, the performance corresponding to the MIDI data is started. Then, when 250 msec has further elapsed, the performance corresponding to the MIDI data whose timing has been adjusted by the timing adjusting means 341 is started. Therefore, of the video data p [k], the VT audio data va [k], the CD audio data ca [k], and the MIDI data m [k], the one that is reproduced in synchronization is the beginning of the video and the music. From 750 msec
After that, that is, p [0.7
5] and later video data, and VT after va [0.75]
They are audio data, CD audio data after ca [0.75], and MIDI data after m [0.75]. As a result, the clock means 2 described with reference to FIG.
Time measured by 20 r [1.0], r [1.5], r
MIDI data m [1.0], m [1.5], m [2.
0] and the corresponding CD audio data ca [1.
0], ca [1.5], ca [2.0] and corresponding V
T audio data va [1.0], va [1.5],
va [2.0] and video data p [1.0], p [1.
5] and p [2.0] are reproduced in synchronization.

As is clear from the above description, according to the multimedia system 700 of this embodiment, the VT
In addition to recording the MIDI data in synchronization with the reproduction of the CD and the CD, the recorded MIDI data can be reproduced in synchronization with the corresponding CD audio data, VT audio data, video data and the like.

Prior to the above-mentioned synchronized reproduction,
The controller 3 receives the identification information (V-ID) of the VT from the video camera 100, and the identification information (V-ID) of the VT stored in the header chunk HT of the SMF recorded in the FD from the FD playback device 8a. If the identification information of the two does not match, the reproduction of the MIDI data recorded in the FD (that is, automatic performance) may be stopped. By adopting such a configuration, it is possible to prevent the problem that the automatic performance that does not match the image is performed and the user feels uncomfortable. In the above description, the VT identification information (V-ID) has been described as an example, but the CD identification information (C-
The ID) may be used to determine whether or not the reproduction of MIDI data should be stopped, or the identification information of the two may be used to make the determination.

In the second embodiment described above, VT
Although the case where the reproduction of the CD audio data stored in the HD and the reproduction of the MIDI data stored in the FD are performed in synchronization with the reproduction of the above, the recording of the video data or the like to the VT is not performed, but the reproduction of the VT. In sync with
It is also possible to reproduce CD audio data stored in HD and MIDI data stored in FD. In this case, the recording unit 110 of the video camera 100 stores video data, VT audio data,
While recording the record information including the video time code,
The video time code is sequentially converted to the time code converter 200.
Supply to. The time code converter 200 converts the sequentially supplied video time code into musical sound time code (MT
C), and the converted MTC is supplied to the controller 3. It should be noted that, in synchronization with the recording of the video data and the like on the VT, the CD audio data stored in the HD is reproduced and the MIDI data stored in the FD is reproduced, the output timing of the MTC and the read timing of the MIDI data. Since the above can be explained in substantially the same manner as in the above-described second embodiment, description thereof will be omitted. In this way, the CD audio data stored in the HD is reproduced in synchronization with the recording of the video data, the VT audio data, etc. for the VT, and the MIDI stored in the FD is reproduced.
The data may be reproduced.

In the second embodiment described above, the case where the CD audio data to be reproduced synchronously is stored in the HD in advance has been described as an example. However, the CD audio data to be reproduced is recorded in the CD drive device 1. If the CD is loaded, the CD audio data can be synchronously reproduced even if the CD audio data is not stored in the HD in advance. Specifically, when an instruction to select a VT to be reproduced is input from the operation unit 130 of the video camera 100, the controller 3
FD the identification information (V-ID) for identifying the VT
Send to the playback device 8a. The FD reproducing device 8a refers to the header chunk HT of the SMF written in the FD, using the identification information (V-ID) as a search key.

The FD reproducing device 8a uses the identification information (VI
When the identification information (V-ID) that matches D) is found, the identification information (C-ID) for identifying the CD recorded in association with the identification information is read and returned to the controller 3. The controller 3 is the FD reproducing device 8a.
When the identification information (C-ID) is received from the CD drive unit 1, an instruction to read the music data including the CD audio data stored in the corresponding CD is sent to the CD drive device 1 together with the identification information (C-ID). The CD drive device 1 reads the music data stored in the corresponding CD according to the instruction of the controller 3, and sends it to the controller 3 via the DSP 2. When the controller 3 receives the music data via the DSP 2, it stores it in the HD. The operation after storing the music data in the HD can be described in the same manner as in the second embodiment described above, and thus will be omitted. By executing the processing described above, it becomes possible to synchronously reproduce the VT, the CD, and the FD even if the CD audio data to be reproduced is not stored in the HD in advance.

Here, the present invention relates to, for example, a video camera,
FD recording device, FD reproducing device, controller, etc.
It can also be implemented in the manner of manufacturing and selling an automatic piano including all the components shown in FIG. According to such an automatic piano, the user reproduces the CD audio data stored in the HD and synchronizes with the MI in synchronization with the reproduction of the video data, the VT audio data, etc. recorded in the VT.
DI data (such as MIDI data obtained by a user playing an automatic piano) can be recorded in the FD, and video data and VT audio can be synchronized with the automatic performance based on the MIDI data recorded in the FD. Data and CD audio data can be reproduced.

C. Third Embodiment Now, as described in the second embodiment, if the multimedia system 700 is used, it is stored in the HD in synchronization with the reproduction of the video data recorded in the VT and the VT audio data. It is possible to reproduce the CD audio data that is recorded, and record the MIDI data (a music piece or the like obtained by the user playing the automatic piano) in the FD. Further, by using the multimedia system 700, the video data and VT recorded in the VT are synchronized with the automatic performance based on the MIDI data recorded in the FD.
The audio data can be reproduced, and the CD audio data stored in the HD can be reproduced.

By the way, the C thus stored in the HD
When the MIDI data recorded in the FD is reproduced (specifically, the automatic performance by the automatic piano 20) in synchronization with the reproduction of the D audio data, for example, C stored in the HD is reproduced.
The reference pitch of D audio data (specifically, the frequency of the reference sound "A") is 443 Hz,
Assuming that the reference pitch of the piano 11 (live piano) used during the automatic performance is 448 Hz, the piano 11 is synchronized with the reproduction of the CD audio data stored in the HD.
Even if the automatic performance is performed by, the beat may occur due to the deviation of the reference pitch, which may cause the user discomfort.

Therefore, in the multimedia system according to the present embodiment, the reference pitch of the piano 11 (in the above example,
There is no beat even if the piano 11 is automatically performed in synchronization with the reproduction of the CD audio data stored in the HD by the control of automatically adjusting the reference pitch of the CD audio data to (448 Hz). It is possible to give the user a pleasant sound.

Now, the controller 3 according to the present embodiment
In addition to having the functions described in the second embodiment, the CD audio data recording processing and the pitch control / playback processing described below are performed.

FIG. 13 is a flowchart showing the CD audio data recording process. The controller 3 is a CD
When it is detected that the CD on which the music data including the CD audio data to be reproduced is recorded is attached to the drive device 1 and the FD for recording is attached to the FD recording device 8 (step Sb1), the CD drive device 1 is notified. , Sends a command to transfer the music data recorded on the mounted CD. CD
The drive device 1 reads the music data recorded on the CD according to the command, and returns it to the controller 3 via the DSP 2. The controller 3 is the CD drive device 1
When the music data including the CD audio data is received from the DSP via the DSP 2, the music data is transferred to the HD (step Sb2).

Then, when proceeding to step Sb3, the controller 3 judges whether or not an instruction for obtaining the reference pitch of the CD audio data has been sent from the operation section 4 or the like. Here, when the instruction to obtain the reference pitch of the CD audio data is not sent from the operation unit 4 or the like (step Sb3; NO), the controller 3 skips steps Sb4 to Sb8 described below and proceeds to step Sb9. On the other hand, when the controller 3 detects that an instruction for obtaining the reference pitch of the CD audio data is sent from the operation unit 4 or the like (step Sb3; YES), the controller 3 proceeds to step Sb4 and performs fast Fourier transform (FFT) on the CD audio data. Fast Fourier Transform) to obtain the sound pressure (SPL; Sound Pressure Level) for each frequency.

Here, FIG. 14 is a diagram illustrating a signal waveform obtained from CD audio data, in which the horizontal axis represents frequency and the vertical axis represents sound pressure. Here, FIG.
Among the signal components shown in FIG. 4, the frequency component that needs to be analyzed in this embodiment is a frequency component that can be the reference pitch (for example, a frequency component near 440 Hz). Therefore, by using a bandpass filter in advance, 440 Hz
Frequency components in the vicinity (for example, (440 ±
Only α) Hz) may be extracted.

Now, the controller 3 operates as shown in FIG.
The sound pressure of each frequency in the vicinity of 40 Hz is scanned (step Sb5), and it is determined whether a sound pressure equal to or higher than a preset predetermined value (threshold value S shown in FIG. 14) is detected (step Sb6). ). When the sound pressure equal to or higher than the predetermined value is not detected (step Sb6; NO), the controller 3 returns to step Sb4 again and redoes the frequency analysis of the CD audio data. On the other hand, when the sound pressure equal to or higher than the predetermined value is detected (step Sb6; YES), the controller 3
Proceeds to step Sb7, recognizes the frequency (443 Hz in FIG. 14) at which the sound pressure equal to or higher than the predetermined value is detected as the reference pitch of the CD audio data, generates an event representing this reference pitch, and generates the FD recording device 8 Identification information (C
-ID) is notified to the FD recording device 8 (step Sb).
8).

When the FD recording device 8 receives the event and the notification, as shown in FIG. 15, the identification information (C-ID) for identifying the CD audio data in the predetermined storage area of the FD and the reference pitch. Stores the event corresponding to. After that, the user operates the operation unit 4 etc. to C
When the controller 3 is notified of a command to record MIDI data (for example, MIDI data obtained by a user playing an automatic piano) in the FD in synchronization with the reproduction of D (step Sb9), the controller 3 Then, the control for the synchronous recording described in the above-described first embodiment is performed (step Sb10), and the process ends. On the other hand, in step Sb9, M is synchronized with the reproduction of the CD.
The command to record the IDI data in the FD is the controller 3
If not notified, the controller 3 ends the process without performing control for synchronous recording.

The above is the details of the CD audio data recording process. In this embodiment, the case where the reference pitch of the CD audio data is automatically detected as described above is described. However, instead of automatically detecting the reference pitch of the CD audio data, for example, the user uses his / her own pitch. To determine the reference pitch of the CD audio data and operate the operation unit 4 or the like (that is, manual operation)
By doing so, the determined reference pitch may be recorded in the FD.

Now, the CD audio data thus recorded on the CD is stored in the HD, and the MIDI data is F
When the piano automatic performance based on the performance data recorded on the FD is performed in synchronization with the reproduction of the CD audio data recorded on the HD after being recorded on the D, the controller 3 performs the pitch control / reproduction processing described below. Run.

FIG. 16 is a flowchart showing the pitch control / reproduction processing. When an instruction to reproduce the CD audio data recorded on the HD and the MIDI data recorded on the FD in synchronization is sent from the operation unit 4 or the like to the controller 3, the controller 3 instructs the FD reproducing device 8a to
An instruction to read the reference pitch of the CD audio data from the FD is sent (step Sc1). FD playback device 8a
According to the instruction, from the FD to the reference pitch (for example,
The event corresponding to (443 Hz) is read out and returned to the controller 3. When the controller 3 receives the event, the controller 3 should emit a sound (“A (La)”) necessary for detecting the reference pitch of the piano 11, for example, for detecting the reference pitch of the piano on a display unit (not shown). A character message (for example, “Mr. User Z, please play the sound of“ La ”!”) Is displayed. After that, when the user presses a specific key of the piano 11 and the sound of "LA" necessary for detecting the reference pitch is pronounced from the piano 11, the controller 3 causes the piano to play through a microphone (not shown) or the like. The sound produced by 11 is received as musical sound data, and the reference pitch (for example, 448 Hz) of the piano 11 is detected (step Sc2).

As a method of detecting the reference pitch of the piano 11, the sound ("A") necessary for detecting the reference pitch of the piano is displayed on the display unit (not shown) as described above.
In addition to displaying a character message or the like that should be sounded, for example, by sending a command to the piano electronic circuit 16 to sound the sound necessary for detecting the reference pitch of the piano and using the drive solenoid group 17, such sound May be controlled to automatically (that is, without the user pressing a specific key of the piano 11). As for the reference pitch of the piano 11, as described above, for example, the musical tone data is subjected to fast Fourier transform to obtain the sound pressure for each frequency, and the frequency at which the sound pressure equal to or higher than a preset predetermined value is detected is determined. By detecting, it is possible to detect the reference pitch of the piano 11.

When the reference pitch of the piano 11 is detected in this way, the controller 3 compares the detected reference pitch of the piano 11 with the reference pitch of the CD audio data to determine the deviation (pitch difference) of the reference pitch. Obtained (step Sc3). Here, since the detected reference pitch of the piano 11 is 448 Hz and the reference pitch of the CD audio data is 443 Hz, the controller 3
It is determined that the reference pitch of the CD audio data should be increased by 5 Hz (= 448 Hz-443 Hz) from the deviation of both reference pitches, and the reading speed of the CD audio data stored in the HD is determined based on the determination result (step Sc4). . Here, FIG. 17 is a diagram illustrating a signal waveform obtained when the CD audio data is read at each reading speed. More specifically, the signal waveform shown in A of FIG. 17 is a signal waveform obtained when reading is performed at a predetermined read speed Vb, and the signal waveforms shown in B and C of FIG. 17 are respectively obtained from the read speed Vb. Faster read speed Vf (> Vb) and slower read speed Vs
It is a signal waveform obtained when read at (<Vb).

Assuming that a sound having a reference pitch of 443 Hz is produced when read at a predetermined read speed Vb, the CD audio data is read at the read speed Vf as shown in FIG. 17B. If issued,
When a sound having a reference pitch higher than 443 Hz (for example, a sound having a reference pitch of 448 Hz) is generated, while the CD audio data is read at the read speed Vs as shown in C of FIG. Is lower than 443 Hz (for example, a sound having a reference pitch of 440 Hz) is generated. Therefore, the controller 3 sets the reference pitch of the CD audio data to 5 Hz based on the deviation of both reference pitches.
(= 448Hz-443Hz) If it is determined that the CD should be raised, the CD stored in the HD based on the determination result.
The reading speed of the audio data is determined to be higher than the reading speed in the case of producing a sound of 443 Hz. The case of lowering the reference pitch of the CD audio data can be described in the same manner as above, and thus the description is omitted.

Then, the controller 3 reads the CD audio data from the HD at the determined reading speed,
An analog audio signal is generated from the read CD audio data and sequentially supplied to the mixer 5 (step Sc5), while MIDI data is sequentially read from the FD through the FD reproducing device 8a based on the reading speed of the CD audio data ( Step Sc6), the read M
The IDI data is sequentially supplied to the piano electronic circuit 16 (step Sc7).

The piano electronic circuit 16 controls the energization of the drive solenoid group 17 according to an event included in the MIDI data sequentially supplied from the controller 3,
The keys and pedals of the piano 11 are driven. This makes M
The piano 11 is automatically played based on the IDI data. When the controller 3 proceeds to step Sc8, the controller 3 determines whether or not the reproduction of the CD audio data for one song and the reproduction of the MIDI data have been completed. When it is determined that the reproduction of the CD audio data for one song and the reproduction of the MIDI data have not ended (step Sc8; NO), the controller 3 returns to step Sc5, and step Sc5 →
A series of processes such as step Sc6 → step Sc7 is repeatedly executed. Then, the controller 3 determines that the reproduction of the CD audio data for one song and the reproduction of the MIDI data are completed (step Sc8; YE).
S), the pitch control / reproduction processing described above is terminated.

As described above, according to the multimedia system of this embodiment, the CD recorded on the HD
M recorded in the FD in synchronization with the reproduction of audio data
CD for automatic piano performance based on IDI data
Reference pitch of audio data (for example, 443Hz)
The reference pitch of the CD audio data is controlled so that the pitch coincides with the reference pitch of the piano 11 (for example, 448 Hz). As a result, even if the piano 11 automatically performs the performance in synchronization with the reproduction of the CD audio data, it is possible to give the user a pleasant sound without a beat.

<Modification 1> In the third embodiment described above, the case where the MIDI data recorded in the FD is reproduced in synchronization with the reproduction of the CD audio data stored in the HD has been described. The VT audio data is stored in the HD instead of the audio data, and in synchronization with the reproduction of the VT audio data stored in the HD, the F
It is also possible to reproduce the MIDI data recorded in D. Note that the processing executed by the controller 3 or the like in such a case can be described in almost the same manner as in the above-described third embodiment, and thus the description thereof will be omitted.

Further, as shown in FIG. 18, CD audio data and VT audio data to be reproduced are stored in a predetermined storage area of the HD, reproduction of the CD audio data stored in the HD and VT audio stored in the HD are stored. The MIDI data recorded in the FD may be reproduced in synchronization with the reproduction of the data. For example, the reference pitch of the piano 11 used when reproducing MIDI data is 448 Hz, and the reference pitch of CD audio data is 44, for example.
The reference pitch of 3 Hz, VT audio data is, for example, 4
In the case of 50 Hz, the controller 3 performs control to raise the reference pitch of the CD audio data by 5 Hz (= 448 Hz-443 Hz) with reference to the reference pitch of the piano 11, and reproduce the reference pitch of the VT audio data by 2 Hz. (= 450Hz-448Hz) The control for lowering and reproducing is performed. In this way, a plurality of types of audio data may be stored in the HD, and the MIDI data recorded in the FD may be played back in synchronization with the playback of all (or some) of these audio data.

[0103]

As described above, according to the present invention, in synchronization with reproduction of video data or recording of video data,
It is possible to reproduce or record a plurality of music data.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a multimedia system according to a first embodiment.

FIG. 2 is a diagram showing a structure of CD audio data according to the same embodiment.

FIG. 3 is a block diagram for explaining a function of the controller according to the same embodiment.

FIG. 4 is a flowchart showing an elapsed time adjustment process according to the same embodiment.

FIG. 5 is a diagram showing a configuration of an SMF according to the same embodiment.

FIG. 6 is a block diagram showing a configuration of a control circuit of the FD recording apparatus according to the same embodiment.

FIG. 7 is a flowchart showing a delta time adjustment process according to the same embodiment.

FIG. 8 is a time chart showing an operation of the multimedia system according to the same embodiment.

FIG. 9 is a flowchart showing an elapsed time adjustment process according to a modified example 1 of the same embodiment.

FIG. 10 is a block diagram showing a configuration of a multimedia system according to a second embodiment.

FIG. 11 is a block diagram showing a configuration of a control circuit of the FD recording apparatus according to the same embodiment.

FIG. 12 is a time chart showing an operation of the multimedia system according to the same embodiment.

FIG. 13 is a flowchart showing a CD audio data recording process in the third embodiment.

FIG. 14 is a diagram exemplifying a signal waveform obtained from the CD audio data according to the same embodiment.

FIG. 15 is a diagram for explaining a storage state of the FD according to the same embodiment.

FIG. 16 is a flowchart showing pitch control / reproduction processing according to the same embodiment.

FIG. 17 is a diagram exemplifying a signal waveform obtained when audio data is read at each reading speed according to the embodiment.

FIG. 18 is a diagram for explaining a storage state of an HD according to Modification 1 of the same embodiment.

[Explanation of symbols]

3 ... Controller, 4 ... Operation part, 5 ... Mixer, 6 ... Amplifier, 7 ... Speaker, 8 ... FD
Recording device, 8a ... FD reproducing device, 10, 20 ...
Automatic piano, 11 ... Piano, 12 ... Key sensor, 13 ... Pedal sensor, 14 ... MIDI event generation circuit, 15 ... Piano sound source, 16 ... Piano electronic circuit, 17 ... Drive solenoid group, 18 ...
Ensemble sound source, 100 ... Video camera, 200
... Time code converter, 210 ... Clock generating means, 211, 220 ... Time measuring means, 212 ... Elapsed time adjusting means, 213 ... CD audio data reproduction control means, 213a ... HD Read control means, 2
13b ... Audio signal generating means, 230 ... Delta time adjusting means, 230 '... Elapsed time adjusting means, 240 ... Delta time generating means, 250 ...
SMF creation / writing control means, 330 ... Event transfer control means, 341 ... Timing adjustment means.

─────────────────────────────────────────────────── ─── Continued Front Page (51) Int.Cl. ⁷ Identification Code FI Theme Coat (Reference) G11B 20/10 321 G11B 20/10 321Z 27/10 27/10 L H04N 5/76 H04N 5/76 Z 5 / 765 5/91 L 5/91 NCF term (reference) 5C052 AA01 AB03 CC02 DD04 DD06 5C053 JA01 JA23 JA26 LA05 LA11 5D044 AB05 AB07 BC01 BC03 BC08 CC04 CC09 DE14 DE25 DE39 DE45 FG09 FG21 FG23 GK12 HL11 5D22 AA08 AA08 AA08 BB16 DC03 5D378 MM35 MM71 MM79 QQ01 QQ08 QQ34 QQ38

Claims (18)

[Claims] 1. Recording information including video data and a series of video time codes representing the elapsed time from the start of recording of the video data is read from the first storage medium, and the video data in the recording information is reproduced. The master device reproduces the first music data stored in the storage means and the second music data recorded in the second storage medium in synchronization with the reproduction of the video data by the master device. A multimedia system comprising a slave device for performing, wherein the slave device acquires a video time code in the recorded information, and converts the video time code into a musical sound time code. A clocking means for counting clocks by counting clocks; and a clocking means stored in the storage means based on a clocking result by the clocking means. That reads the first audio data at a time determined by the first time code in the music data including a series of time codes representing the elapsed time from the start of the first audio data and the music piece that represents the music,
Audio data reproduction control means for controlling to output as a sound, and a plurality of events and each event for instructing performance control of the music recorded in the second storage medium based on the time measurement result by the time measurement means Event output means for outputting the event at a time determined by the timing data in the music data including the timing data for specifying the execution timing of, and an automatic performance for supplying the event output by the event output means to the automatic performance means. A control means and a read timing of the first audio data,
First timing adjusting means for adjusting based on a time difference between a time corresponding to a time code for a musical sound output from the time code converting means and a time corresponding to a time measurement result of the time measuring means; and the event is the event output. The timing supplied from the means to the automatic performance control means is adjusted based on the time difference between the time corresponding to the musical tone time code output from the time code conversion means and the time corresponding to the time measurement result of the time measurement means. A multimedia system comprising: a second timing adjusting means. 2. The first music data is music data recorded in a third storage medium, and the slave device reproduces the video data by the master device and executes the first music data by the slave device. Prior to the synchronous reproduction with the music data and the second music data, it further comprises a synchronous reproduction pre-processing executing means for reading the first music data from the third storage medium and storing it in the storage means. The multimedia system according to claim 1, wherein: 3. The recording information further includes second audio data, wherein the master device reproduces the video data in the recording information and the slave device reproduces the second audio data in the recording information. 2. The multimedia system according to claim 1, wherein the multimedia data is output to a device, and the audio data reproduction control unit further performs control to output the second audio data received from the master device as a sound. . 4. The master device records record information including video data and a series of video time codes representing an elapsed time from the start of recording the video data in a first storage medium, and the slave device is Reproducing the first music data stored in the storage means and the second music data recorded in the second storage medium in synchronization with recording of the recording information by the master device. The multimedia system according to claim 1, wherein: 5. The recording information including the video data and a series of video time codes representing the elapsed time from the start of recording the video data is read from the first storage medium, and the video data in the recording information is reproduced. A multimedia system including a master device and a slave device that reproduces the first music data stored in the storage means and records the second music data in synchronization with the reproduction of the video data by the master device. The slave device obtains a video time code in the recorded information, converts the video time code into a musical sound time code, and counts clocks to perform timekeeping. Clocking means, and a first audio representing the music stored in the storage means based on the clocking result by the clocking means Over data and reading the first audio data at a time determined by the first time code in the music data including a series of time codes representing the elapsed time from the start of the song,
Audio data reproduction control means for performing control for outputting as sound, and read timing of the first audio data,
Timing adjusting means for adjusting on the basis of the time difference between the time corresponding to the musical tone time code output from the time code converting means and the time corresponding to the timing result of the timing means, and detecting means for detecting the operating state of the musical instrument An event generation unit that acquires the operation state from the event generation unit and generates an event that instructs performance control of a music piece according to the acquired operation state; and when the event generation unit generates the event, the time counting unit measures the time. Timing data generating means for generating timing data designating the execution timing of the event based on the time corresponding to the result; time corresponding to the musical tone time code output from the time code converting means; and the time measuring means. Adjust the timing data based on the time difference from the time corresponding to the time measurement result That timing data adjusting means and the event generating means a second said second music data including the timing data adjusted by the plurality of events and the timing data adjusting means which is generated by
And a writing means for writing to the storage medium. 6. The slave device further comprises first reference pitch writing means for obtaining a reference pitch of the first audio data and writing information about the obtained reference pitch in the second storage medium. The multimedia system according to claim 5, characterized in that: 7. The recording information further includes second audio data, the slave device acquires a reference pitch of the second audio data, and the information about the acquired reference pitch is stored in the second storage medium. 7. The multimedia system according to claim 5, further comprising a second reference pitch writing unit for writing the data in the medium. 8. The master device records record information including video data and a series of video time codes representing an elapsed time from the start of recording of the video data in a first storage medium, and the slave device is The reproduction of the first music data stored in the storage means and the recording of the second music data are performed in synchronization with the recording of the recording information by the master device. Multimedia system according to any one of the preceding claims. 9. The recording information including the video data and a series of video time codes representing the elapsed time from the start of recording the video data is read from the first storage medium and the video data in the recording information is reproduced. A multimedia system comprising: a master device; and a slave device that records the first music data and the second music data stored in a storage means in synchronization with the reproduction of the video data by the master device. The slave device obtains a video time code in the recorded information, converts the video time code into a musical sound time code, and a time counting means for counting time by counting clocks. And first audio data representing a music piece stored in the storage means based on the time measurement result of the time measurement means. An audio that controls to read the first audio data at a time determined by the time code in the first music data including a series of time codes representing the elapsed time from the start of the music and output the sound as sound. A data reproduction control means and a read timing of the first audio data,
Timing adjusting means for adjusting on the basis of the time difference between the time corresponding to the musical tone time code output from the time code converting means and the time corresponding to the timing result of the timing means, and detecting means for detecting the operating state of the musical instrument And an event generation means for generating the event for instructing the performance control of the music in accordance with the acquired operation state, and a time corresponding to the musical tone time code output from the time code conversion means. A clock time adjusting means for adjusting the time corresponding to the clock result based on a time difference from the time corresponding to the clock result by the clock means; and when the event is generated by the event generating means, the clock time adjustment Generates timing data that specifies the execution timing of the event based on the time adjusted by the means And timing data generation means, and writing means for writing the second music data including the event generated by the event generation means and the timing data generated by the timing data generation means in a second storage medium. A multimedia system characterized by that. 10. The slave device further comprises first reference pitch writing means for obtaining a reference pitch of the first audio data and writing information about the obtained reference pitch in the second storage medium. The multimedia system according to claim 9, characterized in that: 11. The recording information further includes second audio data, the slave device acquires a reference pitch of the second audio data, and the information about the acquired reference pitch is stored in the second storage medium. Second to write on
The multimedia system according to claim 9 or 10, further comprising: a reference pitch writing unit. 12. The master device records record information including video data and a series of video time codes indicating an elapsed time from the start of recording the video data in a first storage medium, and the slave device is 12. The reproduction of the first music data stored in the storage means and the recording of the second music data are performed in synchronization with the recording of the recording information by the master device. A multimedia system according to any one of the preceding claims. 13. The first music data stored in the storage means in synchronism with the reproduction of recorded information including video data and a series of video time codes representing the elapsed time from the start of recording of the video data. A reproducing device for reproducing and reproducing the second music data recorded in a storage medium, wherein a time code for musical tone is converted from a time code converting means for converting a video time code in the recorded information into a musical sound time code. Receiving means for receiving the clock, clocking means for counting the clock by counting clocks, first audio data representing the music stored in the storing means and the music of the music based on the clocking result by the clocking means. The first audio data at a time determined by the time code in the first music data including a series of time codes representing the elapsed time from the start. Read the
Audio data reproduction control means for controlling output as sound, and a plurality of events for instructing performance control of the music recorded in the storage medium and execution timing of each event based on the timing result by the timing means Event output means for outputting the event at a time determined by the timing data in the music data including the timing data for specifying, and automatic performance control means for supplying the event output by the event output means to the automatic performance means. , Read timing of the first audio data,
A first timing adjusting means for adjusting based on a time difference between a time corresponding to a time code for a musical sound received by the receiving means and a time corresponding to a timing result of the timing means; and the event from the event output means. Second timing for adjusting the timing supplied to the automatic performance control means based on the time difference between the time corresponding to the time code for the musical tone received by the receiving means and the time corresponding to the time measurement result of the time measuring means A reproducing apparatus comprising: an adjusting unit. 14. The reproducing apparatus is stored in the storage means in synchronization with recording of record information including video data and a series of video time codes representing an elapsed time from the start of recording the video data. 14. The reproducing device according to claim 13, wherein the reproducing device reproduces the first music data and the second music data recorded in the storage medium. 15. The first music data stored in the storage means in synchronism with the reproduction of recorded information including video data and a series of video time codes representing the elapsed time from the start of recording of the video data. A reproducing / recording device for reproducing and recording second music data, comprising a receiving means for receiving a musical sound time code from a time code converting means for converting a video time code in the recorded information into a musical sound time code. , Clocking means for counting time by counting clocks, and first audio data representing the music stored in the storage means and the elapsed time from the start of the music based on the timing result by the time counting means. And a sequence of time codes that represent the first audio data at a time determined by the time code in the first music data. And an audio data reproduction control means for performing control for outputting the first audio data,
Timing adjusting means for adjusting based on the time difference between the time corresponding to the musical tone time code received by the receiving means and the time corresponding to the timing result of the timing means; and the detecting means for detecting the operating state of the musical instrument. Event generating means for acquiring an operation state and generating an event for instructing performance control of music according to the acquired operation state; and, when the event is generated by the event generating means, a timing result by the timing means is displayed. Corresponding to timing data generating means for generating timing data designating the execution timing of the event based on the corresponding time, time corresponding to the musical tone time code received by the receiving means, and time counting result by the time measuring means Timing data that adjusts the timing data based on the time difference from the And a writing unit that writes the second music data including a plurality of events generated by the event generating unit and the timing data adjusted by the timing data adjusting unit into a storage medium. Characteristic reproduction / recording device. 16. The reproducing / recording device is stored in the storage means in synchronization with recording of record information including video data and a series of video time codes representing an elapsed time from the start of recording of the video data. The reproduction of the first music data and the recording of the second music data are performed.
The reproducing / recording apparatus according to item 5. 17. The first music data stored in the storage means in synchronism with the reproduction of recorded information including video data and a series of video time codes representing the elapsed time from the start of recording the video data. A reproducing / recording device for reproducing and recording second music data, comprising a receiving means for receiving a musical sound time code from a time code converting means for converting a video time code in the recorded information into a musical sound time code. , Clocking means for counting time by counting clocks, and first audio data representing the music stored in the storage means and the elapsed time from the start of the music based on the timing result by the time counting means. And a sequence of time codes that represent the first audio data at a time determined by the time code in the first music data. And an audio data reproduction control means for performing control for outputting the first audio data,
Timing adjusting means for adjusting based on the time difference between the time corresponding to the musical tone time code received by the receiving means and the time corresponding to the timing result of the timing means; and the detecting means for detecting the operating state of the musical instrument. An event generation means for acquiring an operation state and generating an event for instructing performance control of a music piece in accordance with the acquired operation state; a time corresponding to a musical tone time code received by the receiving means; Based on the time difference from the time corresponding to the time measurement result, the time measurement time adjusting means for adjusting the time corresponding to the time measurement result, and when the event is generated by the event generating means, is adjusted by the time measurement time adjusting means. Timing data that generates timing data that specifies the execution timing of the event based on the time And a writing unit that writes the second music data including the event generated by the event generating unit and the timing data generated by the timing data generating unit into a storage medium. Playback recording device. 18. The recording / reproducing apparatus is stored in a storage means in synchronization with recording of record information including video data and a series of video time codes representing an elapsed time from the start of recording of the video data. The reproduction of the first music data and the recording of the second music data are performed.
7. The reproducing / recording apparatus according to item 7.