JP4062324B2 - Movie playback apparatus and movie playback method - Google Patents

Description

  The present invention relates to a moving image reproducing apparatus and a moving image reproducing method capable of reproducing and displaying a moving image in accordance with music by controlling a time axis for reproducing the moving image.

  Conventionally, a device for reproducing a moving image and a device for playing music according to the tempo have existed. There are also devices that can easily change the playback speed, such as video decks, that simply double the size or reduce the size by half when playing a video. It was.

  However, the conventional apparatus cannot match the music tempo with the moving image playback timing. Also, even if you can synchronize the music tempo and video playback timing at the start and match only the head, if you make the tempo faster in the middle of the song, the video frame will be extra, and if you slow it down, there will be more frames. There was inconvenience such as disappearing. Furthermore, it has been difficult to make a moving image material for loop playback follow a variable tempo.

  In view of the above-described problems of the conventional technology, the present invention provides a moving image playback apparatus and a moving image playback capable of playing back and displaying a moving image in accordance with music to be played, with the time axis being variably controlled even during the playback of the moving image It aims to provide a method.

  According to the main feature of the present invention, in response to a locating command stored in the moving image data memory (70) and arranged in accordance with a desired timing of music data from a plurality of moving image data composed of a plurality of frames representing the moving image. While selecting one moving image data, the reproduction of the selected moving image data is started according to the deferred play command arranged in accordance with the desired timing of the music data, and the moving image is displayed on the display device (15). A video input device (1) for inputting a tempo clock (MIDI clock) (TC) representing a music performance tempo based on music data during reproduction of the video data, and input by the clock input means For controlling the playback speed of moving image data expressed in frames per unit time according to the tempo clock Control signal generating means (71, 73) for generating a raw speed control signal, and a frame based on the reproduction speed control signal generated by the control signal generating means, by using the frame of the moving image data stored in the moving image data memory (70). A moving image data reading means (73) that sequentially reads out and outputs to the display device (15) at a reading rate, and the frame reading rate is a tempo indicating a standard tempo (for example, tempo value = 120) by the clock input means. When a clock is input, it is represented by a predetermined number of frames per unit time (for example, 30 frames / second), and a tempo clock indicating a tempo changed from the standard tempo to positive or negative by the clock is input. When the number of frames per unit time has changed by the amount corresponding to this tempo change from the number of frames per unit time Video reproducing apparatus [Claim 1] is provided which is represented in. Note that the parentheses indicate reference symbols, terms, and the like of the corresponding embodiments for convenience of understanding, and the same applies to the following.

  Further, according to this feature, 1 is determined in accordance with a locate command that is stored in the moving image data memory (70) and is arranged in accordance with a desired timing of the music data among a plurality of moving image data composed of a large number of frames representing the moving image. In order to select one moving image data and start reproduction of the selected moving image data in accordance with a deferred play command arranged in accordance with a desired timing of the music data to display the moving image on the display device (15). A video input method (1) for inputting a tempo clock (MIDI clock) (TC) representing a music performance tempo by music data during reproduction of the video data, and a clock input step Playback of video data expressed in frames per unit time according to the tempo clock input at A control signal generation step (71, 73) for generating a playback speed control signal for controlling the degree of reproduction, and a frame of moving image data stored in the moving image data memory (70) generated by the control signal generation step A moving image data reading step (73) that sequentially reads out and outputs to the display device (15) at a frame reading rate based on the speed control signal, and the frame reading rate is a standard tempo (for example, tempo value = When a tempo clock indicating 120) is input, it is represented by a predetermined number of frames per unit time (for example, 30 frames / second), and indicates a tempo changed from the standard tempo to positive or negative at the clock input step. When a tempo clock is input, the number of frames per unit time corresponds to this tempo change. Video playback method represented only changed per unit time the number of frames [Claim 3] is provided.

  In other words, according to the present invention, one of the plurality of moving image data that is stored in the moving image data memory (70) and includes a plurality of frames representing the moving image is arranged in accordance with the locate command arranged in accordance with the desired timing of the music data. For selecting moving image data and starting reproduction of the selected moving image data in accordance with a deferred play command arranged in accordance with a desired timing of the music data and displaying the moving image on the display device (15). In the moving image reproducing apparatus and moving image reproducing method, during reproduction of moving image data, a tempo clock (MIDI clock) (TC) representing a music performance tempo based on music data is input (1), and a reading speed (in accordance with the input tempo clock ( Each frame of moving image data stored in the moving image data memory (70) is sequentially read at a frame reading rate). Output to put the display device (15) performs regeneration control of the video data (71, 73). Here, in the reproduction control (71, 73) of the moving image data, a reproduction speed control signal for controlling the reproduction speed of the moving image data represented by the number of frames per unit time is generated according to the input tempo clock (71 73), and the frames of the moving image data stored in the moving image data memory (70) are sequentially read out at a frame reading rate based on the generated reproduction speed control signal and output to the display device (15) (73). This frame reading rate is represented by a predetermined number of frames (for example, 30 frames / second) when a tempo clock (TC) indicating a standard tempo (for example, tempo value = 120) is input. When a tempo clock indicating a tempo that has changed positively or negatively from the standard tempo is input, it is represented by the number of frames per unit time that has changed from the predetermined number of frames per unit time by an amount corresponding to this tempo change. The tempo clock representing the performance tempo of the music represents, for example, the reproduction speed of performance data for automatically playing the melody, rhythm, accompaniment, etc. of the music arranged over time. Also, this playback speed control signal changes the read time per predetermined number of frames (unit data) according to the tempo clock when the predetermined number of frames is taken as the unit data of the moving image data. This is time axis control information for controlling the time axis. Therefore, according to this, the playback time of a moving image in a certain section expressed by the number of frames follows the length of a corresponding section (for example, expressed by bars, beats, ticks, etc.) in the music being played. As described above, the reproduction time axis of the moving image is controlled in accordance with the tempo clock representing the music performance tempo. Therefore, even if the performance tempo of the music played during the reproduction of the moving image changes, it becomes possible to reproduce the moving image in accordance with the performance tempo.

  The moving picture reproducing apparatus according to the present invention further includes moving picture data synthesizing means (74) for synthesizing the moving picture data read by the moving picture data reading means (73) and the separately input moving picture data and outputting them to the display device. According to a second aspect of the present invention, there is provided a moving image reproduction method according to the present invention, wherein the moving image data read in the moving image data reading step and the separately input moving image data are combined and displayed. A moving image data synthesizing step (74) to be output to the apparatus can be provided.

  That is, in the moving image playback apparatus and the moving image playback method according to the present invention, the moving image data read by the playback control and the separately input moving image data are combined and output to the display device (15). . In this case, as moving image data to be input separately, for example, moving image data from the video player (14) can be used. According to this, it is possible to output a moving image that further enhances the video effect.

  According to the main feature of the present invention (claims 1 and 3), as described above, a plurality of moving image data composed of a number of frames representing a moving image is stored in the moving image data memory, and the desired timing of the music data is obtained. According to the locate command arranged together, one video data is selected from these video data, and the selected video data is selected according to the deferred play command arranged in accordance with the desired timing of the music data. When the moving image data is being reproduced, a tempo clock representing the music performance tempo based on the music data is input, and the moving image data is displayed at a frame reading rate represented by the number of frames per unit time according to the input tempo clock. Since the frames of the video data are sequentially read from the memory and output to the display device, the music data The desired video data can be played back in synchronization with the playback, and the frame readout rate when playing back the video data consisting of a large number of frames is controlled in real time according to the tempo clock representing the music performance tempo. And the playback time of the video data in a certain section expressed by the number of frames follows the length of the corresponding section in the music data being played. Changes, it becomes possible to reproduce the moving image in accordance with the performance tempo, and to enhance the effects of music performance and moving image reproduction.

  Further, according to the present invention, the moving image data read out by the reproduction control and the separately input moving image data (for example, moving image data from the video player) are combined and output to the display device. It is possible to output a moving image with a further enhanced visual effect (claims 2 and 4).

[Various features]
In addition, the following [1] to [3] can be configured by other features relating to the implementation of various inventions described in this specification in response to the above-described problem (object):
[1] Output timing set information (TS) for designating the coincidence of reproduction timing between at least one music data in a series of music data and at least one video data in a series of video data; And based on the output timing set information (TS), it is stored in the moving image data memory (70) so that the reproduction timing of at least one moving image data matches the reproduction timing of at least one music data. The moving image data read timing is controlled, and the read moving image data is output to the display device (15).

[2] Output timing set information (TS) for designating the coincidence of reproduction timing between at least two music data in the series of music data and at least two video data in the series of video data; And based on the output timing set information (TS), it is stored in the video data memory (70) so that the playback timing of at least two video data matches the playback timing of at least two music data. The read-out moving image data is output to the display device (15) by controlling the read-out timing and read-out speed of the moving image data.

  In these cases [1] and [2], the timing set information (TS) represents a specific performance timing in music data, for example, corresponding to a specific frame of moving image data. In other words, it represents a specific frame of the moving image data in correspondence with a specific performance timing in the music data. And this specific performance timing is represented using the absolute time after a performance start, a bar, a beat, or the number of ticks. As the music data, performance data for automatically playing the melody, rhythm, accompaniment, etc. of the music arranged over time can be used.

  According to the configurations [1] and [2], a desired moving image frame can be reproduced in synchronization with a music performance of a desired portion regardless of a change in music tempo. Further, since the moving image can be reproduced while preventing the joint portion of the moving image from failing, the moving image material for loop reproduction can also be used. Therefore, for example, synchronization of a music accent portion and a certain frame of a moving image can be easily performed in real-time work, so that the range of variations in performance and playback can be further expanded.

  In particular, when a video is played in a loop for VJ (VisuaI Jockey), it is possible to handle any tempo using your favorite loop material without having to prepare several video materials to match the tempo. become. Therefore, the relationship between DJ (Disc Jockey) & VJ, which is difficult to make a prior meeting and editing, can be dealt with flexibly on the VJ side at the performance and playback site.

[3] In the configuration of [1] and [2] described above, the moving image data read by the reproduction control and the separately input moving image data are combined and output to the display device (15). . In this case, as moving image data to be input separately, for example, moving image data from the video player (14) can be used. According to the configuration [3], it is possible to output a moving image that further enhances the video effect.

Furthermore, from another viewpoint, the following (1) to (3) can also be configured:
(1) Movie storage means for storing movie information, control input means for inputting a variable time axis control signal during playback of the movie, and movie playback speed control for controlling the playback speed of the movie in accordance with the time axis control signal A moving picture reading means for reading and outputting moving picture information from the moving picture storage means by expanding and contracting a reading time per information unit according to a reproduction speed, and a moving picture output means for outputting the read moving picture information to a display device Time axis control instruction device. In this video time axis control instruction device, the control input means inputs a tempo clock as a time axis control signal, and the video playback speed control means controls the video playback speed according to the tempo clock, or the control The input means can input a time code as a time axis control signal, and the moving picture playback speed control means can be configured to control the playback speed of the moving picture according to the time code. Further, an operation element for instructing the speed of reproduction of the moving image and a detection means for detecting the operation of the operation element are provided, and the control input means inputs a time axis control signal based on the detected operation of the operation element. Can be configured. In this case, the operation element is preferably a one-dimensional position detection type operation element.

(2) Movie storage means for storing movie information, playback timing designating means for designating a plurality of specific playback timings for playback timing of movie information, and performance for designating a plurality of specific performance timings for performance timing of musical sound information Based on the timing specification means and the specified specific playback timing and specific performance timing, each specific playback timing is adjusted to correspond to each specific performance timing, and a plurality of intermediate playback timings between each specific playback timing are calculated. A moving image comprising: a reproduction timing adjusting unit that performs the operation, a moving image reading unit that reads out the moving image information at the adjusted specific reproduction timing and the calculated intermediate reproduction timing, and a moving image output unit that outputs the read out moving image information to the display device Time axis control instruction device.

(3) In the above-described moving image time axis control instruction device of (1) and (2), moving image composition for combining moving image information separately input and moving image information read from the moving image storage unit by the moving image reading unit A moving picture time axis control instruction device comprising means.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. The following embodiments are merely examples, and various modifications can be made without departing from the spirit of the present invention.

[Hardware configuration]
FIG. 1 shows a block diagram of a hardware configuration of a moving image reproduction control system (also referred to as “moving image time axis control instruction system”) according to an embodiment of the present invention. In this system example, a configuration is employed in which various processes including a moving image time axis control instruction are performed using a personal computer (PC). The system includes a central processing unit (CPU) 1, a read-only memory (ROM) 2, a main body side random access memory (main body RAM) 3, an external storage device 4, a detection circuit 5, a display circuit 6, etc. In addition, a moving image processing circuit 7, a MIDI interface (MIDII / F) 8, a communication interface (communication I / F) 9, and the like are provided, and these devices 1 to 9 are connected to each other via a bus 10.

  The CPU 1 that controls the entire system performs various controls using time measurement by the timer 11 in accordance with a predetermined software program, and particularly manages processing such as a moving image time axis control instruction to be described later. The ROM 2 stores a predetermined control program for controlling this system. These control programs include basic information processing, a program for managing processing such as a moving picture time axis control instruction according to the present invention, various tables, various data, and the like. The main body RAM 3 functions as a temporary storage unit that stores data and parameters necessary for various processes.

  In addition to the hard disk (HD), the external storage device 4 includes a compact disk read only memory (CD-ROM), a flexible disk (FD), a magneto-optical (MO) disk, a digital multipurpose disk (DVD), ZIP, It consists of a device using a portable storage medium such as a semiconductor memory, and can store various control programs and various data. Therefore, programs and various data necessary for moving image time axis control instructions and the like are read not only in the ROM 2 but also from the external storage device 4 into the RAM 4 as necessary, and the processing results are recorded in the external storage device 4. obtain.

  A panel operating device 12 is connected to the detection circuit 5, and the panel operating device 12 includes various switches, keys, and other types of operating devices. These operators are also referred to as panel operators, and are used by a user to input various settings such as a moving image playback mode and instruction information to the system for processing such as a moving image time axis control instruction. For example, the mode switches (M1, M2) are provided for switching the playback mode of moving images, and the ribbon controller (linear operation type pad operator for detecting a one-dimensional position where the operation finger is in contact) is used for music and moving images. It is provided as a real-time operator for operating the traveling speed of the camera in real time.

  The display circuit 6 controls the display state of the main body display 13 constituted by an LCD or the like and various indicators. In relation to the display state of the main body display 13, the user operates the operator of the panel operator device 13, and settings and instructions relating to the moving image time axis control instruction are performed. And the main body system (or main body PC) is comprised by these elements 1-6, 10-13 mentioned above.

  The moving image processing circuit 7 has a dedicated RAM. Connected to the moving image processing circuit 7 are a video player 14 such as a VTR or a video disk player, and a video display 15 having a relatively large screen such as a CRT or projector. The moving image processing circuit 7 processes the video signal (moving image signal) from the image reproducing device 14 to reproduce moving image information at a predetermined reproduction speed, and displays an image based on the moving image information on the image display 15. These moving image processing circuit 7, video player 14 and video display 15 constitute a moving image playback system.

  Note that an A / D converter is built in the input portion of the moving image processing circuit 7, and an analog video signal output from the video reproducer 14 is converted into a digital video signal (moving image) at this input portion. Information). In the moving image processing circuit 7, various processes are performed on the converted digital moving image information. In addition, a D / A converter is built in the output unit of the moving image processing circuit 7, and the moving image information in the digital format processed by the moving image processing circuit 7 is converted into a video signal in the analog format to be displayed on the video display 15. Is output.

Connected to the MIDI interface (I / F) 8 is a MIDI device 16 including a sequencer for generating music data and a personal computer (PC). A tone signal generator 17 is connected to the MIDI device 16.
The sound source circuit 17 forms a tone signal based on the input music data and outputs it. A sound system 18 including a digital signal processor (DSP) and a speaker is connected to the sound source circuit 17.

  The sound system 18 applies various musical effects to the input musical sound signal, and generates musical sounds corresponding to the musical sound signal to which the musical effect is applied. In addition to its own music data (music information), the MIDI device 16 can also output, for example, music data (music information) stored in the ROM 2 or the external storage device 4 to the sound source circuit 17. Therefore, a sound corresponding to a music signal based on music data stored in the ROM 2 or the external storage device 4 is also generated from the sound system 18.

  Here, the music data is, for example, automatic performance data in a standard MIDI file (SMF) format, and the MIDI device 16 reproduces the automatic performance data. When playing back music data (automatic performance data), the playback tempo is controlled in accordance with a time code accompanying the music data, a tempo clock (MIDI clock) command, or an instruction from the real-time controller (12). . The MIDI device 16, the sound source 17, and the sound system 18 constitute a music performance system.

  The time code is a timing signal periodically given with reference to the absolute time from the start of reproduction of automatic performance data. This time code is time information indicating the time from the start of reproduction of automatic performance data. (Hours, minutes, seconds). The tempo clock is a timing signal that is periodically given at time intervals corresponding to the tempo of music, and is given, for example, every 1/24 time length corresponding to a quarter note.

  The communication interface (I / F) 9 is communicably connected to a server computer 20 or the like via a communication network 19 such as a local area network (LAN), the Internet, or a telephone line. Accordingly, various data such as a control program and music data can be downloaded from the server computer 16 or the like to the external storage device 4.

  1 describes the system using the main body PC, but the main body system and the music performance system use an information processing apparatus such as an electronic musical instrument or an automatic performance apparatus having an equivalent processing function, and a moving image is used for this. A video playback control system (video time axis control instruction system) may be configured by connecting a playback system.

[Video playback system]
FIG. 2 shows a block diagram of a moving image processing function unit (moving image reproduction system) in the moving image time axis control instruction system according to one embodiment of the present invention. The moving image processing circuit 7 includes a time axis control unit 71, a moving image recording unit (video sampler) 72, a moving image reproduction unit 73, and a video composition unit 74 in addition to a dedicated RAM 70 dedicated for moving image processing.

  Here, the outline of the moving image reproduction control system (moving image time axis control instruction system) according to the embodiment of the present invention will be briefly described with reference to FIG. In this moving image reproduction control system, when reading moving image data (moving image information) stored in advance in the moving image data memory (dedicated RAM) 70, the time axis for reproducing the moving image data is variably controlled. In a certain mode M1, during reproduction of moving image data, the CPU 1 supplies a time axis control signal corresponding to the tempo of the performance music from the tempo detection mechanisms TC and TM to the time axis control unit 71. The time axis control unit 71 adjusts (controls) the reproduction speed according to the time axis control signal, and the moving image reproduction unit 73 expands and decreases the read time per unit data according to the adjusted reproduction speed, and stores the moving image data in the data memory. The data is read from 70 and output to a display device (display) 15. In another mode M2, a specific music performance timing is set (TS) at a specific frame playback timing of the video data, and the video data playback control system including the CPU 1, the time axis control unit 71 and the video playback unit 73 Based on the timing set information TS, moving image data of a specific frame is read from the data memory 70 at a reproduction timing that matches each performance timing, and is output to the display device 15.

  Hereinafter, the moving image reproduction system of FIG. 2 will be described in detail. First, the moving picture recording unit 72 samples a video signal (moving picture signal) from the video player 14 such as a VTR in advance according to an instruction from the CPU 1 of the main system and records it in a dedicated RAM 70 that functions as a storage of digital moving picture data. To do. That is, in response to a recording start instruction from the panel operator 12, the video signal input from the video player 14 to the moving image processing circuit 7 at that time is recorded as digital moving image data in a predetermined storage area of the dedicated RAM 70. Is done. In this dedicated RAM, a plurality of storage areas can be set, and a plurality of digital moving image data can be stored.

  As described above, the video signal (moving image signal) processed by the moving image processing circuit 7 is converted into a digital signal by an input unit (not shown). Therefore, the moving picture recording unit 72 writes a digital video signal in the dedicated RAM 70 as digital recording data. In writing the digital recording data, the moving picture recording unit 72 may compress the digital moving picture data and write it into the dedicated RAM 70 in order to reduce the amount of data stored in the dedicated RAM 70.

  At the time of moving image reproduction, the CPU 1 instructs the moving image reproduction in various ways. For example, the panel operator 12 is provided with a plurality of moving image selection switches corresponding to a plurality of moving image data stored in the dedicated RAM. The CPU 1 instructs the moving image processing circuit 7 to start reproducing the corresponding moving image data in response to an operation of any moving image selection switch.

  Alternatively, reproduction of moving image data can be instructed by music data reproduced by the MIDI device 16. The MIDI standard defines a MIDI machine control (MMC) standard for controlling digital recorders and VTRs. When MMC is used, one moving image data can be selected from a plurality of moving image data according to the locate command, and reproduction of the moving image data selected according to the deferred / play command can be started. The locate command is a command for instructing the position of the moving image data to be reproduced in the plurality of moving image data. The deferred play command is a command for designating the timing for starting the reproduction of moving image data.

  Further, in synchronism with the reproduction of the music data, a place where it is desired to reproduce the desired moving image data, and a locate instruction and a deferred play instruction may be arranged in accordance with the desired timing of the music data. When the music data is reproduced by the MIDI device 16, a locate command and a deferred play command are respectively fetched at the arranged timing. The fetched locate command and deferred play command are sent to the MIDI interface (I / F) 8. Via CPU 1. The CPU 1 selects the moving image data according to the locate command and controls the moving image processing circuit 7 to start reproducing the moving image data selected according to the deferred play command.

  At the time of moving image reproduction, the moving image reproduction unit 73 selects desired moving image data from a plurality of sampled moving image data recorded in the dedicated RAM 70 in accordance with such an instruction from the CPU 1, and the selected moving image data is displayed on the time axis. Under the control of the control unit 71, it is read out at a predetermined speed and sequentially reproduced. As described above, when the compressed moving image data is stored in the dedicated RAM, the moving image reproducing unit 73 expands the moving image data read from the dedicated RAM 70. Here, the time axis control unit 71 controls the video playback unit 73 based on the music tempo instruction information TC or a specific timing set TS according to the control input from the CPU 1 according to the playback mode (M1, M2), It has a function to control the playback of moving images.

  That is, the time axis control unit 71 performs moving image reproduction at a moving image reproduction speed that is a value that matches the progress of the currently played music, based on the tempo instruction information TC or the specific timing set TS from the CPU 1. An instruction is given to the playback unit 73. Then, the moving image reproduction unit 73 sequentially reads the moving image information from the dedicated RAM 70 at the instructed moving image reproduction speed and reproduces the moving image. The control of the moving image playback speed based on the tempo instruction information and the specific timing set TS will be described in detail later.

  In addition, in the moving image reproduction by the moving image reproducing unit 73, “loop reproduction” in which a part or all of the moving image data selected from the dedicated RAM 70 is repeatedly reproduced by an instruction from the CPU 1 can be used. According to this embodiment, it is possible to perform loop reproduction seamlessly and without failure (the occurrence of image clipping or blank image generation).

  The moving image data reproduced by the moving image reproduction unit 73 is displayed on the screen of the video display 15 at a predetermined reproduction speed via the video composition unit 74. In this example, the video synthesizing unit 74 receives moving image data whose playback speed is controlled (for example, as a main image) from the moving image reproducing unit 73 and reproduces it from the video reproducing unit 14 in real time (so-called “so-called“ The “flowing”) receives other moving image data (for example, as a background image) from the video signal, and synthesizes both moving image data. As a result, an image obtained by combining both moving image data is displayed on the image display 15. The signal supplied to the video display 15 is an analog video signal provided in an output unit (not shown) as described above.

  The video composition unit 74 may synthesize a video (color image) by simply superimposing a playback video with a variable time axis from the video playback unit 73 and a real-time video from the video player 14. Apply various crossfades, etc., take the logical sum or logical product of playback video and real-time video, display both in different screen areas and change each display area as necessary, etc. A video effect can be employed.

[0th playback mode]
In this moving image time axis control instruction system, the moving image processing circuit 7 can be operated in the 0th reproduction mode. In the 0th reproduction mode, as shown in FIG. 2 (upper left), the moving image reproduction speed is changed based on tempo instruction information TC based on an instruction such as a tempo clock (MIDI clock) or time code of music data. That is, when performing using music data, the CPU 1 is performed upon receiving tempo instruction information indicated by a time code difference value (time code time interval) or a time interval between tempo clocks (MIDI clock). A change in the music tempo is detected and this is transmitted to the time axis control unit 71. The time axis control unit 71 controls the moving image reproduction in the moving image reproduction unit 73 based on the tempo instruction information given as a control input from the CPU 1.

  FIG. 3 is a diagram schematically showing how the reading speed for moving image data changes in accordance with the tempo instruction in the 0th playback mode. In the figure, a circle indicates a predetermined moving image data unit [a predetermined number of frames in moving image data. Hereinafter, it is simply referred to as “section”. ] And music data units in music data [a predetermined number of bars. The number of beats or the number of ticks (TICK) may be used. In any case, the number may not be an integer (a value including a decimal number). Hereinafter, it is simply referred to as “section”. The width of the circles simply indicates the reproduction of data in each section or the change in performance density (time amount per information unit) in a visually easy-to-understand manner. The number of ticks indicates the number of unit time intervals representing the time resolution in the sequencer.

  The top row in FIG. 3 represents moving image data stored in the moving image reproduction dedicated RAM 70. On the other hand, the second level shows how the tempo value based on tempo instruction information by an instruction such as a tempo clock changes with respect to the elapsed time t (sec), and “0” on the vertical axis indicates the standard tempo value. (For example, tempo value = 120).

  When music performance starts in accordance with the music data in the music performance systems 16 to 18, moving image reproduction is performed at a speed corresponding to the tempo value designated by the tempo designation information. For example, if the tempo information of music data is set to a tempo instruction value (standard tempo value = 120) corresponding to “0”, in the moving image processing circuit 7, the moving image data stored in the dedicated RAM 70 is stored in the CPU 1. Is read out at a normal moving image reproduction speed, for example, 30 frames / second by the time axis control unit 71 and the moving image reproduction unit 73.

  When the tempo instruction value changes from the standard tempo value of “0”, the changed tempo instruction value is instructed from the CPU 1, and music performance and moving image reproduction are performed based on the instructed tempo instruction value. The tempo instruction value can be obtained from information such as a tempo clock time interval and a time code difference value (time code time interval).

  Accordingly, in the 0th playback mode, when the tempo value changes as in the second stage, the music data is changed according to the changed tempo value [from the specified tempo value to the positive / negative (+,-) tempo value change. With the tempo changed by the corresponding amount), the playback speed of the video data is changed according to the positive / negative (+,-) tempo value change from the normal video playback speed. As shown in the third row, the moving image data is reproduced at a speed corresponding to the tempo value, so that it is possible to obtain an effect that matches the reproduction of the moving image with the music performance. In music data such as SMF, tempo information can be arranged at a timing when the tempo value is desired to be changed. When the music data is reproduced, tempo information appears at that timing and the tempo value is changed. That is, based on the tempo information included in the music data, music performance and moving image playback can be controlled simultaneously.

[First playback mode]
In this moving image time axis control instruction system, when the mode switch M1 in the operation device 12 is operated (see FIG. 2), the moving image processing circuit 7 can be operated in the first reproduction mode. In the first playback mode, during the performance of the music, the CPU 1 receives the tempo change (operation) information TM based on the operation of the real-time operator and detects that there has been an operation for changing the tempo of the music being played. This is transmitted to the time axis control unit 71. The time axis control unit 71 controls moving image reproduction in the moving image reproducing unit 73 based on tempo change (operation) information given as a control input from the CPU 1.

  FIG. 4 is a diagram schematically showing how the reading speed for moving image data changes in accordance with the tempo changing operation in the first playback mode. Also in the figure, the meanings of the circles are the same as described above. As already described, the panel controller 12 is provided with a ribbon controller (a real-time controller for detecting a one-dimensional contact position) in order to change the music performance tempo in real time. 4 represents moving image data stored in the moving image reproduction-dedicated RAM 70. On the other hand, the second stage shows the change in the operation amount of the ribbon controller with respect to the elapsed time t (sec), and “0” on the vertical axis is the origin position of the ribbon controller operation.

  When the operating finger touches the origin position of the ribbon controller, music performance and moving image reproduction are performed at a speed corresponding to the origin position = “0”. That is, in the music performance systems 16 to 18, for example, if the standard tempo value = 120, the performance is performed at the tempo value = 120 corresponding to the origin position. That is, the CPU 1 generates a tempo clock signal having a tempo value = 120 and supplies it to the MIDI device 16, and the MIDI device 16 reproduces music data in synchronization with the supplied tempo clock signal. In this case, in the moving image processing circuit 7, the moving image data stored in the dedicated RAM 70 is converted to a normal moving image reproduction speed, for example, by the time axis control unit 71 and the moving image reproduction unit 73 based on a command from the CPU 1. , 30 frames / second.

  Further, when an operation finger is touched at an arbitrary position of the ribbon controller, an operation amount corresponding to the position of the touched finger (origin position = displacement from “0”) is instructed from the CPU 1, and based on the instructed operation amount. Music performance and video playback are performed. Accordingly, when the ribbon controller is operated as in the second stage, the music data in the MIDI device 16 corresponds to the positive / negative (+, −) operation amount from the specified tempo value according to the operation amount of the ribbon controller. The tempo changed by the minute is played as in the bottom row. Correspondingly, the reading speed of the moving image data is changed from the normal moving image reproduction speed by an amount corresponding to the positive / negative (+, −) operation amount, and the moving image data is shown in the third row. Since the playback is performed at a speed corresponding to the operation amount of the ribbon controller, the playback of the moving image can be matched with the performance of music.

  In the example of FIG. 4, at the time t0, music performance and moving image playback are started, and at the same time, as shown in the second stage, the operation of the ribbon controller is started, and once in the positive direction, the negative direction is operated. After passing through the position = “0”, the operation is performed again in the positive direction. At the time t1, the operation is stopped at the predetermined positive position. At the same time, the operation jumps to the negative predetermined position and operates again in the positive direction. Is shown when the operation is stopped (the operation finger is released from the ribbon controller).

  In this case, as shown in the lowermost part of FIG. 4, the music data in the first section (defined by measures, beats, or ticks) n0 starts to be played back at time t1, as shown in the bottom row of FIG. When the operating finger is released from the controller, the music performance in the section n1 is resumed at a speed corresponding to the negatively touched predetermined position. When the operation is stopped at the time point t2 when the music performance in the section n2 is performed, the music performance is maintained at a speed corresponding to the operation amount when the operation is stopped after the time point t2.

  On the other hand, for the moving image data, as shown in the third row of FIG. 4, the playback speed is controlled in the same manner as the music data. That is, at the time point t0, the moving image data in the first section (defined by the number of frames) p0 of the moving image data is reproduced at a relatively high speed ("steep") according to the controller operation at a certain position in the positive direction. The moving image reproduction is sequentially performed at “slow” and “sudden” speeds according to the controller operation. When the operating finger jumps to a predetermined negative position at time t1, the moving image data of the section p1 is played back at a “slow” playback speed corresponding to the post-jump position, and the playback speed is sequentially increased according to the controller operation. Since the operation amount at the time point t2 after the operation stop is maintained based on the operation stop at the time point t2 when the moving image reproduction of the section p2 is performed, the high speed ( The video data will continue to be played with “Sudden”.

  Accordingly, the moving image data can be reproduced in accordance with the progress of the music performance, with the reproduction speed changing slowly according to the controller operation amount, similarly to the performance speed of the music performance. As shown in FIG. 2 (upper left), in addition to the change in the moving image reproduction speed based on the tempo instruction information such as the time interval of the tempo clock of the music data and the time code difference value (time code time interval), Since the music performance speed and the moving picture reproduction speed are simultaneously changed based on the tempo change information obtained by the real-time operation of the ribbon controller, the effect of performing the music performance and the moving picture reproduction can be enhanced.

[Second playback mode]
In this moving image time axis control instruction system, when the mode switch M2 in the operation device 12 is operated (see FIG. 2), the moving image processing circuit 7 can be operated in the second reproduction mode. In the second playback mode, when playing back a moving image associated with a music performance, the CPU 1 sends a preset TS of specific music performance timing-specific moving image playback timing to the time axis control unit 71 as shown in the upper right of FIG. send.

  Based on the specific timing set TS, the time axis control unit 71 sets the video playback speed to be instructed to the video playback unit 73 so that the specific playback timing of the video data matches the performance timing of the corresponding music performance. The adjustment is made and an instruction is given to the moving image reproduction unit 73 to reproduce the moving image at this speed. Then, the moving image reproduction unit 73 reads the sampled moving image data from the dedicated RAM 70 at the instructed moving image reproduction speed and reproduces the moving image.

  FIGS. 5 to 9 are diagrams schematically showing examples of time axis conversion for explaining the moving image reproduction in the second reproduction mode. 5 to 7 and the second stage of FIG. 8 represent music data used for music performance in the music performance systems 16 to 18, taking into account the tempo indication by time code or tempo clock (MIDI clock). And converted into performance time. In this example, music data “section n0” to “immediately before section n1” have a faster tempo than “section n1” to “section n2”, and this is displayed in the width of a circle. That is, tempo information is arranged at the start point of the section n1 of the music data. The MIDI device 16 starts the music performance from the section n0 with the initial tempo arranged in the music data as the performance tempo, and at the start point of the section n1, the performance tempo is further determined according to the tempo information arranged there. Change to a slower tempo and continue playing music.

  The second stage in FIGS. 5 to 7 and the third stage in FIG. 8 represent the moving picture data stored in the music data dedicated RAM of the moving picture processing circuit 7. In FIGS. Is made coincident with the total performance sections n0 to n2 of the music data for convenience. Such timing synchronization can be controlled by adjusting the timing at which the above-described deferred play command is arranged in the music data. The meanings of the circles in each figure represent each section (data unit) constituting the data, as already described with reference to FIGS. 3 and 4, and the music data has a predetermined number of bars, beats or ticks. It can be defined by (TICK) number [may be not an integer (may be a decimal)], and moving image data can be defined by a predetermined number of frames.

  FIG. 5 shows an example of time-axis conversion when a set (TS) in which moving image information at an arbitrary position in moving image data is associated with performance information at an arbitrary position in music data is used. This time axis conversion is suitable for a production effect in which one point (may be a plurality of points) in the course of moving images and music is matched. In this example, as shown between the uppermost stage and the second stage, a specific section p1 (start point) in the moving image data corresponds to a specific section n1 (start point) of the music data. In this way (“SET”), the start and end positions p0, n0; p2, n2 of both data are not associated (“Free”).

  In this case, each reproduction of moving image data and music data is started at arbitrary time points t0 and t0m, respectively. However, the moving image data is reproduced in the section p1 at the specific time point t1 when the reproduction of the music data in the specific section n1 set to correspond to the moving image data in the specific section p1 is started after the reproduction is started from the time point t0. The time axis for reproducing the moving image data is controlled so that the reproduction of the moving image data is started.

  That is, the start time information indicating the reproduction start times t0 and t0m of the first section p0 and n0 in the moving image data and the music data, and the music data of the section n1 set so as to correspond to the moving image data of the specific section p1. Specific time point information representing the specific time point t1 at which reproduction starts is sent from the CPU 1 to the time axis control unit 71. The time axis control unit 71 can receive the time point information and start the reproduction of the moving image data of the first section p0 at the time point t0 and can start the reproduction of moving image data of the specific section p1 at the specific time point t1. In this manner, the moving image playback unit 73 is controlled. Then, the moving image reproducing unit 73 calculates the reading speed of moving image data between time points t0 and t1, and starts reproducing moving image data of the first section p0 from the time point t0 at the calculated reading speed.

  In addition, the CPU 1 detects a time point t0m at which music data playback should be started, and instructs the MIDI device 16 to start playback. In response to an instruction to start playback, the MIDI device 16 starts playback of music data and supplies a tempo clock signal to the CPU 1 via the MIDI interface (I / F) 8. The CPU 1 corrects the reference time in the time axis control unit 71 based on the supplied tempo clock signal. Here, the “reference time” represents, for example, a reference timing for detecting time points such as time points t0, t1, t2,. That is, the “reference time” is measured before the music performance is started at the time point t0m. When the music performance is started, the “reference time” is measured in synchronization with the music performance thereafter.

  Thereby, at the specific time point t1, the moving image data of the specific section p1 is reproduced in synchronization with the reproduction of the music data of the specific section n1 set in advance so as to correspond thereto. In other words, the reading of moving image data is not fixed (Free), and the speed is adjusted from the reproduction start point t0 to the specific time point t1. Further, after the specific time point t1, the reproduction is performed based on a predetermined reading speed of the moving image data itself. Therefore, the moving image data reproduction end point t2 is, as illustrated, the music performance end point t2m. Does not necessarily match. The section p0 may also be reproduced at the reading speed of the moving image data itself. Thus, in this embodiment, the reproduction timing of the moving image data is controlled so that the moving image data at the specific position (n1) is reproduced at the time point t1 when the music data at the specific position (p1) is reproduced. .

  In FIG. 5, the specific time point t1 is one point. However, a plurality of corresponding positions in the course of moving images and music are set, and two or more reproductions of moving image data and music data at each set position are performed. You may make it correspond at the time. In this case, the time axis (speed) of moving image reproduction from the start of moving image reproduction to a specific time point and between specific time points is controlled. Further, in this embodiment, the description has been made assuming that the MIDI device 16 itself generates a tempo clock signal after the MIDI device 16 starts reproduction in response to a reproduction instruction from the CPU 1, but instead of the MIDI device 16. The CPU 1 may generate a tempo clock signal, and the MIDI device 16 may reproduce music data based on the tempo clock signal supplied from the CPU 1. In that case, the CPU 1 controls the period of the generated tempo clock signal based on the reference time in the time axis control unit 7 and the tempo information (arranged in the music data) reproduced by the MIDI device.

  FIG. 6 shows an example of time-axis conversion when a set (TS) in which video information at an arbitrary position and playback end position in video data is associated with performance information at an arbitrary position and performance end position in music data is used. ing. This time axis conversion is suitable for a production effect in which one point (may be a plurality of points) in the course of moving images and music and the end position coincide with each other. In this example, as shown between the uppermost stage and the second stage, a specific section p1 (start point) in the moving image data corresponds to a specific section n1 (start point) of the music data. (“SET”) and set the moving image data reproduction end (end of the last section p2) to correspond to the music data reproduction end (end of the last section n2) ( "SET" & "FIX") and the sections p0 and n0 of the start positions of both data are not associated ("Free").

  Here, “SET” refers to a setting for synchronizing an arbitrary position in the moving image data with an arbitrary position in the music data. “FIX” refers to a setting for controlling an arbitrary position in the moving image data and an arbitrary position in the music data to be reproduced at predetermined points on the reference time scale, respectively. Therefore, the “arbitrary position” for which “FIX” has been set is fixed at a “predetermined time point” on the reference time scale. For example, even if the playback time of a certain section or position of the video data is edited, the playback time of the video data at the position set to “FIX” remains fixed at “predetermined time” (fixed) Adjusted to remain.) In FIGS. 6 to 8, “SET” is set for all positions indicated by “FIX”.

  In the case of FIG. 6 as well, the moving image data and the music data are started to be reproduced at arbitrary start times t0 and t0m, respectively, but the moving image data is the moving image data of the specific section p1 after the reproduction is started from the start time t0. The time axis of the moving image reproduction is controlled so that the reproduction of the moving image data of the section p1 is started at the specific time point t1 when the reproduction of the music data of the specific section n1 set so as to correspond to is started. Next, after the specific time point t1, the playback time axis is controlled so that the moving image data from the start of playback of the specific section p1 to the end of playback of the final section p2 matches the music performance of the sections n1 to n2.

  That is, the reproduction of the music data in the section n1 set so as to correspond to the start time information indicating the reproduction start times t0 and t0m of the first section p0 and n0 in the moving image data and the music data and the moving picture data in the specific section p1 is performed. Specific time point information representing the specific time point t1 to be started and end time point information representing the end time point t2 at which the reproduction of the music data in the last section n2 is completed are sent from the CPU 1 to the time axis control unit 71. The time axis control unit 71 receives these pieces of time information, starts to reproduce the moving image data of the first section p0 at the time t0, and starts to reproduce the moving image data of the specific section p1 at the specific time t1 and ends at the time t2. Thus, the moving image reproducing unit 73 is controlled so that the reproduction of the moving image data in the last section p2 can be finished. Then, the moving image reproduction unit 73 calculates the first and second reading speeds of the moving image data between the time points t0 to t1 and t1 to t2, and the moving image data of the first section p0 is timed at the calculated first reading speed. The video data is reproduced from t0, and after the specific time t1, the moving image data from the specific interval p1 to the final interval p2 is sequentially reproduced at the calculated second reading speed.

  Thereby, at the specific time point t1, the moving image data of the specific time period p1 is not only reproduced in synchronization with the reproduction of the music data of the specific time period n1 set in advance so as to correspond thereto, but also after the specific time point t1. The moving image data up to the last section p2 is reproduced in synchronization with the reproduction of the corresponding music data. Therefore, as shown in the figure, the period from the reproduction start point t0 that is not set synchronously ("Free") to the specific time point t1 and the period from the specific time point t1 that is set synchronously ("SET") to the reproduction end point t2 are different. The playback speed of video data is adjusted.

  In FIG. 6, the specific time point t1 is one point. However, a plurality of corresponding positions in the course of moving images and music are set, and two or more reproductions of moving image data and music data at each set position are performed. You may make it correspond at the time. In FIG. 6, the end time t2 is matched, but as a modified example, the end time t2 is not matched (each end position is not “SET” but “Free”), and the reproduction start position t0 of the moving image data is set. The reproduction of the moving image and the music performance may be synchronized only from the common start time t0 to the specific time t1 so as to coincide with the music data reproduction start time t0m (each start position is “SET”).

  FIG. 7 shows a case where a set (TS) in which moving image information at a reproduction start position in the moving image data, an arbitrary position in the middle, and a reproduction end position is associated with performance information at the arbitrary position and performance end position in the music data is used. This shows an example of time axis conversion. This time axis conversion is an effect of matching the start and end positions of the moving image and music and one point (may be a plurality of points) in progress, especially the effect of replaying the moving image data repeatedly according to the music data by loop reproduction. Suitable for effect. In this example, as shown between the uppermost stage and the second stage, the playback start position of music data (start of the first section p0) and the end of playback (end of the last section p2) are respectively started to play back music data. It is set so as to correspond to the position of (start of the first section n0) and playback end (end of the last section p2) ("SET" & "FIX"), and a specific section p1 (of the video data) (Start point) is set so as to correspond to (a start point of) a specific section n1 of music data ("SET").

  In this case, at the predetermined start time t0, each reproduction of the moving image and music data in the first sections p0 and n0 set so as to correspond to each other is started and set so as to correspond to the moving image data in the specific section p1. The time axis for reproducing the moving image data is controlled so that the reproduction of the moving image data of the specific section p1 is started at the specific time t1 when the reproduction of the music data of the specific section n1 is started. Next, after the specific time point t1, the time axis for reproducing the moving image data is such that the moving image data from the start of reproduction of the specific section p1 to the end of reproduction of the last section p2 matches the music performance of the sections n1 to n2. Be controlled.

  That is, it is set so as to correspond to the start / end time information indicating the reproduction start time points t0 and t2 of the first and last intervals p0, n0; p2, n2 in the moving image and music data, and the moving image data of the specific interval p1. The specific time point information indicating the specific time point t1 at which the reproduction of the music data in the section n1 starts is sent from the CPU 1 to the time axis control unit 71. The time axis control unit 71 receives these pieces of time information, starts reproducing moving image data of the first section p0 at time t0, starts reproducing moving image data of the specific section p1 at specific time t1, and ends time t2. Thus, the moving image reproducing unit 73 is controlled so that the reproduction of the moving image data in the last section p2 can be finished. Then, the moving image reproduction unit 73 calculates the first and second reading speeds of the moving image data between the time points t0 to t1 and t1 to t2, and the moving image data of the first section p0 is calculated at the time points with the calculated first reading speed. The video data is reproduced from t0, and after the specific time t1, the moving image data from the specific interval p1 to the final interval p2 is sequentially reproduced at the calculated second reading speed.

  As a result, at the start and end times t0 and t2, the reproduction of the moving image data in the first and last sections p0 and p2 is started and ended in accordance with the start and end of the music performance in the first and last sections n0 and n2. The Further, at the specific time point t1, the moving image data in the specific section p1 is reproduced in synchronization with the reproduction of the music data in the specific section n1 set in advance so as to correspond thereto. In the example shown in the figure, the tempo and playback speed increase from a fixed (“SET” & “FIX”) playback start point t0 to a specific time point t1, and the tempo and playback speed decrease after the specific time point t1. ing.

  In FIG. 7, the specific time point t1 is one point. However, a plurality of corresponding positions in the course of the moving image and music are set, and the reproduction and performance of the moving image data and music data at each set position is two or more. You may make it correspond in several time points.

  FIGS. 8 and 9 are generalizations of the modification shown in the description of FIG. 6 or the example of FIG. 7, and the moving image data reproduction start position and moving image information at a plurality of positions in the middle of moving image data are respectively shown in FIGS. 4 shows an example of time axis conversion in the case of using a set (TS) associated with music data reproduction start position and performance information at arbitrary plural positions in the middle of the music data. This time axis conversion is suitable for the effect of rendering a plurality of points in progress from the start position of the moving image and music to each other. The top row in FIG. 8 simply shows the music data using a score, and is not strictly associated with the music data in the second row.

  In the example of FIG. 8, as shown between the second stage and the third stage, at least the moving image data reproduction start (start of the first section p0) position is set to the music data reproduction start (start of the first section n0). A certain plurality of sections p1, p2, p3, p4,... (Starting point) of the moving image data are set to correspond to the positions (“SET” & “FIX”). Are set so as to correspond to specific sections n1, n2, n3, n4,... (Start point) (“SET”).

  In this case, at the predetermined start time t0, the reproduction of the moving image and music data in the first sections p0 and n0 set so as to correspond to each other is started, and is set so as to correspond to the moving picture data in the specific section p1. The time axis for reproducing the moving image data is controlled so that the reproduction of the moving image data information of the specific section p1 is started at the specific time t1 when the reproduction of the music data of the specific section n1 is started. Next, after the specific time point t1, the moving image data is reproduced so that the moving image data in the specific sections p1, p2, p3, p4,... Matches the music data in the specific sections n1, n2, n3, n4,. The time axis is controlled.

  FIG. 9 schematically shows a method of calculating the recording position (frame) of moving image data read from the dedicated RAM 70 with the passage of time in order to control the moving image reproduction time axis. Start time information indicating playback and performance start time t0 of the first section p0, n0 in the moving image and music data, and a section n1 set so as to correspond to the moving image data of the specific sections p1, p2, p3, p4,. , N2, n3, n4,..., N2, n3, n4,..., N2, n3, n4,.

  The time axis control unit 71 receives these pieces of time information, starts to reproduce the moving image data of the first section p0 at the time t0, and the specific sections p1, p2, p3 at the specific time points t1, t2, t3, t4,. , P4,... Is controlled so that the moving image playback unit 73 can be started sequentially. The moving picture reproducing unit 73 calculates the reading speed of moving picture data between time points t0 to t1, t1 to t2, t2 to t3, t3 to t4,. The readout speed to be calculated is indicated by the slope of each line segment at time points t0 to t1, t1 to t2, t2 to t3, t3 to t4,. Then, the corresponding moving image data between time points t0 to t1, t1 to t2, t2 to t3, t3 to t4,.

  Thereby, the reproduction of the moving image data in the first section p0 is started at the start time t0 in accordance with the start of the music performance in the first section n0, and at the specific time points t1, t2, t3, t4,. The moving image data of p1, p2, p3, p4,... is reproduced in synchronization with the reproduction of the music data of specific sections n1, n2, n3, n4,.

  That is, as shown in the moving image time axis conversion example of FIG. 8, moving image data including a large number of frames is stored in the dedicated RAM 70 for music data. When the user specifies that a plurality of points in the data are synchronized, the readout position between the points is automatically interpolated, and the moving image data is slowly (“normal”, “slow”, “sudden”). Play with. Therefore, for example, the positions where there are strong notes on the score (the upper part of FIG. 8) are selected as specific positions of the music performance, and specific moving image frames (red light, blue light, yellow light, etc.) that are stimulating to these performance positions. If you set the screens to be compatible with each other, when a sound related to a specific note is generated, a specific video frame appears, so the sound and video are linked to a concert venue or club. Effective production for audiences such as events.

[Third playback mode]
In this moving image time axis control instruction system, when the mode switches M1 and M2 in the operation device 12 are operated (see FIG. 2), the moving image processing circuit 7 can be operated in the third reproduction mode. In the third playback mode, the CPU 1 operates a real-time operator such as a ribbon controller in addition to a preset specific music performance timing-specific video playback timing set (TS) when playing a moving image accompanying a music performance. Tempo change information (TM) corresponding to is sent to the time axis control unit 71.

  Based on the specific timing set, the time axis control unit 71 makes the specific reproduction timing of the moving image data coincide with the performance timing of the corresponding music performance, and further according to the operation of a real-time operator such as a ribbon controller. Then, the moving image reproducing speed commanded to the moving image reproducing section 73 is adjusted so that the reproducing speed changes, and an instruction is given to the moving image reproducing section 73 to reproduce the moving image data at this speed. Then, the moving image reproduction unit 73 reads the sampled moving image data from the dedicated RAM 70 at the instructed moving image reproduction speed and reproduces the moving image.

  In the third playback mode, as in the second playback mode, not only the moving image data of the specific section matches the music data of the specific section, but also the speed of moving image playback and music performance can be changed according to the operation of the real-time operator. Therefore, various effects can be obtained.

[Various Embodiments]
As described above, the moving image time axis instruction processing according to the present invention has been described according to the specific embodiment, but the embodiment is not limited to the illustrated example. For example, in the embodiment, the input moving image is from a VTR (video tape recorder). However, the present invention is not limited to this, and an apparatus capable of reproducing moving images such as a video output from an LD player, a DVD player, or a PC (personal computer). If so, the type is not limited.

  In the embodiment, the output device is a relatively large video display, but the type and size of CRT, LCD, etc. are not limited. Further, a plurality of output devices such as a monitor for the user (operator) and a display for the audience may be connected.

  In the embodiment, the playback speed of the moving image is controlled by changing the time axis control signal by the operation unit built in the main body system and the moving image processing system itself. However, the playback speed of the moving image is controlled by the time axis control signal from the external control device. May be controlled, and an external control signal input means for this purpose may be provided. Specifically, on the video control software in the PC, the speed of playback is represented by a broken line with numerical instructions, or using an output value represented by a handwritten input curve using an external pointing device such as a mouse or tablet, It may be one that changes the playback speed of the video.

  In the embodiment, the reading of the moving image data is automatically corrected based on the fixed (“FIX”) reproduction start or end point (end). However, the reproduction start or end point (end) is “FIX” or “Free”. Whether to cut the excess part or to compensate for the missing part can be arbitrarily adopted as necessary.

  In the embodiment, an example in which one moving image data is reproduced in synchronization with reproduction of one music data is shown, but a plurality of moving image data may be reproduced in synchronization at a time when each is selected individually. .

  In the embodiment, the MMC command is used to control the reproduction of the moving image data, but a MIDI show control (MSC) command may be used instead. In that case, based on the go (“GO”) command of the MSC, the moving image data may be selected according to the queue (“QUE”) number included in the go command, and playback may be started.

  The specific reproduction timing of the music data may be specified by time on the performance time axis, or may be specified by the number of tempo clocks, the number of measures, the beat, or the like. Also, the specific playback timing of moving image data may be specified by time on the playback time axis, or may be specified by a frame number of the moving image.

  In the embodiment, the music data in which the tempo information is arranged has been described as being associated (SET) with an arbitrary position in the moving image data. However, after the association (SET) is first performed, Tempo information may be arranged at this point. In this way, even if the tempo of the music data is edited, the video data is played back in synchronization with the music data after the tempo edit, so the user does not mind the synchronization gap with the video and the user can worry about the tempo of the music data. Can be edited freely.

  In FIGS. 7 and 8, as a result of the application of the present invention, the tempo of the moving image data and the music data is depicted as being synchronized throughout, but it is not always necessary to completely synchronize so far. It suffices if the desired part of the video data and the desired part of the music data can be synchronized.

  Note that the music data processing method is a method of changing the processing cycle according to the set tempo instruction information, the processing cycle is constant, and the value of timing data in the music data is changed according to the set tempo instruction information The processing method and the processing cycle are constant, and any method may be used, such as a method of changing the timing data counting method in the music data in accordance with the tempo instruction information.

FIG. 1 is a block diagram showing a hardware configuration of a moving image reproduction control system (moving image time axis control instruction system) according to an embodiment of the present invention. FIG. 2 is a block diagram of the moving image processing function unit of the moving image time axis control instruction system according to the embodiment of the present invention. FIG. 3 shows an example of a case where the moving image time axis control is performed by changing the moving image reproduction speed according to the tempo instruction information in the first embodiment of the moving image time axis control instruction system according to one embodiment of the present invention (0th reproduction mode). FIG. FIG. 4 shows an example of the case where the moving image time axis control is performed by changing the moving image reproduction speed by operating the ribbon controller in the moving image time axis control instruction system according to the first embodiment of the present invention (first reproduction mode). FIG. FIG. 5 is a diagram showing a moving image time axis control example [1] when one point is set in the second embodiment (second reproduction mode) of the moving image time axis control instruction system according to the embodiment of the present invention. FIG. 6 is a diagram showing a moving image time axis control example [2] when two points including the end point are set in the second embodiment (second reproduction mode) of the moving image time axis control instruction system according to the embodiment of the present invention. It is. FIG. 7 shows a moving picture time axis control example [3] in the case where three points including the start and end points are set in the second embodiment (second playback mode) of the moving picture time axis control instruction system according to one embodiment of the present invention. FIG. FIG. 8 is a diagram showing a general moving image time axis control example [4] of the second embodiment (second reproduction mode) of the moving image time axis control instruction system according to an embodiment of the present invention. FIG. 9 is a diagram for explaining the calculation of the moving image playback speed in the moving image time axis control example [4] of the second embodiment (second playback mode).

Explanation of symbols

A tempo instruction detection mechanism for detecting a tempo instruction of TC music data;
TM Tempo change detection mechanism to detect music tempo change operation,
TS specific playback and performance timing setting mechanism,
p0, p1, p2, p3, p4 Section (frame) number in the video data,
t0, t1, t2, t3, t4 Read timing of each section (frame),
t0m, t2m Music data read start and end timing,
n0, n1, n2, n3, n4 Section (measure, beat or tick) numbers in music data.

Claims (4)

One moving image data is selected from a plurality of moving image data composed of a large number of frames representing a moving image stored in the moving image data memory in accordance with a locate command arranged in accordance with a desired timing of the music data, and the music A video playback device for starting playback of the selected video data in accordance with a deferred play command arranged in accordance with a desired timing of data and displaying the video on a display device,
A clock input means for inputting a tempo clock representing a performance tempo of music based on the music data during reproduction of the moving image data;
Control signal generating means for generating a reproduction speed control signal for controlling the reproduction speed of moving image data represented by the number of frames per unit time in accordance with the tempo clock input by the clock input means;
Moving image data reading means for sequentially reading out frames of moving image data stored in the moving image data memory at a frame reading rate based on a reproduction speed control signal generated by the control signal generating means and outputting the frames to the display device; Comprising
When the tempo clock indicating the standard tempo is input by the clock input means, the frame reading rate is represented by the number of frames per predetermined unit time, and changes from the standard tempo to positive or negative by the clock input means. When the tempo clock indicating the tempo is input, the moving picture reproducing apparatus is represented by the number of frames per unit time which is changed from the predetermined number of frames per unit time by an amount corresponding to the change in tempo. further,
2. The moving image reproduction according to claim 1, further comprising moving image data synthesizing means for synthesizing the moving image data read by the moving image data reading means and the separately input moving image data and outputting the resultant to the display device. apparatus. One moving image data is selected from a plurality of moving image data composed of a large number of frames representing a moving image stored in the moving image data memory in accordance with a locate command arranged in accordance with a desired timing of the music data, and the music A video playback method in a video playback device for starting playback of the selected video data in accordance with a deferred play command arranged in accordance with a desired timing of data and displaying the video on a display device,
A clock input step of inputting a tempo clock representing a performance tempo of music based on the music data during reproduction of the moving image data;
A control signal generating step for generating a playback speed control signal for controlling the playback speed of moving image data represented by the number of frames per unit time in accordance with the tempo clock input in the clock input step;
A moving image data reading step for sequentially reading out the frames of the moving image data stored in the moving image data memory at a frame reading rate based on the reproduction speed control signal generated in the control signal generating step and outputting the frames to the display device; With
When the tempo clock indicating the standard tempo is input by the clock input means, the frame reading rate is represented by the number of frames per predetermined unit time, and changes from the standard tempo to positive or negative by the clock input means. When the tempo clock indicating the tempo is input, the moving image reproduction method is characterized by being represented by the number of frames per unit time which is changed from the predetermined number of frames per unit time by an amount corresponding to the tempo change. further,
4. The moving image reproduction according to claim 3, further comprising a moving image data synthesizing step of combining the moving image data read in the moving image data reading step and the separately input moving image data and outputting the resultant data to the display device. Method.

Images