JP2006166055A - Device and program for synchronous reproduction of musical sound and image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To minimize the number of image files, when an image is displayed in synchronization with musical sound. <P>SOLUTION: A plurality of image data are recorded into one image file A, and a part of image data specified by an image designation data in the image file are cut out and read out by a cut-out range data. An image data at a position in an image buffer specified with a display position data is overwritten by a read-out cut-out image data C. The image data on the image buffer is displayed on a display unit. Variously switched images can be displayed on the display unit by sequentially reading a cut-out range with a changed timing using the cut-out range data. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a musical sound and image synchronous reproduction apparatus and a synchronous reproduction program which are configured to reproduce musical sounds and images synchronously.

Conventionally, a storage area is set in correspondence with each of a plurality of pad switches, one image data is recorded in the corresponding storage area, and turned on when one of the pad switches is turned on. There is known an image control apparatus that reads and displays image data from a storage area corresponding to a pad switch (see Patent Document 1).
In this image control apparatus, video effect parameters for controlling the image data are recorded separately from the image data, and brightness, hue, display position, display size, and the like are controlled by the video effect parameters. . Specifically, the image data read from the storage area corresponding to the pad switch that is turned on is changed based on the video effect parameter, and the changed image data is displayed on the display.

Further, it is conceivable to display while switching images in accordance with the reproduction of musical sounds by SMF (Standard MIDI File) which is a standard file format when handling music data in MIDI format. In this case, the instruction data indicating the image to be displayed is recorded at the position of the display timing in the SMF, and the instruction data is instructed at the timing when the image display instruction data is read during the SMF reproduction. The image to be read may be read out and displayed. In this case, as many image files as the number of images to be displayed need to be recorded in the storage means in advance.
JP2003-244594

  In such an image control apparatus, when it is desired to sequentially change a part of the displayed image in accordance with the reproduction of the musical sound and to produce an effect like a moving image, the image data is read in units of image files. Therefore, as many image files as the number of movements are required. For this reason, the number of image files used when reproducing performance data such as one SMF increases, and the number of files increases, so that various operations when handling performance data such as SMF become troublesome. was there. For example, when performance data such as SMF is transferred to another recording medium, all image files used by the performance data such as SMF must also be transferred.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a musical sound and image synchronized playback apparatus and a synchronized playback program capable of reducing the number of image files as much as possible when images are displayed in synchronization with musical sounds.

  In order to achieve the above object, the musical sound and image synchronized playback apparatus and synchronized playback program of the present invention cut out clipped image data from an image file within a range specified by the clipped range data in an image reading event, and The most important feature is that the cut-out image data is overwritten to the position of the image data in the image storage means specified by the display position data in the reading event.

  According to the present invention, a plurality of image data to be displayed in one file can be provided. Therefore, the number of image data files can be reduced.

  It is an object of the present invention to provide a synchronized playback apparatus and synchronized playback program for musical sounds and images that can reduce the number of image files as much as possible when displaying images in synchronization with musical sounds. Realized by cutting out the cut image data from the image file in the range specified by the range data and overwriting the cut image data to the image data position in the image storage means specified by the display position data at the image reading event did.

FIG. 1 is a block diagram showing a configuration of a musical sound and image synchronized playback apparatus 1 having a synchronized playback program according to an embodiment of the present invention. The synchronized playback program is executed in the synchronized playback apparatus 1 having substantially the same configuration as that of the personal computer, thereby realizing the synchronized playback apparatus for musical sound and image according to the present invention.
In the synchronized playback apparatus 1 shown in FIG. 1, a CPU (Central Processing Unit) 10 controls the overall operation of the synchronized playback apparatus 1 and executes operation software such as a synchronized playback program. A ROM (Read Only Memory) 11 stores at least operation software such as a synchronous reproduction program executed by the CPU 10, and a RAM (Random Access Memory) 12 stores various data such as a work area of the CPU 10 and image files. A storage area is set. By making this ROM 11 a rewritable ROM such as a flash memory, the operation software can be rewritten and the operation software can be easily upgraded. Further, an image file prepared in advance may be stored in the ROM 11.

  The detection circuit 14 detects an event of the operation element 13 by scanning the operation element 13 such as a panel operation element or a pointer device, and outputs an event output corresponding to the operation element 13 having the event. The display circuit 16 includes a display unit 15 such as a liquid crystal, and displays images on the display unit 15 in synchronization with musical sounds, and displays various setting screens and the like in the synchronized playback apparatus 1 on the display unit 15. The various setting screens are screens on which a user selects performance data using a GUI (Graphical User Interface), and instructs the start and end of playback. The tone generator circuit 18 is a waveform memory tone generator or FM tone generator having a waveform memory, and generates tone waveform data by performing tone generation processing based on the control of the CPU 10. The musical sound waveform data generated in the sound source circuit 18 is supplied to the sound system 17 and converted into an analog signal to be emitted. The sound system 17 can perform volume control and sound quality control. The communication interface (I / F) 20 is an interface for connecting the synchronous playback apparatus 1 to a communication network such as a LAN (local area network), the Internet, or a telephone line, and is connected to the external control device 19 via the communication network. can do. Each of these units is connected to the communication bus 21. As a result, various programs, image data, and the like can be downloaded from the external control device 19. The downloaded program, image data, and the like are stored in the RAM 12 or a hard disk device (not shown).

In the synchronized playback apparatus 1 according to the present invention configured as described above, a plurality of image files and performance data including musical events and image events are recorded in a storage device such as the RAM 12. This performance data is recorded in time series in the order in which various event data are read out, and timing data representing the read timing at which the event should be processed is associated with each event and recorded. The timing data is recorded in the format of the number of MIDI clocks, for example.
The performance data is, for example, SMF (Standard MIDI File), which is a standard file format when handling music data in MIDI format, and both the event for musical sound and the event for image are the event data in MIDI format. ing. In this case, the event for musical sound is various event data for instructing processing related to the sound generation, and specifically, event data such as note-on, note-off, program change, and control change. Yes. When it is determined that the musical sound event has been read out while playing the performance data, the musical sound event is sent to the sound source circuit 18, and the generation process of the musical sound waveform data corresponding to the event is performed. 18 is performed.

  FIG. 2 shows an image of an image file characteristic of the present invention stored in the RAM 12 or the like. As shown in FIG. 2, in the image file A, one piece of image data created by connecting a plurality of image data B is recorded. This image data B is, for example, a part of the image data that is sequentially changed in order to produce a motion when a part of the displayed image is changed in order in accordance with the reproduction of the musical sound, and an effect such as a moving image is desired. It is said. Specifically, a plurality of pieces of image data in which the hand moves sequentially are sequentially recorded as image data B of the image file A in order to display the human hand displayed on the display unit 15 so as to move. Therefore, the plurality of pieces of image data B are cut out and read one by one in the display order from the image file A, and the image data of the hand portion in the image buffer is sequentially rewritten and displayed with the read image data B. The display unit 15 is displayed so that the hand moves.

  By the way, the event for an image is various event data shown below instructing processing related to image display, and is defined and recorded in the format of an exclusive message of the MIDI standard. When it is determined that the event for the image has been read while playing the performance data, the process corresponding to the event for the image is executed by the synchronous playback program. The image event defines an image reading event, a display start event, a display end event, and a special display event described below.

1. Image reading event This event instructs to read image data into the image buffer, and always includes the following three data.
a. Image designation data that designates an image file of image data to be read.
b. Cutout range data that specifies the read range (the cutout range on the image file) of the image file specified by the image specification data.
c. Display position data for designating a display position on the display unit 15 of image data (cut image data) read based on the cut range data.

  A mode of image data read out and displayed by this image reading event is shown in FIG. Assume that the image file A as shown in FIG. 2 stored in the RAM 12 or the like is designated by the image designation data in the image reading event. This image file A is a bitmap image, and the range of the cut image data C in the image file A is designated by the cut range data. The range of the cut-out image data C is a rectangle cut out by the number of dots Xs (horizontal axis) and Ys (vertical axis) from the coordinate position (Xp, Yp) indicated by the number of dots from the start point at the upper left end of the bitmap of the image file A. Specified as a range of. The cut-out image data C cut out in this way indicates the position of the start point of the upper left end of the bitmap of the image file A by the coordinate position (Xa, Ya) indicated by the number of dots from the start point of the upper left end of the display screen in the display unit 15. By displaying, it can be displayed at an arbitrary display position.

2. Display start event This event instructs to display image data recorded in the image buffer. By defining this display start event, an image can be created in advance by an image reading event and recorded in the image buffer, so that display switching of images can be performed quickly. In other words, it is possible to set the display start timing of the image data independently of the image data read by the image read event, thereby causing a time lag in an image obtained by sequentially changing a part of the displayed image according to the reproduction of the musical sound. Thus, it is possible to display on the display unit 15 an image having an effect such as a moving image in synchronization with the musical sound.

3. Display end event This event instructs to erase the image data in the displayed image buffer. After the display by the display start event is displayed, the image based on the image data read into the image buffer is continuously displayed on the display unit 15. Therefore, when the display is completed, the image displayed on the display unit 15 is cleared and the basic is performed. A display end event is defined to return to the image. For this reason, a display end event is always included at the end of the SMF file.

4). Special display event This event indicates that the image data in the image buffer should be set to “black” or “white”. For example, a special display event is inserted when image data is pasted on an entirely black screen by an image reading event.

Next, FIG. 4 shows a flowchart of performance data reading processing executed in the synchronized playback apparatus 1 having the synchronized playback program according to the present invention. This performance data reading process is started at every MIDI clock timing during the playback from the start to the end of the playback of the performance data, and the process corresponding to the type of the read event is started. is doing.
When the MIDI clock timing is reached, the performance data reading process is started, and in step S10, it is determined from the timing data in the performance data whether or not the event to be processed at the current MIDI clock timing is in the performance data being reproduced. Is done. If it is determined that there is an event to be processed at the current MIDI clock timing, the process proceeds to step S11 to determine whether the event type to be processed at the current MIDI clock timing is an event for music or an event for images. Determined. Next, an event process corresponding to the determined event type is executed in step S12. When this process ends, the performance data reading process ends. Here, when there are a plurality of events to be processed at the current MIDI clock timing, all event processing corresponding to the event types determined in the recorded order is sequentially executed.

  If the event determined in step S11 is a musical sound event, the musical sound event read from the performance data in step S12 is sent to the tone generator circuit 18. The tone generator circuit 18 performs processing based on the received musical sound event to generate musical sound waveform data. The musical sound events include events such as note-on (sounding start), note-off (sounding end), program change (tone color change), control change (pitch change and effect addition). The generated musical sound waveform data is supplied to the sound system 17, and a musical sound based on the supplied musical sound waveform data is reproduced and emitted from the sound system 17.

  If the event determined in step S11 is an image event, processing corresponding to the type of image event is executed in step S12. As described above, image event types include image reading events, display start events, display end events, and special display events, and image reading processing, display start processing, display end processing, and special events corresponding to each event. Any one of the display processes is started and executed in step S12.

Here, FIG. 5 shows a flowchart of an image reading process that is started when an event determined as an image event is determined as an image reading event.
When the image reading process is started, the image data of the image file specified by the image specifying data included in the image reading event to be processed this time is read from the storage means such as the RAM 12 and opened in step S20. The image of the opened image file is an image file A shown in FIG. Next, in step S21, image data in a range specified by the cut-out range data included in the image reading event is cut out from the image data of the opened image file. This cut-out range data is the coordinate position (Xp, Yp) indicated by the number of dots in FIG. 3 and the number of dots Xs, Ys. In step S22, the clipped image data (cutout image data) is overwritten to the position indicated by the display position data included in the image reading event in the image data recorded in the image buffer. Thus, the image reading process is completed, and the process returns to the performance data reading process. Here, the image buffer is a buffer (image storage means) for temporarily recording image data (image data representing an image to be displayed on the display unit 15) designated by an image event read from the performance data. And has a storage area in which image data for one screen that can be displayed on the display unit 15 shown in FIG. 3 can be recorded. The image buffer is configured by allocating a storage area of the RAM 12 or by a storage unit provided in the display circuit 16. The display position data is the coordinate position (Xa, Ya) indicated by the number of dots in FIG.

Next, FIG. 6 shows a flowchart of a display start process started when an event determined as an image event is determined as a display start event.
When the display start process is started, a display start instruction is given to the display circuit 16 in step S30, and the image data recorded in the image buffer is displayed on the display unit 15. As a result, the display start process ends, and the process returns to the performance data reading process. Note that an image displayed by the image data clipped by the above-described image reading process and pasted on the image data recorded in the image buffer and displayed on the display unit 15 is shown as cut image data C shown in FIG. Yes.

Next, FIG. 7 shows a flowchart of a display end process started when an event determined as an image event is determined as a display end event.
When the display end process is activated, the image data of the basic screen is read from the storage means such as the ROM 11 and the RAM 12 and supplied to the display circuit 16 in step S40, and the basic image is displayed on the display unit 15. As a result, the display end process ends, and the process returns to the performance data reading process. Here, the basic screen is a screen that is displayed on the display unit when performance data is not being played back, and is a screen for the user to select performance data and instruct the start and end of playback. Yes. The image data of the basic screen is recorded in advance in storage means such as the ROM 11 and RAM 12 independent of the image buffer.

Next, FIG. 8 shows a flowchart of special display processing that is started when the event determined to be an image event is a special display event.
When the special display process is activated, the image data on the image buffer is rewritten to a special display state such as black or white in step S50. As a result, the special display process ends, and the process returns to the performance data reading process. Then, the image reading process described above is performed, and then the display start process is performed, whereby an image based on the image data pasted by the image reading process on the special display screen is displayed on the display unit 15. .

According to the musical sound and image synchronized playback apparatus and synchronized playback program of the present invention described above, only a part of image data can be cut out from one image file and reflected in the image data displayed on the display unit 15. It becomes like this. Therefore, it is possible to record a plurality of image data as one image file, and to cut out part of the image data by shifting the timing from the image file. For this reason, with a very small number of image files, various images displayed on the display unit 15 can be switched and effects such as moving images can be provided. In this case, since the number of image files is reduced, it becomes possible to easily move and copy all image files used together with the SMF when the performance data is SMF.
In the present invention, the image data displayed on the display unit 15 is not created at the timing when the display start is instructed. In the present invention, creation of image data to be displayed on the display unit 15 and start of display of the image data can be independently designated. Thus, since the image data can be created before the display start timing, the image data can be quickly displayed on the display unit 15 when a display start instruction is received. Therefore, a desired image created in advance at the timing of just instructing the display start can be displayed on the display unit 15 without being delayed with respect to the progress of the sound generation.

Note that after the display start event is read out, control may be performed so that the image displayed on the display unit 15 cannot be manually changed. After the display start event is read, every time the data in the image buffer is updated, a new image after update is displayed on the display unit. When the display end event is read, the image buffer The display of the image based on the data may be stopped, and the image before the display start event is read may be displayed.
Further, when an operation program such as a synchronous reproduction program is not stored in the ROM 11, the operation program or data is stored in a hard disk or a CD-ROM and is read into the RAM 12, or is connected to the server computer via the communication interface 20. By connecting and downloading the operation program and data from the server computer, the CPU 10 may be caused to perform the same operation as when the operation program and data are stored in the ROM 11.
Furthermore, the image data and cut-out image data recorded in the image buffer may be recorded in the image buffer after being controlled by a video effect parameter that controls brightness, hue, display position, display size, and the like.

  In the above description, the synchronized playback program according to the present invention is provided in the synchronized playback apparatus for musical sounds and images. However, the present invention is not limited to this, and the synchronized playback of musical sounds and images can be provided in various musical sound related apparatuses. An apparatus can be realized. In this case, the synchronized playback program may be recorded on a recording medium and incorporated in various musical sound related devices.

It is a block diagram which shows the structure of the synchronous reproduction apparatus of a musical sound provided with the synchronous reproduction program of the Example of this invention. It is a figure which shows the image of the image file in the synchronous reproduction apparatus of a musical sound of this invention. It is a figure which shows the image which cuts out and displays a part of image file on the display part in the synchronous reproduction apparatus of the musical sound of this invention, and an image. It is a flowchart of the reading process of the performance data performed in the synchronous reproduction apparatus of a musical sound of this invention. It is a flowchart of the image reading process performed in the synchronous reproduction apparatus of a musical sound of this invention. It is a flowchart of the display start process performed in the synchronous reproduction apparatus of the musical sound of this invention. It is a flowchart of the display completion process performed in the synchronous reproduction apparatus of a musical sound of this invention. It is a flowchart of the special display process performed in the synchronous reproduction apparatus of a musical sound of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Synchronous reproduction apparatus, 10 CPU, 11 ROM, 12 RAM, 13 Operator, 14 Detection circuit, 15 Display part, 16 Display circuit, 17 Sound system, 18 Sound source circuit, 19 External control apparatus, 20 Communication interface, 21 Communication bus

Claims (4)

Storage means for recording an image file and musical performance data including musical events and image events;
A determination unit that determines whether the read event is a music event or an image event when events in the performance data stored in the storage unit are sequentially read;
Sound source means for controlling the sound generation of a musical sound based on the music event when the read event is determined to be a music event and the determination means;
The image event includes image designation data for designating an image file to be read from the previous storage means, cutout range data for designating a read range in the image file designated by the image designation data, and the cutout range. And at least an image reading event having display position data for designating a position for displaying the cut image data cut out based on the data, and the read event is determined by the determination means as an image event. If the event for the image is the image reading event, it is specified by the cut-out range data at the image reading event from the image file specified by the image specifying data at the image reading event. An image specified by the display position data in the image reading event after cutting out the cut image data in a range And display processing means for pasting the cut image data to the position of the image data in 憶 means,
Display means for displaying the image data stored in the image storage means on the display unit at a display start timing;
An apparatus for synchronously reproducing a musical sound and an image, comprising:   The image event includes a display start event that instructs to display the image data recorded in the image storage unit on the display unit. The read event is an image event and the determination unit. 2. The musical sound and image synchronous reproduction apparatus according to claim 1, wherein the timing at which the image event is determined as the display start event is the display start timing. A computer comprising storage means for recording an image file and musical performance data including musical events and image events,
A determination unit that determines whether the read event is a music event or an image event when events in the performance data stored in the storage unit are sequentially read;
Sound source means for controlling the sound generation based on the music event when the read event is determined to be a music event and the determination means;
The image event includes image designation data for designating an image file to be read from the previous storage means, cutout range data for designating a read range in the image file designated by the image designation data, and the cutout range. And at least an image reading event having display position data for designating a position for displaying the cut image data cut out based on the data, and the read event is determined by the determination means as an image event. If the event for the image is the image reading event, it is specified by the cut-out range data at the image reading event from the image file specified by the image specifying data at the image reading event. An image specified by the display position data in the image reading event after cutting out the cut image data in a range Display processing means for pasting the cut image data to the position of the image data in 憶 means,
Display means for displaying the image data stored in the image storage means on the display unit at a display start timing;
A program for synchronized playback of musical sounds and images, characterized in that it is made to function as a program. The image event includes a display start event that instructs to display the image data recorded in the image storage unit on the display unit. The read event is an image event and the determination unit. 4. The program for synchronous reproduction of a musical sound and an image according to claim 3, wherein the timing at which the image event is determined to be the display start event is the display start timing.

Images