JP5040425B2 - Content playback method, playback device, and program - Google Patents

Description

  The present invention relates to a technique for controlling reproduction of content different from that by reproduction of image content.

  Some conventional playback apparatuses that play back content perform playback of content in synchronization with playback of content from other devices. For example, in the playback apparatus described in Patent Document 1, performance information generated by playing externally in association with playback of a recording medium such as a CD is input, and the input performance information is sent from the recording medium at a predetermined interval. Synchronized performance information corresponding to the time information read out in step S1 is created and stored. Then, at the time of the synchronous performance, the time information read from the recording medium is compared with the time information included in the synchronous performance information, and the performance information corresponding to the matching time information is reproduced. As a result, it is possible to easily perform repeated synchronous reproduction of music or the like recorded on a normal recording medium such as a CD and music played by another device.

  However, in such a reproduction method, the content must be stored once. For this reason, rapid synchronized reproduction is impossible. Further, when the stored content is an image, the data amount becomes large. For this reason, it is not necessarily desirable for performing synchronized playback.

Further, when performing synchronized playback in real time, it can be performed by notifying one of the two playback devices (master) to the other (slave) of necessary information. For this purpose, however, the playback apparatus (master) must be provided with a special process or interface, which is not desirable. This is because a reproduction apparatus that is generally widely used does not have such processing and interface and cannot be used. It is almost impossible for a general user to prepare such a special playback device, and even if it is prepared, the economic burden on the user side becomes larger. For this reason, it is considered desirable to use an ordinary playback device as the master, that is, to be able to control the playback of content such as real-time synchronized playback through the content played back by the playback device.
JP-A-5-297867

  An object of the present invention is to provide a technique for controlling the reproduction of content on another reproduction device in real time according to the content reproduced by a reproduction device that is generally spread.

The content playback method of the present invention is a method for playing back another type of content different from the image content by the second playback device in accordance with the playback of the image content by the first playback device. , the screen on which is reproduced by the reproduction of the image content, and insert the image representing the control information in which the second reproducing apparatus according to the reproduction of other content recognizable, by the reproduction of the image content by the first playback device the screen to be reproduced is input to the second playback device, to recognize images representing control information on said screen to a second reproduction device by the input of the screen, the other content on the basis of the recognition result Let it play.

A second aspect of the reproducing apparatus of the present invention includes a screen input means for assumes that renewable different types of content from the image content, inputting a screen to be reproduced by the reproduction of the image content, present in the image input means screen on which the inputted recognizing information recognition means the image representing the control information related to the reproduction of the content, and reproducing means for reproducing the content, the playback of the content by the playback means, information recognition means A reproduction control means for recognizing an image representing the control information recognized by and controlling based on the recognition result .

The playback device of the third aspect further includes image output means for outputting the image input by the image input means in addition to the configuration of the second aspect.

Reproducing apparatus of the fourth aspect, in addition to the configuration in the third embodiment, on the image input means inputs the image plane of the image representing the control information, the image processing to perform processing to place another image The image output means outputs an image processed by the image processing means.

  The program of the present invention includes an image input function for inputting an image reproduced by playing back image content to a computer that can be used as a playback device capable of playing back different types of content from image content, and image input. An information recognition function for recognizing control information related to content reproduction existing in an image input by the function, a reproduction function for reproducing the content, and reproduction of content by the reproduction function as control information recognized by the information recognition function And a reproduction control function for controlling based on this.

  In the present invention, control information related to reproduction of other content is inserted into an image (video) reproduced by reproduction of the image content, and the image reproduced by the reproduction side (master) of the image content is reproduced as the other content. Is input to the playback side (slave), and the slave is made to recognize the control information to control the playback of other contents. For this reason, other content can be synchronized with the reproduction of the image content in real time. Since the master only needs to reproduce the image content, it is possible to use a reproduction apparatus that has been widely used.

  When the image input to the slave is output, the image can be arbitrarily displayed on the slave or a display device capable of inputting the image from the slave. When the control information is deleted (erased) for output, a more appropriate image can be displayed. Depending on the content to be processed, an image to which useful information is further added can be displayed.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
<First Embodiment>
FIG. 1 is a diagram showing a system configuration constructed for content reproduction. As shown in FIG. 1, the system is constructed by connecting an electronic musical instrument 10 to a DVD player 20 and a television (hereinafter “TV”) 30. The electronic musical instrument 10 includes a keyboard 11, a speaker 12, and a liquid crystal display (LCD) 13.

  The playback apparatus according to the first embodiment is mounted on the electronic musical instrument 10. Accordingly, in the system configuration shown in FIG. 1, the video (Video) signal output from the DVD player 20 is input, playback of the music is controlled by the video signal, and the video signal is output to the TV 30 to output an image (video). ) Is displayed.

  FIG. 2 is a functional configuration diagram of the electronic musical instrument. The electronic musical instrument 10 has the following functions and hardware.

  The video decoder 101 decodes the video signal input from the DVD player 20 and reproduces an image. The video buffer 102 is a memory that temporarily stores the reproduced image (image data), and the stored image is output to the barcode detection unit 103 and the video processing unit (OSD) 105.

  In the present embodiment, it is assumed that a two-dimensional barcode as shown in FIG. 3 is inserted as a control information into a necessary image in a series of images on an optical disk such as a DVD that can be played back by the DVD player 20. is doing. Thus, by arranging control information directly in an image, a playback device (master) that can play back an optical disk such as a DVD player 20 can be used as a playback device (master) that is generally widely used. The bar code detection unit 103 performs image analysis, extracts a bar code inserted therein, recognizes information indicated by the bar code, and outputs the recognition result to the sequencer unit 108.

  It is not desirable to display (output) the image with the barcode inserted as it is. For this reason, in the present embodiment, as shown in FIG. 4, processing for arranging another image on the barcode is performed. The video database 104 is a memory storing an image (video) for performing the processing or data for generating the image. Actual processing is performed by the video processing unit 105 and output to the video encoder 106. Thereby, the video output by the video processing unit 105 is converted into a video signal by the video encoder 106 and displayed by being input to the TV 30.

  The ROM 107 stores at least one piece of music data for music playback, for example, in the form of a standard MIDI file (SMF). In the SMF, for each performance event, time information indicating the processing timing and event information indicating the contents of the event are prepared. The sequencer unit 108 reads designated music data from the ROM 107 by a user operating a predetermined switch, and reproduces it.

  In this embodiment, it is assumed that a Start command for instructing the start of reproduction of music data, a Stop command for instructing the end thereof, time information, and the like are notified by a barcode inserted in the image. The time information is represented with reference to an image in which a bar code indicating the Start command is inserted, that is, an absolute time. As a result, the SMF time information also indicates the absolute time.

  Note that the control information represented by the barcode is not limited to the information described above. For example, an instruction to pronounce a sound effect or the like may be used. Or you may instruct | indicate the process which should be performed to image | video itself. The processing as shown in FIG. 4 can be performed from music data, but can also be performed according to control information. Since all such control information corresponds to a command from the master side, it is also referred to as a “command”.

  The sequencer unit 108 starts and ends the reproduction of music data in accordance with the command output from the barcode detection unit 103. Although not shown in FIG. 6 to be described later, when inputting a video signal, music data designated by a barcode is selected.

  The time information output from the barcode detection unit 103 is processed by the synchronization processing unit 108a, and the processing timing of the event data in the song data is corrected based on the time information. By the correction, the music data is reproduced in synchronization with the reproduction of the optical disk by the DVD player 20. A command for instructing the start of the tone generation or the muting thereof is output to the assigner unit 110.

  The keyboard processing unit 109 is for responding to key pressing and key release on the keyboard 11. When the key is pressed, a command for instructing the start of tone generation corresponding to the key is generated, and when the key is released, a command for canceling the tone corresponding to the key is generated. The generated command is output to the assigner unit 110.

  The assigner unit 110 inputs a command from the sequencer unit 108 or the keyboard processing unit 109 and causes the sound source unit 111 to generate waveform data for generating a tone that can be generated. When the waveform data is input to the DAC & amplifier circuit 112, a musical sound is emitted from the speaker 12. In order to notify the user of the key to be pressed, the LED corresponding to the key is turned on.

  The barcode detection unit 103, the video processing unit 105, the sequencer unit 108, the keyboard processing unit 109, and the assigner unit 110 are actually used for, for example, a CPU, a storage device that stores a program executed by the CPU, and a work. This is realized by a storage device or the like. The CPU performs the following process by executing the program, and realizes the functions of these units. This will be specifically described with reference to FIGS.

  FIG. 5 is a flowchart of the bar code detection interrupt process. The interrupt process is started by an interrupt signal generated every time the video decoder 101 outputs one video. The interrupt process will be described in detail with reference to FIG.

  First, in step SA1, the video output from the video decoder 101 is analyzed. In the subsequent step SA2, it is determined whether or not a barcode is detected by the analysis. If a barcode is inserted in the video, the determination is yes, and in step SA3, the control information indicated by the barcode is stored in the buffer, and in step SA4 another image is deleted to delete the barcode on the video. Is processed by overwriting and output to the video encoder 106, and then the interruption process is terminated. If not, that is, if it is not necessary to perform video processing, the determination is no and the interrupt process is terminated here.

  FIG. 6 is a flowchart of sequencer timer interrupt processing. The interrupt process is a process for reproducing music data that is activated by an interrupt signal generated by a timer mounted on the CPU at a time interval corresponding to the minimum time resolution of time information, for example. Next, the interrupt process will be described in detail with reference to FIG.

  First, in step SB1, it is determined whether or not there is barcode control information in the buffer. If the command (control information) is stored in the buffer by the barcode detection interrupt process executed immediately before, the determination is YES and the process proceeds to step SB2. Otherwise, the determination is NO and the step Transition to SB7.

  In step SB2, the type of command is determined. If the command is a Start command, it is determined so and the process proceeds to step SB3, where a music start process for starting reproduction of music data stored in the ROM 107 is executed. Thereafter, the process proceeds to step SB7. By executing the music start process, a variable Δt, which will be described later, is initialized and 0 is substituted.

  If the command stored in the buffer is a Stop command, it is determined to that effect and the process proceeds to step SB4 to execute a song end process for ending reproduction of the song data. After the execution, the process proceeds to step SB7.

  If the command stored in the buffer is time information, it is determined so and the process proceeds to step SB5, where the time information is substituted into the variable AT. In the following step SB6, 0 is substituted for the variable Δt. After the substitution, the process proceeds to step SB7. The variables AT and Δt are prepared for realizing synchronized reproduction of music data in a form matched to time information.

  In step SB7, the buffer is cleared, and it is determined whether or not music data is being reproduced. If the reproduction is in progress, the determination is yes and the process proceeds to step SB8. Otherwise, the determination is no and the interrupt process is terminated here.

  In step SB8, the value of variable Δt is incremented and the value of variable T is decremented. In step SB9 that moves to thereafter, it is determined whether or not the value of the variable T is 0 or less. When it is time to process the next event data, the value of the variable T becomes 0 or less. From this, when it becomes the timing, determination will be YES and it will transfer to step SB10, and when that is not right, determination will be NO and will transfer to step SB13.

  In step SB10, event data to be processed next is processed. In the subsequent step SB11, the time information added to the next event data is read from the ROM 107 and substituted into the variable T. In step SB12 to be transferred thereafter, the value obtained by subtracting the added value of each value of the variable AT and the variable Δt from the previous value to the variable T (= T− (AT + Δt)), that is, the time previously substituted. A value indicating the time until the timing indicated by the information is substituted. Thereby, synchronized playback is realized in real time. The process proceeds to step SB13 after the substitution.

The time information is inserted into the image at a certain frequency. If the time information is not stored in the buffer for a predetermined time, there is a possibility that there is no video signal input itself. Therefore, in step SB13, it is determined whether or not the time information is stored in the buffer for a predetermined time or more. If the predetermined time or more has elapsed since the last storage, the determination is YES, and after performing the song end process in step SB14, the interrupt process ends. Otherwise, the determination is no and the interrupt process is terminated here.
<Second Embodiment>
In the first embodiment, the music data is synchronously reproduced in a manner that perfectly matches the time information indicated by the barcode inserted in the image. In such synchronous reproduction, the insertion interval of the bar code indicating the time information and the deviation of the time measured on the reproduction side of the music data during that time directly affect the reproduction of the music data. The shift may be relatively large if the insertion interval is a relatively long time. A relatively large shift may change the pronunciation interval between musical sounds to make it feel unnatural. For this reason, the second embodiment avoids such a reproduction problem.

  The playback apparatus according to the second embodiment is mounted on an electronic musical instrument, and the configuration of the electronic musical instrument is basically the same as that of the first embodiment. The operation is basically the same for the most part. For this reason, the reference numerals used in the first embodiment are used as they are, and only the parts different from the first embodiment will be described in detail.

  In the second embodiment, the deviation is monitored, and if the magnitude (the deviation is positive or negative, which is an absolute value here) is out of the allowable range, the music data is reproduced. The time measured for management is corrected so as to approach the time indicated by the time information. As a result, the reproduction of song data that makes it feel unnatural is avoided by avoiding a modification that causes the time taken for managing the reproduction of song data to change suddenly and relatively large. Yes.

  In the second embodiment, the synchronization processing unit 108a of the sequencer unit 108 shown in FIG. 2 performs time management as described above, and the sequencer unit 108 performs song data at the processing timing corrected by the time management. Process the event data inside. Thereby, the reproduction (synchronous reproduction) of music data that does not always feel unnatural is realized.

  As a process executed by the CPU in order to reproduce such song data, the sequencer timer interrupt process shown in FIG. 6 is partially different from the first embodiment. Therefore, the interrupt processing in the second embodiment will be described in detail with reference to FIGS. The interrupt processing is also realized by the CPU executing a program, as in the first embodiment.

  7 and 8, the same or basically the same processing steps as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment (FIG. 6). Here, a description will be given in a manner focusing on the processing steps with the reference numerals according to the first embodiment.

  In the second embodiment, the time for reproduction management is measured using the variable T. The initialization of the variable T, that is, the substitution of 0 is performed at the time of execution of the music start process at step SB3. After the execution, the buffer is cleared at step SC1, and the process proceeds to step SB7. When the music end process of step SB4 or the process of step SB5 is executed, the process proceeds to step SC1 after the execution.

  If YES is determined in step SB7, the process proceeds to step SC2, and the value of the reproduction management variable T is incremented. In the next step SC3, it is determined whether or not there is barcode time information. If the time information indicated by the barcode is stored in the buffer before being cleared in step SC1, the determination is yes and the process proceeds to step SC4. In that case, step SB5 is executed, and the time information is substituted into the variable AT. Otherwise, the determination is no and the process moves to step SB13. In the second embodiment, the process proceeds to step SC8 in FIG. 8 when NO is determined in step SB13.

  In step SC4, the time indicated by each value of the variables T and AT is compared, and the time difference (deviation) between them is calculated. In the following step SC5, it is determined whether or not the time difference is outside the allowable range. If the time difference is within the allowable range, the determination is no, and the value of the variable AT is cleared in step SC6, that is, 0 is substituted, and then the process proceeds to step SC8 in FIG. Otherwise, the determination is yes, and in step SC7, the value of the variable T is brought closer to the value of the variable AT for a predetermined time. In other words, if the value satisfies the relationship T> AT, the value of the variable T is made smaller. If T <AT, the variable T is changed so as to increase the value, and the process proceeds to step SC8.

  The allowable range is determined assuming a time interval that does not normally exist as a sound generation start interval of adjacent musical sounds, and specifically, is a time range corresponding to, for example, ± 64th notes. The predetermined time is, for example, a time corresponding to 1/2 (128th note). The allowable range and the predetermined time may be fixedly set in advance, but the setting information is prepared for each piece of music data, or is automatically set by analyzing the music data. Also good. The predetermined time may be set at any time so that, for example, the time until the next event data is to be processed is taken into account, the processing interval between the event data and the immediately preceding event data does not change significantly. good. By setting such an allowable range and a predetermined time, it is possible to avoid the reproduction of music data that feels unnatural.

  In step SC8 in FIG. 8, it is determined whether or not the time (absolute time) indicated by the value of the variable T matches the time indicated by the time information (event time) added to the next event data. For example, if the time indicated by the variable T is equal to or longer than the time indicated by the time information, the determination is YES, the next event data is processed in step SC9, and the event data is added to the next event data in step SC10. After reading the time information (event time) from the ROM 107, the interrupt process is terminated. Otherwise, the determination is no and the interrupt process is terminated here, assuming that the timing for processing the next event data has not come. The time information (event time) to be compared with the value of the variable T at step SC8 is read at step SC10.

  In the present embodiment (first and second embodiments), the video input from the master (equivalent to the DVD player 20 in the system configuration of FIG. 1) is displayed on the TV 30 which is another display device. However, this is because an arbitrary display device can be used for video display. However, such a display need not be performed. That is, you may enjoy the image | video which a master displays. Further, the content reproduced by the master side may be other than the image content. However, in order to reduce the load on the slave side, it is desirable to play content on the master side that is heavier than the slave side.

  A program for realizing the above-described playback apparatus may be distributed by being recorded on a recording medium such as a CD-ROM, a DVD, or a removable flash memory. Part or all of the program may be distributed via a communication network such as a public network. In such a case, the user can apply the present invention to a computer by acquiring the program and loading it into a computer that can be used as a playback device. Therefore, the recording medium may be accessible by a device that distributes the program.

It is a figure which shows the system configuration constructed | assembled for content reproduction | regeneration. It is a block diagram of the electronic musical instrument carrying the reproducing | regenerating apparatus by this Embodiment. It is a figure which shows the example of an image in which control information was inserted. It is a figure which shows the example of a process of the image in which control information was inserted. It is a flowchart of a barcode detection interruption process. It is a flowchart of the timer interruption process for sequencers. It is a flowchart of the timer interruption process for sequencers (2nd Embodiment). 6 is a flowchart of sequencer timer interrupt processing (second embodiment: continued).

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Electronic musical instrument 20 DVD player 30 Television 101 Video decoder 102 Video buffer 103 Barcode detection part 104 Video database 105 Image processing part 106 Video encoder 107 ROM
108 Sequencer part 110 Assigner part 111 Sound source part 112 DAC & amplifier circuit

Claims (5)

A method for playing back other types of content different from the image content by the second playback device in accordance with the playback of the image content by the first playback device,
On screen reproduced by the reproduction of said image content, and insert the image representing the control information related to the reproduction of recognizable the other content,
The screen to be reproduced by the reproduction of the image content by the first playback device is input to the second reproduction device,
To recognize images representing control information on said screen to said second reproduction device by the input of the screen to perform the playback of the other content on the basis of the recognition result,
A content reproduction method characterized by the above. In a playback device capable of playing back different types of content from image content,
An image input means for inputting a screen to be reproduced by the reproduction of the image content,
Present in the image input unit screen on which the inputted recognizing information recognition means the image representing the control information related to reproduction of the content,
Playback means for playing back the content;
Reproduction control means for causing the reproduction means to recognize the image representing the control information recognized by the information recognition means and controlling the reproduction based on the recognition result ;
A playback apparatus comprising: Image output means for outputting a screen input by the image input means;
The playback apparatus according to claim 2, further comprising: Wherein the image input means on the image representing the control information on the screen entered, further comprising image processing means for performing processing to place another image, and
The image output means outputs the image processed by the image processing means;
The reproducing apparatus according to claim 3. In a computer that can be used as a playback device capable of playing back different types of content from image content,
An image input function for inputting a screen to be reproduced by the reproduction of the image content, the present on-screen input by the image input function, the recognizing information image representing control information related to the reproduction of the content Recognition function,
A playback function for playing back the content;
A reproduction control function for causing the reproduction of the content by the reproduction function to recognize an image representing the control information recognized by the information recognition function and controlling based on the recognition result ;
A program to realize

Images