JP2010193062A - Program, apparatus and method of reproducing video and musical piece - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a program, apparatus and a method of reproducing video and musical pieces capable of giving a user a desire to purchase even the same musical piece two or more times and matching and reproducing the musical piece and the video image when switching and reproducing the musical piece and the video corresponding to an operation by the user. <P>SOLUTION: In a state that the video is reproduced on a display of a display part using first video data, the reproduction of second video data is started from a reproduction start position determined corresponding to the playback position of musical piece data on the basis of the detection of contact to a contact sensor, and the reproduction state of the video using the first video data is switched to the reproduction state of the video using the second video data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video and music playback program and a video and music playback device for playing back music and video, and more specifically, a display device including at least a display device such as a liquid crystal display and an audio output interface such as an earphone jack. The present invention relates to a video and music playback program, and a video and music playback device in which a video displayed on the display device is switched to another video when the displayed image is touched through an overlapped touch panel. Further, the present invention relates to a video and music playback program and a video and music playback device connected to a network such as the Internet and downloaded from an external server when switching the video to be displayed on the display device.

(Transition from music media sales to download sales)
Traditionally, music has gone from the record era to the cassette tape era, and further through the era of MD, CD, and laser disc. I was trying to play as needed.

  Also, portable players that are portable from the Walkman (registered trademark) of Sony Corporation, which prevailed all at once in the 1970s, to iPod (registered trademark) of Apple Inc. are changing. As factors for this, in recent years, flash memory that does not require a backup power supply has been reduced in price as a medium for storing music, and an environment in which everyone can use the Internet has been established.

  By the way, there are mainly the following two methods for storing music data in the memory in the iPod described above. In the first method, the user first purchases a music CD, and then uses the conversion software to convert the music of the music CD into data on the hard disk in the personal computer, and the iPod connected to the personal computer via USB. Is stored in the memory.

  In the second method, a personal computer and an external music download site for iPod are connected via a communication line such as the Internet, and music data to be purchased from a plurality of music data is operated by operating a keyboard on the personal computer. Select and download the purchased music data to the hard disk in the personal computer on the condition that electronic money or card payment is performed for the amount set in advance for each music data, and then connect the iPod and personal computer with the USB cable In the connected state, the downloaded music data is transferred to the iPod memory.

  Generally, software called iTunes is used to transfer music data to an iPod memory. Also, a portable player manufactured and sold by another manufacturer, which is also a player having a semiconductor memory inside and storing music data in the semiconductor memory. When using such a portable player, In many cases, dedicated software prepared in advance by each manufacturer is used.

(Game program using music)
When the inventor of the present invention created the present invention, the inventor investigated whether an application for switching video and music by touching the screen had already been filed.

  As a result, a game program having a content that the sound changes when a moving image is touched was hit (for example, see Patent Document 1).

  Also, in the past, only a uniform movie was displayed, but what was proposed to change this to a plurality of previously prepared movies when operated by the player during movie playback was hit (for example, , See Patent Document 2).

  The switching of the movie presented in the above-mentioned Patent Document 2 is prepared in advance in various games such as role playing games, simulation games, action games, etc., immediately after the game software is activated and when it becomes an important scene in the progress of the story. Although beautiful image data is continuously displayed on a display as a moving image (movie), application to this movie has been proposed. The paragraph number 0048 of Patent Document 2 states that “all of the plurality of movie data is generated as frame images at any time, and the display image is changed by selecting the frame image to be output to the display. It may be. "

JP 2005-198974 A JP 2005-034499 A

  In any of the first method and the second method described above, favorite music data can be stored in the memory of a portable player such as iPod. However, when the second method is used, In addition to downloading music data, video data that is reproduced in synchronization with the music data and title image data such as album jackets can often be downloaded. For this reason, if the display device capable of displaying video data as well as the music playback function is provided like iPod Touch, the video related to the music data can be displayed on the display device. Becomes only in the case of music data purchased by the second method.

  In addition, when selecting a movie or a music piece on a personal computer or a mobile phone, an image or icon representing the music piece such as the album jacket as described above is displayed and an image or an icon is selected. In view of the current situation where methods for selecting music and music are becoming popular, the second method described above is expected to become more common in the future.

  Furthermore, from the form in which physical recording media such as CDs are sold as before, through a communication line such as the Internet without using a physical recording medium, anytime, anywhere using a personal computer, The fact that the music has been purchased from the music download site has also become a factor, and it has become possible to individually purchase only the music data desired by the user. In the case of such a purchase form, as compared with the case of purchasing an LP or a non-single CD album in which a plurality of songs are stored as before, only the song data that is desired is purchased, so the purchase is inexpensive. On the other hand, from the point of view of the seller, it is not possible to sell songs that have been sold in a cross-selling manner, only the songs that have been advertised can be sold, and songs that are not advertised cannot be sold, in other words, must be advertised. A new problem has arisen that even if it is a good song, it cannot be sold easily.

  In addition, until now, the product of music content has been sold only with music, but recently, even memory cards such as micro SD cards are becoming cheaper and larger capacity, It has been sold as music content linked to video. In other words, not only listening to music with your ears, but also using your eyes to view the video, music content comes from a form that is sold only with music as in the past, and music and video come together. It has become a form that is sold.

  The first object of the present invention is to reproduce music contents that can give the user a desire to download the same music twice or three times and store it in memory. It is to provide a playback program for.

  A second object of the present invention is to provide a reproduction program that enables zapping viewing from the sensation of simply reproducing conventional music. Zapping as used herein refers to an act similar to the act of viewing while frequently switching channels with a remote controller when viewing TV, and refers to the act of switching and viewing music or video according to user operations.

  The reason why Patent Document 1 and Patent Document 2 are listed as conventional technologies in this specification is completely different from the present invention in terms of the solution problem and purpose, as described below. This is based on the idea that it is possible to prevent the judgment that there is a similar point and clarify the difference in advance by listing it as prior art.

  Patent Document 1 discloses a game program that changes the sound when a moving image is touched as described above. However, as described in the first purpose above, the music content is encouraged to be motivated to purchase twice or three times, or the zapping act while listening to the music as stated in the second purpose. There is no disclosure or suggestion of adding new enjoyment to the music viewing so that it can be performed. Patent Document 1 is an invention in a completely different technical field, in which sound changes when a moving image displayed on a game screen is touched.

  Therefore, there is no disclosure of a configuration related to a playback link that links playback of music and video. The playback link is information that is referred to in order to prevent the video from being played back from the video playback position that is mismatched with the music when the video is switched while the music is being played back. In this invention, since it is an important element in the music content to prevent the video from being played back from the video playback position that is mismatched with the music, the playback position that matches the music playback position when switching Thus, it is possible to control whether to reproduce another video from the reproduction link.

  In Patent Document 2, a movie (movie) of an important scene in the progress of the story is changed by the player's operation. However, due to its nature, a movie that follows the story of the game is prepared in advance. It can be estimated that something that deviates from the story is not prepared in advance. Furthermore, such a program is not made to purchase the same game software, but the music content which is the object of the present invention is purchased twice or three times, or is listed as the second purpose. As described above, there is no disclosure or suggestion of adding new enjoyment to the music viewing so that the zapping action can be performed while listening to the music.

  Also, at the time of movie playback in Patent Document 2, some music may be played back, but since there is no clear disclosure regarding this point, there is no description about the link between video and music. Therefore, when switching videos, there is no disclosure or suggestion of how to switch between the reproduced music and the video in order to avoid mismatching.

  The above first and second objects are the main objects of the present invention. Even if the music is the same, it is reluctant to make additional purchases by changing or changing the video. The present invention has an additional object in addition to the above-described first and second objects due to the following new problems that occur accompanying the following.

(Ingenuity of the sales form when selling the video accompanying the same music in a different form)
Traditionally, the reason for purchasing general music is that it was purchased when I listened to the theme song of a TV program anime or drama, or a song or song associated with a commercial when I heard it on a song program etc. Or maybe you like the singer and buy it whenever a new song is released.

  However, as the inventor of the present invention thinks, when trying to purchase many times by the variation of the video accompanying the same music, what is assumed as a new problem is that the music is as before It is how to appeal to users who like it because they like it, and various types of video that will accompany the song. For example, with an appeal such as “This video has changed compared to what was previously attached to the song,” the user has no awareness or motivation to make a new purchase. there is a possibility.

  In order to solve this problem, the present inventor separated the video and the music from the viewpoint of creating a trial period and separating the video and the music instead of creating the video itself in advance and selling it as a set with the music. Focused on the point of sale. The latter viewpoint is intended to solve the above problem from a financial viewpoint.

  For example, in the case of the former point of view, the usage state after purchase is limited, and an approach of charging a user who wants to use it even after the usage period has passed, and similarly the usage state is limited. We assumed an approach in which the advertiser is paid for the cost of the trial period. In either case, as in the past, the producer side prepares in advance a video that is considered to be matched to the music.

  As an approach different from this method, it is configured so that music and video possessed by the user can be appropriately linked, and various music and video can be linked and synchronized by user-like production. There is something to do. In other words, it was assumed that another approach would be to sell a song with a video link production program. Using the link production program with the video accompanying the music, the user links the desired video to the music so that the original music content can be produced.

  As shown in an embodiment to be described later, when the number of videos that can be linked to one piece of music is four, for example, the video is sequentially written in four working memories and output to the display. Then, the video to be written to the working memory is made by the user. And if you let the original music content compete, you can enjoy creating multiple original music content for one type of music, and you can purchase the music content many times I thought.

  The present invention achieves the above first and second objects, and is a completely new music video playback program and music made by technical creation in order to solve various problems caused by this by various approaches. Provided are a video playback device, a music video playback program and a music video playback device connected to a network.

  The present invention has been made in view of the above points, and the object of the present invention is to give a user a desire to purchase a plurality of times even for the same piece of music, and to the user's operation. Accordingly, an object of the present invention is to provide a music video playback program, a music video playback device, and a music video playback method capable of matching and playing back music and video when switching music and video.

The first invention is
(A) In a storage unit in which music content data including music data for reproducing one music and video data for reproducing video of a plurality of patterns associated with the one music is stored. A process of reading the stored music content data and reproducing a first video of the plurality of patterns of video in synchronization with the reproduction of the one music;
(B) When a contact with a contact sensor unit that detects contact from the outside is detected in a state in which a video is reproduced on the display unit using the first video data, A process of determining a reproduction start position of second video data different from the first video data in accordance with a reproduction position of the music data being reproduced, of the pattern of video,
(C) The process of (b) based on the detection of the contact with the contact sensor unit in the state where the video is reproduced on the display of the display unit using the first video data. The playback of the second video data is started from the playback start position determined by the above, and the playback state of the video using the first video data is switched to the playback state of the video using the second video data. And processing.

  According to this invention, when switching from the video using the first video data to the video using the second video data, the playback start position of the second video data corresponding to the playback position of the music data to be played back Therefore, it is possible to reduce the risk of mismatching between the reproduced video by the second video data and the music in progress. For example, in the case of the invention of this configuration, if the first video data is a video of a musician who is singing a song, the second video data is a video with different clothes and back decorations of the musician. And In this case, when switching from the video of the first video data to the video of the second video data, when the video playback position is always played back from the beginning, the voice of singing the music by the playback of the music data and the movement of the mouth However, according to the present invention, such a problem can be avoided.

  It should be noted that the video switching is executed by an interrupt process, and the first video is not reproduced on the display of the display unit, but the first video data processing is preferably continued. For example, processing such as reading of the first video data and expansion to the memory continues even when it is not reproduced on the display. A situation where the video and the music are not synchronized (synchronized) even when switching to return the reproduced video to the original first video data by continuing the processing in this way. Can be avoided.

  It should be noted that the storage unit described above is not limited as long as it can store music content data in both cases of download reproduction and streaming reproduction.

further,
Music reproduction position information indicating a plurality of reproduction positions from the start to the end of reproduction of the music;
A storage unit for storing playback link data including video playback position information indicating a plurality of playback positions from the start to the end of playback of the video;
The reproduction link data is preferably composed of video identification information for identifying a video and video playback position information of the video in association with at least one piece of music playback position information.

  Here, the “playback position” is not limited as long as it can identify the place where the playback is performed, and may be determined by the playback time or playback time, or by the number of frames of music data or video data. . Further, “at least one piece of music reproduction position information” may be used, and it is not necessary to define reproduction link data for all pieces of music reproduction position information.

Furthermore,
When playing back the first video, read the playback link data,
While reproducing the first video data in synchronization with the reproduction of the one music piece,
When playing back the second video, read the playback link data,
It is preferable that the second video data is reproduced in synchronization with the reproduction of the one music piece.

In addition to the first invention described above, the second invention
When the reproduction of the video using the second video data is completed, a process of switching to the video reproduction using the first video data is executed.

  That is, it has a function of automatically switching to a video using the first video data when the playback using the second video data is completed. According to this, for example, when a video sung by an artist is reproduced based on the first video data, touching the face portion of the artist causes the video to expand that face portion to be the second video. It may be configured such that when the reproduction of the second video is completed and the reproduction of the facial image is completed, the previous first video is automatically reproduced. In this case, the playback position for playing back the first video at the stage where the playback of the second video has ended may be configured to play back from a playback position determined in advance according to the progress of music playback.

  Also in this case, during the reproduction of the second video, the processing of the video data of the first video, the processing such as reading and expansion to the memory is continued, and the video to be displayed on the display is changed from the second video to the A changeover switch for switching to one video may be used. Further, during playback of the second video, processing such as processing, reading, and development of the video data of the first video may be interrupted. In such a case, since the processing of the video data of the first video is not necessary, there is an advantage that the burden on the CPU can be reduced.

further,
When a contact with the contact sensor unit is detected in a state in which a video is played on the display of the display unit using the first video data, the music playback position information at the next playback position is When it does not exist, it is preferable that the first video data is reproduced up to the reproduction position where the music reproduction position information exists.

Furthermore,
Using the second video data based on the fact that contact with the contact sensor unit is detected in a state where video is being reproduced on the display of the display unit using the first video data. When the video is played back, the reading of the first video data is interrupted,
Next, based on the detection of contact with the contact sensor unit in a state where the second video data is used to reproduce the video on the display of the display unit, the first video data is When the video is reproduced using the first video data read from the reproduction position where the music reproduction position information exists.

In addition to the first invention, the third invention
In a case where a preset time has elapsed since switching from video playback using the first video data to video playback using the second video data, the first video data is Switch to the playback of the video used.

  Furthermore, the control for causing the computer to execute the process of stopping the timing is performed on the condition that the preset key code is input during the period from the start of the timing to the preset time. Is preferred.

In addition to the first invention, the fourth invention
An authentic key code indicating that the music content is authentic and associated with the music content;
Processing for storing payment request information for requesting payment for continuously playing video using the second video data;
After the video using the second video data is reproduced, the process of reading the payment request information from the storage unit and displaying it on the display as a payment request information image;
Through the communication interface that communicates with the outside, on the condition that payment permission data indicating permission of payment is input by contact with the contact sensor unit after the payment request information is displayed, the payment permission data, A process of transmitting the user ID code and the music content identification code;
A process of receiving a key code generated based on the transmitted payment authorization data, the user's ID code, and the music content identification code;
A process of determining whether or not the received key code matches the genuine key code;
When it is determined that the received key code matches the genuine key code, the payment request information is deleted, and the timing process for determining whether or not a preset time has elapsed is canceled to obtain the second video data. And a process of continuously playing back the used video.

  The above-described genuine key code is a code indicating that the music content is authentic and associated with the music content. The payment request information is information for requesting payment for continuously playing video using the second video data.

  According to the above-described third and fourth inventions, the second video is restricted from being viewed in a timely manner, and a user can examine and purchase the video, while the music content is sold from the side. Then, it is possible to expect a substantial advertising effect while eliminating the economic demerits of being listened to in the future without paying the fee.

In addition to the first invention, the fifth invention
Storing the number of times of contact with the contact sensor unit in order to switch from video playback using the first video data to video playback using the second video data;
When the number of times of contact reaches a preset number associated with the music content, the reproduction processing of the music content is stopped.

  The number of times of contact with the contact sensor unit may be the same storage unit that stores music content data, or may be a physically separated storage unit. According to the fifth aspect of the invention, by restricting the trial listening by the number of times of contact from the time counting methods as in the third and fourth inventions described above, the internal clock to be auditioned is distorted and the trial listening time is illegally prolonged. Such a fear can be reduced.

In addition to the first invention, the sixth invention
Storing advertisement data for displaying an advertisement on the display, and the number of contact times corresponding to an advertisement set in accordance with the advertisement,
When playing back video using the first video data or playing back video using the second video data, the advertisement data is added and displayed on the display;
When the number of touches reaches the number of touches corresponding to the advertisement, the video reproduction is stopped.

  That is, when the first video or the second video stored in the storage unit is played back, advertisement data is added and displayed, and the video cannot be played when the number of contacts set for each advertisement data is reached. Control the computer to

  According to the sixth invention, an advertising fee is collected in advance from the advertiser for each advertising data, and the first video and the second video can be reproduced and switched by the number of contacts set in accordance with the advertising fee. For example, music content such as TV commercials can be distributed to users free of charge to increase the advertising effect, and playback of music content can be restricted according to advertising fees. There is a possibility that copyright protection can contribute.

In addition to the first invention, the seventh invention
Storing advertisement data for displaying an advertisement on the display and identification data for identifying the advertisement;
When playing back video using the first video data or playing back video using the second video data, the advertisement data is added and displayed on the display;
The accumulated display count of the displayed advertisement is stored in the storage unit in correspondence with the identification data of the advertisement displayed after being contacted with the contact sensor unit, and the accumulated display count for each advertisement is Transmit via a communication interface for communication.

  According to the seventh invention, by transmitting the cumulative display count for the identification number of the advertisement data to a data collection server or the like as an external advertisement performance data processing device via the communication interface, for example, the data In the collection server, the advertising fee can be calculated according to the cumulative display count, and the advertising fee according to the calculation result can be collected from the advertiser, and the actual number of advertisements can be reported to the advertiser as a number. Such as various advertisement effect display data such as can be used in an external advertisement performance data processing apparatus.

In addition to the first invention, the eighth invention
The video includes at least one object that can be displayed on the display,
Reproduction position data on the display when an image including at least one object is reproduced, and object corresponding image data in which an image is associated with the reproduction position data are stored in the storage unit,
When a video including the at least one object is being played, when the touch sensor unit is touched, contact with a region where one object of the at least one object is displayed In some cases, switching to playback of the video associated with the one object is performed.

In addition to the first invention, the ninth invention
If it is determined that the playback time of the predetermined video is longer than the playback time of the music, the video data corresponding to the playback time of the music in the video data of the predetermined video is sent to the contact sensor unit. Select according to the contact of
The selected video data is stored in the storage unit.

In addition to the first invention, the tenth invention
Video data for reproducing at least one of the plurality of patterns of video is stored in a video data server that is communicably connected via a communication interface that communicates with the outside.
When video data stored in the video data server is used in response to contact with the contact sensor unit, video data is downloaded from the video data server via the communication interface,
The downloaded video data is reproduced on the display.

  Even if it is the same song, it can give the user the desire to purchase multiple times, and when the song or video is switched and played according to the user's operation, the song and the video are matched and played. it can.

It is a block diagram which shows the structure and function of the video music reproduction apparatus. 1 is a front view showing an appearance of a video and music playback device 100. FIG. It is a flowchart which shows the subroutine for performing the video reproduction switching process 1. FIG. It is a time chart which shows the aspect by which a video is switched by the video reproduction switching process 1. FIG. It is a figure which shows a mode that it switches from reproduction | regeneration of the image | video A to reproduction | regeneration of the image | video B on LCD180 according to the operator touching the touch panel 130. FIG. It is a table which shows the content of the reproduction | regeneration link data used by the image | video reproduction | regeneration switching process 1. FIG. It is a flowchart which shows the subroutine for performing the video reproduction switching process 2. FIG. It is a time chart which shows the aspect by which a video is switched by the video reproduction switching process 2. FIG. It is a table which shows the content of the reproduction | regeneration link data used by the image | video reproduction | regeneration switching process 2. FIG. It is a flowchart which shows the subroutine for performing a video reproduction automatic switching process. It is a time chart which shows the aspect by which a video is switched by video reproduction | regeneration automatic switching processing. It is a table which shows the content of the reproduction | regeneration link data used by video reproduction | regeneration automatic switching processing. It is a flowchart which shows the subroutine which performs the process of 1st payment (settlement process 1) with respect to the image | video to reproduce | regenerate. 3 is a time chart showing a settlement process 1; It is a flowchart which shows the subroutine which performs the process of 2nd payment (settlement process 2) with respect to the image | video to reproduce | regenerate. It is a flowchart which shows the subroutine which performs the process of 3rd payment (settlement process 3) with respect to the image | video to reproduce | regenerate. It is a flowchart which shows the subroutine which performs an advertisement display / non-display process. It is a flowchart which shows the subroutine of an object display corresponding | compatible process. It is a table which shows the content of the object display position data used by an object display corresponding | compatible process. It is a figure which shows the example of the image | video displayed on LCD180 by performing an object display corresponding | compatible process. It is a flowchart which shows the subroutine of a reproduction time adjustment process. It is a time chart which shows the specific example of a reproduction time adjustment process. It is a flowchart which shows the subroutine of the video reproduction switching process 3. FIG. 10 is a time chart showing a specific example of video reproduction switching processing 4; 10 is a time chart showing a specific example of video reproduction switching processing 5; It is a figure which shows the example of the image | video displayed on LCD180 by performing an object display corresponding | compatible process. It is a table which shows the content of the reproduction | regeneration link data used by the image | video reproduction | regeneration switching process 5. FIG.

  Embodiments of the present invention will be described below with reference to the drawings. In the following, as an embodiment of the present invention, a portable video playback device and a music playback device (hereinafter referred to as a video music playback device) are shown. Terminal)), a mobile phone capable of video playback and music playback, and a computer capable of video playback and music playback. In any case, any method can be used as long as it can control the reproduction of the music using the music data and can control the reproduction of the video using the video data.

<<< Configuration of Video Music Reproducing Device 100 >>>
FIG. 1 is a block diagram showing the configuration and function of the video and music playback device 100.

<CPU 110 and storage device 120>
As shown in FIG. 1, the video and music playback device 100 includes a CPU 110 (central processing unit 110) and a storage device 120. The CPU 110 and the storage device 120 constitute the basic computer that controls the video and music playback device 100. The CPU 110 executes various system programs based on clock pulses generated by a system timer (not shown). For example, the system program includes a video playback program, a music video playback program, a video switching program, an object display program, a touch determination program, and the like. Various subroutines to be described later are configured by combining these system programs.

  The storage device 120 includes a ROM (Read Only Memory) (not shown) and a RAM (Random Access Memory) (not shown). The storage device 120 stores music data, video data, video control data, video control data, playback link data, and the like. The CPU 110 is electrically connected to the storage device 120 via a data bus (not shown) and an address bus (not shown). By doing in this way, CPU110 can read the various data memorize | stored in the memory | storage device 120, or can memorize | store predetermined data.

<Touch panel 130>
Further, the CPU 110 is electrically connected to the touch panel 130 via a data bus (not shown) and an address bus (not shown). The touch panel 130 includes a panel that can be operated by an operator, and an operation signal is supplied to the CPU 110 in accordance with the operation of the operator. CPU110 performs various processes according to the supplied operation signal. For example, as described later, the video to be reproduced is switched.

<Communication interface 140>
Furthermore, the CPU 110 is electrically connected to the communication interface 140 via a data bus (not shown) and an address bus (not shown). The communication interface 140 is for communicating with a storage device or server external to the video and music playback device 100. Via this communication interface 140, new music data and video data can be downloaded, and various information and data can be transmitted and received.

<Timer 150>
The CPU 110 is electrically connected to the timer 150 via a data bus (not shown) and an address bus (not shown). The timer 150 is used for measuring time, and in particular, for measuring the reproduction time of music.

<Audio interface 160>
The audio interface 160 is electrically connected to an audio generator such as a headphone (not shown) or a speaker (not shown) outside the video and music playback device 100. The audio interface 160 converts the digital signal supplied from the CPU 110 into an analog signal. The audio interface 160 amplifies the converted analog signal into a predetermined audio signal so as to emit audio from the connected headphones or speakers.

<Graphic controller 170>
The graphic controller 170 converts various data supplied from the CPU 110 into video signals to be supplied to a liquid crystal display 180 (hereinafter referred to as LCD 180). Work buffers 172A to 172D and a frame buffer 174 (not shown) are electrically connected to the graphic controller 170.

<Working buffers 172A-D>
The work buffer 172 is a buffer mainly used for intermediate processing for displaying an image, such as buffering for rendering or buffering materials such as texture used for rendering. In the present embodiment, as many work buffers 172 as the number of videos to be reproduced are provided. For example, when playing back four videos A, B, C, and D, the work buffer 172 includes a work buffer 172A for the video A, a work buffer 172B for the video B, and a work for the video C. It consists of four work buffers, a work buffer 172C and a work buffer 172D for video D. When the video data of video A is read from the storage device 120, the video data is expanded in the work buffer 172A each time it is read, and the contents of the work buffer 172A are updated. Similarly, when the video data of video B is read from the storage device 120, the video data is expanded in the work buffer 172B and the contents of the work buffer 172B are updated each time the video data is read. Similarly, for the video C and the video D, the contents of the work buffer 172C and the work buffer 172D are updated each time video data is read.

<Frame buffer 174 (frame memory)>
The frame buffer 174 (not shown) is electrically connected to the graphic controller 170. The frame buffer 174 temporarily stores raster data of an image displayed on the LCD 180 by the graphic controller 170. The raster data temporarily stored in the frame buffer 174 is read out in a predetermined order under the control of the graphic controller 170 and transmitted to the LCD 180 as a video signal.

  As described above, in this embodiment, the video and music playback device 100 includes the four work buffers 172A to 172D corresponding to the four video data. The graphic controller 170 generates data to be temporarily stored in the frame buffer 174 using the data temporarily stored in these four work buffers 172A to 172D. In this way, raster data to be finally displayed on the LCD 180 can be generated from the data developed in the work buffers 172A to 172D and temporarily stored in the frame buffer 174.

  For example, when one image is always displayed on the LCD 180, one of the four work buffers 172A to 172D is selected, and the data of the selected work buffer 172 is transmitted to the frame buffer 174. . Further, when two or more images are displayed on the LCD 180 in an overlapping manner, one corresponding to the image to be displayed is selected from the four work buffers 172A to 172D, and the data in the work buffer 172 is stored. The size to be displayed is adjusted, and the adjusted data is transmitted to the frame buffer 174.

<Playback link data>
As described above, reproduction link data is stored in the storage device 120. The reproduction link data is data that defines the correspondence between the reproduction position of music data and the reproduction position of video data. By reproducing the music while referring to the reproduction link data and reproducing the video, the video can be reproduced so as to match the music to be reproduced. Specific contents and configuration of the reproduction link data will be described later.

  In this specification, the reproduction position is information indicating a reproduction place in music data or video data, and may be information that can uniquely specify the reproduction place. For example, the number of frames of music data or video data may be used as the music or video playback time or playback time. Separately from the reproduction time, reproduction time, and number of frames, information that can be counted as the music or video is reproduced may be used as the reproduction position. In any case, any information can be used as long as it can be uniquely specified according to the reproduction of music or video.

  The playback link data may be provided in advance by a content provider such as music data or video data, or may be created by the user of the video music playback device 100. When using reproduction link data provided by a content provider, music data and video data are acquired by downloading or the like, and reproduction link data is also acquired. By doing in this way, a music and an image | video can be matched and reproduced | regenerated easily. In addition, when the user of the video and music playback device 100 can create the playback link data, the degree of freedom of the combination of the music and the video can be increased. In particular, the user of the video and music playback device 100 has recorded. Even when music data or video data taken by a user of the video music playback device 100 is used, music and video can be played back in alignment.

<Video playback / music playback>
In this specification, the reproduction of the video means that the video is displayed on the LCD 180 so that the video can be visually recognized, and the reproduction of the music means that a headphone (not shown) or a speaker (see FIG. (Not shown).

<<< Appearance of Video Music Reproducing Device 100 >>>
FIG. 2 is a front view showing the appearance of the video and music playback device 100.

  2A shows a mode in which the liquid crystal display 180 and the touch panel 130 are integrally formed, and FIG. 2B shows a mode in which the liquid crystal display 180 and the touch panel 130 are formed separately. In any aspect, according to the operation of the operator, the music desired by the operator can be reproduced and the video associated with the music to be reproduced can be reproduced.

  2 is a portable video and music playback device capable of communication. As described above, the video and music playback device according to the present invention is a portable terminal device (PDA (personal personal digital assistant)) that can communicate. An information terminal)), a mobile phone capable of video playback and music playback, or a computer capable of video playback and music playback.

<<< Processing in Video Music Reproducing Device 100 >>>
In the video and music playback device 100, video is played back in conformity with the music to be played back by playing back the video together with the music to be played back. Specifically, as will be described later, the music and the video can be matched and played by playing the video while referring to the playback link data stored in the storage device 120.

<< Video playback switching process 1 >>
A process of switching the video to be played back based on the operation of the operator is executed. In this video playback switching process 1, video data of a plurality of videos, for example, four videos A, B, C, and D are always read from the storage device 120 over the time during which the music is played back, The work buffers 172A to 172D are expanded and updated (see FIG. 4). In this way, even if the video playback is immediately switched based on the operation of the operator, the video can be smoothly played back, and even when the video playback is switched, the music and the video are consistent. Can be achieved.

  FIG. 3 is a flowchart showing a subroutine for executing the video reproduction switching process 1. FIG. 4 is a time chart showing how video is switched by video playback switching processing 1. FIG. 5 is a diagram illustrating a state in which playback of video A is switched to playback of video B in response to the operator touching touch panel 130. FIG. 6 is a table showing the contents of the playback link data used in the video playback switching process 1.

  As described above, FIG. 3 is a flowchart showing a subroutine for executing the video reproduction switching process 1. This subroutine is called and executed from a main routine (not shown) at a predetermined timing by various functions such as the arithmetic function of the CPU 110 of the video and music playback device 100. Note that the process of counting the playback time of music and the process of counting the number of music frames are independently performed by a separate subroutine (not shown).

  First, it is determined whether or not the operator has touched the touch panel 130 (step S11). When it is determined that the operator does not touch the touch panel 130 (NO), this subroutine is immediately terminated.

  On the other hand, when it is determined that the operator has touched the touch panel 130 (YES), the reproduction link data shown in FIG. 6 is read (step S13). Next, it is determined whether or not reproduction link data exists at a time later than the current music reproduction time (step S15). The music playback time is a time measured by starting the timer 150 on the basis of the time when the music playback is started. When it is determined that there is no reproduction link data at a time later than the current music reproduction time (NO), this subroutine is immediately terminated.

  On the other hand, when it is determined that the playback link data exists at a time later than the current music playback time (YES), it is determined whether it is the timing to switch the video playback (step S17). When it is determined that it is not the timing to switch the video reproduction (NO), the process of step S17 is repeated.

  On the other hand, when it is determined that it is time to switch the reproduction of the video (YES), the video identification number for identifying the video currently being reproduced is stored (step S19). For example, the video identification number of video A is “1”, the video identification number of video B is “2”, the video identification number of video C is “3”, and the video identification number of video D is “4”. When the video B is being reproduced, the video identification number “2” is stored. As described above, by storing the video identification number of the video currently being played back, it is possible to facilitate the process of switching to the playback of the original video. The process of switching to the original video playback will be described later.

  Next, using the video indicated by the playback link data read out in the process of step S13 and the playback position of the video, switching is performed so that the video is played back from the playback position (step S21), and this subroutine is finished. To do.

  The video playback switching process 1 shown in FIG. 3 will be described with reference to FIG. In the time chart shown in FIG. 4, time flows from top to bottom. In FIG. 4, the portion displayed as “touch” indicates that the operator touched the touch panel 130 and operated. For example, in response to the operator touching the touch panel 130 at the music playback time t13, the playback of the video A is switched to the playback of the video B (see FIG. 5). As described above, the time is the reproduction time of the music. As shown in FIG. 4, the reproduction of the music starts at the music reproduction time t0, and the reproduction of the music ends at the music reproduction time tend. As described above, the criteria for judging the playback time and time are performed using the playback time of the music. Note that the reproduction of the music starts at the music reproduction time t0 by using the first data of the music data, and ends at the music reproduction time tend by using the last data of the music data.

  As shown in FIG. 4, the music reproduction is started at the music reproduction time t0. As described above, in the video playback switching process 1, the video data of the four videos A, B, C, and D are always read from the storage device 120 over the time during which the music is played, and the four operations are performed. It is expanded and updated in the buffers 172A to 172D. For this reason, as shown in FIG. 4, at the music playback time t0, the playback of the music starts and the video data of the four videos A, B, C, and D are also read from the storage device 120 and the four tasks. Update in the buffers 172A to 172D is started. In the video playback switching process 1 shown in FIG. 4, reading from the storage device 120 and updating in the four work buffers 172A to 172D are started at the music playback time t0, but the music playback time t0. Then, among the four work buffers 172A to 172D, the work buffer 172A is selected, and the video A is reproduced on the LCD 180.

  As shown in FIG. 6A, the playback link data in the video playback switching process 1 is for 11 music playback times t0, t11, t12, t13, t14, t15, t16, t17, t18, t19, and t20. The relationship between the video to be switched and the playback position (video playback time described later) of the video to be switched is defined. As shown in FIG. 6A, the video and video playback positions are determined only for the five music playback times t0, t13, t15, t18, and t20. Not all videos and video playback positions are defined. As described above, the reproduction link data is for reproducing the video in conformity with the reproduction of the music. Therefore, for the music reproduction time that cannot be reproduced in conformity, the reproduction position of the video and the video. Cannot be determined. In the playback link data table shown in FIG. 6 (a), the music playback time at which the video and the video playback position cannot be determined is shown with hatching.

  In the playback link data table shown in FIG. 6A, video A is selected at music playback time t0, and playback position 0 of video A, that is, the beginning of video data of video A (video playback time t0). Played from. Note that the video A at the music reproduction time t0 is selected without being touched by the operator on the touch panel 130 and reproduced on the LCD 180.

  At music playback time t13, video B and video playback time t13 of video B are defined as playback link data. Video playback time t13 of video B means that video B is played from data at a position corresponding to video playback time t13 in video data of video B. That is, video B is played back from a playback position that has passed video playback time t13 from the beginning of video data of video B (video playback time t0).

  At the music playback time t15, the video D and the video playback time t15 of the video D are defined as playback link data. Video playback time t15 of video D means that video D is played back from data at a position corresponding to video playback time t15 in video data of video D. That is, video D is played back from a playback position that has passed video playback time t15 from the beginning of video data of video D (video playback time t0).

  Furthermore, at the music playback time t18, the video C and the video playback time t18 of the video C are defined as playback link data. Video playback time t18 of video C means that video C is played from data at a position corresponding to video playback time t18 in video data of video C. That is, video C is played back from a playback position that has passed video playback time t18 from the beginning of video data of video C (video playback time t0).

  Furthermore, at the music playback time t20, the video A and the video playback time t20 of the video A are defined as playback link data. Video playback time t20 of video A means that video A is played from data at a position corresponding to video playback time t20 in video data of video A. That is, video A is played back from a playback position that has passed video playback time t20 from the beginning of video data of video A (video playback time t0).

  As described above, the video playback times of the videos A to D indicate the playback times measured from the start of playback of these videos to the time and the playback positions of the video data of these videos. Similarly, the music playback time of the music also indicates the playback time measured from the time when the music playback is started until that time, and the playback position of the music data of the music. By using the video playback time of the video and the music playback time of the music, it is possible to form a correspondence (playback link data) that combines the playback position of the video data and the playback position of the music data.

  As described above, the playback link data is for playing back video in conformity with music playback. The matching between the reproduction of the music and the reproduction of the video is to match the content of the music being played back with the content of the video being played back. For example, an image of a content that a person sings or dances to match the rhythm of the music being reproduced. Also, depending on the volume of the music being played, the change in the volume, the frequency, or the change in frequency, the brightness, saturation, etc. may change, or the subject such as the character may change. Make things into images. In this way, what characterizes the contents of music, changes thereof, and timings of change are associated with those that characterize contents of videos, changes, and timings of changes. For example, the feature amount of the music, the change of the feature amount, the timing when the feature amount changes, the change of the feature amount or the feature amount of the video, or the timing when the feature amount changes are associated with each other. What is necessary is just to use what is used by the speech recognition technique, for example as the feature-value of a music. Moreover, what is necessary is just to use what is used by the image recognition technique, for example as the feature-value of an image | video. A plurality of types of such videos are stored in the storage device 120 and prepared, and the playback link data indicating the correspondence is stored in the storage device 120, so that the content of the music being played back can be played back. It is possible to match the content of the existing video.

  The playback link data table shown in FIG. 6 (a) described above uses the music playback time (music playback time) and video playback time (video playback time) as the music playback position and video playback position. It is determined. As described above, the playback position of the music and the playback position of the video only need to be information that can uniquely identify the location of the playback of the music or video, and not only the music playback time and video playback time but also the music data The number of frames (music frame number) or the number of video frames of video data (video frame number) may be used as the playback position. FIG. 6B is a table of playback link data determined using the number of music frames and the number of video frames (music frame number and video frame number) as playback positions.

  As shown in FIG. 6B, the playback link data in the video playback switching process 1 is for 11 music frame numbers f0, f11, f12, f13, f14, f15, f16, f17, f18, f19, and f20. The relationship between the video to be switched and the playback position (number of video frames (video frame number)) of the video to be switched is defined. Similar to the playback link data table shown in FIG. 6A, the video and the video playback position are defined only for the five music frame numbers f0, f13, f15, f18, and f20. The video and the playback position of the video are not determined for all 21 music frame numbers. Also in the playback link data table shown in FIG. 6B, the number of music frames for which the video and the video playback position cannot be determined are indicated by hatching.

  In the reproduction link data table shown in FIG. 6B, the video A is selected with the number of music frames f0, and the playback position 0 of the video A, that is, the beginning of the video data of the video A (the number of video frames f0). Played from. Similar to the reproduction link data table shown in FIG. 6A, the video A at the music frame number f0 is selected without being touched by the operator on the touch panel 130 and reproduced on the LCD 180.

  In the music frame number f13, the video B and the video frame number f13 of the video B are determined as reproduction link data. The video frame number f13 of the video B means that the video B is reproduced from data at a position corresponding to the video frame number f13 in the video data of the video B. In other words, the video B is played back from a playback position that has passed the video frame number f13 from the head (video frame number f0) of the video data of the video B.

  In the music frame number f15, the video D and the video frame number f15 of the video D are determined as reproduction link data. The video frame number f15 of the video D means that the video D is reproduced from data at a position corresponding to the video frame number f15 in the video data of the video D. That is, the video D is played back from a playback position where the video frame number f15 has elapsed from the head (video frame number f0) of the video data of the video D.

  Further, in the music frame number f18, the video C and the video frame number f18 of the video C are determined as reproduction link data. The video frame number f18 of the video C means that the video C is reproduced from data at a position corresponding to the video frame number f18 in the video data of the video C. That is, the video C is played back from the playback position where the video frame number f18 has elapsed from the head of the video data of the video C (video frame number f0).

  Furthermore, in the music frame number f20, the video A and the video frame number f20 of the video A are defined as reproduction link data. The video frame number f20 of the video A means that the video A is reproduced from data at a position corresponding to the video frame number f20 in the video data of the video A. That is, the video A is played back from a playback position where the video frame number f20 has elapsed from the head (video frame number f0) of the video data of the video A.

  As described above, the number of video frames of the videos A to D indicates the number of video frames (video frame number) counted by playing from the time when playback of these videos is started to that time. In addition, the playback position of the video data of these videos is also shown. Similarly, the music playback time indicates the number of music frames (music frame number) counted from the time when music playback was started to the time of playback, and the music data of the music. The playback position is also shown. By using the number of video frames and the number of music frames, it is possible to form a correspondence relationship (playback link data) that combines the playback position of video data and the playback position of music data.

  In the playback link data table shown in FIG. 6B, for the sake of simplicity, the number of music frames f13 corresponds to the number of video frames f13, and the number of music frames f15 corresponds to the number of video frames f15. The music frame number f18 corresponds to the video frame number f18, and the music frame number f20 corresponds to the video frame number f20 so that the music frame number and the video frame number are the same. Actually, the number of music frames and the number of video frames may differ depending on the data size, the data compression rate, and the like. However, in FIG. 6B, the same number of frames is used for convenience.

  In the reproduction link data shown in FIGS. 6A and 6B, five music reproduction times t0, t13, t15, t18, t20 (five music frame numbers f0, f13, f15, f18). , F20) only shows the video and video playback position defined, but it is preferable that the music playback time and the number of music frames with the video and video playback position are large. By increasing the number of music playback times and the number of music frames in which the video and the video playback position are determined, the timing for switching the video can be increased. Therefore, when the operator touches the touch panel 130, the video can be quickly displayed. The video can be switched, and the switching response can be improved.

  Using the reproduction link data shown in FIGS. 6A and 6B described above, the video can be switched and reproduced in accordance with the operation of the operator. In the following, video switching will be described according to the time chart of FIG. In the time chart shown in FIG. 4, a portion where the video is actually played is indicated by a thick solid line arrow extending downward, and although the video is not actually played back, the four videos A, The portions where the B, C, and D video data are expanded and updated in the four work buffers 172A to 172D are indicated by bold broken arrows extending downward. In addition, video switching locations are indicated by thin line arrows A1, B1, C1, and D1 extending in the horizontal direction. The state in which the music is being played is indicated by a thin solid line arrow extending downward.

  Further, in the time chart shown in FIG. 4, 21 music playback times t0 to t20 and 21 music frame numbers f0 to f20 are shown at equal intervals. Since it is determined in order to reproduce the video in conformity with the reproduction of the music, it is not necessary to determine it at equal intervals.

  Also, as shown in the time chart of FIG. 4, the video data of the four videos A, B, C, and D so that the playback times of the four videos A, B, C, and D coincide with the playback times of the music. Are preliminarily adjusted and stored in the storage device 120. In particular, it is preferable to edit and adjust the video data so that the reproduction of the music ends and the content of the video also ends. For example, by deleting at least one place in the middle of the video data and shortening the video playback time, the video playback time may coincide with the music playback time. In this way, by storing and playing back the adjusted video data, the content of the video ends as soon as the music playback ends, regardless of whether the video A, B, C or D ends. Thus, the video can be reproduced, and the content of the music and the content of the video can be matched and terminated.

  The video data as described above may be adjusted in advance by a video data provider (for example, a content provider) and provided to an operator (user) of the video music playback device 100. Alternatively, the operator of the video and music playback device 100 may use the video data provided by the video data provider.

  In the video playback switching process 1, the playback time of the four videos A, B, C, and D may be longer than the playback time of the music. In this case, the reproduction of the video is terminated together with the termination of the reproduction of the music. At the end of the music playback, the content of the music cannot be matched with the content of the video, but the video data provider or the operator of the video music playback device 100 saves time and effort for adjusting the video data. Therefore, it is possible to make the operator of the video and music playback device 100 enjoy the playback of music and video simply and easily.

  First, when the operator touches the touch panel 130 and operates before the music playback time t13 or the music frame number f13 (see FIG. 4), the music playback time t13 or the music frame number f13 is set. When it arrives (step S17 in FIG. 3), the video A is switched to the video B (see arrow A1 in FIG. 4), and the video B is played back from the video playback time t13 and the number of video frames f13 (step S21 in FIG. 3). . Note that if the operator does not touch and operate the touch panel 130 before reaching the music playback time t13 or the music frame number f13, the video A is continuously played back on the LCD 180.

  Next, when video B is being played, if the operator touches the touch panel 130 and operates before the music playback time t15 or the music frame number f15 (see FIG. 4). When the music playback time t15 or the music frame number f15 is reached (step S17 in FIG. 3), the video B is switched to the video D (see arrow B1 in FIG. 4), and the video playback time t15 or video frame number of the video D is changed. Playback is started from f15 (step S21 in FIG. 3). If the operator does not touch the touch panel 130 before the music playback time t15 or the music frame number f15, the video B is continuously played back on the LCD 180.

  Further, when the video D is being played and the operator touches the touch panel 130 and operates before the music playback time t18 or the music frame number f18 (see FIG. 4), When the music playback time t18 and the music frame number f18 are reached (step S17 in FIG. 3), the video D is switched to the video C (see arrow C1 in FIG. 4), and the video playback time t18 of the video C and the video frame number f18. (Step S21 in FIG. 3). If the operator does not touch the touch panel 130 and does not operate before reaching the music playback time t18 or the music frame number f18, the video D is continuously played back on the LCD 180.

  Furthermore, when the operator touches the touch panel 130 and operates at the time before the music playback time t20 or the music frame number f20 (see FIG. 4) while playing the video C, When the music playback time t20 and the music frame number f20 are reached (step S17 in FIG. 3), the video C is switched to the video A (see arrow D1 in FIG. 4), and the video playback time t20 and the number of video frames of the video A are displayed. Playback is started from f20 (step S21 in FIG. 3). If the operator does not touch the touch panel 130 before the music playback time t20 or the music frame number f20, the video C is continuously played back on the LCD 180.

  Finally, when the video A is being played, if the music playback time tend and the number of music frames fend are reached, the playback of the music is terminated by finishing playing back all of the music data. As described above, in the video playback switching process 1, the video data of the four videos A, B, C, and D are pre-adjusted so that the music playback ends and the video content also ends. preferable. For this reason, even if any of the four videos A, B, C, and D is being played back, the playback of the video can be ended so that the content of the video ends with the end of the playback of the music.

  If the playback time of the four videos A, B, C, and D is longer than the playback time of the music, the playback of the video may be ended together with the end of the playback of the music.

  As described above, when the music is being reproduced by executing the processing according to the subroutine shown in FIG. 3 with reference to the reproduction link data shown in FIGS. 6A and 6B described above. Even if the video to be reproduced is switched, the risk of mismatching between the music and the video can be reduced, and the consistency between the content of the music and the content of the video can be maintained and switched.

  The video switching process is executed by an interrupt process. That is, for example, when the video A is switched to the video B, the video A is not reproduced on the LCD 180, but the video data of the video A is read from the storage device 120 so that the video data is always continued. The contents are expanded in the work buffer 172A, and the contents of the work buffer 172A are updated. In this way, even when switching to return to playback of video A by continuing to read and expand the video data, the processing is not delayed, so the video and the music can be synchronized. it can. Therefore, it is possible to avoid a situation in which the video and the music are not synchronized due to the delay of the video data processing.

  As described above, in the video playback switching process 1, video data is always read from the storage device 120 and expanded in the work buffer 172 and updated regardless of the video being played back on the LCD 180. Showed things. However, as described above, when the video data is always read from the storage device 120 or the video data is expanded in the work buffer 172, other processing may be delayed by the processing. In addition, since this process is continuously performed, the amount of heat generated by the process may increase or the power consumption may increase. In particular, when the video and music playback device 100 is portable, since it is battery-powered, the time that can be driven by the battery is shortened, and as a result, when the number of times of charging is increased, the battery life is shortened. To do.

  For this reason, when the video is switched and the reproduction of the video is interrupted, the process of reading the video data of the interrupted video from the storage device 120 and the process of developing the video data in the work buffer 172 are also interrupted. In this way, the amount of heat generated can be suppressed, the power consumption can be reduced, the time that can be driven by the battery is lengthened, and the life of the battery can be extended.

  It should be noted that at the time before reaching the music playback time (t13, t15, t18, or t20) or the number of music frames (f13, f15, f18, or f20) defined in the playback link data, the operator uses the touch panel 130. When operated by touching, the video data of the video determined as the playback link data is read in advance from the storage device 120 and developed in the work buffer 172, so that the music playback time (t13, t15, When t18 or t20) or the number of music frames f13, f15, f18 or f20) is reached, the playback of the switched video can be started immediately, and the video can be switched smoothly and played back.

  As shown in FIG. 6, as the information about the video of the playback link data, only the name indicating the video is shown, such as “Video A” and “Video C”. Information necessary for reading video data, such as a file name indicating the video data and an address where the video data is stored.

  Further, although the reproduction link data shown in FIG. 6 shows only the information related to the video and the information of the reproduction position, other information may be included in the reproduction link data. For example, data for adjusting the volume when playing music, data for adjusting the hue, saturation, brightness, etc. of the video may be stored. In this way, it is possible to play music with a volume corresponding to the content of the video to be played back, or to adjust the visual balance of the video played back by switching.

  The above-described videos A, B, C, and D correspond to “videos of a plurality of patterns”.

<< Video playback switching process 2 >>
As described above, in the video playback switching process 1, video data whose video playback time matches the music playback time is used, or video data whose video playback time is longer than the music playback time is used. Showed things. On the other hand, video data whose video playback time is shorter than the music playback time can also be used. Even in such a case, it is necessary to reproduce the video in alignment with the reproduction of the music. A process using video data whose video playback time is shorter than the music playback time will be described below as video playback switching process 2.

  FIG. 7 is a flowchart showing a subroutine for executing the video reproduction switching process 2. FIG. 8 is a time chart showing how video is switched by video playback switching processing 2. FIG. 9 is a table showing the contents of the playback link data used in the video playback switching process 2.

  As described above, FIG. 7 is a flowchart showing a subroutine for executing the video reproduction switching process 2. This subroutine is called and executed from a main routine (not shown) at a predetermined timing by various functions such as the arithmetic function of the CPU 110 of the video and music playback device 100. Note that the process of counting the playback time of music and the process of counting the number of music frames are independently performed by a separate subroutine (not shown).

  First, it is determined whether or not the operator has touched the touch panel 130 (step S31). When it is determined that the operator has touched the touch panel 130 (YES), a process of switching the video to be played is executed according to the operation by touching the touch panel 130 (step S33), and this subroutine is finished. The process for switching the video to be played in response to the touch on the touch panel 130 may be performed in the same manner as the video playback switching process 1 described above. That is, the playback link data is read out, and the video to be played back is switched according to the video and the playback position (music playback time and the number of music frames) in the playback link data, and the video is played back.

  When it is determined that the operator is not in contact with the touch panel 130 in the determination process in step S31 described above (NO), the reproduction link data shown in FIG. 9 is read from the storage device 120 (step S35). Next, it is determined whether or not reproduction link data exists at a time later than the current music reproduction time (step S37). The music playback time is a time measured by starting the timer 150 on the basis of the time when the music playback is started. When it is determined that there is no reproduction link data at a time later than the current music reproduction time (NO), this subroutine is immediately terminated.

  On the other hand, when it is determined that the playback link data exists at a time later than the current music playback time (YES), it is determined whether it is the timing to end the playback of the video (step S39). When it is determined that it is not time to end the reproduction of the video (NO), it is determined whether or not the video data has already been developed in the work buffer 172 (step S41). If it is determined that the video data has already been developed in the work buffer 172 (YES), the process returns to step S39 described above. In this way, it is possible to wait until the timing to switch and play the video, and when the timing to switch and play the video is reached, the video data has already been developed in the work buffer 172. The video can be played immediately.

  On the other hand, when it is determined that the video data has not yet been developed in the work buffer 172 (NO), whether or not there is a video that can be searched for and switched to the playback link data read in the process of step S35. Is determined (step S43). If it is determined that there is no video that can be switched (NO), the process returns to step S39 described above.

  On the other hand, when it is determined in step S43 that there is a video that can be switched (YES), the video data of the video that can be switched is read from the storage device 120 and developed in the work buffer 172. Then, preparation for reproducing the video is made (step S45), and the process returns to step S39 described above. In this way, when it is time to switch and play the video, the video can be played back immediately.

  When it is determined in the above-described determination processing in step S39 that it is the timing to end the reproduction of the video (YES), a video that can be switched, that is, a video that is developed in the work buffer 172 and is in a standby state is displayed. Switching to playback (step S47), the present subroutine is terminated.

  After the processing in step S47 described above is executed, the video data of the switched video is continuously read from the storage device 120, developed in the work buffer 172, and the work buffer 172 is constantly updated. In this way, the switched video can be continuously played back.

  By executing the video playback switching process 2 shown in FIG. 7, when video data whose playback time is shorter than the playback time of the music is used, The video to be played can be switched and played back. This video switching is executed according to the playback link data, so even if the video playback time is shorter than the music playback time, the video is matched to the music being played back. You can switch and play.

  The video playback switching process 2 shown in FIG. 7 will be described with reference to FIG. Also in the time chart shown in FIG. 8, time flows from top to bottom. Also in the time chart shown in FIG. 8, the time is the reproduction time of the music. As shown in FIG. 8, the reproduction of the music starts at time t0, and the reproduction of the music ends at the music reproduction time tend. As described above, the criteria for judging the playback time and time are performed using the playback time of the music. In the example shown in FIG. 8, among the four videos A, B, C, and D, the videos whose playback time is shorter than the playback time of the music are the three videos B, C, and D. The playback time is the same as the music playback time.

  Similar to FIG. 4, the music reproduction starts at the music reproduction time t <b> 0. In the video playback switching process 2, video is played back using video data whose video playback time is shorter than the music playback time. Therefore, as shown in FIG. 8, at the music playback time t0, the playback of the music is started, and first, the video data of the video A is read from the storage device 120, developed into the work buffer 172A, and updated. At the music reproduction time t0, the work buffer 172A is selected, and the video A is reproduced on the LCD 180.

  As shown in FIG. 9A, the playback link data in the video playback switching process 2 is the video to be switched and the playback position (video) of the video to be switched for the five music playback times t0, t21, t22, t23 and t24. (Reproduction time). As described above, the video reproduction switching process 2 reproduces a video using video data shorter than the music reproduction time. For this reason, the playback link data in the video playback switching process 2 defines other video to be switched and played while maintaining consistency when playback of one video is finished. In the playback link data table shown in FIG. 9A, the video A is selected at the music playback time t0, and the playback position 0 of the video A, that is, the beginning of the video data of the video A (video playback time t0 or Video A is reproduced on the LCD 180 from the video frame f0).

  At the music playback time t21, the video B and the video playback time t0 of the video B are defined as playback link data. Video playback time t0 of video B means that video B is played from data at a position corresponding to video playback time t0 in video data of video B. That is, video B is played back from the beginning (video playback time t0) of video data of video B.

  At the music playback time t22, the video D and the video playback time t0 of the video D are defined as playback link data. Video playback time t0 of video D means that video D is played back from data at a position corresponding to video playback time t0 in video data of video D. That is, the video D is played back from the beginning (video playback time t0) of the video data of the video D.

  Furthermore, at the music playback time t23, the video C and the video playback time t0 of the video C are defined as playback link data. Video playback time t0 of video C means that video C is played from data at a position corresponding to video playback time t0 in video data of video C. That is, video C is played back from the beginning (video playback time t0) of video data of video C.

  Furthermore, at the music playback time t24, the video A and the video playback time t24 of the video A are defined as playback link data. The video playback time t24 of the video A means that the video A is played back from data at a position corresponding to the video playback time t24 in the video data of the video A. That is, the video A is played back from the playback position that has passed the re-video raw time t24 from the beginning (video playback time t0) of the video data of the video A.

  As described above, at the music playback time t21, the video B is played from the beginning of the video data of the video B (video playback time t0), and at the music playback time t22, the video D is the top of the video data of the video D ( The video C is played back from the beginning (video playback time t0) of the video data of the video C at the music playback time t23. In this video playback switching process 2, video is played back using video data whose video playback time is shorter than the music playback time, so that the video playback time (t21, t22) for video B, C, and D, respectively. Alternatively, playback may be started from the beginning of the video data from t23). On the other hand, since the video A has the same playback time as the music playback time, at the music playback time t24, playback that has passed the video playback time t24 from the beginning of the video data of the video A (video playback time t0). Video A is reproduced from the position.

  The playback link data in the video playback switching process 2 is also for playing back video in conformity with music playback. The matching between the reproduction of the music and the reproduction of the video is to match the content of the music being played back with the content of the video being played back. In particular, the playback link data in the video playback switching process 2 is used to match audio playback and video playback even when switching to another video when playback of one video ends. is there.

  The playback link data table shown in FIG. 9A also uses the music playback time (music playback time) and video playback time (video playback time) as the music playback position and video playback position. It is determined. As described above, the playback position of the music and the playback position of the video only need to be information that can uniquely identify the location of the playback of the music or video, and not only the music playback time and video playback time but also the music data The number of frames (music frame number) or the number of video frames of video data (video frame number) may be used as the playback position. FIG. 9B is a table of playback link data determined using the number of music frames and the number of video frames (music frame number and video frame number) as playback positions.

  As shown in FIG. 9B, the playback link data in the video playback switching process 1 is the video to be switched and the playback position of the video to be switched (video) for the five music frame numbers f0, f21, f22, f23 and f24. The relationship with the number of frames (video frame number) is determined. In the playback link data table shown in FIG. 9B, video A is selected with the number of music frames f0, and playback position 0 of video A, that is, the beginning of video data of video A (video frame number f0). Therefore, the video A is reproduced on the LCD 180.

  In the music frame number f21, the video B and the video frame number f0 of the video B are determined as reproduction link data. The video frame number f0 of the video B means that the video B is reproduced from data at a position corresponding to the video frame number f0 in the video data of the video B. That is, video B is played back from the beginning (video frame number f0) of video data of video B.

  In the music frame number f22, the video D and the video frame number f0 of the video D are determined as reproduction link data. The video frame number f0 of the video D means that the video D is reproduced from data at a position corresponding to the video frame number f0 in the video data of the video D. That is, the video D is reproduced from the head (video frame number f0) of the video data of the video D.

  Furthermore, in the music frame number f23, the video C and the video frame number f0 of the video C are determined as reproduction link data. The video frame number f0 of the video C means that the video C is reproduced from data at a position corresponding to the video frame number f0 in the video data of the video C. That is, the video C is reproduced from the head of the video data of the video C (the number of video frames f0).

  Furthermore, in the music frame number f24, the video A and the video frame number f24 of the video A are determined as reproduction link data. The video frame number f24 of the video A means that the video A is reproduced from data at a position corresponding to the video frame number f24 in the video data of the video A. That is, the video A is played back from the playback position where the video frame number f24 has elapsed from the head (video frame number f0) of the video data of the video A.

  As described above, at the music frame number f21, the video B is reproduced from the head (video frame number f0) of the video data of the video B, and at the music frame number f22, the video D is the head of the video data of the video D ( The video C is played from the head of the video data of the video C (the video frame number f0). As described above, in this video playback switching process 2, video is played back using video data whose playback time is shorter than the playback time of the music. Therefore, for the videos B, C, and D, the number of music frames ( From f21, f22 or f23), the video data may be reproduced from the beginning. On the other hand, since the video A has the same playback time as the music playback time, in the music frame number f24, the music frame number f24 has elapsed since the beginning of the video data of the video A (video frame number f0). Video A is reproduced from the position.

  In the reproduction link data table shown in FIG. 9B, for the sake of simplicity, the number of music frames and the number of video frames are the same so that the number of music frames f24 corresponds to the number of video frames f24. As shown. Actually, the number of music frames and the number of video frames often differ depending on the data size, data compression rate, and the like, but in FIG. 9B, the same number of frames is used for the sake of simplicity.

  Using the reproduction link data shown in FIG. 9A and FIG. 9B described above, it is possible to switch and reproduce the video corresponding to the operation of the operator. In the following, video switching will be described with reference to the time chart of FIG. In the time chart shown in FIG. 8, the portion where the video is actually played back is indicated by a thick solid line arrow extending downward, the video is not played back, and the four videos A, B , C and D video data are not read, and the portions where the contents of the four work buffers 172A to 172D are not updated are indicated by vertical thin broken lines. In addition, video switching points are indicated by thin line arrows A2, B2, C2, and D2 extending in the horizontal direction. The state in which the music is being played is indicated by a thin solid line arrow extending downward.

  First, the video A is selected at the music playback time t0 or the number of music frames f0, and the video A from the playback position 0 of the video A, that is, from the beginning of the video data of the video A (video playback time t0 or video frame f0). Is reproduced on the LCD 180.

  Thereafter, when the operator touches the touch panel 130 and operates before the music playback time t21 or the music frame number f21 (see “touch” in FIG. 8), the music playback time t21 or music When the frame number f21 is reached (step S33 in FIG. 7), the video A is switched to the video B (see arrow A2 in FIG. 8), and the video B is played from the video playback time t0 and the video frame number f0 (FIG. 7). Step S47). Note that if the operator does not operate by touching the touch panel 130 before reaching the music playback time t21 or the music frame number f21, the video A is continuously played back on the LCD 180.

  Next, when the video B is being played back, the video data of the video D is read out to switch to the video D before the music playback time t22 or the music frame number f22 is reached, and the work buffer is read. The image is expanded to 172D and prepared for reproduction (step S45 in FIG. 7). Thereafter, when the music playback time t22 or the music frame number f22 is reached (see FIG. 8), the video B is switched to the video D (see arrow B2 in FIG. 8), and from the video playback time t0 of the video D or the video frame number f0. Playback is performed (step S47 in FIG. 7).

  If the music playback time t22 or the music frame number f22 is reached while the video B is being played back, the video B is read to the end of the video data, and the playback of the video B ends. That is, the total reproduction time of the video data of the video B is configured to be t22-t21, or the total number of frames of the video B is configured to be f22-f21.

  Further, when the video D is being played back, the video data of the video C is read out and switched to the video C at the time before reaching the music playback time t23 or the music frame number f23, and the work buffer 172C is used. To prepare for reproduction (step S45 in FIG. 7). Thereafter, when the music playback time t23 and the music frame number f23 are reached (see FIG. 8), the video D is switched to the video C (see arrow C2 in FIG. 8), and the video playback time t0 of the video C and the video frame number f0. (Step S47 in FIG. 7).

  If the music playback time t23 or the music frame number f23 is reached while the video D is being played back, the video data of the video D is read to the end and the playback of the video D ends. That is, the total reproduction time of the video data of the video D is configured to be t23-t22, or the total number of frames of the video B is configured to be f23-f22.

  Furthermore, when the video C is being played back, the video data of the video A is read out to switch to the video A at a time before reaching the music playback time t24 or the music frame number f24, and the work buffer The image is expanded to 172A and prepared for reproduction (step S45 in FIG. 7). Thereafter, when the music playback time t24 or the music frame number f24 is reached (see FIG. 8), the video C is switched to the video A (see arrow D2 in FIG. 8), and the video playback time t24 or the video frame number f24 of the video A is changed. (Step S47 in FIG. 7).

  If the music playback time t24 or the number of music frames f24 is reached while the video C is being played back, the video C is read to the end of the video data, and the playback of the video C ends. That is, the total reproduction time of the video data of the video C is set to t24-t23, or the total number of frames of the video B is set to f24-f23.

  Finally, when the video A is being played, if the music playback time tend and the number of music frames fend are reached, the playback of the music is terminated by finishing playing back all of the music data.

  As described above, when the reproduction of the music is completed by executing the processing according to the subroutine shown in FIG. 7 with reference to the reproduction link data shown in FIGS. 9A and 9B described above. Even if the video to be reproduced is switched, the risk of mismatching between the music and the video can be reduced, and the consistency between the content of the music and the content of the video can be maintained and switched.

  In addition, when switching to playback of another video after the end of playback of one video, video data of the other video is read from the storage device 120 at a time before switching to another video, and the work buffer 172, the playback immediately after switching can be made smooth and playback can be started immediately.

  Furthermore, when the operator touches the touch panel 130 and operates before the music playback time t21 or the music frame number f21, when the music playback time t21 or the music frame number f21 is reached, Switch playback from video A to video B. Thus, since it is not necessary to reproduce the image A after switching the reproduction to the image B, it is not necessary to read out the image data of the image A from the storage device 120 and develop it in the work buffer 172A. It is preferable to interrupt reading and development of video data. By doing so, it is possible to suppress the amount of heat generation required for processing, reduce power consumption, extend the battery drive time, and extend the life of the battery. Even if the reading and development of the video data are interrupted in this way, as described above, the video data of the video A is read before the music playback time t24 and the number of music frames f24, and the work data is used. Since it is expanded in the buffer 172A and prepared for reproduction, reproduction immediately after switching can be made smooth. In addition, since the processing is not delayed, the video and the music can be synchronized. Therefore, it is possible to avoid a situation in which the video and the music are not synchronized due to the delay of the video data processing.

<< Video playback automatic switching process >>
In the video playback switching process 2 described above, when the playback of the three videos B, C, and D whose playback time is shorter than the playback time of the music ends, the video is switched one after another with reference to the playback link data. Indicated. As described above, the playback time of the video A is the same as the playback time of the music. Thus, in the video playback switching process 2, in order to switch the video using the four videos A, B, C and D, the playback link data defined for the four videos A, B, C and D are required, It is necessary to use a storage device having a storage capacity capable of storing video data of four videos A, B, C, and D, or to determine reproduction link data for the four videos A, B, C, and D in advance. was there.

  FIG. 10 is a flowchart showing a subroutine for executing the video reproduction automatic switching process. FIG. 11 is a time chart showing a mode in which video is switched by video playback automatic switching processing.

  As described above, FIG. 10 is a flowchart showing a subroutine for executing the video reproduction automatic switching process. This subroutine is called and executed from a main routine (not shown) at a predetermined timing by various functions such as the arithmetic function of the CPU 110 of the video and music playback device 100. Note that the process of measuring the playback time of music and the process of counting music frames are independently performed by a separate subroutine (not shown).

  First, it is determined whether or not the video reproduction has ended (step S51). When it is determined that the video reproduction has not ended (NO), this subroutine is immediately ended.

  On the other hand, when it is determined that the reproduction of the video has ended (YES), it is determined whether there is a video identification number for identifying the video that was originally reproduced (step S53). The identification number is stored by the process of step S19 in FIG. For example, when the originally reproduced video is video A, “1” is stored as the video identification number. When the originally reproduced video is video B, “2” is stored as the video identification number. When the originally reproduced video is video C, “3” is stored as the video identification number. When the originally reproduced video is video D, “4” is stored as the video identification number. Note that when the video is played for the first time after the music is played back, the video that was originally played back does not exist, so there is no video identification number.

  When it is determined in the determination process in step S53 that there is no identification number for identifying the originally played video (NO), that is, when the video is played for the first time after playing the music, Exit the subroutine.

  On the other hand, when it is determined in the determination process of step S53 that there is an identification number for identifying the video that was originally reproduced (YES), that is, when at least a plurality of videos are reproduced after the music is started to be reproduced, The video that has been reproduced is switched to be reproduced (step S55), and this subroutine is terminated.

  The video playback automatic switching process shown in FIG. 10 will be described with reference to FIG. Also in the time chart shown in FIG. 11, time flows from top to bottom. In FIG. 11, a portion where “touch” is displayed indicates that the operator touched the touch panel 130 and operated. For example, in response to the operator touching the touch panel 130 at time t31 or time t33, the playback of the video A is switched to the playback of the video B (see FIG. 11). Also in FIG. 11, the time is the reproduction time of the music. As shown in FIG. 11, the reproduction of the music starts at time t0, and the reproduction of the music ends at the music reproduction time tend. As described above, the criteria for judging the playback time and time are performed using the playback time of the music.

  The time chart shown in FIG. 11 shows a case in which video A or video B can be played along with music playback. The playback time of video A is the same as the playback time of music, while the playback time of video B is shorter than the playback time of music.

  As shown in FIG. 12 (a), the playback link data in the video playback automatic switching process includes the video to be switched and the playback position (video playback time) of the video to be switched for the three music playback times t0, t31 and t33. A relationship has been established. In the playback link data table shown in FIG. 12A, video A is selected at music playback time t0, and playback position 0 of video A, that is, the beginning of video data of video A (video playback time t0). Played from. Note that the video A at the music reproduction time t0 is selected without being touched by the operator on the touch panel 130 and reproduced on the LCD 180.

  At music playback time t31, video B and video playback time t0 of video B are defined as playback link data. At the music playback time t33, the video B and the video playback time t0 of the video B are defined as playback link data. The video playback time t0 of the video B described above means that the video B is played back from data at a position corresponding to the video playback time t0 in the video data of the video B. That is, video B is played back from the beginning (video playback time t0) of video data of video B.

  The playback link data used in the video playback automatic switching process is also for playing back video in conformity with the playback of music. The matching between the reproduction of the music and the reproduction of the video is to match the content of the music being played back with the content of the video being played back. For example, an image of a content that a person sings or dances to match the rhythm of the music being reproduced.

  The table of the playback link data shown in FIG. 12A described above uses the music playback time (music playback time) and video playback time (video playback time) as the music playback position and video playback position. It is determined. As described above, the playback position of the music and the playback position of the video only need to be information that can uniquely identify the location of the playback of the music or video, and not only the music playback time and video playback time but also the music data The number of frames (music frame number) or the number of video frames of video data (video frame number) may be used as the playback position. FIG. 12B is a table of playback link data determined using the number of music frames and the number of video frames (music frame number and video frame number) as playback positions.

  As shown in FIG. 12B, the playback link data used in the video playback automatic switching process is the video to be switched for the three music frame numbers f0, f31 and f33, and the playback position of the video to be switched (number of video frames (video Frame number)).

  In the playback link data table shown in FIG. 12B, video A is selected with the number of music frames f0, and playback position 0 of video A, that is, the beginning of video data of video A (video frame number f0). Played from. Similar to the reproduction link data table shown in FIG. 12A, the video A at the music frame number f0 is selected without being touched by the operator on the touch panel 130 and reproduced on the LCD 180.

  In the music frame number f31, the video B and the video frame number f0 of the video B are determined as reproduction link data. Further, in the music frame number f33, the video B and the video frame number f0 of the video B are determined as reproduction link data. The video frame number f0 of the video B means that the video B is reproduced from data at a position corresponding to the video frame number f0 in the video data of the video B. That is, video B is played back from the beginning (video frame number f0) of video data of video B.

  Using the playback link data shown in FIGS. 12A and 12B described above, the video can be switched and played in response to the operation of the operator, and when the playback ends, the original video is automatically restored. Switch to playback. In the following, video switching will be described according to the time chart of FIG. In the time chart shown in FIG. 11, a portion where the video is actually reproduced is indicated by a thick solid arrow extending downward, and a portion where the video is switched is indicated by a thin line arrow A3 extending horizontally. , B3, A4 and B4. The state in which the music is being played is indicated by a thin solid line arrow extending downward.

  First, when the operator touches the touch panel 130 and operates before the music playback time t31 or the music frame number f31 (see FIG. 11), the music playback time t31 or the music frame number f31 is reached. When it arrives, the video A is switched to the video B (see arrow A3 in FIG. 11), and the video B is played back from the video playback time t0 and the number of video frames f0. If the operator does not touch the touch panel 130 before the music playback time t31 or the music frame number f31, the video A is continuously played back on the LCD 180.

  When the music playback time t32 or the music frame number f32 is reached after switching from the video A to the video B (see FIG. 11), the video B is switched to the video A (returning from the video B to the video A) (FIG. 11). The video A is reproduced from the video reproduction time t32 and the number of video frames f32 (step S55 in FIG. 10).

  Furthermore, when the operator touches the touch panel 130 and operates before the music playback time t33 or the music frame number f33 after returning from the video B to the playback of the video A (see FIG. 11). ) When the music playback time t33 or the music frame number f33 is reached, the video A is switched again from the video B (see arrow B3 in FIG. 11), and the video B is played back from the video playback time t0 or the video frame number f0. . Note that if the operator does not operate by touching the touch panel 130 before reaching the music playback time t33 or the music frame number f33, the video A is continuously played back on the LCD 180.

  When the music playback time t34 or the music frame number f34 is reached after switching from the video A to the video B and playing (see FIG. 11), the video B is switched again to the video A (returning from the video B to the video A) ( The video A is reproduced from the video reproduction time t34 and the number of video frames f34 (see step S55 in FIG. 10).

  If the music playback time t32 or the music frame number f32 is reached while the video B is being played, or if the music playback time t34 or the music frame number f34 is reached, the end of the video data of the video B is displayed. Reading to the tail is finished, and the reproduction of the video B is finished. That is, the total reproduction time of video data of video B is t32-t31 (= t34-t33), or the total number of frames of video B is f32-f31 (= f34-f33). It is configured.

  Finally, when the video A is being played, if the music playback time tend and the number of music frames fend are reached, the playback of the music is terminated by finishing playing back all of the music data.

  In the example shown in the time chart of FIG. 11 described above, the case where the same video B is switched and reproduced at both time t31 and time t33 is shown. That is, while the case where the video B is played twice while the music is being played is shown, it may be switched to a different video at time t31 and time t33. For example, it may be switched to video B at time t31 and switched to video C at time t33.

  In this way, when the video being played ends, it is immediately switched to the video that was originally played and played. By doing in this way, it is possible to switch the video to be reproduced with reference to the simply configured reproduction link data, thereby facilitating the processing.

  In the determination process of step S51 of FIG. 10 described above, it is determined whether or not the video playback has ended, and the video playback is switched. However, a predetermined time has elapsed since the start of the playback of one video. Sometimes, it may be switched to playback of another video. In this way, for example, when one video is a product for sale, if you want to continuously play back the entire video, make a payment for the one video, Let the operator purchase the video. In this way, the operator can be motivated to purchase one video, and only a part of the one video can be played to an operator who has not made a payment. By not doing so, it is possible to protect the one video by preventing the entire one video that is a sale product from being reproduced.

<< Payment processing 1 >>

  FIG. 13 is a flowchart showing a subroutine for performing a settlement process for the music to be reproduced. FIG. 14 shows a timing chart of the settlement process for the music to be reproduced. Also in the video and music playback device 100 in which this settlement processing is executed, the playback link data described above is stored in the storage device 120, and when playing back a music by referring to the playback link data, Video can be played back to match playback.

  As described above, FIG. 13 is a flowchart showing a subroutine for executing the settlement process 1. This subroutine is called and executed from a main routine (not shown) at a predetermined timing by various functions such as the arithmetic function of the CPU 110 of the video and music playback device 100.

  In the subroutine of FIG. 13, first, it is determined whether or not the payment request information is stored in the storage device 120 (step S61). The payment request information is information for requesting the operator to make a payment. If payment is not made, the payment request information is stored in the storage device 120 and if payment is made, the storage device is stored. It is deleted from 120 and does not exist.

  When it is determined in the determination process of step S61 that the payment request information is erased and does not exist in the storage device 120 (NO), the subroutine is immediately terminated. This means that the operator has already settled the music and the settlement request information is deleted and does not exist. In this way, when payment is made and the payment request information does not exist, the music piece for which payment has been made can be played back by the video and music playback device 100 without any restriction on the playback time. it can. On the other hand, in a state where payment is not made and payment request information exists, the time that can be played back by the video music playback device 100 is limited for the target music. That is, when the reproducible time is exceeded, the music is not output.

  When it is determined that the payment request information is stored in the storage device 120 and exists in the determination process in step S61 described above (YES), a predetermined time has elapsed since the reproduction of the music started. It is determined whether or not it has been done (step S63). Timing for determining whether or not a predetermined time has elapsed since the reproduction of the music is started is performed by starting the timer 150 described above simultaneously with the reproduction of the music and counting the timer value of the timer 150. Can do.

  If it is determined in step S63 that the predetermined time has not elapsed since the music reproduction was started (NO), the subroutine is immediately terminated. On the other hand, when it is determined that a predetermined time has elapsed since the reproduction of the music has started (YES), the payment request information stored in the storage device 120 is read and the payment request information is displayed on the LCD 180 ( Step S65). In this way, by displaying the payment request information, the playback of the video that is being performed along with the playback of the music will be hindered to some extent, and by making the operator visually recognize the presence of the payment request information, the payment is still made. It is possible to make the operator clearly recognize that the process is not over.

  Next, it is determined whether or not the operator has touched the touch panel 130 (step S67). This touch on the touch panel 130 means that the operator has accepted payment. When it is determined in step S67 that the operator has not touched the touch panel 130 (NO), that is, when the operator does not make a payment, the subroutine is immediately terminated.

  On the other hand, when it is determined that the operator has touched the touch panel 130 (YES), that is, when the operator recognizes that payment is to be made, the communication interface 140 receives the payment permission data, the user ID, and the music identification code. To the server (not shown) (step S69). Here, the settlement permission data is information indicating that the operator has accepted settlement, and is information generated when it is determined in step S67 that the operator has touched the touch panel 130. is there. Further, the user ID is information for identifying the operator, and is used for accurately performing settlement without omission or duplication. This user ID may be uniquely assigned to the operator or may be uniquely assigned to the video and music playback device 100. When assigned to the video and music playback device 100, the operator is indirectly identified through the video and music playback device 100. In any case, the user ID is information for identifying the operator in order to make an accurate settlement. Furthermore, the music identification code is information for identifying the music that the operator intends to settle.

  The server to which the payment permission data, the user ID, and the music identification code are transmitted by the process of step S69 described above generates a key code based on these pieces of information. This key code is information indicating that the operator has settled for the music that he wishes to settle. In particular, the key code is preferably generated using a user ID or a music identification code and used as information unique to the user ID or the music identification code in order to prevent unauthorized use. By generating such a key code, even if the same key code is used, different video and music playback devices 100 can prevent the same music from being played back and prevent the music from being used illegally. .

  After executing the process of step S69 described above, the key code generated by the server is received (step S71), and it is determined whether or not the received key code matches the authentic key code (step S73). Here, the genuine key code is a key code generated by the video and music playback device 100. The genuine key code is generated by the video and music reproduction device 100 by the same algorithm as the key code generation by the server described above. Therefore, in principle, the key code generated by the server matches the genuine key code generated by the video and music playback device 100. However, there is a high possibility that the key code obtained or generated by some kind of fraud etc. will not match the genuine key code. In this way, it is possible to prevent unauthorized use of music by determining whether or not the key code generated by the server matches the authentic key code generated by the video and music playback device 100.

  When it is determined that the received key code does not match the authentic key code in the determination process in step S73 described above (NO), that is, when it is determined that the received key code is invalid, the reproduction is performed. The reproduction of the existing music is stopped (step S75), and this subroutine is terminated. On the other hand, when it is determined that the received key code matches the authentic key code (YES), that is, when it is determined that the received key code is authentic, the payment request information stored in the storage device 120 is stored. Is deleted (step S77), and this subroutine is terminated. By deleting the payment request information through the processing in step S77, the restriction on the reproduction time of the music can be released.

  FIGS. 14A and 14B are time charts showing the settlement process. In the example of the time chart shown in FIG. 14, the music is the object of settlement, and the video is not the object of settlement.

  In FIG. 14A, first, the reproduction of the music is started at the music reproduction time t0, and the reproduction of the video A is also started. Next, when the operator touches the touch panel 130 and operates before the music playback time t41 (see FIG. 14), when the music playback time t41 is reached, the video A to the video B (See arrow A5 in FIG. 14), and the music B is reproduced from the music reproduction time t0. If the operator does not touch the touch panel 130 and does not operate before the music playback time t41, the video A is continuously played back.

  Thereafter, when the music playback time reaches the music playback time t42, it is determined that a predetermined time has elapsed since the music playback started (step S63 in FIG. 13), and the settlement request information is displayed on the LCD 180. (Step S65 in FIG. 13), accordingly, when the operator touches the touch panel 130 (Step S67 in FIG. 13), the payment request information is deleted by the payment (Step S77 in FIG. 13). The restriction on the music playback time is released, and the music playback is continued after the music playback time t42 (see the solid arrow in FIG. 14A).

  On the other hand, in FIG. 14B, the same applies until the music playback time reaches the music playback time t0 to t42. When the music playback time reaches the music playback time t42, it is determined that a predetermined time has elapsed since the music playback started (step S63 in FIG. 13), and settlement request information is displayed on the LCD 180 ( Step S65 in FIG. When the payment request information is displayed on the LCD 180 but the operator does not touch the touch panel 130 (step S67 in FIG. 13), the payment request information exists without being settled (FIG. 13). Step S77). When the payment request information exists, the music reproduction is interrupted after the music reproduction time t42 (see step S75 in FIG. 13 and the thin broken line in FIG. 14B).

  As described above, the contact of the operator with the touch panel 130 at the music playback time t41 (before the music playback time t41) is an operation for switching from the video A to the video B. On the other hand, the contact of the operator with the touch panel 130 at the music playback time t42 is a settlement operation for enabling the music to be played back continuously (step S67 in FIG. 13). As described above, various operations can be performed only by touching the touch panel 130, and the operation can be prevented from becoming complicated. In addition,

  By executing the processing as described above, even if the music playback time is limited, at least a part of the music can be played, so the operator determines whether the music is truly the desired music. An opportunity that can be provided can be provided, and the operator can be prevented from spending unnecessarily.

  Also, if the music playback time is limited and payment is not made, the music provider will stop playing the music, so that the music provider can play the music without paying it. It is possible to expect a substantial advertising effect while eliminating it.

  Furthermore, in the above-described example, the case where the key code transmitted from the server is compared with the authentic key code has been shown. However, the release information such as the password is transmitted from the server, and the operator receives the release information from the touch panel 130. By inputting, the restriction on the reproduction time of the music may be released. In this case, when an operation for settlement processing for music is performed on the touch panel 130, release information such as a password is transmitted from the server, and the operator operates the touch panel 130 to transmit from the server. Enter the released release information. When the release information is input from the touch panel 130, the video and music playback device 100 deletes the payment request information to release the restriction on the playback time of the music.

  Moreover, although FIG. 13 and FIG. 14 described above show a case where only the music is charged and set as a payment target, videos such as video A and video B may be set as payment targets. In this case, if the settlement is not completed even after a predetermined reproduction time has elapsed, the reproduction on the LCD 180 is stopped. By doing in this way, not only the music provider but also the profit of the video provider can be protected.

<< Settlement process 2 >>
FIG. 15 is a flowchart showing a subroutine for performing the second settlement process (settlement process 2) on the video to be reproduced. Note that in the second settlement process, video is the subject of settlement. This subroutine is also called and executed from a main routine (not shown) at a predetermined timing by various functions such as the arithmetic function of the CPU 110 of the video and music playback device 100. Also in the video and music playback device 100 in which the settlement process 2 is executed, the playback link data described above is stored in the storage device 120, and when playing back a music by referring to the playback link data, Video can be played back to match playback.

  First, it is determined whether or not the payment request information is stored in the storage device 120 (step S81). This payment request information is information for requesting the operator to make a payment, as in the payment process 1, and is stored in the storage device 120 when no payment is made. When the payment is made, it is deleted from the storage device 120 and does not exist.

  When it is determined in the determination process of step S81 described above that the payment request information is erased in the storage device 120 and does not exist (NO), the subroutine is immediately terminated. This means that the operator has already settled the video and the settlement request information has been deleted and does not exist. In this way, when payment is made and the payment request information does not exist, the video music playback device does not limit the number of times the video is played, as will be described later, for the video for which payment has been made. 100 can be played. On the other hand, when payment is not made and payment request information exists, the number of times that the video music playback device 100 can play back the target video is limited. That is, when the number of times that the video can be reproduced free of charge is exceeded, the video is prevented from being reproduced.

  When it is determined in the determination process in step S81 that the payment request information exists in the storage device 120 (YES), it is determined whether or not the operator has touched the touch panel 130 (step S83). The touch on the touch panel 130 is an operation for switching a video to be reproduced. If it is determined in the determination process in step S83 that the operator is not touching the touch panel 130 (NO), that is, if it is determined that an operation for switching the video to be reproduced is not performed, this subroutine is immediately ended.

  On the other hand, when it is determined in the determination process in step S83 that the operator has touched the touch panel 130 (YES), that is, when it is determined that an operation for switching the video to be reproduced has been performed, the number of times of contact is increased by 1 (step S85). ). Next, it is determined whether or not the number of times of contact has become greater than a predetermined number (step S87). When it is determined that the number of times of contact is less than or equal to the predetermined number (NO), this subroutine is immediately terminated.

  On the other hand, when it is determined that the number of times of contact has become greater than the predetermined number (YES), the same processes of steps S65 to S77, S88 and S89 as in FIG. 13 are executed. Note that, as described above, in the second settlement process, video is a target of settlement. Therefore, in step S88, when the operator permits settlement, settlement permission data, user ID, and video identification are performed. The code is transmitted to the server (not shown) via the communication interface 140. This video identification code is information for identifying a video to be settled by the operator. In this way, in the second settlement process, by transmitting the video identification code to the server, the key code is generated by the server, transmitted from the server, and compared with the authentic key code.

  In step S89, when the key code does not match the genuine key code, the video reproduction is stopped.

  When the operator touches the touch panel 130, an operation for switching the video to be reproduced is performed. Therefore, by counting the number of times the operator has touched the touch panel 130, the number of times that the video has been reproduced can be acquired, and it can be determined whether or not to make a payment.

  In the settlement process 1 described above, the music playback time is limited, and when the settlement is made, the music playback time (trial listening time) restriction is canceled. However, in the case of managing the settlement using the playback time of the music, by adjusting the time indicated by the internal clock (not shown) that constitutes the video music playback device 100, the audition time is lengthened and illegal. There is also a possibility of being regenerated.

  On the other hand, in the settlement process 2, the number of times that the operator touches the touch panel 130, that is, the number of times of video reproduction is counted, and the video can be reproduced free of charge until the number exceeds the predetermined number. In this way, even if the number of times of video playback is limited, it can be played at least once, thus giving the operator an opportunity to determine whether or not the video is truly the desired video. It is possible to prevent the operator from spending unnecessarily.

  In addition, by limiting the number of video playbacks and not making payments, the video cannot be played back, eliminating the economic disadvantage of video providers that video will be played back without payment. However, it can also be expected to have a substantial advertising effect. Furthermore, by adjusting the time indicated by the internal clock constituting the video and music playback device 100, it is possible to lengthen the listening time and prevent the video from being played back illegally.

  Also, in the settlement process 2, the case where the key code transmitted from the server is compared with the authentic key code has been shown. However, cancellation information such as a password is transmitted from the server, and the operator receives the cancellation information from the touch panel 130. By inputting, the restriction on the reproduction time of the music may be released. In this case, when an operation for processing the settlement of the music is performed on the touch panel 130, release information such as a password is transmitted from the server, and the operator operates the touch panel 130 to transmit it from the server. Enter the release information. When the release information is input from the touch panel 130, the video and music playback device 100 deletes the payment request information to release the restriction on the playback time of the music.

<< Settlement process 3 >>
FIG. 16 is a flowchart showing a subroutine for performing the third settlement process (settlement process 3) on the video to be reproduced. In this settlement process 3 as well, video is the target of settlement. In addition, the same code | symbol was attached | subjected about the process similar to FIG. This subroutine is also called and executed from a main routine (not shown) at a predetermined timing by various functions such as the arithmetic function of the CPU 110 of the video and music playback device 100. Even in the video and music playback device 100 in which the settlement process 3 is executed, the playback link data described above is stored in the storage device 120. When playing back a music by referring to the playback link data, Video can be played back to match playback.

  First, it is determined whether or not the payment request information is stored in the storage device 120 (step S81). This payment request information is information for requesting the operator to make a payment, as in the payment process 1 and the payment process 2, and is stored in the storage device 120 when no payment is made. In the case where it exists and is settled, it is erased from the storage device 120 and does not exist.

  When it is determined in the determination process of step S81 described above that the payment request information is erased in the storage device 120 and does not exist (NO), the subroutine is immediately terminated. This means that the operator has already settled the video and the settlement request information has been deleted and does not exist. In this way, when the payment is made and the payment request information is not present, when the payment video is played back, only the video is played without displaying the advertisement as will be described later. can do. On the other hand, when the payment request information exists without payment, a predetermined advertisement is displayed along with the reproduction of the target video. In other words, a predetermined advertisement is displayed when the number of times that the video can be reproduced for free is exceeded.

  When it is determined in the determination process in step S81 that the payment request information exists in the storage device 120 (YES), the LCD 180 reproduces the video and displays the advertisement (step S91). In this way, by displaying the advertisement, the contents of the advertisement can be visually recognized by the operator, and the video being played is not yet settled even though it is charged. Can be recognized by the operator. As described above, when the payment is made, the payment request information does not exist in the storage device 120, so that the advertisement is not displayed on the LCD 180.

  When the process of step S91 described above is executed, the same steps S83 to S87 and steps S65 to S77, S88 and S89 as in FIG. 15 are executed.

  As described above, since the video is also the target of payment in the third payment process, in step S88, when the operator permits the payment, the payment permission data, the user ID, and the video identification are accepted. The code is transmitted to the server (not shown) via the communication interface 140. This video identification code is information for identifying a video to be settled by the operator. In this way, in the second settlement process, by transmitting the video identification code to the server, the key code is generated by the server, transmitted from the server, and compared with the authentic key code. In step S89, when the key code does not match the genuine key code, the video reproduction is stopped.

  The number of touches of the touch panel 130 in the second settlement process (settlement process 2) and the third settlement process (settlement process 3) described above is preferably determined according to the advertisement fee. For example, for each advertisement that can be displayed on the LCD 180, an advertising fee is collected in advance from the advertiser, and the number of touches on the touch panel 130 is determined according to the advertising fee. The contact count determined in this way is stored in the storage device 120 together with the video data, and is read and used as the “predetermined count” in the determination process in step S87. By doing in this way, the frequency | count which can be reproduce | regenerated for free can be defined according to an advertisement fee.

  In addition, with respect to the reproduction of the video, it is borne by the advertising fee collected from the advertiser until the settlement is made, and when the settlement is made, the operator is borne by the settlement.

  Furthermore, in the above-described example, the advertisement is handled as a separate item from the video and music, but the advertisement may be handled as one video and music. For example, video and music such as TV commercials can be distributed and played back to the video music playback device 100 free of charge, so that the advertising effect can be improved and the video and music can be played according to the advertising fee. By limiting the number of times that playback is possible, the possibility of protecting the copyright of the video also arises.

  Also, in the payment process 3, the case where the key code transmitted from the server is compared with the authentic key code has been shown. However, cancellation information such as a password is transmitted from the server, and the operator receives the cancellation information from the touch panel 130. By inputting, the restriction on the reproduction time of the music may be released. In this case, when an operation for settlement processing for music is performed on the touch panel 130, release information such as a password is transmitted from the server, and the operator operates the touch panel 130 to transmit from the server. Enter the released release information. When the release information is input from the touch panel 130, the video and music playback device 100 deletes the payment request information to release the restriction on the playback time of the music.

<< Advertisement display / non-display process >>
FIG. 17 is a flowchart showing a subroutine for performing advertisement display / non-display processing. In the third settlement process (settlement process 3) described above, a case where one type of advertisement is displayed has been described. However, a plurality of types of advertisements may be displayed. This subroutine is also called and executed from a main routine (not shown) at a predetermined timing by various functions such as the arithmetic function of the CPU 110 of the video and music playback device 100. Even in the video and music playback device 100 in which this advertisement display / non-display processing is executed, the playback link data described above is stored in the storage device 120, and when the music is played back by referring to the playback link data. The video can be played back to match the playback of the music.

  First, on the LCD 180, a plurality of advertisements are displayed along with the playback (step S101). The plurality of advertisements may be displayed on the LCD 180 at the same time, or may be displayed so as not to overlap in time. The correspondence between the position where each of these advertisements is displayed and the identification number for identifying the advertisement is stored in advance in the storage device 120, and the position of the advertisement displayed on the LCD 180 is designated. Thus, the identification number of the advertisement displayed at the position can be identified, and the type of advertisement displayed on the LCD 180 can be specified.

  Next, it is determined whether or not the operator has touched the touch panel 130 (step S103). The touch on the touch panel 130 is an operation for the operator to specify the displayed advertisement. When it is determined that the operator does not touch the touch panel 130 (NO), this subroutine is finished.

  On the other hand, when it is determined that the operator has touched the touch panel 130 (YES), the position (area) of the touch panel 130 touched by the operator is acquired, and the correspondence relationship stored in the storage device 120 is referred to. Thus, an identification number for identifying the advertisement is determined (step S105). Next, the display frequency of the displayed advertisement is integrated for each advertisement identification number (step S107). The advertisement identification number and the number of times the displayed advertisement is displayed are transmitted to the server (step S109), and this subroutine is terminated.

  By doing so, the server can receive the advertisement identification number and the number of times the advertisement is displayed on the LCD 180, and can count the number of times the displayed advertisement is displayed for each advertisement. Furthermore, in the data, the advertising fee can be calculated according to the total number of times the advertisement is displayed, and the advertising fee can be collected from the advertiser. A system that can report to the advertiser as numbers can be constructed, and the server can be used as various advertising effect display systems.

<< Object display handling process >>
A plurality of objects associated with the video may be displayed on the LCD 180, and when the object is designated by the operator, the video associated with the object may be reproduced.

  In this specification, an object can be visually recognized as a predetermined target (such as an object or a person) when a video is played on the LCD 180 of the video and music playback device 100, and is previously captured as a video. And those that are displayed in a superimposed manner or displayed at the same time as video playback. In particular, a so-called icon or the like that represents an associated video content as a small image is an object. In this way, the video to be reproduced can be clearly shown to the operator.

  FIG. 18 is a flowchart showing a subroutine of object display correspondence processing. FIG. 19 is a table showing the contents of the object display position data used in the object display correspondence processing. FIG. 20 is a diagram illustrating an example of an image displayed on the LCD 180 by executing the object display corresponding process. Note that also in the video and music playback device 100 in which the object display corresponding processing is executed, the playback link data described above is stored in the storage device 120, and when the music is played back by referring to the playback link data, the music The video can be played back to match the playback of the video.

  As described above, FIG. 18 is a flowchart showing a subroutine for executing the object display correspondence processing. This subroutine is also called and executed from a main routine (not shown) at a predetermined timing by various functions such as the arithmetic function of the CPU 110 of the video and music playback device 100.

  In the subroutine shown in FIG. 18, first, it is determined whether or not the object is reproduced on the LCD 180 (step S111). When it is determined that the object is not reproduced on the LCD 180 (NO), this subroutine is immediately terminated.

  On the other hand, when it is determined that the object is being reproduced on the LCD 180 (YES), it is determined whether or not the operator has touched the touch panel 130 (step S113). When it is determined that the operator does not touch the touch panel 130 (NO), this subroutine is immediately terminated.

  On the other hand, when it is determined that the operator has touched the touch panel 130 (YES), the music playback time when the operator has touched the touch panel 130 is acquired (step S115). This process can be performed by starting the timer 150 described above simultaneously with the reproduction of music and counting the timer value of the timer 150. Note that the processing in step S115 only needs to be able to acquire the timing that has elapsed since the start of music playback, so the number of music frames may be counted and acquired from the start of music playback.

  Next, the position where the operator touches the touch panel 130 is obtained from the output signal of the touch panel 130, and the position of the operator touching the touch panel 130 is referred to by referring to object display position data shown in FIG. The object displayed at the position is acquired (step S117). Next, referring to the object display position data, switching to the reproduction of the video associated with the object (step S119), the present subroutine is terminated.

  FIG. 19 is a table showing the contents of the object display position data used in the object display correspondence processing. In the example shown in FIG. 19, the object is shown in the first column, the video associated with the object is shown in the second column, the video playback time is shown in the third column, and the fourth column is shown in the third column. The object playback position on the touch panel 130 at the indicated video playback time is shown.

  As shown in the object display position data of FIG. 19, the object is composed of a plurality of objects such as O1 and O2. In the object display position data shown in FIG. 19, for the sake of clarity, symbols such as O1 and O2 are used as identification information for identifying an object. However, in order to facilitate processing, only numerical values are used. The identification information may be configured.

  In the case of the video and music playback device 100 in which the LCD 180 and the touch panel 130 are integrally formed as shown in FIG. 2A, the playback position of the object on the touch panel 130 and the playback position of the object on the LCD 180. Therefore, the object reproduction position shown in the fourth column of the object display position data may be the object reproduction position on the LCD 180.

  Further, as described above, instead of using the music playback time, the number of music frames may be used to determine the timing that has elapsed since the music playback started. When this number of music frames is used, the third column of the object display position data shown in FIG. 19 may be the number of music frames instead of the video playback time.

  The third and fourth columns of the object display position data shown in FIG. 19 are searched using the acquired music playback time by the processing of step S115 described above, and the object corresponding to the music playback time is played back. An object having a position is selected, and an object existing at the reproduction position is acquired by referring to the first column by the process of step S117. Next, in the process of step S119, the third column of the object display position data is read, and a video associated with the acquired object is acquired and switched to playback of the video.

  For example, as shown in FIG. 20, the video A is reproduced on the entire LCD 180. Further, on the LCD 180, one person is displayed on the front side of the video A on the right side of the LCD 180, and a single tree is displayed on the front side of the video A on the left side of the LCD 180. In the example shown in FIG. 20, the area of the person's head is set as the object O1, and the entire tree is set as the object O2.

  Therefore, as shown in FIG. 20, when the operator touches the touch panel 130 and touches the area of the person's head, the image C associated with the object O1 is reproduced by selecting the object O1. Is done. Further, when the operator touches the touch panel 130 and touches the area of the tree, the video B associated with the object O2 is reproduced by selecting the object O2.

  By using the object display position data shown in FIG. 19, the position of the object is stored in advance together with the music playback time, so even if the object moves with the music playback time, the touch panel 130 is touched. The position of the object at the music playback time when it is played can be acquired, and switching to playback of the video associated with the object can be made accurately.

<< Playback time adjustment process >>
In some cases, the video and music playback device 100 plays back the video desired by the operator. However, in this case, it is necessary to adjust the reproduction time of the video in order to be combined with the music to be reproduced together with the video. Hereinafter, a process for adjusting the playback time of the video will be described.

  FIG. 21 is a flowchart showing a subroutine of reproduction time adjustment processing. FIG. 22 is a time chart showing a specific example of the reproduction time adjustment processing.

  As described above, FIG. 21 is a flowchart showing a subroutine for executing the reproduction time adjustment process. This subroutine is also called and executed from a main routine (not shown) at a predetermined timing by various functions such as the arithmetic function of the CPU 110 of the video and music playback device 100. Note that also in the video and music playback device 100 in which this playback time adjustment processing is executed, the playback link data described above is stored in the storage device 120, and when the music is played back by referring to the playback link data, The video can be played back to match the playback of the video.

  In the reproduction time adjustment subroutine shown in FIG. 21, first, the video data prepared by the operator is temporarily stored in the working memory of the storage device 120 (step S121).

  Next, it is determined whether or not the playback time of the video prepared by the operator is longer than the playback time of the music played back together with the video prepared by the operator (step S123). When it is determined that the playback time of the video prepared by the operator is longer than the playback time of the music played together with the video prepared by the operator (YES), a part of the video prepared by the operator should be selected. This is displayed on the LCD 180 (step S125).

  Next, it is determined whether or not the start time of a part of the video prepared by the operator has been input by operating the touch panel 130 (step S127). The start time is input by displaying on the LCD 180 a long image showing the entire playback time of the video prepared by the operator, and the operator operates the touch panel 130 at a location corresponding to the start time of the displayed image. Can be done. If it is determined in step S127 that the start time has not been input (NO), the process returns to step S127.

  On the other hand, if it is determined in step S127 that the start time has been input (YES), whether or not a part of the end time of the video prepared by the operator has been input by operating the touch panel 130. Is determined (step S129). The input of the end time corresponds to the end time of the displayed image, as shown in the above-described step S127, by displaying on the LCD 180 a long image showing the entire playback time of the video prepared by the operator. The location can be input by the operator operating the touch panel 130. If it is determined in step S129 that the end time has not been input (NO), the process returns to step S129.

  In the processing of steps S127 and S129 described above, a long image showing the entire playback time of the video prepared by the operator is displayed on the LCD 180, and the start time and end time can be designated for the image. As a result, the entire reproduction time, the start time, and the end time can be visually and clearly shown, and the operation of the operator can be facilitated.

  On the other hand, when it is determined in step S129 that the end time has been input (YES), the difference between the start time input in step S127 and the end time input in step S129. Is determined to be less than or equal to the playback time of the music to be played back (step S131). When it is determined that the difference between the start time and the end time is longer than the playback time of the music to be played back (NO), the process returns to step S127 described above.

  On the other hand, when it is determined that the difference between the start time and the end time is less than or equal to the playback time of the music to be played back (YES), the start time to the end time in the video data of the video prepared by the operator The data up to is cut out (step S133). With this process, video data from the start time t5s (see FIG. 22A) to the end time t5e (see FIG. 22A) of the video prepared by the operator can be cut out.

  Next, the video data cut out in the process of step S133 is stored in the video data memory of the storage device 120 (step S135), and this subroutine is finished.

  By executing the processes of steps S123 to S135 described above, even when the playback time of the video prepared by the operator is longer than the playback time of the music to be played back, the playback time of the music to be played back is adapted. The video playback time can be adjusted.

  For example, as shown in FIG. 22A, when the playback time of the video prepared by the operator is longer than the playback time TPB of the music to be played back, the start time t5s of the video prepared by the operator is By specifying the end time t5e, the data from the start time t5s to the end time t5e is cut, and the playback time of the video prepared by the operator is the same as the playback time TPB of the music to be played (t5e-t5s). (See FIG. 22B). The playback start time t0 of the clipped video corresponds to the start time t5s of the video before cropping, and the playback end time TPB corresponds to the start time t5e of the video before cropping. By doing so, it is possible to match the start and end of music playback by starting playback from the playback start time t0 of the clipped video and ending playback at the playback end time TPB. For this reason, it is possible to reproduce the video in the same manner as when the reproduction is started from the start time t5s of the video before cutting and the reproduction is ended at the end time t5e.

  When it is determined in the above-described determination processing in step S123 that the playback time of the video prepared by the operator is equal to or shorter than the playback time of the music (NO), it is determined whether or not to confirm with the video prepared by the operator. Judgment is made (step S137). Although this determination has already determined at least one video to be played in conjunction with the playback of the music, the video playback time is shorter than the music playback time, and there is a time margin in the video playback processing. This is for determining whether or not to add a video to be reproduced. Hereinafter, a video that has been determined to be played in conjunction with the playback of the music is referred to as a first video. Further, the video to be added is referred to as a second video. Therefore, the determination process in step S137 specifically adds the second video so that the first video is played first and then the second video is played back along with the playback of the music. It is for judging whether to do.

  More specifically, the operator confirms whether or not there is an intention to play both the first video and the second video to which the second video is added as the video to be played along with the music playback. Execute the process to make it happen. For example, whether or not there is a second video to be added is displayed on the LCD 180 and the operator operates the touch panel 130, so that there is a second video to be added or a second video to be added is displayed. Let the operator choose not to. When the operator selects “there is a video to be added”, the determination process in step S137 determines “YES”, and when the operator selects “there is no video to be added”, the step S137. In the determination process, it is determined that “NO”.

  When it is determined in the determination process in step S137 that the video prepared by the operator is confirmed (NO), that is, the operator selects “no video to be added” and does not add the second video. When only the first video is selected, this subroutine is immediately terminated.

  On the other hand, when it is determined in step S137 that the video prepared by the operator is not confirmed (YES), that is, the operator selects “there is a video to be added”, and only the first video is displayed. If the second video is added, the video data of the video (second video) newly prepared by the operator is stored in the video data memory of the storage device 120 (step S139). The reproduction time is calculated and displayed on the LCD 180 (step S141). Here, the remaining playback time is a time obtained by subtracting the sum of the playback time of the first video (t5e-t5s) and the playback time of the second video from the playback time of the music.

  Next, it is determined whether or not the second video can be used according to the remaining reproduction time calculated by the process of step S141 (step S143). For example, when the remaining playback time calculated in the process of step S141 is zero or more, it is determined that the second video can be used, and when the remaining playback time is negative, the second video is used. It is determined that cannot be performed.

  When it is determined in step S143 described above that the second video can be used (YES), the process is further performed in step S121 to add a new video (third video). return. In this case, since the video data of the second video is already stored in the video data memory of the storage device 120 in the process of step S139, the second video is read by reading the video data. Can be played.

  On the other hand, when it is determined that the second video can be used in the determination processing in step S143 described above (NO), this subroutine is immediately terminated.

  In the above-described processing, the processing for cutting out the predetermined range is not shown for the data of the second video, but the start time and the end time of the second video data are also the same as the first video data. And data from the start time to the end time may be cut out and used. In this way, the playback time of the second video can be adjusted, and the second video can be played after the first video is played in alignment with the playback time of the music. .

  Thus, by executing the playback time adjustment process, the video data of the video desired by the operator can be edited according to the operation of the operator, so that not only the purchased video data but also the operator can shoot Thus, the prepared video data can also be set as a playback target in the video and music playback device 100 so that it can be played back along with the playback of the music.

  In the example described above, the video data prepared by the operator is cut out only at one continuous location, and the cut out video data is associated with the start and end of music playback. In the video data prepared by the operator, a plurality of portions are cut out, connected in the order desired by the operator, and the generated video data is associated with the start of music playback and the end of playback. Such editing may be possible. By doing so, it is possible to reproduce the music with improved consistency between the video and the video.

  Furthermore, it is preferable that not only video data but also reproduction link data can be edited. For example, in the case of video data generated by cutting out a plurality of locations from video data prepared by the operator and connecting them, playback link data is generated so as to correspond to the connected playback positions, and the music data is generated. Make it correspond. In this way, even when video data generated by concatenation and other video data are switched and played back, the music and video can be played with improved consistency.

<< Video playback switching process 3 >>
FIG. 23 is a flowchart showing a subroutine of the video reproduction switching process 3. This subroutine is also called and executed from a main routine (not shown) at a predetermined timing by various functions such as the arithmetic function of the CPU 110 of the video and music playback device 100.

  First, it is determined whether or not the operator has touched the touch panel 130 (step S151). When it is determined that the operator does not touch the touch panel 130 (NO), this subroutine is immediately terminated.

  On the other hand, when it is determined that the operator has touched the touch panel 130 (YES), predetermined reproduction link data (not shown) is read (step S153). Next, it is determined whether or not reproduction link data exists at a time later than the current music reproduction time (step S155). The music playback time is a time measured by starting the timer 150 on the basis of the time when the music playback is started. When it is determined that there is no reproduction link data at a time later than the current music reproduction time (NO), this subroutine is immediately terminated.

  On the other hand, when it is determined that the playback link data exists at a time later than the current music playback time (YES), it is determined whether it is the timing to switch the video playback (step S157). When it is determined that it is not the timing to switch the video reproduction (NO), it is determined whether or not there is video data to be switched (step S159). This determination is to determine whether or not the video data to be switched is already stored in the storage device 120. When it is determined that there is video data to be switched (YES), the process returns to step S157 described above.

  On the other hand, when it is determined that there is no video data to be switched (NO), the video data to be switched is downloaded from the server via the communication interface (step S161), and the downloaded video data is stored in the storage device 120. (Step S163), and the process returns to Step S157 described above.

  As described above, even when the video data of the video to be switched and played back does not exist in the storage device 120, the video data is downloaded from the server in advance, so that the video data can be quickly acquired and the video is played back. It can be played back in time and the video can be played back smoothly. In this way, since the video is downloaded from the server before switching the video reproduction, the processing can be switched smoothly without delay.

  When it is determined in the above-described determination processing in step S157 that it is the timing to switch the playback of the video (YES), the video data indicated by the playback link data is read (step S165), and the video indicated by the playback link data and the playback thereof Switching to playback from the position (step S167), reading of the video data of the video that was originally played back from the storage device 120 is interrupted (step S169), and this subroutine is terminated.

  Video data stored in advance in a server that is communicably connected via a communication line is downloaded as needed and played back on the LCD 180 of the video music playback device 100. In this way, even when the capacity of the storage device 120 is small, only the video data necessary at that time can be stored in the storage device 120, and a plurality of videos can be switched and reproduced. it can. Further, even video of video data not stored in the storage device 120 can be quickly obtained and reproduced if desired by the operator.

  In the example described above, the video and music playback device 100 downloads and plays back video and music. However, streaming playback may be used. In the present specification, download playback refers to starting playback after downloading the entire video data and music data. By doing so, it is possible to smoothly reproduce the video and music without depending on the communication state. On the other hand, streaming playback means playback at the same time as downloading video data and music data. In this way, the waiting time until the entire video data or music data is downloaded can be reduced.

  After streaming the video data, the operator may determine whether the video data is to be officially stored in the storage device 120. When the storage device 120 is officially stored, the video data that has been streamed and reproduced can be officially stored in the storage device 120 when a predetermined amount is charged for the charge. In this case, it is preferable to store information indicating that payment has been made on the server side. In this way, even if the operator accidentally deletes the video data from the video music playback device 100, processing is performed so that the video data can be downloaded again without being charged anew. Can do.

  On the other hand, when the payment is refused for the billing, the video data reproduced by streaming is deleted from the video music reproducing apparatus 100. In particular, in this case, the video music playback device 100 performs a process of writing zero (zero clear) or a process of writing a random number in an area where the video data that has been streamed is temporarily stored. The video data can be restored to prevent unauthorized playback.

  In this way, it is possible to prevent the video data from being illegally stored in the storage device 120 and to determine whether or not the video is truly desired by the operator. However, since only the video data that is truly desired can be left at the operator's hand, it is economical for the operator.

<< Video playback switching process 4 >>
FIG. 24 is a time chart showing a specific example of the video reproduction switching process 4. Also in the time chart of FIG. 24, the portion where the video is actually played is indicated by a bold solid line arrow extending downward, and the four videos A, B, The portions where the C and D video data are expanded and updated in the four work buffers 172A to 172D are indicated by bold broken arrows extending downward. In addition, video switching locations are indicated by thin line arrows A6, B6, C6 and D6 extending in the horizontal direction. The state in which the music is being played is indicated by a thin solid line arrow extending downward.

  The music playback starts at the music playback time t0, and the music playback ends at the music playback time tend. As described above, also in the video reproduction switching process 4, the reproduction time and time are determined using the reproduction time of the music. Note that the reproduction of the music starts at the music reproduction time t0 by using the first data of the music data, and ends at the music reproduction time tend by using the last data of the music data.

  As shown in FIG. 24, the music reproduction starts at the music reproduction time t0. As described above, in the video playback switching process 4, the video data of the four videos A, B, C, and D are always read from the storage device 120 over the time during which the music is played, and the four operations are performed. It is expanded and updated in the buffers 172A to 172D. For this reason, as shown in FIG. 24, at the music playback time t0, the playback of the music starts, and the video data of the four videos A, B, C, and D are also read from the storage device 120 and the four tasks. Update in the buffers 172A to 172D is started. In the video playback switching process 4 shown in FIG. 24, reading from the storage device 120 and updating in the four work buffers 172A to 172D are started at the music playback time t0, but the music playback time t0. , The work buffer 172A among the four work buffers 172A to 172D is selected, and the video A is played back on the LCD 180 from the video playback time t0.

  Also in this video playback switching process 4, the video data of the four videos A, B, C, and D are such that the playback times of the four videos A, B, C, and D coincide with the playback times of the music. Suppose that it is adjusted in advance and stored in the storage device 120. In particular, it is preferable to edit and adjust the video data so that the reproduction of the music ends and the content of the video also ends. For example, by deleting at least one place in the middle of the video data and shortening the video playback time, the video playback time may coincide with the music playback time. In this way, by storing and playing back the adjusted video data, the content of the video ends as soon as the music playback ends, regardless of whether the video A, B, C or D ends. Thus, the video can be reproduced, and the content of the music and the content of the video can be matched and terminated.

  As described above, the playback times of the four videos A, B, C, and D are adjusted in advance so as to coincide with the playback time of the music, but as described later, this video playback switching process 4 Then, the playback position for playing back the video may be returned to the past playback position. When the playback position is returned to the past playback position, the playback time of the video may not match the playback time of the music. Further, in the following description, the music and video playback positions are mainly described using the music playback time and video playback time, but the number of music frames and the number of video frames may be used as the playback position.

  First, when the operator touches the touch panel 130 and operates before the music playback time t61 (music frame number f61) (see FIG. 24), the music playback time t61 and the music frame number f61. Is reached, the video A is switched to the video B (see arrow A6 in FIG. 24), and the video B is reproduced from the video reproduction time t61 and the number of video frames f61. Note that if the operator does not touch and operate the touch panel 130 before the music playback time t61 or the music frame number f61, the video A is continuously played back.

  Next, when the operator touches the touch panel 130 and operates the video B at a time before reaching the music playback time t63 or the music frame number f63 while playing the video B (see FIG. 24). When the music reproduction time t63 and the music frame number f63 are reached, the video B is switched to the video C (see arrow B6 in FIG. 24), and the video C is reproduced from the video reproduction time t63 and the video frame number f63. Note that if the operator does not touch and operate the touch panel 130 before the music playback time t63 or the music frame number f63, the video B is continuously played back.

  Further, when the video C is being played, if the operator touches the touch panel 130 and operates before the music playback time t64 or the music frame number f64, the music playback time t64 or When the number of music frames f64 is reached, the video C is switched to the video D (see arrow C6 in FIG. 24), the video D is played from the video playback time t64 and the video frame number f64, and finally the video D is played back. When the music playback time tend and the number of music frames fend are reached, the playback of the music is finished and the playback of the video D is also finished by finishing the playback of all the music data.

  On the other hand, if the operator does not operate by touching the touch panel 130 before the music playback time t64 or the music frame number f64, the video C is continuously played back. When the video C is continuously played back, if the operator touches the touch panel 130 and operates before the music playback time t65 or the music frame number f65, the music playback time t65. When the number of music frames f65 is reached, the video C is switched to the video B (see arrow D6 in FIG. 24), and the video B is played from the video playback time t62 and the video frame number f62. If the operator does not operate by touching the touch panel 130 at a time before reaching the music playback time t65 or the music frame number f65, the video C is continuously played, and finally, When the music playback time tend and the number of music frames fend are reached while the video C is being played back, the playback of the music is completed and the video C is also played back by finishing the playback of all the music data. finish.

  As described above, in the video playback switching process 4, when switching from video C to video B when the music playback time t65 or the number of music frames f65 is reached (see arrow D6 in FIG. 4), video of video B is displayed. Returning to the reproduction time t62 and the video frame number f62, the video B is reproduced from the video reproduction time t62 and the video frame number f62. In this way, when switching from video C to video B at the music playback time t65 and the music frame number f65, as in the video playback time t62 and the video frame number f62 of the video B, the past video playback position is reached. Play back. Even if the video B is played back up to the video playback time t62 and the video frame number f62, the music reaches tend and fend after the music playback time t65 and the music frame number f65. Until all of the music data is reproduced, when the tend and fend are reached, the reproduction of the music is terminated. With the end of the reproduction of this music, the reproduction of the video B is also ended.

  Also in this video playback switching process 4, music playback and video playback are matched with reference to playback link data (not shown). In particular, in the video playback switching process 4, as described above, the video B is played back to the video playback time t62 and the video frame number f62 at the music playback time t65 and the music frame number f65. At the time of playback time t65 and the number of music frames f65, the content of video C at the time of video playback time t65 and video frame number f65 matches the content of video B at the time of video playback time t62 and video frame number f62. Thus, it is necessary to determine reproduction link data in advance. By defining the playback link data in this way, when the music playback time t65 or the number of music frames f65 is reached, the video playback can be performed even if the video B is played back to the video playback time t62 or the video frame number f62. The video B can be reproduced in conformity with the above.

<< Video playback switching process 5 >>
FIG. 25 is a time chart showing a specific example of the video reproduction switching process 5. In the time chart of FIG. 25 as well, the part where the video is actually played is indicated by a thick solid line arrow extending downward, and although the video is not actually played back, the four videos A, B, The portions where the C and D video data are expanded and updated in the four work buffers 172A to 172D are indicated by bold broken arrows extending downward. In addition, video switching locations are indicated by thin line arrows A7, B7, C7 and D7 extending in the horizontal direction. The state in which the music is being played is indicated by a thin solid line arrow extending downward.

  In the time charts of FIG. 4, FIG. 8, FIG. 11, FIG. 14 or FIG. 24 described above, the order (video A → video B → In the case of the video C), the video is switched to another video and played back. However, in the time chart shown in FIG. 25, when contact with the touch panel 130 is detected prior to the music playback time t73, switching to the video C or switching to the video D different from the video C occurs. It is particularly characterized in that it is configured as described above. That is, as shown in the time chart of FIG. 24, when the touch on the touch panel 130 is triggered, the video is not only switched to the video C and played, but the video is also touched on the touch panel 130 at the same music playback time t73. There is a case where the video is switched to a video D different from C and reproduced.

  As shown in FIG. 25, the video B is played back particularly at the music playback time t73. When this video B is played back, the video B is played back by switching to a different video depending on the position or area in contact with the touch panel 130. Those that do are preferred. For example, as shown in FIG. 26, a person and a tree are displayed on the LCD 180, and when the person touches the image area (object O1) of the person's head, the image is switched to video C and reproduced. When the image area (object O2) is touched, the image D is switched to be reproduced. By doing in this way, the video to be played can be made different according to the position or area in contact with the touch panel 130,

  By using the time chart as shown in FIG. 25, it is possible to branch from one video B to various different videos for playback at a certain music playback time, for example, music playback time t73. Therefore, it is possible to prevent the same video from always being played back even if the switching operation is performed, and it is possible to view a variety of videos and enjoy the merit that it is difficult to get bored.

  First, as shown in FIG. 25, for music playback, music playback starts at music playback time t0, and music playback ends at music playback time tend. Thus, also in the video reproduction switching process 5, the reproduction time and time are determined using the reproduction time of the music. Note that the reproduction of the music starts at the music reproduction time t0 by using the first data of the music data, and ends at the music reproduction time tend by using the last data of the music data.

  As shown in FIG. 25, the music reproduction starts at the music reproduction time t0. As described above, in the video playback switching process 5, the video data of the four videos A, B, C, and D are always read from the storage device 120 over the time during which the music is played, and the four operations are performed. It is expanded and updated in the buffers 172A to 172D. For this reason, as shown in FIG. 25, at the music playback time t0, the playback of the music starts, and the video data of the four videos A, B, C, and D are also read from the storage device 120 and the four operations. Update in the buffers 172A to 172D is started. In the video playback switching process 5 shown in FIG. 25, reading from the storage device 120 and updating in the four work buffers 172A to 172D are started at the music playback time t0, but the music playback time t0. , The work buffer 172A among the four work buffers 172A to 172D is selected, and the video A is played back on the LCD 180 from the video playback time t0.

  Also, in this video playback switching process 5, the video data of the four videos A, B, C, and D are such that the playback times of the four videos A, B, C, and D coincide with the playback times of the music. Suppose that it is adjusted in advance and stored in the storage device 120. In particular, it is preferable to edit and adjust the video data so that the reproduction of the music ends and the content of the video also ends. For example, by deleting at least one place in the middle of the video data and shortening the video playback time, the video playback time may coincide with the music playback time. In this way, by storing and playing back the adjusted video data, the content of the video ends as soon as the music playback ends, regardless of whether the video A, B, C or D ends. Thus, the video can be reproduced, and the content of the music and the content of the video can be matched and terminated.

  As described above, the playback times of the four videos A, B, C and D are adjusted in advance so as to coincide with the playback time of the music, but the video playback switching process 5 also plays back the video. In some cases, the playback position to be returned is returned to the past playback position. When the playback position is returned to the past playback position, the video playback time may not match the music playback time. Further, in the following description, the music and video playback positions are mainly described using the music playback time and video playback time, but the number of music frames and the number of video frames may be used as the playback position.

  First, when the operator touches the touch panel 130 and operates before the music playback time t71 (music frame number f71) (see FIG. 25), the music playback time t71 and the music frame number f71. , The video A is switched to the video B (see the arrow A7 in FIG. 25 and the playback link data in FIG. 27), and the video B is played back from the video playback time t71 and the number of video frames f71. Note that if the operator does not operate by touching the touch panel 130 before reaching the music playback time t71 or the music frame number f71, the video A is continuously played back on the LCD 180.

  Next, when video B as shown in FIG. 20 is played back, when the operator touches the touch panel 130 and operates before the music playback time t73 or the music frame number f73. (See FIG. 25) is processed as follows.

  When the operator touches the position (X1, Y1) in the area of the touch panel 130 at a time before reaching the music playback time t73 or the music frame number f73, the playback link data shown in FIG. 27 is referred to. The image displayed at the position (X1, Y1) is the object O1. Further, referring to the reproduction link data shown in FIG. 27, it is acquired that the object O1 is linked to the video reproduction time t73 of the video C. Therefore, when the operator touches the position (X1, Y1) in the area of the touch panel 130, the video B is switched to the video C (see the arrow B7 in FIG. 25 and the playback link data in FIG. 27). The video C is played back from the video playback time t73.

  Similarly, when the operator touches the position (X2, Y2) in the area of the touch panel 130 at a time before reaching the music playback time t73 or the music frame number f73, the playback link data shown in FIG. With reference to this, it is acquired that the image displayed at the position (X2, Y2) is the object O2. Further, referring to the reproduction link data shown in FIG. 27, it is acquired that the object O2 is linked to the video reproduction time t73 of the video D. Therefore, when the operator touches the position (X2, Y2) in the area of the touch panel 130, the video B is switched to the video D (see the arrow D7 in FIG. 25 and the reproduction link data in FIG. 27). The video D is played back from the video playback time t73.

  Note that the reproduction link data shown in FIG. 27 shows only the position (X1, Y1) of the object O1 and the position (X2, Y2) of the object O2 at the time of the music reproduction time t73 for simplicity. It is preferable that the position of the object O1 and the position of the object O2 at a plurality of music playback times at times other than the music playback time t73 and the music frame number f73 are determined in detail and used as playback link data. In this way, even when the object O1 or the object O2 is displayed on the LCD 180 in such a manner that the object O1 or the object O2 moves as the music playback time advances, the position of the object O1 or the object O2 is accurately determined. The video to be switched can be accurately determined according to the type of the object.

  In the playback link data shown in FIG. 27, only the music playback time and video playback time are used as playback positions, but as described above, playback link data is used as playback positions using music frames and video frames. It may be determined.

  Note that if the operator does not operate by touching the touch panel 130 before reaching the music playback time t73 or the music frame number f73, the video B is continuously played back on the LCD 180.

  Switching to the video C according to the arrow B7 described above, and when playing the video C from the music playback time t73 and the music frame number f73, at a time earlier than the music playback time t74 and the music frame number f74, When the operator touches the touch panel 130 and operates, when the music playback time t74 or the music frame number f74 is reached, the video C is switched to the video B (the arrow C7 in FIG. 25 and the playback link data in FIG. 27). If the music playback time tend and the music frame number fend are reached when the video B is played back, and finally the video B is played back, When the reproduction of all the data is finished, the reproduction of the music is finished and the reproduction of the video B is also finished. On the other hand, if the operator does not operate by touching the touch panel 130 before reaching the music playback time t74 or the music frame number f74, the video C is continuously played back.

  When the video D is reproduced from the music playback time t73 and the music frame number f73 by switching to the video D according to the arrow D7 described above, when the music playback time tend and the music frame number fend are finally reached, When all the music data has been reproduced, the reproduction of the music is completed and the reproduction of the video D is also completed.

  In the time chart shown in FIG. 25 described above, among the objects displayed on the LCD 180, the video to be switched is determined in accordance with the object specified by the operator. Alternatively, it may be determined according to the history of the video that has been played so far, or may be determined according to the history of the number of times that the operator (user) has touched the touch panel 130 so far.

  For example, when it is determined by lottery processing, a random number is generated, and an image to be switched may be determined according to the value of the random number. In the process using the history of the reproduced video, the number of times of reproduction for each reproduced image is counted every time the image is reproduced, and the counted number is stored in the storage device 120 and the storage device 120 is stored. The video to be switched may be determined in accordance with the value of the number of times read from. Furthermore, the process using the history of the number of times touching the touch panel 130 counts the number of times the touch panel 130 is touched for each video played in response to the touch, and stores the counted number in the storage device 120. The video to be switched may be determined according to the value of the number of times read from the storage device 120. In particular, in the process of determining the video to be switched according to the history of the number of times played or the history of the number of touches, the playback is permitted when the number of histories exceeds a predetermined number. A video may be included. In this way, in order to be able to play the video, it is necessary to repeatedly play other videos, for example, advertising videos, so that the possibility of enhancing the advertising effect arises, and at the cost of the video Can be made playable.

  By deciding according to the history of the video that has been played so far, it is possible to play back the video with the freshness felt by the operator by switching and playing the video that has been played less frequently. On the other hand, the video that has been played many times is preferentially switched and played as the video preferred by the operator, so that the video can be played with the operator feeling a sense of security.

<< Game execution process >>
When playing the above-described music, the process of switching and playing the video in alignment with the music being played can also be used as the process of executing the game. In FIG. 20 and FIG. 26 described above, the object image is displayed together with the reproduction of the video, and when the operator (user) touches the area of the object image, the video is switched to another video and reproduced. It was. Instead of such an object image, an image indicating a button or switch for allowing the operator to select a process is displayed as an object image, and the image indicating the button or switch is touched (contact with the touch panel 130). Thus, the processing can be branched according to the contact of the operator. For example, an image of a button of “Yes” or “YES” and an image of a button of “No” or “NO” are displayed on the LCD 180, and the area where the image is displayed is touched to the operator (to the touch panel 130). ), It is possible to execute processing according to “Yes” or “YES”, or to execute processing according to “No” or “NO”.

  By performing the processing as described above, when playing the music of the game as BGM (background music), it is possible to switch and play the game video in accordance with the music being played. it can. Furthermore, since the switching of the video of the game can be selected according to the operation of the operator, the interactiveness (bidirectionality) of an interactive game or the like can be improved, and a character for selection can be selected. Since it is displayed, the contents of the process can be clearly indicated to the operator (user).

  The image of the object used in FIG. 20 and FIG. 26 described above, the image of the button “Yes” or “YES”, or the image of the button “No” or “NO” are shown in FIG. Data ”is stored in the storage device 120. A process that can branch the process in response to an operator touching an image of an object is stored in the storage device 120 as the “image object display program” shown in FIG.

100 video and music playback device 110 CPU
120 Storage Device 130 Touch Panel 140 Communication Interface 150 Timer

Claims (30)

(A) In a storage unit in which music content data including music data for reproducing one music and video data for reproducing video of a plurality of patterns associated with the one music is stored. A process of reading the stored music content data and reproducing a first video of the plurality of patterns of video in synchronization with the reproduction of the one music;
(B) When a contact with a contact sensor unit that detects contact from the outside is detected in a state in which a video is reproduced on the display unit using the first video data, A process of determining a reproduction start position of second video data different from the first video data in accordance with a reproduction position of the music data being reproduced, of the pattern of video,
(C) The process of (b) based on the detection of the contact with the contact sensor unit in the state where the video is reproduced on the display of the display unit using the first video data. The playback of the second video data is started from the playback start position determined by the above, and the playback state of the video using the first video data is switched to the playback state of the video using the second video data. Processing,
A music video playback program characterized in that the program is executed on a computer.   (D) When the reproduction of the video using the second video data is completed, a process of switching to the video reproduction using the first video data is executed by a computer. 1. The music video playback program according to 1.   (E) If a preset time has elapsed since the playback of the video using the first video data was switched to the playback of the video using the second video data, the first The music video reproduction program according to claim 1 or 2, wherein a computer performs a process of switching to video reproduction using video data. The storage unit
An authentic key code indicating that the music content is authentic and associated with the music content;
Payment request information for requesting payment for continuously playing video using the second video data;
(F) after the video using the second video data is reproduced, reading the payment request information from the storage unit and displaying the payment request information image on the display as a payment request information image;
(G) The payment permission is obtained through a communication interface that communicates with the outside on condition that payment permission data indicating permission of payment is input by touching the contact sensor unit after the payment request information is displayed. A process of transmitting data, a user ID code, and a music content identification code;
(H) a process of receiving a key code generated based on the transmitted payment permission data, the user ID code, and the music content identification code;
(I) a process of determining whether or not the received key code matches the genuine key code;
(J) When it is determined that the received key code matches the genuine key code, the payment request information is deleted, and the time counting process for determining whether or not a preset time has elapsed is canceled to A process for continuously playing video using video data;
The music video reproduction program according to claim 3, wherein the program is executed by a computer. The storage unit stores the number of times of contact with the contact sensor unit in order to switch from reproduction of video using the first video data to reproduction of video using the second video data,
(K) When the number of times of contact reaches a preset number associated with the music content, a process of stopping the reproduction processing of the music content is executed by a computer. Item 3. The music video playback program according to Item 1 or 2. The storage unit stores advertisement data for displaying an advertisement on the display, and an advertisement-corresponding contact count set according to the advertisement,
(L) a process of adding the advertisement data and displaying it on the display when playing back video using the first video data or playing back video using the second video data;
(M) When the number of times of contact has reached the number of contacts corresponding to the advertisement,
The music video reproduction program according to claim 5, wherein the program is executed by a computer. The storage unit stores advertisement data for displaying an advertisement on the display, and identification data for identifying the advertisement,
(N) a process of adding the advertisement data and displaying on the display when playing back video using the first video data or playing back video using the second video data;
(O) The accumulated display count of the displayed advertisement is stored in the storage unit in correspondence with the identification data of the advertisement displayed after being contacted with the contact sensor unit, and the cumulative display count for each advertisement is A process of transmitting via a communication interface that communicates with the outside;
The music video reproduction program according to claim 5, wherein the program is executed by a computer. The video includes at least one object that can be displayed on the display,
The storage unit stores playback position data on the display when a video including the at least one object is played back, and object-corresponding video data in which a video is associated with the playback position data.
(P) When a video including the at least one object is being played, and when touched by the contact sensor unit, to an area where one of the at least one object is displayed The music video playback program according to claim 1, wherein when the contact is, the computer performs a process of switching to playback of the video associated with the one object. (Q) If it is determined that the playback time of the predetermined video is longer than the playback time of the music, the video data corresponding to the playback time of the music among the video data of the predetermined video is Processing to select according to the contact with the sensor unit;
(R) processing for storing the selected video data in the storage unit;
The music video reproduction program according to claim 1, wherein the music video reproduction program is executed by a computer. Video data for reproducing at least one of the plurality of patterns of video is stored in a video data server that is communicably connected via a communication interface that communicates with the outside.
(S) When using video data stored in the video data server in response to contact with the contact sensor unit, a process of downloading video data from the video data server via the communication interface;
(T) a process of playing the downloaded video data on the display;
10. The music video reproduction program according to claim 1, wherein the music video reproduction program is executed by a computer. A music video playback means for playing music and video,
A storage unit that stores music content data composed of music data for playing back one piece of music and video data for playing back images of a plurality of patterns associated with the one piece of music;
A video playback unit comprising a display on which the video is played;
A music playback unit comprising a speaker from which the music is played;
A contact sensor for detecting contact from the outside;
A CPU that reproduces an image selected from the images of the plurality of patterns and reproduces the music, and the CPU includes:
(A) In a storage unit in which music content data including music data for reproducing one music and video data for reproducing video of a plurality of patterns associated with the one music is stored. A process of reading the stored music content data and reproducing a first video of the plurality of patterns of video in synchronization with the reproduction of the one music;
(B) When a contact with a contact sensor unit that detects contact from the outside is detected in a state in which a video is reproduced on the display unit using the first video data, A process of determining a reproduction start position of second video data different from the first video data in accordance with a reproduction position of the music data being reproduced, of the pattern of video,
(C) The process of (b) based on the detection of the contact with the contact sensor unit in the state where the video is reproduced on the display of the display unit using the first video data. The playback of the second video data is started from the playback start position determined by the above, and the playback state of the video using the first video data is switched to the playback state of the video using the second video data. And a music video reproduction means. The CPU
(D) When the reproduction | regeneration of the image | video using the said 2nd video data is complete | finished, the process switched to reproduction | regeneration of the image | video using the said 1st video data is performed. Music video playback means. The CPU
(E) If a preset time has elapsed since the playback of the video using the first video data was switched to the playback of the video using the second video data, the first The music video playback means according to claim 11 or 12, wherein a process of switching to video playback using video data is executed. The storage unit
An authentic key code indicating that the music content is authentic and associated with the music content;
Payment request information for requesting payment for continuously playing video using the second video data;
The CPU
(F) after the video using the second video data is reproduced, reading the payment request information from the storage unit and displaying the payment request information image on the display as a payment request information image;
(G) The payment permission is obtained through a communication interface that communicates with the outside on condition that payment permission data indicating permission of payment is input by touching the contact sensor unit after the payment request information is displayed. A process of transmitting data, a user ID code, and a music content identification code;
(H) a process of receiving a key code generated based on the transmitted payment permission data, the user ID code, and the music content identification code;
(I) a process for determining whether or not the received key code matches the genuine key code;
(J) When it is determined that the received key code matches the genuine key code, the payment request information is deleted, and the time counting process for determining whether or not a preset time has elapsed is canceled to A process for continuously playing video using video data;
The music video playback means according to claim 13, wherein: The storage unit stores the number of times of contact with the contact sensor unit in order to switch from reproduction of video using the first video data to reproduction of video using the second video data,
The CPU
(K) When the number of times of contact reaches a preset number associated with the music content, a process of canceling the reproduction process of the music content is executed. Or the music video reproducing means according to 12; The storage unit stores advertisement data for displaying an advertisement on the display, and an advertisement-corresponding contact count set according to the advertisement,
The CPU
(L) a process of adding the advertisement data and displaying it on the display when playing back video using the first video data or playing back video using the second video data;
(M) When the number of times of contact has reached the number of contacts corresponding to the advertisement,
The music video playback means according to claim 15, wherein: The storage unit stores advertisement data for displaying an advertisement on the display, and identification data for identifying the advertisement,
The CPU
(N) a process of adding the advertisement data and displaying on the display when playing back video using the first video data or playing back video using the second video data;
(O) The accumulated display count of the displayed advertisement is stored in the storage unit in correspondence with the identification data of the advertisement displayed after being contacted with the contact sensor unit, and the cumulative display count for each advertisement is stored in the storage unit. A process of transmitting via a communication interface that communicates with the outside;
The music video playback means according to claim 15, wherein: The video includes at least one object that can be displayed on the display,
The storage unit stores playback position data on the display when a video including the at least one object is played back, and object-corresponding video data in which a video is associated with the playback position data.
The CPU
(P) When a video including the at least one object is being played, and when touched by the contact sensor unit, to an area where one of the at least one object is displayed 18. The music video playback means according to claim 11, wherein the music video playback means executes a process of switching to playback of the video associated with the one object when the contact is made. The CPU
(Q) If it is determined that the playback time of the predetermined video is longer than the playback time of the music, the video data corresponding to the playback time of the music among the video data of the predetermined video is Processing to select according to the contact with the sensor unit;
(R) processing for storing the selected video data in the storage unit;
19. The music video playback means according to claim 11, wherein the music video playback means is executed. Video data for reproducing at least one of the plurality of patterns of video is stored in a video data server that is communicably connected via a communication interface that communicates with the outside.
The CPU
(S) When using video data stored in the video data server in response to contact with the contact sensor unit, a process of downloading video data from the video data server via the communication interface;
(T) a process of playing the downloaded video data on the display;
20. The music video playback means according to claim 11, wherein the music video playback means is executed. Reproduction of the one piece of music using music content data comprising music data for reproducing one piece of music and video data for playing back images of a plurality of patterns associated with the one piece of music A step of reproducing a first video of the plurality of patterns of video in synchronization with
When contact with a contact sensor unit that detects contact from the outside is detected in a state where an image is reproduced on a display of the display unit using the first image data, the plurality of patterns are Determining a reproduction start position of second video data different from the first video data in the video according to a reproduction position of the music data being reproduced;
Playback determined by the determining step based on detection of contact with the contact sensor unit in a state in which video is played on the display of the display unit using the first video data Starting playback of the second video data from a start position, and switching from a video playback state using the first video data to a video playback state using the second video data. A music video playback method characterized by the above.   The music video playback according to claim 21, further comprising a step of switching to video playback using the first video data when playback of the video using the second video data is completed. Method.   In a case where a preset time has elapsed since switching from video playback using the first video data to video playback using the second video data, the first video data is The music video playback method according to claim 21 or 22, further comprising a step of switching to playback of the video used. The music content data, the authentic music key code indicating that the music content is authentic, and the video using the second video data are continuously reproduced. Storing payment request information for requesting payment for storage in a storage unit;
After the video using the second video data is played back, reading the payment request information from the storage unit and displaying it on the display as a payment request information image;
Through the communication interface that communicates with the outside, on the condition that payment permission data indicating permission of payment is input by contact with the contact sensor unit after the payment request information is displayed, the payment permission data, Transmitting a user ID code and a music content identification code;
Receiving a key code generated based on the transmitted payment permission data, the user's ID code, and the music content identification code;
Determining whether the received key code matches the genuine key code;
When it is determined that the received key code matches the genuine key code, the payment request information is deleted, and the time counting process for determining whether or not a preset time has elapsed is canceled to obtain the second video data. A process of continuously playing the video used,
The music video playback method according to claim 23, comprising: Storing in the storage unit the number of times of contact with the contact sensor unit in order to switch from video playback using the first video data to video playback using the second video data;
A step of stopping the reproduction processing of the music content when the number of times of contact reaches a preset number of times associated with the music content;
The music video playback method according to claim 21, wherein the music video playback method is provided. Storing the advertisement data for displaying the advertisement on the display and the number of contact corresponding to the advertisement set according to the advertisement in the storage unit;
A step of adding the advertisement data and displaying it on the display when playing back video using the first video data or playing back video using the second video data;
When the number of contacts reaches the number of contacts corresponding to the advertisement, stopping the reproduction of the video;
26. The music video playback method according to claim 25, comprising: Storing in the storage unit advertisement data for displaying an advertisement on the display and identification data for identifying the advertisement;
A step of adding the advertisement data and displaying it on the display when reproducing the video using the first video data or reproducing the video using the second video data;
The accumulated display count of the displayed advertisement is stored in the storage unit in correspondence with the identification data of the advertisement displayed after being contacted with the contact sensor unit, and the accumulated display count for each advertisement is Transmitting via a communication interface for communicating;
26. The music video playback method according to claim 25, comprising: The video includes at least one object that can be displayed on the display,
Storing the playback position data on the display when the video including the at least one object is played back, and the object corresponding video data in which the video is associated with the playback position data in the storage unit;
When a video including the at least one object is being played, when the touch sensor unit is touched, contact with a region where one object of the at least one object is displayed There is a step of switching to playback of the video associated with the one object,
The music video playback method according to any one of claims 21 to 27, characterized by comprising: If it is determined that the playback time of the predetermined video is longer than the playback time of the music, the video data corresponding to the playback time of the music in the video data of the predetermined video is sent to the contact sensor unit. Selecting according to the contact of,
Storing the selected video data in the storage unit;
29. The music video playback method according to claim 21, further comprising: When video data stored in the video data server is used in response to contact with the contact sensor unit, at least one of the images of the plurality of patterns via a communication interface that communicates with the outside. Downloading video data from a video data server storing video data for reproducing two videos; and
Playing the downloaded video data on the display;
30. The music video playback method according to any one of claims 21 to 29, comprising:

Images