JP2005122664A - Audio data recording device, audio data reproducing device, and audio data recording and reproducing device, and audio data recording method, audio data reproducing method and audio data recording and reproducing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve more complicated image processing while reducing an information processing amount in reproduction in a device having a function of displaying a image matched to music as one of the additional functions in music reproduction. <P>SOLUTION: Unlike a conventional method, the music data are not analyzed concurrently with reproduction processing for music data, instead, music data analysis information is prepared and is stored on an analysis information storage section 16 in storing the music data as a music content file on the HDD 14 into an audio data recording and reproducing device 1. Consequently, the image synchronized with the music is generated to reduce the information processing amount in reproduction. The music data analysis information includes the quickness of a musical composition (BPM), time-spaced feature position (key point) such as a change in a bar in the musical composition, characteristic frequency information such as a base line and cymbals. The music data analysis information is analyzed in advance for preparation, and is distributed together with a music file in music delivery using a network 3 or the like. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an audio data recording device, an audio data reproducing device, an audio data recording / reproducing device, an audio data recording method, an audio data reproducing method, and an audio data recording / reproducing method, and according to music as an additional function during music reproduction. The present invention relates to an audio data recording apparatus, an audio data reproducing apparatus, an audio data recording / reproducing apparatus, an audio data recording method, an audio data reproducing method, and an audio data recording / reproducing method having a function of displaying a graphic.

  With the widespread use of digitization technology, various content data including music data, video data, and the like have been digitally recorded or reproduced on a recording medium. If an information processing device such as a personal computer (PC) is used, music content and video content downloaded from a recording medium such as a CD (Compact Disc) or DVD (Digital Versatile Disc) are ripped, dubbed or downloaded via a network. It can be recorded on a recording medium and managed. In recent years, the capacity of PC hard disks has been increasing, and users have been able to manage enormous content on PCs and the like.

  A personal computer (PC) can reproduce music content recorded on a recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc). As an additional function in music reproduction, for example, music Some also have a “visualizer” function that displays graphics. In this case, graphic processing in which the shape and color change in synchronization with music is performed using the analysis result obtained by performing frequency analysis of audio data during reproduction. As a conventional technique for converting sound into an image, a technique is known in which the movement of the sound of each part of a song corresponds to the movement of an image having a basic shape (see, for example, Patent Document 1). .

  Here, as an example, a content playback apparatus capable of storing and playing back music content as a file is shown. As a matter of course, this apparatus includes a display unit for displaying a graphic. FIG. 6 schematically shows a music playback mechanism in the content playback apparatus 100.

  The content reproduction apparatus 100 includes a drive 102 that reproduces music data recorded on a CD 101, an encoder 103 that converts the reproduced music data into a format for accumulating the music data, and an HDD (hard disk drive) 104 that accumulates the music data. A decoder 105 that decodes music data stored in the HDD 104, an analysis processing unit 106 that performs frequency analysis or the like on the decoded signal, and an acoustic reproduction processing unit 107 that outputs the analyzed music data , A D / A conversion circuit 108 and a speaker 109. In addition, a visualizer 110 that generates display image parameters according to the analysis result, an image reproduction processing unit 111 for outputting an image, a display device driver 112, and a display panel 113 are provided. These are comprehensively controlled by the control unit 114.

  The content reproduction apparatus 100 converts music data input from the CD 101 into a format to be recorded on the HDD 104 by the encoder 103, and generally compresses music data analysis information at the same time. The music data is stored in the HDD 104 as a content file. At the time of reproduction, the content file recorded in the HDD 104 is decompressed into music data by the decoder 105. The music data expanded by the decoder 105 is sent to the sound reproduction processing unit 107 and the analysis processing unit 106. The music data sent to the analysis processing unit 106 is analyzed and then sent to the visualizer 110. The visualizer 110 generates a graphic that matches the music data based on the analysis result, and sends the graphic to the video reproduction processing unit 111. The video reproduction processing unit 111 displays a graphic according to the music data on the display panel 113 via the display device driver 112.

  Furthermore, in recent years, an operation database that records the motion of a joint object, a pitch detection unit that detects a pitch, a volume detection unit that detects a volume, and a motion generation unit that extracts a motion data from the motion database and generates a motion are provided. A music image conversion device has also been proposed in which an image having more specificity than the technique disclosed in Patent Document 1 is moved in synchronization with music according to changes in the volume and pitch of music data. (For example, refer to Patent Document 2).

  However, since the graphic processing performed in conjunction with the music playback processing requires real-time properties, depending on the processing capability of this playback device, there may be a time lag between the video and the sound, or the sound may be cut off. There is. Conventionally, in order to prevent this, particularly when the decoder 105, the analysis processing unit 106, and the visualizer 110 are implemented as software, these processings can be sufficiently covered by the processing capability of the control unit 114. It was necessary to hold down to the amount. Therefore, only comparatively light analysis processing can be performed, and the analysis information obtained is limited. In particular, the analysis process generally has a large amount of processing, and if the analysis process such as frequency analysis is made heavy in order to complicate the graphic process, a graphic delay occurs with respect to the reproduced sound depending on the performance of the PC. , It was difficult to synchronize graphics with music, and sometimes the sound was interrupted. In addition, since the number of processes executed during music content playback increases, the music playback process may be affected.

JP-A 63-184875 JP-A-8-293039

  The present invention has been made in view of the above-described conventional situation, and in an apparatus having a function of displaying an image according to music as one of additional functions at the time of music playback, the amount of information processing at the time of playback is reduced. However, an object of the present invention is to provide an audio data recording apparatus and audio data recording method, and an audio data reproducing apparatus and audio data reproducing method that enable more complicated image processing.

  An audio data recording apparatus according to the present invention includes an input means for inputting music data, a music data analysis means for analyzing the input music data and extracting music data analysis information, and a music data storage means for storing music data. And an analysis information storage means for storing the music data analysis information extracted from the music data, whereby the music data analysis means extracts music data analysis information representing the characteristics of the music data, and the music data storage means stores the music data in the music data storage means. And the music data analysis information is recorded in the analysis information storage means. The audio data recording apparatus includes a reproducing unit that reproduces music data recorded on a recording medium, and the music data analyzing unit extracts music data analysis information from the music data reproduced by the reproducing unit. Further, there is a case where network connection means for connecting to a network is provided, and music data is analyzed for music data acquired via the network.

  The recording method corresponding to the audio data recording apparatus according to the present invention includes an input process of inputting music data, a music data analysis process of analyzing the input music data and extracting music data analysis information, and the music data as music data. A music data accumulating step for accumulating in the accumulating means; and a storing step for storing the music data analysis information extracted from the music data in the analysis information storing means.

  An audio data reproducing apparatus according to the present invention includes music data accumulating means for accumulating music data, music data reproducing means for reproducing music data, and analysis information storing means for storing music data analysis information extracted from the music data. And an image display processing means for displaying an image in accordance with the music data, and an image reproduction control means for changing the image information of the image in accordance with the music data analysis information. The image information of the displayed image is changed according to the music data analysis information representing the characteristics of the music data.

  A reproduction method corresponding to the audio data reproduction apparatus according to the present invention includes a step of reproducing music data stored in the music data storage means, an image display step of displaying an image in accordance with the reproduced music data, An image reproduction control step of changing the image information of the image according to the music data analysis information extracted from the music data and stored in the analysis information storage means.

  Further, as another form of the audio data reproducing apparatus, input means for inputting music data divided in advance and music data analysis information, music data reproducing means for reproducing music data, and music data according to the music data Image display processing means for displaying an image, and image reproduction control means for changing the image information of the image in accordance with the music data analysis information. In this case, the audio data reproducing apparatus is provided with network connection means for connecting to the network, and the music data and the music data analysis information are acquired via the network.

  A reproduction method corresponding to another embodiment of the audio data reproduction apparatus includes an input step of inputting music data and music data analysis information obtained by analyzing the music data, a music data reproduction step of reproducing the music data, And an image display process for displaying an image according to the music data, and an image reproduction control process for changing the image information of the image according to the music data analysis information.

  The audio data recording / reproducing apparatus according to the present invention includes an input means for inputting music data, a music data analysis means for analyzing the input music data and extracting music data analysis information, and music data for storing music data. Storage means; analysis information storage means for storing music data analysis information extracted from music data; music data reproduction means for reproducing music data; image display processing means for displaying an image in accordance with the music data; And an image reproduction control means for changing the image information of the image according to the data analysis information, whereby the music data analysis means extracts music data analysis information representing the characteristics of the music data, and the music data is stored in the music data storage means. Accumulate and record music data analysis information in the analysis information storage means. The audio data recording / reproducing apparatus includes reproducing means for reproducing music data recorded on a recording medium, and the music data analyzing means extracts music data analysis information from the music data reproduced by the reproducing means. Further, there is a case where network connection means for connecting to a network is provided, and music data is analyzed for music data acquired via the network.

  In addition, this method includes an input process of inputting music data, a music data analysis process of analyzing the input music data and extracting music data analysis information, and analyzing the music data analysis information extracted from the music data as analysis information A storage step of storing in the storage means; a music data storage step of storing music data in the music data storage means; a music data playback step of playing back the music data; and an image display step of displaying an image in accordance with the music data; And an image reproduction control step of changing the image information of the image according to the music data analysis information.

  In particular, in the present invention, the music data analysis information includes at least one of music tempo, bar break, time break, and characteristic frequency information, and the music data has a time break position and a characteristic frequency position. Information serving as an index is added and stored in the analysis information storage means.

  According to the present invention, a music analysis process with a large amount of processing is generally performed at the stage of creating a content file, so that a more complicated analysis can be performed than a conventional method that is played back simultaneously with the analysis process. And the amount of processing during playback can be reduced. Thereby, troubles such as sound interruption during reproduction can be prevented. Also, by extracting and storing music information such as music tempo, contents can be classified according to the characteristics of the music, entertainment can be improved, and a new service can be proposed to the user.

  The present invention is an apparatus having a function of displaying an image in accordance with music as one of the additional functions at the time of music playback, and does not analyze the music data together with the music data playback processing as in the prior art. When storing as a music content file in a recording medium in the apparatus, by creating music data analysis information, it is possible to perform more complicated image processing while reducing the amount of information processing during playback. The music data analysis information includes tempo (BPM) of music, temporal characteristic positions (key points) such as changes in bars in the music, characteristic frequency information such as baseline and cymbal, and the like. This music data analysis information can be analyzed in advance and prepared separately from the music data, and can be distributed together with the music file when music distribution is performed using a network or the like.

  Hereinafter, an audio data recording / reproducing apparatus shown as a specific example of the present invention will be described in detail with reference to the drawings. FIG. 1 shows an audio data recording / reproducing apparatus 1.

  The audio data recording / reproducing apparatus 1 includes a drive 11 for reproducing music data recorded on the recording medium 2 and a network interface (I / F) 12 for connecting to the network 3, and can acquire music data from the outside. It is like that.

  The audio data recording / reproducing apparatus 1 also includes an encoder 13 that encodes the reproduced music data into a format for storing the music data, an HDD (hard disk drive) 14 that stores the encoded music data as a music file, Performs frequency analysis etc. on the music data and analyzes it to obtain the tempo (BPM) of the music, the temporal feature positions (key points) such as changes in bars in the music, and the characteristic frequency information such as the baseline and cymbal. The music data analysis part 15 which extracts the music data analysis information to include, and the analysis information storage part 16 which memorize | stores the music data analysis information obtained as a result of the analysis are provided.

  Further, the audio data recording / reproducing apparatus 1 has a decoder 17 for decoding the music data stored in the HDD 14 and a sound reproduction for outputting the music data as a structure for reproducing the music and reproducing the image in accordance with the music reproduction. A processing unit 18, a D / A conversion circuit 19, and a speaker 20 are provided. In addition, an image reproduction control unit 21 having a visualizer function for changing image information (parameters) of an image to be displayed according to music data analysis information, an image reproduction processing unit 22 that performs processing for outputting an image, and a display A device driver 23 and a liquid crystal display panel (LCD) 24 as a display unit are provided. Each of these components is centrally controlled by a control unit 25 including a CPU, a ROM storing processing programs, and a RAM as a work area of the CPU.

  The audio data recording / reproducing apparatus 1 manages music contents by ripping or dubbing music contents from a recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc) to an HDD 14 incorporated in the information processing apparatus. It is possible to manage music content provided by a music distribution service (EMD: Electronic Music Distribution) using a network such as

  As an encoding process in the encoder 13, for example, DPCM (Differential Pulse Code Modulation), ADPCM (Adaptive Differential Pulse Code Modulation), ATRAC (Adaptive TRansform Acoustic Coding) (registered trademark) standard, MPEG (Moving Picture Experts Group) / Audio General purpose high efficiency encoding such as standard audio signal encoding can be used. In this specific example, ATRAC is used.

  In FIG. 1, the recording medium 2 includes a CD-ROM, a CD-R (Compact Disc-Recordable), a CD-RW (Compact Disc-Rewritable), a DVD that realizes a larger storage capacity, and a DVD. DVD-R, DVD + RW, and DVD-R / W that can be recorded. Here, it is CD. Examples of the network 3 include a LAN (Local Area Network) connected by wireless or wired connection, a network such as a so-called Internet.

  The audio data recording / reproducing apparatus 1 compresses music data input from the recording medium 2 or input from the network 3 in the ATRAC format by the encoder 13 and stores it in the HDD 14 as a content file. At this time, the music data analysis unit 15 analyzes the music data analysis information, and the analysis result is stored in the analysis information storage unit 16 as an analysis information storage file. Note that the analysis information storage unit 16 and the HDD 14 may not be prepared separately. For example, the analysis information storage 16 may be a storage area virtually provided on the HDD 14.

  At the time of reproduction, the content file recorded in the HDD 14 is decoded by the decoder 17, and the music data decoded by the decoder 17 is sent to the image reproduction control unit 21. The image reproduction control unit 21 generates image information based on the music data analysis information stored in the analysis information storage unit 16 in association with the music data, and sends the image information to the video reproduction processing unit 22. The video reproduction processing unit 22 performs reproduction processing for reproducing an image with image information generated based on the music data analysis information while synchronizing with the music data. As a result, a graphic adapted to the music data is displayed on the display panel 24 via the display device driver 23.

  Next, analysis of music data will be described. ATRAC divides a digital audio signal into coding units that are delimited at regular intervals, examines the distribution of waveform components by orthogonal transformation from the signal waveform in each delimited unit, and eliminates signals that are unnecessary for hearing. Thus, this is an encoding method that rationalizes the data amount by extracting only 0.3 Mbit of a signal of a level necessary for auditory per unit. The music data analysis unit 15 can use information obtained when the music data is encoded by the encoder 13 in the analysis of the music data.

  The tempo of the music is obtained, for example, by detecting a drastic change (attack sound) in the characteristics of the acoustic signal by the rising or falling edge of the acoustic signal. The tempo of the music can be calculated from the period of the acoustic signal extracted characteristically like the attack sound and the time information of the content. In addition, frequency information having characteristic signal characteristics such as a baseline or a cymbal is extracted as an attack sound. This music data analysis information can be analyzed in advance and prepared separately from the music data, and can be distributed together with the music file when music distribution is performed using a network or the like.

  When the tempo changes in one piece of music, the changing time position is extracted together with the tempo. FIG. 2 shows tempo information, which is one piece of music data analysis information extracted as an analysis information file. This analysis information file is stored corresponding to each content.

  In addition, the music data analysis information includes a bar break and a time break in the music. The bar breaks and the time breaks are information that can be used to realize additional functions during music playback. In this specific example, the bar breaks and the temporal breaks (key points) are determined as shown in FIG.

For example, in this specific example based on ATRAC, the digital audio signal to the orthogonal transformation signals over a period of time T ₁ examine the distribution of waveform components. Also the same processing for another period T _2, examining the distribution of waveform components. For these two arbitrary periods, a sum of characteristic frequency spectra (x _1or2 = Σ (characteristic frequency spectrum)) is further obtained for a predetermined frequency band, and the two are compared. When this is larger than the threshold A, it is assumed that the change in the acoustic signal is characteristic and is a key point (Equation (1)). Further, the characteristic acoustic signal extracted as the attack sound described above is extracted as a key point as it is.

| X ₁ −x ₂ |> A (1)

  However, this fixed section does not necessarily correspond to the encoding unit in view of the bar breaks and the temporal breaks (key points). The music data analysis unit 15 adds information serving as an index to the time delimiter position and the characteristic frequency position to the music data and stores them in the analysis information storage unit 16.

  The music data analysis information analyzed as described above is stored in the analysis information storage unit 16 as an analysis information file in association with the music data.

  FIG. 4 shows an analysis information file of music data stored in the analysis information storage unit 16. Thus, by storing the file storing the music data analysis information in association with the music data, the contents can be classified based on the music data analysis information. For example, as shown in FIG. 4, songs extracted as a result of analysis having a tempo of 132 BPM can be classified as the same category A. As a result, classification by “tempo” can be performed in addition to classification by “artist”, “genre”, and the like. For example, there is a method of describing additional information for content information by text data or the like as a part of the data format, but in this specific example, since it can be classified based on analysis information extracted directly from music data, Suggest new ways to enjoy music.

  As an example, the content classified into the category of the same tempo can be continuously reproduced without interruption between songs by connecting and reproducing the content by crossfading. In addition to this operation, if images are connected by crossfading, the images can be synchronized with the switching of music.

  In addition, when the above-described key points are added to the music data analysis information, if the playback image is changed at the key points, the image can be displayed at the key point position only by playing the music data. And a predetermined process can be performed on the image. Thereby, when music data is reproduced, an image as if it has been edited in advance based on the music data can be displayed with a low load processing amount.

  Next, another specific example of the present invention will be described with reference to FIG. This is an example in which the audio data recording / reproducing apparatus 1 performs streaming reproduction of music content transmitted via a network. As shown in FIG. 5, music data of music content and music data analysis information related to the music data are prepared in advance as analysis information files in the server 4. The server 4 includes, for example, a configuration corresponding to the encoder 13 and the music data analysis unit 15 in the audio data recording / playback apparatus 1, and can extract music data analysis information of each music content by performing the same processing as described above. it can. The server 4 transmits to the audio data recording / reproducing apparatus 1 the music data and analysis information file of the music content requested for distribution from the audio data recording / reproducing apparatus 1 via the network 3. At this time, the server 4 transmits the analysis information file before the music data.

  Therefore, in this specific example, since the music data to be transmitted can be reproduced sequentially, the image processed using the music data analysis information prepared in the server 4 can be reproduced in synchronization with the music data. Processing in the audio data recording / reproducing apparatus 1 during reproduction is greatly reduced. Preparing and providing music data analysis information separately from the music data main body can be applied not only for streaming delivery but also for providing music data on a recording medium such as a CD or DVD. .

  Further, when key points are added to the music data analysis information as described above, for example, the designated URL is accessed in accordance with the tempo change point or key point indicated in the analysis information file. Image display such as acquiring image data and displaying it at the next key point and predetermined processing can be performed on the image. Thereby, when music data is reproduced, an image as if it has been edited in advance based on the music data can be displayed with a low load processing amount.

  As described above, according to the audio data recording / reproducing apparatus 1, generally, analysis processing of music data having a large amount of processing is performed at the stage of creating a music content file, so that it has been reproduced simultaneously with the analysis processing. More complicated analysis processing can be performed than this method. Also, since the audio data recording / reproducing apparatus 1 does not perform analysis processing during reproduction, the processing amount during content reproduction can be greatly reduced. As a result, it is possible to prevent troubles such as sound interruption during reproduction caused by an increase in processing amount. In addition, by extracting and storing music information such as music tempo, it is possible to classify the contents according to the characteristics of the music, and this classification can be applied to various ways of content selection and playback, improving entertainment. To do.

  The present invention can be applied to any application in which an additional function is operated in synchronization with music data analysis information representing characteristics of music data extracted at the stage of recording on a recording medium. For example, when displaying photo data using the slide function, the process for switching the photos is reduced by matching the timing of switching the photos to the feature points of the extracted music data, and the photos are natural and smooth according to the music. Switching processing can be realized.

It is a block diagram explaining the audio data reproducing | regenerating apparatus shown as a specific example of this invention. It is a figure explaining the tempo information which is one of the music data analysis information extracted as an analysis information file. It is a figure explaining the example of extraction of a bar break and a time break (key point) in an audio data recording / reproducing apparatus. It is the figure which showed typically the example of the analysis information file memorize | stored in the analysis information storage part. It is a schematic diagram explaining the example which carries out the streaming reproduction of the music content sent via a network with an audio data recording / reproducing apparatus. It is a block diagram explaining the conventional content reproduction apparatus.

Explanation of symbols

1 audio data recording / reproducing device, 2 recording medium, 3 network,
11 Driver, 12 Network I / F, 13 Encoder,
14 HDD, 15 music data analysis unit, 16 analysis information storage unit,
17 decoder, 18 sound reproduction processing unit, 19 D / A conversion circuit,
20 speakers, 21 visualizers, 22 image playback processing units,
23 Display device driver, 24 Liquid crystal display panel, 25 Control unit

Claims (19)

Input means for inputting music data;
Music data analysis means for analyzing the input music data and extracting music data analysis information;
Music data storage means for storing the music data;
An audio data recording device comprising: analysis information storage means for storing music data analysis information extracted from the music data. A reproducing means for reproducing the music data recorded on the recording medium;
2. The audio data recording apparatus according to claim 1, wherein the music data analysis means extracts music data analysis information from music data reproduced by the reproduction means. Network connection means for connecting to the network,
2. The audio data recording apparatus according to claim 1, wherein the music data analysis means extracts music data analysis information from music data received from the network connection means.   2. The audio data recording according to claim 1, wherein the music data analysis means extracts music data analysis information including at least one of music tempo, bar break, time break, and characteristic frequency information. apparatus.   2. The audio data recording apparatus according to claim 1, wherein the music data analysis means adds information serving as an index to the music data at a time delimiter position and a characteristic frequency position and stores the information in the analysis information storage means. . Music data storage means for storing music data;
Analysis information storage means for storing music data analysis information extracted from the music data;
Music data playback means for playing back the music data;
Image display processing means for displaying an image in accordance with the music data;
An audio data reproduction apparatus comprising: an image reproduction control unit that changes image information of the image according to the music data analysis information.   7. The audio data reproduction according to claim 6, wherein the music data analysis means extracts music data analysis information including at least one of music tempo, bar break, time break, and characteristic frequency information. apparatus.   7. The audio data reproducing apparatus according to claim 6, wherein the music data analyzing means adds information serving as an index to the music data at a time delimiter position and a characteristic frequency position and stores the information in the analysis information storage means. . Input means for inputting music data and music data analysis information obtained by analyzing the music data;
Music data playback means for playing back the music data;
Image display processing means for displaying an image in accordance with the music data;
An audio data reproduction apparatus comprising: an image reproduction control unit that changes image information of the image according to the music data analysis information. Network connection means for connecting to the network,
10. The audio data reproducing apparatus according to claim 9, wherein the music data and music data analysis information are acquired via a network. Input means for inputting music data;
Music data analysis means for analyzing the input music data and extracting music data analysis information;
Music data storage means for storing the music data;
Analysis information storage means for storing music data analysis information extracted from the music data;
Music data playback means for playing back the music data;
Image display processing means for displaying an image in accordance with the music data;
An audio data recording / reproducing apparatus comprising: an image reproduction control unit that changes image information of the image in accordance with the music data analysis information. A reproducing means for reproducing the music data recorded on the recording medium;
12. The audio data recording / reproducing apparatus according to claim 11, wherein the music data analyzing means extracts music data analysis information from the music data reproduced by the reproducing means. Network connection means for connecting to the network,
12. The audio data recording / reproducing apparatus according to claim 11, wherein the music data analysis means extracts music data analysis information from music data received from the network connection means.   12. The audio data recording according to claim 11, wherein the music data analysis means extracts music data analysis information including at least one of music tempo, bar break, time break, and characteristic frequency information. Playback device.   12. The audio data recording / reproducing apparatus according to claim 11, wherein the music data analysis means adds information serving as an index to the music data at a time delimiter position and a characteristic frequency position and stores the information in the analysis information storage means. apparatus. An input process for inputting music data;
A music data analysis process for analyzing the input music data and extracting music data analysis information;
A music data storage step of storing the music data in a music data storage means;
And a storage step of storing in the analysis information storage means music data analysis information extracted from the music data. Reproducing the music data stored in the music data storage means;
An image display step for displaying an image in accordance with the music data to be reproduced;
An audio data reproduction method comprising: an image reproduction control step of changing image information of the image in accordance with music data analysis information extracted from the music data and stored in analysis information storage means. An input step of inputting music data and music data analysis information obtained by analyzing the music data;
A music data playback process for playing back the music data;
An image display step for displaying an image according to the music data;
And an image reproduction control step of changing the image information of the image in accordance with the music data analysis information. An input process for inputting music data;
A music data analysis process for analyzing the input music data and extracting music data analysis information;
A storage step of storing music data analysis information extracted from the music data in the analysis information storage means;
A music data storage step of storing the music data in a music data storage means;
A music data playback process for playing back the music data;
An image display step for displaying an image according to the music data;
And an image reproduction control step of changing the image information of the image according to the music data analysis information.

Images