JP4687297B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus and the like for performing switching display of image data synchronized with music reproduced from music data.

  There is a so-called “slide show” technique in which image data (video data) is switched and displayed in synchronization with reproduction of music data. As a technique related to this slide show, for example, the following is known.

  In Patent Literature 1, rhythm analysis processing of music data including frequency analysis and pattern analysis is performed to generate timing information including timing information for performing image switching and image processing, and then each timing of the analyzed music data A technique is disclosed in which link information obtained by linking a series of image data is generated and an image that matches the rhythm of the music is reproduced by referring to the generated link information.

  Patent Document 2 discloses a technique for extracting a tempo based on the strength and frequency of music data (musical sound data) and setting a slide effect time such as fade-in / fade-out of image display in accordance with the extracted tempo. It is disclosed. In the case of such technology, the tempo of music and the slide effect are increased by increasing the slide effect time for fast-tempo music (music) and conversely for the slow-tempo music. And consistency.

In Patent Document 3, predetermined music changes such as fade-in / fade-out and tempo change are detected from the contents of music data, and based on the detected music changes, fade-in, fade-out, cross-fade, mosaic, A technique is disclosed in which predetermined video effects such as wipe, flash, and color tone change are added to image data (video data).
JP 2000-148107 A JP 2003-249051 A JP 2004-240077 A

  As described above, in the conventional slide show, the rhythm of the music data to be reproduced is analyzed, and display control of the image data is performed based on the analyzed rhythm. However, depending on the music data, there is a part that makes rhythm analysis difficult. In such a portion where the rhythm is not accurately analyzed, there is a problem that the reproduction of the image data becomes unnatural reproduction that does not match the tempo and timing of the music.

  In view of the above circumstances, an object of the present invention is to enable natural switching display of image data even in a portion where accurate rhythm analysis cannot be performed.

In order to solve the above-mentioned problem, the invention described in claim 1
Image data storage means for storing image data (for example, the RAM 60 of FIG. 1);
Music data storage means for storing music data (for example, ROM 50 in FIG. 1);
Detection means (for example, the CPU 10 in FIG. 1) for detecting the rhythm of the music reproduced from the music data stored in the music data storage means;
A determination means (for example, the CPU 10 in FIG. 1) for determining reliability indicating the appropriateness of the rhythm at each timing based on the rhythm detected by the detection means;
By switching and displaying the image data stored in the image data storage means based on the rhythm detected by the detection means, a switching display synchronized with the reproduction of the music data stored in the music data storage means is performed. And a switching display control means for controlling the switching display mode to be different between a portion in which the reliability determined by the determination means in the music data satisfies a predetermined low reliability condition and a portion that does not satisfy the predetermined low reliability condition. (For example, CPU 10 in FIG. 1);
An image processing apparatus (for example, the image processing apparatus 1 in FIG. 1).

The invention according to claim 2 is the image processing apparatus according to claim 1,
The determining means determines a value determined based on an autocorrelation value of music data at each timing as a reliability.

The invention according to claim 3 is the image processing apparatus according to claim 1 or 2,
The switching display control means performs switching display with a slow screen rewriting speed when reproducing a portion of the music data in which the reliability determined by the determination means satisfies a predetermined low reliability condition. And

The invention according to claim 4 is the image processing apparatus according to claim 1 or 2,
The switching display control means reduces the switching display frequency of image data during reproduction of a portion of the music data in which the reliability determined by the determination means satisfies a predetermined low reliability condition. And

The invention according to claim 5 is the image processing apparatus according to claim 3,
The switching display control unit is configured to display a screen for a portion of the music data in which the reliability determined by the determination unit does not satisfy the low reliability condition and the sound pressure level satisfies a predetermined low level condition. It is characterized by switching display with a slow rewrite speed.

The invention according to claim 6 is the image processing apparatus according to claim 4,
The switching display control means switches the portion of the music data in which the reliability determined by the determination means does not satisfy the low reliability condition and the sound pressure level satisfies a predetermined low level condition. The display frequency is reduced.

The invention described in claim 7
A program for causing a computer to switch and display image data synchronized with music that has been reproduced from music data (for example, the synchronous reproduction program 51 in FIG. 1),
A detection function (for example, steps S33 to S41 in FIG. 11) for detecting the rhythm of the music from which the music data is reproduced;
A determination function (for example, step S43 in FIG. 11) for determining reliability indicating the rhythm accuracy at each timing based on the rhythm detected by the detection function;
By switching and displaying the image data based on the rhythm detected by the detection function, switching display synchronized with the reproduction of the music data is performed, and among the music data, the determination unit determines A switching display control function (for example, steps S9 to S15 in FIG. 10) for controlling the switching display mode to be different between a portion where reliability satisfies a predetermined low reliability condition and a portion where reliability is not satisfied ;
Is a program for causing the computer to realize the above.

  According to the first or seventh aspect of the invention, the rhythm of the music from which the music data is reproduced is detected, and the reliability indicating the rhythm accuracy at each timing based on the detected rhythm is determined. Then, based on the detected rhythm, switching display synchronized with music data reproduction is performed, and the mode of switching display is controlled based on the determined reliability. That is, since the image data switching display mode is controlled based on the reliability of the detected rhythm, even if the music data includes a portion where the rhythm is not properly detected, the natural image data A switching display can be realized.

  According to the invention described in claim 2, the reliability is determined based on the autocorrelation value of the music data at each timing based on the detected rhythm.

  According to the third aspect of the present invention, when the portion of the music data in which the determined reliability satisfies the predetermined low reliability condition is reproduced, switching display with a slow screen rewriting speed is performed. Therefore, for example, when the detected rhythm deviates from the original rhythm, this deviation causes the image data switching display to deviate from the original rhythm, but the reliability satisfies the low reliability condition, that is, the rhythm is For parts that are determined not to be detected properly, screen rewriting is performed more slowly than other parts, making the user feel the image data rewriting display shift due to the detected rhythm shift. And a natural switching display can be realized.

  According to the fourth aspect of the present invention, when reproducing a portion of the music data in which the determined reliability satisfies the predetermined low reliability condition, the switching display frequency of the image data is reduced. Therefore, for example, frequent switching of image data in music with a slow tempo will give the user a sense of incongruity, but the switching frequency of the image data is reduced for portions where the rhythm is determined not to be detected properly. In other words, since the image data is not frequently switched compared to other parts, the natural image data can be reproduced without giving the user a sense of incongruity that the image data is frequently switched even though the music is slow in tempo. Switching display is realized.

  According to the fifth aspect of the present invention, a portion of the music data in which the determined reliability does not satisfy the predetermined low reliability condition and the sound pressure level satisfies the predetermined low level condition is displayed on the screen. Switching display with slow rewriting speed is performed. Therefore, even if it is determined that the rhythm is accurately detected, the sound pressure level satisfies the low level condition, that is, the portion where the sound pressure level is low, the screen rewriting is performed slowly. For example, a more natural image data switching display is realized in which a slow screen rewrite changes to a quick rewrite in a portion where the volume greatly changes from a low volume to a high volume.

  According to the invention described in claim 6, a portion of the music data in which the determined reliability does not satisfy the predetermined low reliability condition and the sound pressure level satisfies the predetermined low level condition, Data switching display frequency is reduced. Therefore, even if it is determined that the rhythm is properly detected, the image data is not frequently switched compared to the other portions for the portion where the sound pressure level is low. Natural image data switching display is realized without giving the user a sense of incongruity that the image data is frequently switched at a low volume.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[Configuration of image processing apparatus]
FIG. 1 is a block diagram of an image processing apparatus 1 according to this embodiment. The image processing apparatus 1 performs switching display of image data in synchronization with the reproduction of music data. According to the figure, the CPU 10, the input unit 20, the display unit 30, and the audio output unit 40 are displayed. The ROM 50 and the RAM 60 are provided.

  The CPU 10 reads out a program stored in the ROM 50 at a predetermined timing and develops it in the RAM 60, and performs instructions, data transfer, and the like to each unit constituting the image processing apparatus 1 based on the program. In particular, in the present embodiment, a synchronized playback process (see FIG. 10) according to the synchronized playback program 51 is executed.

  Specifically, in the synchronous playback process, the CPU 10 first performs a rhythm detection process on the music data 62 stored in the RAM 60, and uses the data relating to the detected rhythm and its reliability. A certain rhythm data 67 is generated.

  Specifically, in the rhythm detection process, the CPU 10 divides the music data 62 into a plurality of frames as shown in FIG. FIG. 2 is a diagram showing a part of a speech waveform based on the music data 62. As shown in the figure, the CPU 10 divides the audio waveform based on the music data 62 into a plurality of frames F1, F2,... Here, the time M and the frame length N may be set appropriately. For example, M = 0.5 [seconds] and N = 4 [seconds] are set. However, M <N.

  Next, the CPU 10 derives an autocorrelation function of the audio signal of each divided frame. Specifically, for each frame, the audio signal is passed through a low-pass filter to block high-frequency components and extract low-frequency components. Next, an FFT (Fast Fourier Transform) operation is performed on the extracted low-frequency component signal to calculate a Fourier spectrum, and further squared to calculate a power spectrum. Then, an inverse FTT operation is performed on the calculated power spectrum to derive an autocorrelation function of the audio signal of the frame.

  FIG. 3 is a graph showing an example of the autocorrelation function for one frame. In the figure, the horizontal axis represents the number of samples, that is, time, and the vertical axis represents the correlation function value.

  Further, as shown in FIG. 4, the horizontal axis of this graph is converted into BPM (Beat Per Minute) which is a unit representing the tempo of the music. FIG. 4 is a graph showing an example in which the horizontal axis of the autocorrelation function graph is converted from the number of samples into BPM. Conversion from the number of samples to BPM is performed as follows. That is, assuming that the length N of one frame is “N [seconds]” and the number of samples in deriving the autocorrelation function is “X”, one sample is at time t = M / X [seconds]. Equivalent to. That is, the x-th sample corresponds to time T = t × x [seconds], which is expressed as BPM as 60 / T = 60 / (t × x). For example, when N = 4 [seconds] and X = 2052, one sample corresponds to time t = 0.0001949 [seconds] (= 4/2052). The autocorrelation function for each frame with the horizontal axis as BPM is stored in the RAM 60 as frame autocorrelation data 63.

  Subsequently, the CPU 10 calculates a BPM candidate value for each frame based on the derived autocorrelation function for each frame. Specifically, for each frame, within a predetermined BPM range of the derived autocorrelation function, a predetermined number (for example, 5) is selected in order from the peak value of the autocorrelation value in the maximum. Then, the BPM value corresponding to each selected peak value is set as the BPM candidate value in the frame. The predetermined BPM range is set to, for example, about “50 to 200” which is a BPM range of general music.

  The calculated BPM candidate value is stored in the RAM 60 as frame analysis data 64 together with the corresponding peak value. FIG. 5 shows an example of the data structure of the frame analysis data 64. According to the figure, the frame analysis data 64 stores a plurality of (5 in the figure) BPM candidate values 64b and peak values 64c in association with each frame 64a obtained by dividing the music data 62.

  When the BPM candidate values are calculated for each frame, the CPU 10 groups the calculated BPM candidate values so as to form a group of approximate values. Then, for each group, an average value of BPM candidate values and a cumulative value of peak values are calculated.

  The BPM candidate values grouped here are stored in the RAM 60 as BPM determination data 65. FIG. 6 shows an example of the data configuration of the BPM determination data 65. According to the figure, the BPM determination data 65 stores the BPM candidate value 665 and the peak value 65c belonging to each group 65a in association with each other, and the average 65d of the BPM candidate value 65b and the peak value 65c. Is stored.

  This figure also shows a case based on the frame analysis data 64 shown in FIG. 5, in which a BPM candidate value close to “62” is grouped, and a BPM candidate value close to “123” is grouped. The second group is grouped into a third group obtained by grouping BPM values close to “246”.

  Next, the CPU 10 determines a group having the largest accumulated peak value from among the plurality of groups, and determines an average value of BPM candidate values of the determined group as the BPM of the music data 62. For example, in the case of FIG. 6, the second group having the largest accumulated peak value “287.1” is determined, and the average value “123.8” of the BPM candidate values of the second group is determined as the BPM of the music data 62. It is determined. The BPM determined here is stored in the RAM 60 as BPM determined value data 66.

  When the BPM of the music data 62 is determined, the CPU 10 subsequently calculates the beat interval L based on the determined BPM. The beat interval L is given by the reciprocal of BPM. For example, when BPM is “120”, the beat interval L is L = 0.5 (= 60 [s] / 120).

  Subsequently, the CPU 10 determines the beat position R in the music data 62. Specifically, with reference to the frame autocorrelation data 63, one frame including a peak value having the maximum sound pressure level (amplitude of the speech waveform) through all the frames among a plurality of frames obtained by dividing the music data 62. Select. Next, as shown in FIG. 7, the position with the maximum sound pressure level in the selected frame is set as the reference position of the beat in the music data 62. FIG. 7 shows audio waveforms for a plurality of frames including the position of the maximum peak value based on the music data 62.

  Then, the CPU 10 determines the beat position R at the calculated beat interval L with respect to the determined beat reference position for the entire speech waveform based on the music data 62.

  Thereafter, the CPU 10 calculates a reliability indicating the accuracy of the rhythm at the determined position R of each beat. Specifically, for each beat position R, a frame including the beat position R is selected from a plurality of frames obtained by dividing the music data 62 as shown in FIG. FIG. 8 shows a part of a speech waveform based on the music data 62. Here, since the frames are generated while being shifted by time M, there are a plurality of frames as frames including the position R of a certain beat. For example, in the case of the figure, three frames Fn + 1, Fn + 2, and Fn + 3 are selected as frames including the beat position R.

  Then, the CPU 10 calculates an autocorrelation value corresponding to the determined BPM of the music data 62 for each selected frame, and multiplies the average value of the calculated autocorrelation values by a predetermined coefficient, The reliability at the position R is assumed.

  The reliability of the beat position R calculated in this way is associated with the beat position R and stored in the RAM 60 as rhythm data 67 corresponding to the music data 62. FIG. 9 is a diagram illustrating an example of the data configuration of the rhythm data 67. As shown in the figure, the rhythm data 67 stores a beat position 67a and a reliability 67b in association with each other. Here, the beat position 67a is expressed by a value in which the head position (reproduction start position) of the corresponding music data 62 is zero.

  When the rhythm detection process is finished, the CPU 10 starts to reproduce the music data 62, and outputs music based on the music data 62 from the audio output unit 40, and synchronizes with the music by referring to the generated rhythm data 67. Thus, an image based on the image data 61 is switched and displayed on the display unit 30.

  Specifically, each time the beat position R specified by the rhythm data 67 arrives, the reliability corresponding to the arrived beat position R is specified by the reliability threshold data 68. It is determined whether or not a certain threshold value is exceeded, and switching to the next image is displayed at a screen rewriting speed according to the determination result. The screen rewriting speed refers to the amount of screen rewriting per short time, and the faster the screen rewriting speed, the faster the next image is rewritten. In this embodiment, it is assumed that there are two screen rewriting speeds: a normal speed and a speed slower than the normal speed. That is, when the reliability exceeds the threshold, the screen display is switched with the screen rewriting speed set to the normal speed. When the reliability does not exceed the threshold value, the screen display is switched with the screen rewriting speed set as a slow speed.

  Here, as an example of a combination of screen rewriting at a normal screen rewriting speed and screen rewriting at a slower speed than normal, for example, there is a combination of full screen rewriting and fade-in / fade-out. Moreover, it is good also as implement | achieving normal rewriting speed and slow rewriting speed | velocity | rate by using the same screen rewriting technique, such as a dissolve and a wipe, and making time required for the completion differ in both.

  In FIG. 1, the input unit 20 includes, for example, a switch, a keyboard, a mouse, various sensors, and the like, and outputs an operation signal corresponding to an operation by a user to the CPU 10.

  The display unit 30 includes a monitor such as an LCD (Liquid Crystal Display), an ELD (Electro Luminescence Display), or a PDP (Plasma Display Panel), and an image according to an image signal input from the CPU 10, In the embodiment, images based on the image data 61 are switched and displayed according to the control of the CPU 10.

  The audio output unit 40 includes a D / A converter, an amplifier, a speaker, and the like, and outputs audio according to an audio signal input from the CPU 10, particularly music based on the music data 62 in the present embodiment. .

  The ROM 50 stores system programs and application programs for the image processing apparatus 1, programs and data for realizing the present embodiment, and the like. In particular, in this embodiment, a synchronized playback program 51 for realizing synchronized playback processing (see FIG. 10) is stored.

  The RAM 60 is used as a work area of the CPU 10 and temporarily stores programs and data read from the ROM 50 and data processed by the CPU 10. In particular, in the present embodiment, the image data 61, the music data 62, the frame autocorrelation data 63, the frame analysis data 64, the BPM determination data 65, the BPM determination value data 66, the rhythm data 67, the reliability Degree threshold data 68 is stored.

  The image data 61 is data composed of a plurality of image groups, for example, data read from an external information storage medium such as a memory card, or data taken from another device via a communication line such as the Internet. . The music data 62 is data read from an external information storage medium such as a CD, or data read from another device via the communication line described above.

[Process flow]
Next, the flow of processing in the image processing apparatus 1 will be described.
FIG. 10 is a flowchart for explaining the flow of the synchronous reproduction process. This process is realized by the CPU 10 executing a process according to the synchronized playback program 51 when a playback instruction is input from the input unit 20, for example.

  According to the figure, the CPU 10 first designates music data 62 and image data 61 to be reproduced in accordance with an input instruction from the input unit 20 (step S1). Then, rhythm detection processing for the designated music data 62 is executed to generate rhythm data 67 corresponding to the music data 62 (step S3).

  FIG. 11 is a flowchart for explaining the flow of the rhythm detection process. According to the figure, the CPU 10 divides the target music data 62 into a plurality of frames of length N shifted by time M (step S31).

  Next, an autocorrelation function is derived for the audio signal of each divided frame (step S33). Then, for each frame, a BPM candidate value and its peak value are calculated within a predetermined BPM range in the derived autocorrelation function (step S35). That is, as described above, a predetermined number of BPM values having a maximum peak value within a predetermined BPM range in the autocorrelation function of each frame are set as BPM candidate values.

  Subsequently, the CPU 10 determines the BPM of the music data 62 based on the calculated BPM candidate value and peak value of each frame (step S37). That is, as described above, the calculated BPM candidate values of each frame are grouped together to approximate each other, and a group having the maximum peak value is selected from the plurality of groups. Then, the average value of the BPM candidate values of the selected group is determined as the BPM of the music data 62.

  Thereafter, the CPU 10 calculates the beat interval L by calculating the reciprocal of the determined BPM (step S39). Then, the position where the sound pressure level is maximized in the frame including the maximum peak value is calculated as the reference position of the beat in the music data 62, and each beat has an interval L of beats with the reference position of the beat as a reference. The position R is calculated (step S41).

Subsequently, the CPU 10 calculates the reliability at the calculated position R of each beat (step S43). That is, as described above, for each beat position R, an average value of autocorrelation values corresponding to the BPM of the music data 62 in each frame including the beat position R is multiplied by a predetermined coefficient, The reliability of the beat position R is used. Then, the rhythm data 67 is generated by associating the position R of each beat with the calculated reliability.
When the above processing is performed, this rhythm detection processing is completed.

  When the rhythm detection process ends, the CPU 10 starts playing the designated music data 62 (step S5). Then, the next process is repeated until the music reproduction ends (step S7: NO).

  That is, when the timing of the beat position R specified by the rhythm data 67 arrives (step S9: YES), the CPU 10 determines that the reliability associated with the beat position R in the rhythm data 67 is the reliability threshold value. It is determined whether or not a threshold value specified by the data 68 is exceeded.

When the reliability exceeds the threshold (step S11: YES), the CPU 10 switches from the current display image displayed on the display unit 30 to the next image at a normal rewriting speed (step S13). If the reliability does not exceed the predetermined threshold (step S11: NO), the screen is switched to the next image at a screen rewriting speed slower than the normal speed (step S15).
Then, when the reproduction of the music data 62 is finished (step S7: YES), this synchronous reproduction process is finished.

[Concrete example]
Here, a specific example in the present embodiment will be described.
FIG. 12 is a graph showing rhythm data generated based on certain music data 62. The horizontal axis represents the elapsed time [seconds] from the start of reproduction of the music data 62, and the vertical axis represents the reliability. The reliability at the position R is plotted. In the figure, when the reliability threshold is “80”, the range of “40 to 100 [seconds]” in the first half of the music to be played, the vicinity of “310 [seconds]” in the second half, and just before the end In the vicinity of “430 [seconds]”, the reliability becomes smaller than this threshold value, and the image rewriting speed becomes slow.

[Action / Effect]
As described above, according to the present embodiment, the beat position R that is the timing of the rhythm of the music from which the music data 62 is reproduced is calculated by the rhythm detection process, and the accuracy of the detected rhythm at each beat position R is shown. The reliability is calculated, and rhythm data 67 in which the beat position R is associated with the reliability is generated. When the music data 62 is reproduced, every time the beat position R specified by the rhythm data 67 arrives, the reliability corresponding to the beat position R exceeds the threshold specified by the reliability threshold data 68. If it exceeds, the screen is switched to the next image at a normal screen rewriting speed, and if not, the screen is switched at a screen rewriting speed slower than the normal speed.

  Therefore, although the detected rhythm may deviate from the original rhythm, the screen rewriting speed is slowed for the portion where the reliability is low and the threshold is not exceeded, so that the user feels uncomfortable in the switching display due to the detected rhythm shift. Can be prevented. That is, even when the music data 62 includes a portion in which the rhythm is not properly detected, natural switching display of the image data 61 can be realized.

[Modification]
The applicable embodiments of the present invention are not limited to the above-described embodiments, and can be appropriately changed without departing from the spirit of the present invention.

(A) Reliability In the above-described embodiment, for example, the reliability at each beat position R is a value based on the autocorrelation value corresponding to the BPM of each frame including the beat position R. A value obtained by adding a value based on the sound pressure level at the position R of the beat to the value based on the autocorrelation value may be used.

  FIG. 13 is a diagram showing an example of the rhythm data 67A in this case. As shown in the figure, the rhythm data 67A stores a beat position 67a, a value 67c based on an autocorrelation value, a sound pressure level 67d, and a reliability 67e in association with each other. The reliability 67e is the sum of the value 67c based on the autocorrelation value and the sound pressure level 67d.

  FIG. 14 is a diagram illustrating an example of a graph based on the rhythm data 67A. In the figure, the horizontal axis is the elapsed time from the start of reproduction of the music data 62, and the vertical axis is plotted with values based on autocorrelation values at each beat position R, sound pressure levels, and reliability values. .

  In FIG. 14, near 90 [seconds], the sound volume becomes almost zero for a moment and the rhythm is interrupted. Then, the rhythm is resumed and the sound volume is recovered at once. In this case, the derived autocorrelation value once decreases at that portion and then gradually recovers. For this reason, by calculating the reliability based not only on the autocorrelation value but also on the sound pressure level, the slow image rewriting speed is restored to the normal speed when the once reduced volume is restored again. A more natural switching display of the image data 61 such as a change is realized.

  Further, in the spectrogram of the audio signal, when the value of a certain frequency suddenly increases and the state continues for a predetermined time, it can be estimated that the position is a code change point. Also, the position where the chord changes is likely to be the beat position. Therefore, the degree of coincidence between the beat position R calculated from the autocorrelation function and the chord change position estimated from the spectrogram may be used as the reliability.

(B) Rhythm detection processing In the above-described embodiment, when the music data 62 is reproduced, the rhythm detection processing for the music data 62 is executed to generate the corresponding rhythm data 67. May be generated in advance. Specifically, for example, before the music data 62 is reproduced, when the music data 62 is read, a rhythm detection process is performed to generate the corresponding rhythm data 67. When the music data 62 is reproduced, the rhythm data 67 is generated in advance. The rhythm data 67 is read out and used.

(C) Image switching display timing In the above-described embodiment, when the music data 62 is reproduced, the switching display of the image data 61 is performed every time the beat position R arrives. Switching display may be performed every time the position R of beats arrives (for example, every 10 beats).

(D) Mode of Switching Display In the above-described embodiment, the screen rewriting speed is controlled in two stages, that is, a normal speed and a speed slower than the normal speed, as control of the switching display mode of the image data 61. However, the screen rewriting speed may be set to three or more and may be switched to three or more stages. In this case, a plurality of threshold values corresponding to the number of screen rewriting speeds are provided, and the screen rewriting speed is determined according to which threshold value the reliability is between.

  Further, although the image rewriting speed is changed as the control of the switching display mode of the image data 61, the switching display frequency may be changed. Specifically, for example, when the music data 62 is reproduced, when the image is switched every time the beat position R arrives, the reliability at the arrived beat position R does not exceed a predetermined threshold. During the subsequent multiple beat positions R (for example, between 10 beats), the image switching frequency is not displayed, so that the image switching frequency is lowered in the low reliability portion. Let

1 is a block configuration diagram of an image processing apparatus. Explanatory drawing of a frame division | segmentation. An example of the graph of an autocorrelation function. An example of the graph of an autocorrelation function. Data structure example of frame analysis data. The data structural example of the data for BPM determination. Explanatory drawing of determination of the position R of a beat. Explanatory drawing of the reliability determination in the position R of a beat. Data configuration example of rhythm data. The flowchart of a synchronous reproduction process. The flowchart of the rhythm detection process performed during a synchronous reproduction process. An example of a graph based on rhythm data. The data structural example of the rhythm data in a modification. An example of the graph based on the rhythm data in a modification.

Explanation of symbols

1 Image processing apparatus 10 CPU
20 Input unit 30 Display unit 40 Audio output unit 50 ROM
51 Synchronous playback program 52 Rhythm detection program 60 RAM
61 Image data 62 Music data 63 Frame autocorrelation data 64 Frame analysis data 65 BPM determination data 66 BPM determination value data 67 Rhythm data 68 Reliability threshold data

Claims (7)

Image data storage means for storing image data;
Music data storage means for storing music data;
Detecting means for detecting the rhythm of the music reproduced from the music data stored in the music data storage means;
Determination means for determining reliability indicating the rhythm accuracy at each timing based on the rhythm detected by the detection means;
By switching and displaying the image data stored in the image data storage means based on the rhythm detected by the detection means, a switching display synchronized with the reproduction of the music data stored in the music data storage means is performed. And a switching display control means for controlling the switching display mode to be different between a portion in which the reliability determined by the determination means in the music data satisfies a predetermined low reliability condition and a portion that does not satisfy the predetermined low reliability condition. When,
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the determination unit determines a value determined based on an autocorrelation value of music data at each timing as the reliability.   The switching display control means performs switching display with a slow screen rewriting speed when reproducing a portion of the music data in which the reliability determined by the determination means satisfies a predetermined low reliability condition. The image processing apparatus according to claim 1 or 2.   The switching display control means reduces the switching display frequency of image data during reproduction of a portion of the music data in which the reliability determined by the determination means satisfies a predetermined low reliability condition. The image processing apparatus according to claim 1 or 2.   The switching display control unit is configured to display a screen for a portion of the music data in which the reliability determined by the determination unit does not satisfy the low reliability condition and the sound pressure level satisfies a predetermined low level condition. The image processing apparatus according to claim 3, wherein switching display with a slow rewriting speed is performed.   The switching display control means includes an image for a portion of the music data in which the reliability determined by the determination means does not satisfy the low reliability condition and the sound pressure level satisfies a predetermined low level condition. The image processing apparatus according to claim 4, wherein the data switching display frequency is reduced. A program for causing a computer to perform switching display of image data synchronized with music that has been reproduced.
A detection function for detecting a rhythm of music obtained by playing the music data;
A determination function for determining reliability indicating the rhythm accuracy at each timing based on the rhythm detected by the detection function;
By switching and displaying the image data based on the rhythm detected by the detection function, switching display synchronized with the reproduction of the music data is performed, and among the music data, the determination unit determines A switching display control function for controlling the state of the switching display to be different between a portion where the reliability satisfies a predetermined low reliability condition and a portion where the reliability does not satisfy ,
A program for causing the computer to realize the above.

Images