JP2007188561A - Automatic generation device for slide show with music, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make the ending time of an arbitrary musical piece and the ending of reproduction of arbitrary number of sheets of images reproduced by a slide show in agreement or in approximately agreement, and perform switching of images by making them to synchronize with the tune break of the musical piece. <P>SOLUTION: Arbitrary music data to be used as BGM of a slide show are input by a music data reception part 50 and a plurality of still image sheets to be displayed in the slide show are input by a still image reception part 56. A tune break detection part 52 detects a tune break of a musical piece, based on input music data. A slide show automatic generation part 54 makes at least the tune break as the switching timing of the still image, based on reproduction time of the music data, a plurality of still images and the tune break of the musical piece, determines the switching timing of still images so that reproduction time per sheet become almost uniform, and switches a plurality of still images in order according to switching timing decided above to output together with reproduction of musical piece based on music data. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an automatic slide show generation apparatus with music and a program, and more particularly to a technique for performing a slide show in which a plurality of still images are reproduced in order according to an arbitrary piece of music.

  Conventionally, as a program for performing this kind of slideshow, music is played, and the boundaries of music phrases (bars) are used as image switching timings, and images are switched for each bar to synchronize music with images. What has been proposed has been proposed (Patent Documents 1 and 2).

  Japanese Patent Application Laid-Open No. 2004-228620 discloses a technique for calculating the display time per image by dividing the musical performance time by the number of images to be reproduced in a slide show, and switching the image every display time.

On the other hand, Patent Documents 3 to 5 disclose techniques for detecting the tonality, chorus section, and music structure of a music piece from given musical score information and the like, and notifying the conversion point.
JP 2002-260340 A JP 2003-288094 A JP-A-5-273976 JP 2004-233965 A JP 2004-184769 A

  However, when switching images for each measure of music as described in Patent Documents 1 and 2, usually, the number of measures in the music does not match the number of images to be reproduced in a slide show that is arbitrarily designated, and the music There arises a problem that it is not possible to finish all the images in the middle of the music or in the music.

  On the other hand, as described in Cited Document 2, when the performance time of a music is divided by the number of images to be reproduced in a slide show to calculate the display time per image, and when switching images for each display time, Although it is possible to make the end point coincide with the end point of reproduction of all images, there is a problem in that the image switching timing and the music conversion point cannot be matched. Note that the boundaries between the measures of the music are not necessarily the music tone conversion points at which the music tone is converted.

  The present invention has been made in view of such circumstances. The end point of an arbitrary piece of music and the end of playback of an arbitrary number of images to be played back in a slide show are made to coincide with each other, and synchronized with the music tone conversion point of the piece of music. It is an object of the present invention to provide an automatic slide show generation apparatus with music and a program capable of switching images.

  In order to achieve the above object, an automatic slide show generation apparatus with music according to claim 1 is a music reception means for receiving arbitrary music data, and a music tone conversion check for detecting a music tone conversion point of music based on the received music data. Each of the still images based on the output means, the still image receiving means for receiving a plurality of still images, the playback time of the music based on the received music data, the number of the still images, and the detected music tone conversion point. Switching timing determining means for determining switching timings of the plurality of still images so that the received still images are switched in synchronization with at least the detected music tone conversion point, and the receiving The music is played based on the music data that has been recorded, and the composite is played according to the determined switching timing. Are of the slideshow automatic generation means for automatically generating a slide show with music image for reproducing switch the pictures in order, comprising the.

  That is, the music tone conversion point of the music is detected based on arbitrary music data used as the background music of the slide show. Based on the playback time of this music, an arbitrary number of still images to be played back in a slide show, and the music tone conversion point of the detected music, at least the music tone conversion point is set as a still image switching timing, and playback per one image is performed. Still image switching timing is determined so that the times are substantially equal. Then, when playing back music based on the received music data, by switching a plurality of still images in order according to the determined still image switching timing, the end of music and the end of playback of an arbitrary number of images are matched. Or can be substantially matched, and the still image can be switched in synchronization with the music tone conversion point of the music.

  As shown in claim 2, in the slide show automatic generation apparatus with music according to claim 1, the music tone conversion point detecting means is based on the received music data, the tone of the music, the chorus section, and the music structure Is detected, and at least one of the change point of the tonality of the detected music, the start point of the chorus section, and the change point of the music structure is detected as a music change point.

  The slide show generation information input means capable of inputting slide show generation information relating to the order of reproduction of the received still images as set forth in claim 1 or 2, The slide show automatic generation means, when the slide show generation information is input from the slide show generation information input means, determines the playback order of the plurality of still images according to the input slide show generation information and generates a slide show image with music. It is characterized by automatic generation.

  4. The slide show automatic generation apparatus with music according to claim 3, wherein the slide show generation information includes information on importance of the plurality of still images, and the slide show automatic generation means is configured to generate the slide show. A slide show image with music by determining the order of reproduction of the plurality of still images in order to reproduce a still image having high importance based on the slide show generation information input from the information input means in synchronization with the detected music tone conversion point. Is automatically generated.

  5. The slide show automatic generation apparatus with music according to claim 4, wherein the slide show automatic generation means has a high degree of importance from a start point of a music chorus section among the detected music tone conversion points. It is characterized in that a slide show image with music is automatically generated by determining the order of reproduction of the plurality of still images in order to reproduce still images. In other words, when there are multiple music tone conversion points, the most important music tone conversion point is the starting point of the chorus section of the music, and a still image (in the case of multiple images) having a high importance in synchronization with this music tone conversion point. Plays the most important still image). Also, the number of still images with high importance is greater than the number of music tone conversion points of the music, and high importance still images that cannot be synchronized with the music tone conversion points are played back in the chorus section of the music. Also good.

  6. The slide show automatic generation apparatus with music according to claim 1, wherein the slide show automatic generation means is a predetermined number of times from the time of the music tone conversion point detected by the music tone conversion point detection means. The section up to the time just before is set as a switching effect section for performing image switching with an image switching special effect, and a moving image with an image switching special effect using at least the still image of the music tone conversion point in the switching effect section. It is characterized by generating.

  That is, by playing a still image at a music tone conversion point immediately after playing a moving image with an image switching special effect, it is possible to produce an image switching special effect along with a change in the music tone.

  As shown in claim 7, in the slide show automatic generation apparatus with music according to claim 6, the switching effect section includes a measure immediately before the conversion point from a time of the music conversion point detected by the music conversion point detection means. It is characterized by a section up to the start time.

  The automatic slide show generation program with music according to claim 8 receives a function of receiving arbitrary music data, a function of detecting a music tone conversion point of music based on the received music data, and receiving an arbitrary plurality of still images. A function for determining the switching timing of each still image based on a function, a playback time of the music based on the received music data, the number of the still images, and the detected music tone conversion point, and at least the detected music tone A function of determining the switching timing of the plurality of still images so that the received still images are switched in synchronization with a conversion point, and playing back music based on the received music data, and according to the determined switching timing A slide show image with music for switching and playing back the plurality of still images in turn is automatically generated. It is characterized in that to realize a function to be generated, to a computer.

  According to the present invention, it is possible to match or almost match the end point of an arbitrary song and the end of playback of an arbitrary number of still images to be played back in a slide show, and switch the still image in synchronization with the music tone conversion point of the song. be able to.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of a slide show automatic generation apparatus with music and a program according to the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing a hardware configuration example of a personal computer provided with a function as a slide show automatic generation apparatus with music according to the present invention.

  As shown in FIG. 1, a personal computer 1 includes a central processing unit (CPU) 10 that mainly controls the operation of each component, a main memory 12 that stores a control program for the device, and serves as a work area when the program is executed. Stores the operating system (OS) of the personal computer 1, device drivers of peripheral devices connected to the personal computer 1, various application software including the slide show automatic generation program with music according to the present invention, user still image data, music data, and the like. Hard disk device 14, DVD-R / RW device 16, display memory 18 for temporarily storing display data, a CRT monitor for displaying images, characters, etc. by image data, character data, etc. from the display memory 18 A monitor device 20 such as a liquid crystal monitor, a sound card 22, and a speaker 24 A keyboard 26, a mouse 28 as a position input device, a mouse controller 30 for detecting the state of the mouse 28 and outputting signals such as the position of the mouse pointer on the monitor device 20 and the state of the mouse 28 to the CPU 10, and a digital camera 2 and an interface 32 such as a USB (Universal Serial Bus) that can input image data and the like, and a bus 34 that connects the above-described components.

  The personal computer 1 having the above-described configuration is well-known except for the slide show automatic generation program with music according to the present invention, which is stored in the hard disk device 14, and thus detailed description of each component will be omitted. This slide show automatic generation program with music is installed in the personal computer 1 by setting a CD-ROM on which the program is recorded in the DVD-R / RW device 16 of the personal computer 1 or through a network via communication means (not shown). be able to.

  FIG. 2 is a functional block diagram showing functions of the personal computer 1 as an automatic slide show generation apparatus with music.

  As shown in the figure, the automatic slide show generation apparatus with music mainly includes a music data reception unit 50, a music tone conversion point detection unit 52, a slide show automatic generation unit 54, a still image reception unit 56, and a slide show generation information input unit 58. It is configured.

  The music data receiving unit 50 is designated when arbitrary music data is designated from music data stored in the hard disk device 14 or stored in a CD or the like set in the DVD-R / RW device 16. Receive music data. That is, the user inputs music data to be used as background music (BGM) of the slide show by the music data receiving unit 50.

  The music data input by the music data reception unit 50 is output to the music tone conversion point detection unit 52 and the slide show automatic generation unit 54. Here, the music data is preferably not a binary information of the sound itself such as a WAV file, but a musical score data of a so-called musical score information format such as a MIDI file. When the music data is information of the sound itself, musical score data having time information accurately synchronized with the music data is also handed over, or is converted into musical score data using a known automatic musical transcription means. Note that the conversion to musical score data may be performed by the music data reception unit 50 or the music tone conversion point detection unit 52.

  When one or a plurality of folders in which a plurality of still image files are stored, or a plurality of still image files are designated, the still image receiving unit 56, or Accepts the specified still image file. That is, the user inputs a plurality of still images to be displayed in a slide show by the still image receiving unit 56.

  A plurality of still image files input by the still image reception unit 56 are output to the slide show automatic generation unit 54.

  The slide show generation information input unit 58 inputs slide show generation information related to the arrangement order (reproduction order) at the time of slide show reproduction of a plurality of still images received by the still image reception unit 56, and specifies the arrangement order. In addition, it is possible to rank the importance of a plurality of still images, or to perform presence / absence of image processing (trimming, color correction, etc.).

  For example, as shown in FIG. 3, when 50 still images are received (file names: DSCF0001.JPG to DSCF0050.JPG), the slide show generation information input unit 58 uses the shooting date / ascending order (descending order) as the arrangement order. , File number, ascending order (descending order), etc. can be specified. As shown in FIG. 3, when the shooting date / ascending order is designated as the arrangement order, 50 still images are arranged so that the shooting date is changed from the oldest to the newest based on the shooting date recorded as the tag information of the image file. The order of image playback (arrangement order) is determined.

  Further, the slide show generation information input unit 58 can rank, for example, importance (large), importance (medium), importance (small), etc. as still image importance. This still image importance level information is used as rearrangement information (still image arrangement information) that has priority in the arrangement order. Note that the importance level information of a still image is associated with the file name of the still image. The slide show generation information specified in this way is output to the slide show automatic generation unit 54.

  The music tone conversion point detection unit 52 analyzes the composition of the music from the input music data and detects the music tone conversion point. That is, the music tone conversion point detection unit 52 detects the tonality of the music from the input music data, detects the chorus section, detects the music composition, etc., and changes the tone of the music (C major to A minor, etc.). The start point of the chorus section and the change point of the music structure are detected as the tone change point. An algorithm for tonality determination is disclosed in Japanese Patent Laid-Open No. 5-273976 (Patent Document 3), and an algorithm for detecting a chorus section is disclosed in Japanese Patent Laid-Open No. 2004-233965 (Patent Document 4). Is disclosed in Japanese Patent Application Laid-Open No. 2004-184769 (Patent Document 5), and the music tone conversion point detection unit 52 can detect the music tone conversion point by applying the above-described known technique. it can.

  FIG. 4 shows an example of the structure of the music piece A. As shown in the figure, this musical piece A is composed of 96 measures, and further includes 8 measures (8 intro, A melody, B melody, chorus, interlude, A melody, B melody, chorus). ). The music tone conversion point detection unit 52 detects a boundary point between each passage as a music tone conversion point and also detects a chorus section. Note that the music tone conversion point is not limited to the boundary point between the passages. For example, when the tone changes in the middle of one measure, the change point of the tone is detected as the music tone conversion point.

  Information on the music tone conversion point (time information) of the music detected by the music tone conversion point detection unit 52 is output to the slide show automatic generation unit 54.

  The slide show automatic generation unit 54 converts the music playback time (performance time) based on the music data input from the music data receiving unit 50, the number of still images (still image files) input from the still image receiving unit 56, and the tone conversion. Switching timing determining means for determining the switching timing of each still image based on the music tone conversion point information input from the point detection unit 52, and playing back music based on the input music data; A slide show image with music for switching and reproducing a plurality of still images in order according to the determined switching timing is automatically generated. In particular, the slide show automatic generation unit 54 sequentially switches the still images so that the reproduction time per still image becomes substantially equal, and at least at the music tone conversion point of the music, the still image is synchronized with the music tone conversion point. This is characterized in that the image is switched so as to be switched.

  Also, the slide show automatic generation unit 54 determines the arrangement order so as to reproduce images with high importance based on the slide show generation information input from the slide show generation information input unit 58 at the music tone conversion point and the chorus section of the music. I have to.

  Next, the operation of the slide show automatic generation unit 54 will be described in more detail.

  FIG. 5 is a flowchart mainly showing a flow of processing of the slide show automatic generation unit 54.

  In FIG. 5, the music tone conversion point detection unit 52 detects the music tone conversion point of the music from the input music data as described above (step S10), and information on the music tone conversion point (position of the music tone conversion point on the score, The type of music tone conversion point (the head position of the chorus section) is passed to the slide show automatic generation unit 54 (step S12).

  The slide show automatic generation unit 54 acquires the length (performance time) of the music from the tag information of the music file of the music data input from the music data reception unit 50, and from the image file input from the still image reception unit 56, (Number of image files) is acquired (step S14). Then, the reproduction time (slideshow switching cycle) per still image is determined from the length of the acquired music and the number of still images (step S16).

  For example, when the length of the music is 3 minutes (180 seconds) and the number of still images is 50, the slide show switching cycle is 3.6 (= 180/50) seconds.

  6 and 7 are diagrams showing an embodiment of image switching timing in music, and schematically show image switching timing on the staff.

  6 and 7, the image switching timing before moving indicates the timing when sequentially switching a plurality of still images in the slide show switching cycle obtained in step S <b> 16, and the image switching timing after moving is determined in step S <b> 16. The timing when a plurality of still images are sequentially switched at the obtained slide show switching cycle and the still images are switched in synchronization with the music tone conversion point is shown.

  Returning to FIG. 5, in step S18, the music tone conversion point measure of the music (if the music tone conversion point is at the head of the measure as shown in FIG. 6) is indicated by a diagonal line immediately before the music tone conversion point. It is determined whether or not the image switching timing (slide show switching) before the movement is included. When an image switching timing is included in a measure at a music tone conversion point (in the case of FIGS. 6A and 6B and FIGS. 7A and 7B), an image switching timing included in the measure is included. Is shifted to the music tone conversion point (step S20). In addition, the image switching timing after the music tone conversion point is also shifted (translated) by the shift amount at this time. When two or more image switching timings are included in the measure of the music tone conversion point (in the case of FIGS. 6B and 7B), the first image switching timing is shifted to the music tone conversion point.

  On the other hand, if no image switching timing is included in the measure of the music conversion point, it is determined which of the image switching timings before and after the music conversion point is close to the music conversion point (step S22). . If the image switching timing immediately before the music tone conversion point is closer, the image switching timing is shifted to the music tone conversion point (step S24) (see FIGS. 6C and 7C), If the image switching timing immediately after the music tone conversion point is closer, the image switching timing is shifted to the music tone conversion point (step S26) (see FIGS. 6D and 7D).

  Subsequently, it is determined whether or not the music tone conversion point of the music is the last one in the music (step S28). If it is not the last music tone conversion point, the process returns to step S18 to perform processing for shifting the image switching timing to the music tone conversion point. If it is the last music tone conversion point, the arrangement of the image switching timing is completed (step S30). ).

  Next, the arrangement order (reproduction order) of a plurality of still images is determined based on the slide show generation information input from the slide show generation information input unit 58 (step S32). That is, as the slide show generation information (arrangement order information), for example, when the shooting date / ascending order is specified as shown in FIG. 3, a plurality of still images are arranged in order from the oldest shooting date to the newest. If no order is specified, the default order is used.

  Subsequently, it is determined whether or not the importance level information is included in the slide show generation information (step S34). If there is importance level information, the still image having the importance level information (in the case where there are a plurality of levels, the still image having the highest importance level) is used as the music tone conversion point corresponding to the start point of the chorus section of the music. Arrangement is performed, and the arrangement order determined in step S32 is rearranged (step S36). Also, if there are multiple still images with importance information, they are rearranged so that they are played back in the chorus section, or at a tone conversion point other than the tone conversion point corresponding to the start point of the chorus section. Rearrange to play.

  When the image switching timing is determined as described above and a still image to be switched at each image switching timing is determined, the slide show automatic generation unit 54 reproduces the music based on the input music data and determines the determined According to the switching timing, a slide show moving image for automatically switching and reproducing the plurality of still images input from the still image receiving unit 56 in the determined arrangement order is generated and output.

  In addition, the slide show automatic generation unit 54 performs image switching using a still image that is switched at the tune conversion point in the section from the beginning of the music bar to the tune conversion point (the section indicated by diagonal lines in FIGS. 6 and 7). Generate and output an image switching video with special effects. As the image switching special effect (transition), known ones such as zoom-in / zoom-out, fade-in / fade-out, and wipe can be applied. Further, a moving image with an image switching special effect may be generated based on two still images before and after the section. Thus, by displaying the moving image with the image switching special effect in the bar section before the music tone conversion point of the music, there is an effect of attracting the viewer to the image of the music tone conversion point.

  When switching images at the image switching timing shown in FIGS. 6 and 7, the image switching timing is shifted in accordance with the music tone conversion point of the music. Therefore, the playback time of the image immediately before the image to be switched at the music tone conversion point (slideshow) The switching cycle is different from the slide show switching cycle obtained in step S16 in FIG. In addition, in order to uniformly shift the image switching timing after the music tone conversion point (in the case of FIGS. 6A to 6C and FIGS. 7A to 7C, it is shifted to the end side of the music, and FIG. ) And FIG. 7D (to shift to the beginning of the music), it is not possible to match the end of the music with the end of display of the last image in the arrangement order until the end of the music. In some cases, an image that cannot be displayed is generated, or the display of the last image in the arrangement order ends before the end of the music.

  In order to solve this problem, every time the image switching timing is shifted at the music tone conversion point of the music, the slideshow switching cycle is recalculated based on the length of the music after that and the remaining number of still images, and the image switching timing is set. Just decide.

  Further, in this embodiment, the section from the music tone conversion point of the music to the beginning of the bar (the section indicated by the hatched lines in FIGS. 6 and 7) is the playback period of the moving image with the image switching special effect. Depending on the length of the video, the playback period of the video with the special effect of image switching is also variable, so that the switching starts from the beginning of the bar and gradually proceeds until the music conversion point, and the switching is completed at the music conversion point. It is necessary to adjust to.

  In addition, you may make it record the data of the slide show moving image with a music produced | generated automatically on DVD-R and CD-R with a DVD-R / RW apparatus. In this embodiment, the case where a slide show video with music is automatically generated by software using a personal computer as an example has been described, but may be realized by hardware.

FIG. 1 is a block diagram showing a hardware configuration example of a personal computer provided with a function as a slide show automatic generation apparatus with music according to the present invention. FIG. 2 is a functional block diagram showing functions of the personal computer as an automatic slide show generation apparatus with music. FIG. 3 is a chart used for explaining the slide show generation information. FIG. 4 shows an example of the structure of an arbitrary music piece. FIG. 5 is a flowchart mainly showing a processing flow of the slide show automatic generation unit. FIG. 6 is a diagram showing an embodiment of image switching timing in music, and is a diagram schematically showing image switching timing on the staff. FIG. 7 is a diagram showing another embodiment of the image switching timing in the music, and is a diagram schematically showing the image switching timing on the staff.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... PC, 2 ... Digital camera, 10 ... Central processing unit (CPU), 12 ... Main memory, 14 ... Hard disk device, 16 ... DVD-R / RW device, 20 ... Monitor device, 22 ... Sound card, 24 ... Speaker, 32 ... interface, 50 ... music data reception unit, 52 ... music tone conversion point detection unit, 54 ... slide show automatic generation unit, 56 ... still image reception unit, 58 ... slide show generation information input unit

Claims (8)

Music receiving means for receiving arbitrary music data;
Music tone conversion point detecting means for detecting a music tone conversion point of the music based on the received music data;
Still image receiving means for receiving any plurality of still images;
Means for determining the switching timing of each still image based on the playback time of the music based on the received music data, the number of still images, and the detected music tone conversion point, and at least the detected music tone conversion point Switching timing determining means for determining switching timing of the plurality of still images so that the received still images are switched in synchronization;
A slide show automatic generation means for automatically generating a slide show image with music for sequentially playing back the plurality of still images according to the determined switching timing and playing back the music based on the received music data;
A slide show automatic generation apparatus with music, comprising:   The music tone conversion point detecting means detects at least one of the tonality of the music, the chorus section, and the music structure based on the received music data, and the change point of the tonality of the detected music, the chorus section 2. The slide show automatic generation apparatus with music according to claim 1, wherein at least one of a start point of music and a change point of the music structure is detected as a music tone change point.   The apparatus further comprises slide show generation information input means capable of inputting slide show generation information related to the order of reproduction of the received still images, and the slide show automatic generation means receives the slide show generation information from the slide show generation information input means. 3. The slide show automatic generation apparatus with music according to claim 1, wherein a slide show image with music is automatically generated by determining a playback order of the plurality of still images according to the input slide show generation information.   The slide show generation information includes information on the importance of the plurality of still images, and the slide show automatic generation unit selects a still image having a high importance based on the slide show generation information input from the slide show generation information input unit. 4. A slide show automatic generation apparatus with music according to claim 3, wherein a slide show image with music is automatically generated by determining the playback order of the plurality of still images so as to be played back in synchronization with the detected music tone conversion point. .   The slide show automatic generation means determines the playback order of the plurality of still images from the start point of the chorus section of the music among the detected music tone conversion points, and determines the playback order of the still images with music. 5. The slide show automatic generation apparatus with music according to claim 4, wherein a slide show image is automatically generated.   The slide show automatic generation means sets a section from the time of the music tone conversion point detected by the music tone conversion point detection means to a time before a predetermined time as a switching effect section for performing image switching with an image switching special effect, 6. The slide show automatic generation apparatus with music according to claim 1, wherein a moving image with an image switching special effect using at least a still image at the music tone conversion point is generated in the switching effect section.   7. The slide show with music according to claim 6, wherein the switching effect section is a section from a time of a music tone conversion point detected by the music tone conversion point detecting means to a start time of a measure immediately before the conversion point. Automatic generator. A function to accept arbitrary music data,
A function of detecting a music tone conversion point of the music based on the received music data;
The ability to accept any number of still images,
A function for determining the switching timing of each still image based on the playback time of the music based on the received music data, the number of the still images, and the detected music tone conversion point, and at least the detected music tone conversion point A function of determining the switching timing of the plurality of still images so that the received still images are switched in synchronization;
A function of automatically generating a slide show image with music for playing back music based on the received music data and switching the plurality of still images in order according to the determined switching timing;
A slide show automatic generation program with music, which is realized by a computer.

Images