JP2013200750A - Image processing apparatus, image processing method and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that can recursively determine an optimal combination of music and images at the time of automatic creation of a slide show and improve the quality of the slide show.SOLUTION: There is provided an image processing apparatus including: an effect interval setting unit setting multiple effect intervals; an image assigning unit assigning the image data to each of the multiple effect intervals; an effect setting unit setting an effect to each of the multiple effect intervals; an effect resetting deciding unit deciding whether an effect interval satisfies a condition for realizing its effect, and causing the effect to be reset if the effect interval thereof does not satisfy the condition; and a consistency deciding unit deciding, if the effect is reset, consistency between the effect interval in which the effect is reset and other effect intervals.

Description

  The present disclosure relates to an image processing device, an image processing method, and a computer program.

  A technique for generating a content called a slide show that sequentially switches and displays a plurality of photos while playing music is disclosed. When creating a slide show automatically, the mainstream technique is to apply an image to a template prepared in advance. However, depending on the image, the image display effect (image effect) determined by the template may not be suitable, or it may be optimal for the image. There was a problem that image effects could not be selected. Techniques for solving such problems have been disclosed (see Patent Documents 1 to 3, etc.).

  Patent Document 1 discloses a technique for automatically creating a slide show synchronized with music of higher quality by creating a display template based on music analysis and optimally selecting assigned images and cutout areas. . Patent Document 2 discloses a technique for selecting a more optimal image or region by using region extraction information in an image when applying an image to a slide show template. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for automatically creating a slide show in which a necessary reproduction music length is determined from a combination of an image and an image effect, and the image effect is synchronized with the music.

JP 2008-084021 A JP 2007-243411 A JP 2005-210350 A

  However, in the technique of Patent Document 1, when there is no image that matches the image effect that is desired to be used, the effect cannot be exhibited, and the completeness as a slide show is lowered. In the technique of Patent Document 2, the quality of the entire slide show is deteriorated depending on the context of the selected image or image effect. In the technique of Patent Document 3, the image effect is switched at a timing that is musically different from the timing at which the image effect is originally intended to be switched. For example, the same image effect continues at the moment of transition from B melody to rust.

  Accordingly, the present disclosure provides a new and improved image processing apparatus, image processing method, and image processing apparatus capable of recursively determining an optimal combination of music and images when automatically creating a slide show to improve the quality of the slide show. To provide a computer program.

  According to the present disclosure, for each of the effect section setting unit that sets a plurality of effect sections from a predetermined section that continuously displays image data, and each of the plurality of effect sections set by the effect section setting unit An image allocating section for allocating the image data, an effect setting section for setting an effect for each of the plurality of effect sections set by the effect section setting section, and the effect setting section for each of the plurality of effect sections. It is determined whether or not the effect set for each of the conditions satisfies the conditions for realizing each effect using image metadata corresponding to the image data allocated by the image allocation unit. If there is, the effect setting section has an effect section. When the effect setting section resets the effect by the effect reset determining section and the effect reset determining section, the consistency between the effect section in which the effect is reset and another effect section is set. An image processing apparatus is provided that includes a consistency determination unit for determining.

  According to such a configuration, the effect section setting unit sets a plurality of effect sections from the predetermined section for continuously displaying the image data, and the image assignment unit sets each of the plurality of effect sections set by the effect section setting unit. Assign image data to. The effect setting section sets an effect for each of the plurality of effect sections set by the effect section setting section, and the effect resetting determination section displays the effect set by the effect setting section for each of the plurality of effect sections. Whether or not the conditions for realizing each effect are satisfied is determined using image metadata corresponding to the image data allocated by the image allocation unit, and if there is an effect section that does not satisfy the conditions, the effect setting unit Reset the effect in the effect section. When the effect setting unit resets the effect by the effect reset determining unit, the consistency determining unit determines the consistency between the effect section in which the effect is reset and another effect section. As a result, such an image processing apparatus can recursively determine the optimal combination of music and images when automatically creating a slide show, thereby improving the quality of the slide show.

  According to the present disclosure, each of the effect section setting step for setting a plurality of effect sections from the predetermined section for continuously displaying image data, and each of the plurality of effect sections set in the effect section setting step. An image assigning step for assigning image data to an effect, an effect setting step for setting an effect for each of the plurality of effect sections set in the effect section setting step, and the plurality of effect sections in the effect setting step Whether or not the effect set for each of the images satisfies the conditions for realizing each effect is determined using image metadata corresponding to the image data allocated in the image allocation step, and the conditions are satisfied. There is no effect section The effect reset determining step for resetting the effect in the effect section, and when the effect is reset in the effect setting step by the effect reset determining step, the effect section in which the effect is reset and another effect section A consistency determination step of determining consistency between the image processing method and the image processing method.

  According to the present disclosure, the effect section setting step for setting a plurality of effect sections from the predetermined section for continuously displaying the image data on the computer, and the plurality of effects set in the effect section setting step. An effect setting step for setting an effect for each of the sections, an image assigning step for assigning image data to each of the plurality of effect sections set in the effect section setting step, and the plurality of the plurality of effects in the effect setting step It is determined using the image metadata corresponding to the image data allocated in the image allocation step whether the effects set for each of the effect sections satisfy the conditions for realizing each effect, and the conditions Does not meet F If there is an effect section, an effect reset determining step for resetting the effect in the effect section and an effect section in which the effect is reset when the effect is reset in the effect setting step by the effect reset determining step. And a consistency determination step of determining consistency between the program and other effect sections. Is provided.

  As described above, according to the present disclosure, a new and improved image processing apparatus capable of recursively determining an optimal combination of music and images and improving the quality of a slide show when automatically creating a slide show. An image processing method and a computer program can be provided.

2 is an explanatory diagram illustrating a functional configuration of an image processing apparatus 100 according to an embodiment of the present disclosure. FIG. It is explanatory drawing which shows a mode that a music feature-value is acquired using 12 sound analysis technology. It is explanatory drawing which shows the example of a relationship between a music theory and chord progression. It is explanatory drawing which shows the example of the theme of a slide show. It is explanatory drawing which shows the example which set the effect area of the slide show. It is explanatory drawing which shows the example of the tree structure comprised by event clustering and the similar analysis result. When creating a slideshow automatically, the optimal combination of music and images can be recursively determined to improve the quality of the slideshow It is explanatory drawing which shows the example of the allocation process of the image to an effect area. It is explanatory drawing which shows the example of the reset of an effect. It is explanatory drawing which shows the example of the effect reset. It is explanatory drawing which shows the example which selected the image from the candidate images. It is explanatory drawing which shows the example of cut-out of the display range by the display range cut-out part 124. FIG. It is explanatory drawing which shows the example which implement | achieved the effect which uses several images from one image. It is explanatory drawing which shows the example of the animation to which the parameter was set. It is explanatory drawing which shows the example of the animation by which the parameter of the key frame was set. It is explanatory drawing which shows the example of a setting of the information which decorates a slide show. 5 is a flowchart showing an operation of the image processing apparatus 100 according to an embodiment of the present disclosure. It is explanatory drawing which shows the example which selects the effect match | combined with the music. It is explanatory drawing which shows the example which selects the effect match | combined with the music. It is explanatory drawing which shows an example of the correspondence of a music tone and an effect. It is explanatory drawing which shows the example of a detection of partial modulation. It is explanatory drawing which shows another example of a detection of partial modulation.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

The description will be made in the following order.
<1. One Embodiment of the Present Disclosure>
[Functional configuration of image processing apparatus]

<1. One Embodiment of the Present Disclosure>
[Functional configuration of image processing apparatus]
First, a functional configuration of an image processing apparatus according to an embodiment of the present disclosure will be described with reference to the drawings. FIG. 1 is an explanatory diagram illustrating a functional configuration of the image processing apparatus 100 according to an embodiment of the present disclosure. Hereinafter, the functional configuration of the image processing apparatus 100 according to an embodiment of the present disclosure will be described with reference to FIG.

  The image processing apparatus 100 illustrated in FIG. 1 is an apparatus having a function of automatically generating a slide show from image data and music data. Note that “slide show” refers to content that sequentially switches and displays a plurality of photos while playing music.

  As illustrated in FIG. 1, the image processing apparatus 100 according to an embodiment of the present disclosure includes a music analysis unit 102, a music metadata holding unit 104, a playback segment selection unit 106, an effect segment setting unit 108, Image candidate assignment unit 110, image analysis unit 112, image metadata holding unit 114, effect setting unit 116, image selection unit 118, effect resetting determination unit 120, slide show quality determination unit 122, display range A cutout unit 124, a timing adjustment unit 126, and a slide show output unit 128 are included.

  The music analysis unit 102 analyzes the music data and outputs the analysis result as metadata. The metadata output by analyzing the music data to be played back in the slide show is used to cut out a playback section by a playback section selection unit 106, which will be described later, and to set an image effect according to the music by the effect section setting unit 108. . The music analysis unit 102 outputs metadata obtained by analyzing the music data to the music metadata holding unit 104. The music metadata holding unit 104 holds music metadata obtained by analyzing music data by the music analysis unit 102. The music metadata held in the music metadata holding unit 104 is referred to by the playback section selection unit 106 and the effect section setting unit 108.

  In this embodiment, in order to create a slide show synchronized with music, a 12-tone analysis technique of a music tone analysis algorithm is used. FIG. 2 is an explanatory diagram showing a state in which music feature values are acquired using a twelve-tone analysis technique. As shown in FIG. 2, music features such as chorus, key for chord, chord for chord, time signature, beat position, beat strength, etc. are acquired by 12-tone analysis technology. It is possible. The music analysis unit 102 extracts and acquires these music feature amounts in order to determine effect switching timing and the like later.

  The playback section selection unit 106 selects the playback section in the music data to be played back using the music metadata of the music data played back in the slide show, which is held by the music metadata holding unit 104. The playback section selection unit 106 supplies information on the selected playback section to the effect section setting unit 108.

  In a slide show that is synchronized with music, you may play a photo along with a single song, or you may play a photo across multiple songs. If you want to send it to a friend, you may want to create a short slideshow with a portion of a song. In such a case, the playback section selection unit 106 cuts out a part of the music data, for example, 30 seconds just before the chorus position as a slide show playback section, using the music feature amount obtained by the 12 sound analysis technique. May be. FIG. 2 shows a case where a part of music is set as a slideshow playback section.

  In addition, there are cases where there is a characteristic in music theory in the chord information obtained by the twelve-tone analysis of the music analysis unit 102 and the chord progression. For example, in music theory, there are somewhat unstable chords called subdominants, unstable chords called dominants, and stable chords called tonics. FIG. 3 is an explanatory diagram showing an example of the relationship between music theory and chord progression.

  Of these chords, the chord transitions from dominant to tonic makes an impressive pattern that makes the listener feel stable from anxiety, so combining them with effects in a slideshow improves expression The effect can be expected. In addition to the above, areas where chords change frequently, unique chords that attract the listener's attention, areas where beats are concentrated, etc. are also effective when creating slideshows automatically Available to: Therefore, the playback section selection unit 106 may hold the area information and supply it to the effect section setting unit 108.

  The effect section setting unit 108 sets a plurality of effect sections from predetermined sections in which image data is continuously displayed as a slide show. The effect section setting unit 108 uses, for example, music metadata of music data to be played back in a slide show, which is held by the music metadata holding unit 104, and a playback section of music data supplied from the playback section selection unit 106. Sets the image effect interval at. The effect section setting unit 108 supplies information on the set effect section to the image candidate assignment unit 110.

  In the present embodiment, the effect section setting unit 108 sets the following four main factors for determining the effect section in order of priority.

(1) Essential effect section for each theme When the effect section is determined, the effect section setting unit 108 first prioritizes the required effect section for each set theme. The theme here refers to a world view of a slide show, and is set, for example, when the user instructs to automatically create a slide show.

  FIG. 4 is an explanatory diagram illustrating an example of a slide show theme. Figure 4 shows a simple slideshow, a monochrome slideshow, a slideshow in which photos are projected on film, a main slideshow with image effects focused on human faces, and photos drifting through the universe Slideshows that make full use of 3D, etc. exist, and if the theme is different, the atmosphere of the slideshow changes greatly. Note that these slide shows are different from simple templates in that the theme itself does not have an effect switching timing.

  Depending on the theme, specific image effects such as intro and ending are prepared, and those effect sections are determined with priority over anything. These specific image effects may be a fixed number of seconds or the first one measure, etc., and vary depending on the theme and effect type.

(2) Timing at which the music composition is switched Next, the effect section setting unit 108 prioritizes the timing at which the music composition is switched in the music metadata when determining the effect section. Examples of the timing at which the music composition is switched include time signature switching (4/4 → 3/4), key changing (C → G), and melody switching (B melody → rust). At these timings, the musical atmosphere changes greatly, so it is effective to switch image effects at that timing.

  The effect section setting unit 108 does not determine the effect section itself when using the timing at which the music composition is switched, but prioritizes the setting of the effect section that crosses the timing later. The effect is divided at the timing when the music composition is switched.

(3) Areas that are impressive in terms of chord progression In the description of the music metadata described above, an impressive chord progression pattern, a characteristic chord, and the like have been extracted. Are determined, the areas having these chord progression patterns and characteristic chords are used preferentially as effect sections. The effect section setting unit 108 may change the chord progression pattern and the characteristic chord depending on the theme and the music composition (such as rust and A melody) at that time.

(4) Switching pattern for each base theme The factors (1) to (3) may hardly exist depending on the theme or music. In such a case, if the effect section setting unit 108 sets the effect section at a constant interval (for example, in units of one measure), there is a possibility that the slide show becomes monotonous.

  Therefore, the effect section setting unit 108 sets the base effect section at various intervals and timings when determining the effect section. The interval and timing differ depending on the theme set for the slide show, and may be set with reference to the mood feature amount of the music obtained by analyzing the music data. For example, the effect section setting unit 108 changes the effect section and the effect switching timing according to the music composition, such as setting the effect section interval short in the chorus portion of the music and setting the effect section longer in the interlude section. May be. In this way, by changing the effect section and the effect switching timing according to the music composition, an effect that does not make the slide show monotonous appears.

  In general, music is often repeated or changed in units of four measures, eight measures, and sixteen measures, so that such knowledge is reflected in the setting of the effect section by the effect section setting unit 108. It is effective.

  FIG. 5 is an explanatory diagram showing an example in which the effect section setting unit 108 sets the effect section of the slide show with the priorities (1) to (4) described above. These effect sections shown in FIG. 5 may be reset later, but first, with the effect section set here as a framework, subsequent image assignment and effect settings are performed.

  The image candidate assigning unit 110 assigns an image to each effect section set by the effect section setting unit 108 from among a group of images prepared for display in a slide show. The image assignment to each effect section by the image candidate assigning unit 110 will be described in detail later.

  The image analysis unit 112 performs image analysis processing on the image group prepared for displaying the slide show. The image analysis unit 112 acquires feature amounts such as analysis feature amounts related to faces, similarities, and colors and general Exif information for the image group prepared for displaying the slide show. Groups are generated and stored with various granularities so that they can be easily used when assigning images used in a slide show to each effect section. The image analysis unit 112 outputs image metadata obtained as a result of the image analysis processing to the image metadata holding unit 114. The image metadata holding unit 114 holds the image metadata output from the image analysis unit 112. The image metadata held by the image metadata holding unit 114 is referred to by the effect setting unit 116, the image selection unit 118, the display range cutout unit 124, and the like.

  Since there are many cases in which images are displayed in the order of shooting date and time in a slide show, for example, if a tree using event clustering or similar analysis results disclosed in Japanese Patent Application Laid-Open No. 2007-94762 is constructed, the images are used in later processing. It becomes easy. FIG. 6 is an explanatory diagram illustrating an example of a tree structure configured by event clustering and a similar analysis result for an image group. For the similarity analysis, information such as the image URL, the size of the image, the shooting date and time, the latitude and longitude of the captured image, the face feature amount, the similar feature amount, the composition feature amount, and the color feature amount may be used.

  Further, group generation may be performed using face detection or face identification technology instead of shooting date order. For example, Japanese Patent Application Laid-Open No. 2011-133888 discloses a technique for performing group generation based on grouping of persons and human relationships.

  The effect setting unit 116 sets an effect for each effect section set by the effect section setting unit 108. The effect setting unit 116 first sets an expected effect for each effect section set by the effect section setting unit 108. The expression “expected” here is the best effect to express the excitement of the specified theme or song, but when the image is assigned, it is an effect that does not match that image, This is because if the effects before and after are reset, if the connection between the effects is bad, there is a possibility that it will be changed later. First, the effect setting unit 116 is caused to set an effect that seems to be optimal as a slide show, and if there is a need to change, the effect setting unit 116 is set to the highest effect. The condition for selecting the effect type is in accordance with the effect section setting method described above.

(1) Essential effect section for each theme In the effect section determined for each theme, an effect corresponding to the uncondition is set. The effect section determined for each theme is, for example, an intro part or an ending part of a slide show.

(2) Timing at which the music composition is switched At the timing at which the music composition is switched, the effect section is not set by the effect section setting unit 108, but the effect type assigned immediately after the music composition is switched is set. Have a role. At this timing, the effect setting unit 116 assigns a flashy effect immediately after switching from B melody to rust, or assigns an effect that causes the entire screen to flash, thereby making the excitement stand out.

(3) Area that is impressive in terms of chord progression In this area that is impressive in terms of chord progression, the effect setting unit 116 assigns effects that are meaningful in terms of music theory. The effect setting unit 116, for example, provides a decorative effect for a special chord that attracts the listener's attention, a connecting effect that causes the unstable chord to be expected later, and a slide show for a stable chord. Select a flashy effect that will be decided in step 1, and a moving effect in a section where chords change frequently. As described above, by setting the effect according to the chord progression, the image processing apparatus 100 according to the present embodiment superimposes the musical impression and the visual impression, and makes an attractive slide show that is more suitable for music. Can be realized.

(4) Switching pattern for each base theme In this base pattern, the effect setting unit 116 indicates that various effects appear, and the same effects are not consecutively arranged, and effects that are well connected are arranged. Set the effect in consideration. Further, the effect setting unit 116 sets the effect according to the composition of the music, for example, by selecting a flashy effect for chorus, a normal effect for a normal melody, and an adult effect for an interlude. .

  When setting these expected effects, the effect setting unit 116 does not always select the same effect if the conditions are the same, but randomly selects from a plurality of effects even if the conditions are the same. You may make it give width. This is because when the user creates a slide show many times, it is not always the same slide show, but it is often better to create a different slide show each time.

  FIG. 7 is an explanatory diagram illustrating an example in which the effect setting unit 116 sets expected effects. In FIG. 7, the effect interval setting unit 108 sets three effect intervals, an expected interval, a texture interval, and an ending, and the effect setting unit 116 zooms toward each face with respect to each effect interval. The figure shows a state in which an effect for displaying a single photo in a timely manner and an effect for displaying a group photo or a collage are set.

  Here, when it is difficult to display all the images input to the image processing apparatus 100 with respect to the playback time of the slide show such as one piece of music to be played back in the slide show or a part of the music for 30 seconds. There is. In such a case, it is required to select a necessary number of images from the input image group.

  Furthermore, in many cases, there are user requests to display images in order of shooting date and time in a slide show. If such a request requires only the required number of images to be displayed from the beginning, The last photo in the input image group may not come out at all.

  Therefore, the image candidate assigning unit 110 may execute processing for determining which image is assigned to the effect section set by the effect section setting unit 108. FIG. 8 is an explanatory diagram showing an example of image allocation processing to the effect section by the image candidate allocation unit 110. FIG. 8 shows an example in which an image is assigned to an effect section using the tree constructed as shown in FIG.

  As shown in FIG. 8, the image candidate assigning unit 110 compares the tree of images prepared in advance with the entire slide show, and performs processing of assigning images from which to where in which effect section. This assigned image group is referred to as a “selectable image candidate” in a certain effect section.

  When assigning image candidates to the effect sections, the image candidate assigning unit 110 may assign the effect sections equally to the effect sections in the order of the shooting date and time, or may assign the effect sections in the ratio of the number of seconds of the effect sections. Further, the image candidate assigning unit 110 may assign the image candidate so as to satisfy the required number of expected effects set in the effect section. However, regarding an effect section in which an image is selected from the whole regardless of the time series of shooting date and time, such as intro or ending, it is desirable that the image candidate assigning unit 110 specially set all images as image candidates.

  In addition, when assigning image candidates to the effect section by various methods, the image candidate assigning unit 110 preferably assigns as large a group as possible to the effect section in the tree configuration. For example, the image candidate assigning unit 110 assigns image candidates in units of similar groups when the number of similar groups is larger than the number of effect sections, and assigns in such groups when assignment is possible in a larger group. . This is useful when you decide which images to display in a slide show later, and you want to avoid displaying unrelated photos at the same time or want to display similar photos in a row. This is to make it easy to select a correct image.

  Conversely, if the number of input images is small with respect to the length of the song, the same image can be assigned to a plurality of effect sections in an overlapping manner. However, in this case, only the same photograph appears with different effects, and as a result, the quality of the slide show may be lowered. In such a case, the slide show quality determination unit 122 (described later) causes the effect section setting unit 108 to set an effect section, for example, to set a longer effect section (with a smaller number of effect sections), or to select a playback section. The unit 106 may perform processing such as shortening the playback section of the music to be played back.

  The image selection unit 118 selects an image to be displayed with the effect set in each effect section by the effect setting unit 116 from the images assigned to each effect section by the image candidate assignment unit 110.

  The effect resetting determination unit 120 determines whether or not the effect expected by the image selected by the image selection unit 118 can be realized. The effect resetting determination unit 120 determines whether or not the expected effect can be realized, for example, whether the number of photographs is sufficient, whether a necessary number of faces exist, or a necessary number of similar images exist. , Decide whether to meet.

  FIG. 9 is an explanatory diagram showing an example of effect resetting. In a certain excitement section, the effect setting unit 116 sets the effect in anticipation of the effect of zooming the face, but there is no photograph with a face in the candidate images assigned by the image candidate assigning unit 110. The effect resetting determination unit 120 causes the effect setting unit 116 to reset another effect. Also, in a certain section, the effect setting unit 116 has set an effect that requires three similar images. However, only two similar images exist in the candidate image, so the effect resetting determination is made. The unit 120 causes the effect setting unit 116 to reset an effect that uses two similar images instead.

  As in the latter example, when the effect resetting determination unit 120 causes the effect setting unit 116 to reset the effect, a rule may be provided to reset the effect as close to the original effect as possible. This is to make the entire slide show as close as possible to the initially expected atmosphere.

  FIG. 10 is an explanatory diagram showing an example of the reset effect. The effect to be reset is determined in a direction that reduces the number of photos, the number of necessary faces, and the number of necessary similar images in a direction that reduces the condition of whether there are faces or similar images. Like that. In FIG. 10, the effect of displaying three similar images is set first, but the condition is not satisfied. Therefore, the effect of displaying two similar images is set next, and further, the condition is not satisfied. An example of setting an effect for displaying a simple image is shown.

  However, even if the number of candidate images is less than the number required for the expected effect, the effect may not be reset. For example, using a composition analysis technique, it is possible to express as if there are a plurality of photos from one photo when actually reproducing in a slide show. Therefore, whether or not the expected effect can be realized even if the required number of images is not reached can be checked by referring to the image feature amount. A method of extracting the display range using the composition analysis technique will be described later.

  The slide show quality determination unit 122 determines the quality of the entire slide show from the effect set by the effect setting unit 116 and the image selected by the image selection unit 118. In particular, when the effect reset determination unit 120 resets the effect, the slide show quality determination unit 122 determines again the quality of the entire slide show from the reset effect and the image selected by the image selection unit 118. .

  As a result of the effect resetting determination unit 120 resetting the effects in the individual effect sections as described above, the effect of the connection between the previous and next effects becomes worse, and the effects used in the entire slide show are biased. There is a possibility that too few images will be displayed. Therefore, when at least one effect is reset, the slide show quality determination unit 122 checks, for example, the connection of the previous and subsequent effects, the bias of the used effect, and the ratio of the number of used images to the population.

  If the slide show quality determination unit 122 determines that the connection between the previous and next effects has deteriorated, the effect can be reset, but if the effect is reset, the same check is performed again. As a result, when the slide show quality determination unit 122 determines that the slide show quality cannot be maintained with the reset effect (for example, the effect is too biased or the number of images used is too small with respect to the parameter). The process returns to the process of changing the image assignment assigned by the image candidate assignment unit 110 or causing the effect interval setting unit 108 to change the setting of the effect interval. By recursively creating a slide show in this way, the image processing apparatus 100 can recursively determine the optimal combination of music and images and improve the quality of the slide show.

  Based on the effect section in the slide show, the applied image effect, and the candidate for the image to be displayed, the image selection unit 118 selects an image actually displayed according to the image effect from the candidate images. select.

The image selection unit 118 may use the following method according to the effect when selecting a required number of images from the candidate images.
(1) Random selection from candidate images (2) Select from candidate images in time series (3) Select images with faces preferentially (4) Select similar images

(1) Random selection from candidate images This method is used when a plurality of images are displayed in an effect and the selected images do not need to be particularly related to each other. For example, it is used when it is desired to display a large number of images among all input images by intro or ending.

(2) Selection in Time Series from Candidate Images This method is used when it is desired to select images that are as distant as possible when displaying a plurality of images in the effect. For example, when selecting three images from candidate images, the image selection unit 118 first divides the candidate image into three according to the shooting date and time, and selects one image from each of the divided image groups. As a result, it is possible to select an image in which a similar scene is not displayed as much as possible. When selecting an image, an image with the longest shooting date and time may be selected, or may be selected at random from the divided image group.

(3) Preferentially selecting an image with a face This method is used when an effect focused on a face is used. When there is an image having a required number of faces in the candidate image, the image selection unit 118 selects only the required number from the images, and the image having the face is less than the required number Selects an insufficient number of images from the remaining images in addition to the image having the face. Note that when selecting an image, the image selection unit 118 may select an image with a face as large as possible, or may randomly select a required number of images from which a face exists.

(4) Selecting consecutive images in similar group This method is used when it is desired to display similar images. For example, when there is an effect in which a plurality of images are flipped one after another like a flip book, it is effective to continuously display images taken by the continuous shooting function of the camera. In order to realize such an effect, the image selection unit 118 selects a necessary number of consecutive images with shooting dates and times from an image group designated as the same similar group based on the image analysis result. If there are no more similar groups than the required number, the image selection unit 118 may select the required number of consecutive images from among the image candidates. As for which similar group is selected, the similar group having the largest number of sheets may be used, or a random similar group may be selected.

  FIG. 11 is an explanatory diagram illustrating an example in which the image selection unit 118 selects an image from candidate images using the above-described selection method. FIG. 11 shows two examples. In the first example, when the effect set in the effect section of the image selection target is a face cutout effect that cuts out and displays a face part from an image, the image selection unit 118 performs the following operation from the given image group: It selects an image with a face and cuts out the largest face using an effect. In the second example, when the effect set in the effect section for image selection is a three-effect that displays three images in succession, the image selection unit 118 gives the given image group. The three most similar images are selected and displayed in chronological order.

  As described above, the image selection unit 118 selects an image suitable for the image effect set in the image selection target effect section from the candidate images.

  As described above in the setting of the expected effect by the effect setting unit 116, some random elements may be inserted when using the image selection unit 118 and the respective image selection methods. By including a random element, when a user creates a slide show many times, a different photo appears each time and a different impression can be given.

  The display range cutout unit 124 cuts out the display range of the image selected by the image selection unit 118. As described above, in each effect section, the image to be displayed and the image effect to be applied have been determined. In the image effect, not only the image is displayed as it is, but also in the area of a specific aspect ratio. There are cases that must be done. For example, when the original 4: 3 aspect ratio image must be displayed at 16: 9 during slideshow playback, or you want to display 4: 3 and 3: 4 aspect ratios, It is necessary to cut out a necessary display range from the original image by some method.

  When it is necessary to display an image in a specific aspect ratio region, the display range cutout unit 124 can cut out an arbitrary aspect ratio region under various conditions by using a composition analysis technique. . FIG. 12 is an explanatory diagram illustrating an example of the display range cutout by the display range cutout unit 124. For example, as illustrated in FIG. 12, when a display area having a vertically long aspect ratio is required from a horizontally long image, the display range clipping unit 124 determines the position of the face or the like when the face is detected in the selected image. The optimum area can be cut out using the orientation information, and when the face is not detected, the optimum area can be cut out using the target object information.

  To cut out the display area using composition analysis, the Hinomaru composition cut out mainly on the face and the attention area, the three-part composition that is generally beautiful as a photograph, the zoom-up that cuts out the face and the attention area as much as possible, There is a diagonal composition that makes parent and child photos look beautiful. The display range cutout unit 124 can cut out with various compositions according to the theme and the type of effect, using the cutout method of these display areas.

  Further, in the effect resetting described above, it has been stated that the effect resetting may not be performed even if the number of candidate images is less than the number required for the expected effect. If the number of candidate images is less than the number required for the expected effect, the display range cutout unit 124 cuts out the image using the composition analysis technique, so that it is as if there are a plurality of photos. It is also possible to achieve effects such as

  FIG. 13 is an explanatory diagram illustrating an example in which the display range cutout unit 124 realizes an effect that uses a plurality of images from one image by cutting out an image using a composition analysis technique. FIG. 13 illustrates a case where two photos are required for an effect to be displayed, but only one image is selected by the image selection unit 118. In such a case, the display range cutout unit 124 uses a composition analysis technique to cut out a part in which a child is shown and a part in which parents are shown from one image and apply them to the effect. As a result, it is possible to realize an effect as if there are a plurality of photographs.

  Of course, in the case of an effect that does not need to be displayed in an area having a specific aspect ratio, the clipping process by the display range clipping unit 124 may be skipped.

The timing adjustment unit 126 adjusts the display timing of the effect. In the above description, since the image to be displayed and the image effect are determined in each effect section, the timing adjustment unit 126 adjusts the display timing of the determined effect. Each effect has parameters of “show”, “start”, “end”, and “hide” disclosed in Japanese Patent Application Laid-Open No. 2011-181103, for example. The meaning of each parameter is as follows.
・ Show… The timing when the effect lifetime starts and the initial state is displayed ・ start… The timing when the effect starts mainly animation ・ end… The timing when the effect ends animation ・ hid… The lifetime of the effect ends And when it is discarded

  For example, the timing adjustment unit 126 sets start and end to coincide with the start and end of the effect section, sets the show one beat before the start, and sets the hide one beat after the end, so that each effect overlaps by one beat. It is possible to make settings such as animation.

  FIG. 14 is an explanatory diagram illustrating an example of an animation in which parameters are set by the timing adjustment unit 126. The trapezoid shown in FIG. 14 represents a state in which an image is displayed by a set effect. In other words, the photo starts to appear at the time of the show, the photo is clearly displayed at the time of the start, the animation with the effect is performed from the start to the end, the photo starts to gradually disappear from the end to the hide, It means that the photo disappears completely at that time.

  In addition, depending on the effect (for example, an effect in which photographs slide one after another in one effect), there are cases where it is desired to arbitrarily set the timing of the animation. The timing adjustment unit 126 can set a parameter called a key frame for such an effect, and can perform fine timing adjustment of the animation in the effect.

  FIG. 15 is an explanatory diagram illustrating an example of an animation in which key frame parameters are set by the timing adjustment unit 126. FIG. 15 shows the effect set to start the slide of the photograph at the time “keyframe1” and to end the slide of the photograph at the time “keyframe2”.

  The slide show output unit 128 outputs the slide show whose effect display timing is adjusted by the timing adjustment unit 126 to one file.

  All the basic information for realizing the slide show has been prepared up to the timing adjustment unit 126, but the image processing apparatus 100 decorates the slide show before outputting the slide show to one file by the slide show output unit 128. Information to be set may be set. For example, by detecting the main colors of the photos used in the slide show, a unique expression can be performed depending on the set theme. This color detection may be performed by the slide show output unit 128.

  FIG. 16 is an explanatory diagram illustrating an example of setting information for decorating a slide show. In the example shown in FIG. 16, it is possible to improve the appearance of the slide show by estimating the main colors of the photo group used in each effect section and changing the color of the light source according to the main colors. ing. For example, in the photograph on the left side of FIG. 16, since red is mainly used, the slide show output unit 128 detects red as a representative color and uses red as a light source in the section. Further, for example, in the photograph on the right side of FIG. 16, since green is mainly used, the slide show output unit 128 detects the green color as a representative color in the section and uses green as the light source. .

  The configuration of the image processing apparatus 100 according to an embodiment of the present disclosure has been described above. Next, the operation of the image processing apparatus 100 according to an embodiment of the present disclosure will be described.

[Operation of image processing apparatus]
FIG. 17 is a flowchart illustrating an operation of the image processing apparatus 100 according to an embodiment of the present disclosure. The flowchart illustrated in FIG. 17 illustrates an operation when the image processing apparatus 100 according to an embodiment of the present disclosure generates a slide show. Hereinafter, the operation of the image processing apparatus 100 according to an embodiment of the present disclosure will be described with reference to FIG.

  When the image processing apparatus 100 generates a slide show, the image processing apparatus 100 first uses the music metadata stored in the music metadata holding unit 104 to reproduce the music that is reproduced when the slide show is displayed. A section is cut out (step S101). The playback segment selection unit 106 executes the segmentation of the playback segment.

  When the music playback section is cut out in step S101, the image processing apparatus 100 subsequently sets an effect section from the music playback section (step S102). The effect section setting unit 108 executes setting of the effect section.

  When the effect section is set in step S102, the image processing apparatus 100 determines whether the number of effect sections set in step S102 is sufficient (step S103). This process is executed by the effect section setting unit 108, and whether or not the number of effect sections is sufficient may be determined by whether or not the number of effect sections is equal to or greater than a certain ratio with respect to the total number of images.

  If the result of determination in step S103 is that the number of effect sections is sufficient, then the image processing apparatus 100 assigns image candidates to the effect sections set in step S102 (step S104). The image candidate assignment is performed by the image candidate assignment unit 110, and the image candidate is assigned to the effect section using the image metadata held in the image metadata holding unit 114.

  When image candidates are assigned to the effect sections in step S104, the image processing apparatus 100 subsequently sets expected effects (expected effects) for the set individual effect sections (step S105). The effect setting unit 116 executes setting of an expected effect for each effect section.

  When the expected effect is set for each effect section in step S105, the image processing apparatus 100 determines whether or not the set expected effect can be realized for all the effect sections with the assigned image (see FIG. Step S106). The effect reset determination unit 120 determines whether this can be realized.

  If it is determined in step S106 that there is an effect that cannot be realized, the image processing apparatus 100 resets the effect for the effect section in which the effect that cannot be realized is set (step S107). This effect resetting is executed by the effect setting unit 116.

  If there are no unrealizable effects for all effect sections, the image processing apparatus 100 subsequently determines whether or not the effects have been reset in step S107 (step S108). The slide show quality judgment unit 122 determines whether or not the effect has been reset.

  If it is determined in step S108 that the effect in step S107 has not been reset, the image processing apparatus 100 subsequently selects an image displayed by the effect set in each effect section (step S109). . This display image selection is executed by the image selection unit 118.

  When the image displayed by the effect set in each effect section is selected in step S109, the image processing apparatus 100 subsequently sets the cutout area of the image selected in step S109 (step S110). The display range cutout unit 124 sets the cutout area of the selected image. If it is not necessary to cut out the display range of the image selected in step S109, the process in step S110 may be skipped.

  When the image clipping region is set in step S110, the image processing apparatus 100 subsequently generates a slide show using the set effect sections, the effects set in each effect section, and the images selected in each effect. (Step S111). The slide show is generated by the slide show output unit 128, and any file format can be used as the file format to be output as the slide show. Note that the image processing apparatus 100 may adjust the display timing of the effect before the slide show generation process in step S111. The adjustment of the display timing of this effect is executed by the timing adjustment unit 126.

  On the other hand, as a result of the determination in step S108, if the effect resetting in step S107 has been executed even once, the image processing apparatus 100 subsequently executes effect resetting according to the context (step S112). Specifically, the slide show quality determination unit 122 looks at the entire slide show and instructs the playback section selection unit 106, the effect section setting unit 108, the image candidate assignment unit 110, and the effect setting unit 116 to perform resetting.

  When the effect resetting according to the context is executed in step S112, the image processing apparatus 100 subsequently determines whether the effect being used is biased (step S113). The slide show quality determination unit 122 determines whether or not the effect is biased. For example, an effect that produces a similar effect is continuously used, or an effect that closes up the face is continuously used. It is determined whether the effect is biased by using a criterion such as whether it is used.

  As a result of the determination in step S113, when it is determined that there is no bias in the effect being used, the image processing apparatus 100 subsequently determines that the images input to the image processing apparatus 100 for creating a slide show have a certain ratio or more. It is determined whether it can be used (step S114). This determination is performed by the slide show quality determination unit 122. In the present disclosure, the determination process in step S114 may be skipped.

  As a result of the determination in step S114, if it is determined that the image can be used more than a certain ratio, the image processing apparatus 100 proceeds to the display image selection process in step S109. On the other hand, if it is determined that the effect being used is biased as a result of the determination in step S113, or if it is determined that the image cannot be used more than a certain percentage as a result of the determination in step S114. The image processing apparatus 100 determines whether the image candidate has been reassigned to the effect section (step S115). If the image candidate has not been reassigned to the effect section, the image processing apparatus 100 executes processing subsequent to the image candidate assignment processing in step S104. On the other hand, if the image candidate has been reassigned to the effect section, the image processing apparatus 100 determines whether or not the effect section has been reset (step S116). If the effect section has been reset, the image processing apparatus 100 executes the processing after the playback section cut-out processing in step S101. If the effect section has not been reset, the image processing apparatus 100 executes the processing after the effect section setting processing in step S102. To do.

  As described above, the image processing apparatus 100 according to the embodiment of the present disclosure selects a playback section of music to be played in accordance with a slide show, sets an effect section, assigns image candidates, sets an effect, and displays an image with an effect. It is possible to improve the quality of the slide show by recursively executing the selection.

[Effect section and effect settings using music analysis results]
Here, an effect section and effect setting processing using the result of music analysis will be described with further examples. As described above, the image processing apparatus 100 according to the present embodiment performs setting of effect sections and effects using the result of music analysis.

  The effect section setting unit 108 detects a non-diatonic chord, a dominant end, and a position where chords are frequently switched from the music key and chord progression obtained as a result of the music analysis, and acquires time information thereof. The effect section setting unit 108 also acquires a partial modulation position and a modulation position from the result of music analysis.

  The effect setting unit 116 determines whether the music to be played in accordance with the slide show is major or minor from the result of the music analysis, and selects a theme. The effect setting unit 116 selects an effect candidate including a plurality of effects from the determination result of the music tone. For example, the effect setting unit 116 selects a bright effect group for major music and a dark effect group for minor music. Note the image processing apparatus 100, for each effect, impart whether the look in advance which feature as a parameter, holds the parameters.

  Then, the effect setting unit 116 selects an effect suitable for the feature from the effect candidates, and sets the effect to be executed at an appropriate time and timing.

  FIG. 18 is an explanatory diagram illustrating an example in which the image processing apparatus 100 according to the present embodiment selects an effect according to music. FIG. 18 shows a case where, as a result of analyzing the music reproduced in accordance with the slide show, it is found that the key is the C major code and the code changes from F → G7 → Gsus4 → G7 → C. .

  In the example of music shown in FIG. 18, an unusual Gsus4 is used as a non-diatonic chord, and a measure in which this chord is used is a measure having an impressive phrase. Therefore, the image processing apparatus 100 selects an effect that is likely to remain in the impression of the person who sees the slide show in the effect section in which the Gsus 4 is used.

  In the example of music shown in FIG. 18, after a non-diatonic chord, a dominant end is used in which the chord advances four times from G7 (V) → C (I) and ends. Dominant termination is a chord progression that gives viewers a sense of termination, so it is effective to use a definite effect in a slideshow. Accordingly, the image processing apparatus 100 uses an effect that uses three photographs in the effect section in which G7 is used, and uses an effect that firmly shows one photograph in the effect section in which C is used.

  Another example is shown. FIG. 19 is an explanatory diagram illustrating an example in which the image processing apparatus 100 according to the present embodiment selects an effect according to music. In FIG. 19, as a result of analyzing the music to be played back in accordance with the slide show, it is found that the key is the C major code and the code changes from F → G7 → C → Bm7b5 → E7 → Am → C. Shows about.

  In the example of music shown in FIG. 19, partial modulation is made to Am. Since Am is a code that gives an inseparable impression, the image processing apparatus 100 sets an effect to be displayed in monochrome in an effect section in which the code advances in the order Bm7b5 → E7 → Am → C. As a result, it is possible to assign an unsatisfactory image to the slide show in the part that is partially modulated to Am.

  Thus, by setting an appropriate effect according to the chord progression, the image processing apparatus 100 according to the present embodiment can create a slide show in which photos are displayed with an effect that matches the tune. As described above, in order to set the appropriate effect in accordance with the chord progression, the image processing apparatus 100, for each effect, impart whether the look in advance which feature as a parameter, the parameter Keep it.

  FIG. 20 is an explanatory diagram illustrating an example of a correspondence relationship between a tune and an effect. Thus, by assigning the correspondence between music features and effects as parameters and holding the parameters, the image processing apparatus 100 creates a slide show in which photos are displayed with effects that match the music. can do.

  For example, a parameter is given to an effect so that an effect that attracts interest is used in a portion where a chord frequently switches at a short timing (eg, in bar units). Also, for example, in the part where the dominant ends, a parameter is given to the effect so that a moving effect, an effect that makes it possible to see one piece firmly, and an effect that makes one piece look good are used. Is done.

  Next, a detection example of partial modulation will be described. FIG. 21 is an explanatory diagram illustrating an example of partial modulation detected by the image processing apparatus 100. If the code normally circulates tonic, subdominant, dominant, and the key is a C major chord, then the tonic is C, Em, Am, the subdominant is Dm, F, and the dominant is G7, Bm (−5). If the key is Am (A minor code), the tonic is Am, C, the sub-dominant is Dm, Bm (−5), F, G7, and the dominant is Em.

  Here, when F → G7 → C → Bm (−5) → Em → Am → C, it can be seen that partial modulation of Am occurs at C → Bm (−5). If the key is a C major chord, since the subdominant is F, it should proceed to F, but proceed to Bm (−5), which is a subdominant when the key is Am.

  FIG. 22 is an explanatory diagram illustrating another detection example of partial modulation by the image processing apparatus 100. If the key is Am (A minor code) and the melodic minor scale, the tonic is Am, C (+5), F # m (-5), the subdominant is Dm, Bm, the dominant is Em, G # m ( -5).

  Here, when F → G7 → C → E7 → Am, it can be seen that partial modulation is performed to Am at C → E7. In the example illustrated in FIG. 22, the image processing apparatus 100 can detect partial modulation that results in a cut image.

  The image processing apparatus 100 according to the present embodiment can automatically select an effect based on the tone of an arbitrary song, such as a pop effect for a bright song and a nostalgic effect for a dark song. For example, the image processing apparatus 100 can select an impressive effect between beats using a non-diatonic chord that is an impressive melody while listening to a song, and after that it tends to be a chorus section. If it becomes a chorus section, it is possible to select a video effect that ends in the chorus section with expectation.

  Since the image processing apparatus 100 according to the present embodiment can detect a part with a sense of end by music analysis, a definite effect that assigns a certain amount of chords with less movement and a single photo is assigned. You can make the slide show sharper.

  Since the image processing apparatus 100 according to the present embodiment can detect a part of a song that is moving up, such as partial modulation or modulation, by music analysis, the video can be excited at the same time as the music swells. You can select an effect that will be used as a showcase.

  In addition, the image processing apparatus 100 according to the present embodiment can detect bars in which chords frequently change in units of beats by music analysis. Therefore, in such a bar, an effect of intense switching such that photographs are switched one after another is applied. Can match music and video effects.

<2. Summary>
As described above, the image processing apparatus 100 according to an embodiment of the present disclosure sets an effect section and an effect suitable for the effect section from music metadata obtained by analyzing the music. When setting the effect section, the image processing apparatus 100 extracts a characteristic portion in the music. Then, the image processing apparatus 100 assigns image data to each effect section, determines whether the assigned image data satisfies the effect condition, and determines an image to be displayed by the effect if it satisfies the condition. If not, it is recursively set with image data assignment, effect setting, and effect section setting.

  When the effect setting is performed again, the image processing apparatus 100 determines the consistency of the effect between the effect section in which the effect is reset and another effect section in which the effect is not reset. If there is an effect section that is not matched as a result of determining the consistency of the effect, the image processing apparatus 100 further sets the effect so that the matching is achieved.

  Accordingly, the image processing apparatus 100 according to an embodiment of the present disclosure can automatically generate a slide show from music metadata obtained by analyzing music and image metadata obtained by analyzing image data. The quality of the generated slide show can be improved.

  Note that the operation of the image processing apparatus 100 according to the embodiment of the present disclosure described above may be performed by hardware or software. When performed by software, for example, a CPU or other control device provided in the image processing apparatus 100 reads out and sequentially executes computer programs stored in a recording medium such as a ROM provided in the image processing apparatus 100. It may be performed depending on the situation.

  In addition, each component of the image processing apparatus 100 according to the embodiment of the present disclosure described above may be mounted on any hardware as long as each hardware can transmit and receive information to and from each other via a network. Further, a process performed by a certain processing unit may be realized by a single piece of hardware, or may be realized by a distributed process using a plurality of pieces of hardware.

  The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present disclosure belongs can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present disclosure.

In addition, this technique can also take the following structures.
(1)
An effect section setting section for setting a plurality of effect sections from a predetermined section for continuously displaying image data;
An image assigning unit that assigns the image data to each of the plurality of effect sections set by the effect section setting unit;
An effect setting section for setting an effect for each of the plurality of effect sections set by the effect section setting section;
Whether or not the effect set by the effect setting unit for each of the plurality of effect sections satisfies a condition for realizing each effect is determined using image metadata corresponding to the image data allocated by the image allocation unit. If there is an effect section that does not satisfy the condition, an effect reset determination unit that resets the effect of the effect section to the effect setting unit,
When the effect setting unit resets the effect by the effect reset determining unit, a consistency determining unit that determines the consistency between the effect section in which the effect is reset and another effect section,
An image processing apparatus comprising:
(2)
The image processing apparatus according to (1), wherein the effect section setting unit sets an effect section using music metadata obtained by analyzing music data reproduced when the image data is displayed.
(3)
The image processing device according to (2), wherein the effect section setting unit extracts a characteristic section from the music metadata and sets the section as an effect section.
(4)
The effect setting unit according to any one of (1) to (3), wherein the effect setting unit sets an effect using music metadata obtained by analyzing music data reproduced when the image data is displayed. Information processing device.
(5)
The image processing apparatus according to (4), wherein the effect setting unit selects an effect that matches the characteristics of the music data played in the effect section set by the effect section setting unit.
(6)
The consistency determining unit causes the effect setting unit to reset the effect when the consistency between the effect segment for which the effect has been reset and another effect segment is not achieved, from (1) above The image processing apparatus according to (5).
(7)
The consistency determining unit causes the image allocation unit to reallocate image data if consistency cannot be obtained in all effect sections even if the effect setting unit resets the effect, the image allocation unit reallocates image data. Image processing apparatus.
(8)
The consistency determining unit causes the effect interval setting unit to reset the effect interval if the consistency cannot be obtained in all the effect intervals even if the effect setting unit resets the effect, (6) or The image processing apparatus according to (7).
(9)
A playback section selection unit that selects a playback section of music data that is played back when the image data is displayed;
If the consistency determination unit does not achieve consistency in all effect sections even if the effect setting unit resets the effect, the consistency determination section sets the playback section of the music data to be played back when the image data is displayed. The image processing apparatus according to any one of (6) to (8), wherein the reproduction section selection unit is changed.
(10)
(1) to (9), further comprising an image selection unit that selects an image to be displayed in the effect section by the effect set by the effect setting unit from among images allocated to the effect section by the image allocation unit. An image processing apparatus according to any one of the above.
(11)
The image processing apparatus according to any one of (1) to (10), further including an image metadata holding unit that holds image metadata obtained by analyzing the image data.
(12)
The image processing according to any one of (1) to (11), further including a music metadata holding unit that holds music metadata obtained by analyzing music data reproduced when the image data is displayed. apparatus.
(13)
An effect section setting step for setting a plurality of effect sections from a predetermined section for continuously displaying image data;
An image assignment step for assigning image data to each of the plurality of effect sections set in the effect section setting step;
An effect setting step for setting an effect for each of the plurality of effect sections set in the effect section setting step;
The image metadata corresponding to the image data assigned in the image assignment step is used to determine whether the effect set for each of the plurality of effect sections in the effect setting step satisfies a condition for realizing each effect. If there is an effect section that does not satisfy the condition, an effect reset determination step for resetting the effect in the effect section,
When the effect is reset in the effect setting step by the effect reset determination step, a consistency determination step of determining consistency between the effect interval in which the effect is reset and another effect interval;
An image processing method comprising:
(14)
On the computer,
An effect section setting step for setting a plurality of effect sections from a predetermined section for continuously displaying image data;
An effect setting step for setting an effect for each of the plurality of effect sections set in the effect section setting step;
An image assignment step for assigning image data to each of the plurality of effect sections set in the effect section setting step;
The image metadata corresponding to the image data assigned in the image assignment step is used to determine whether the effect set for each of the plurality of effect sections in the effect setting step satisfies a condition for realizing each effect. If there is an effect section that does not satisfy the condition, an effect reset determination step for resetting the effect in the effect section,
When the effect is reset in the effect setting step by the effect reset determination step, a consistency determination step of determining consistency between the effect interval in which the effect is reset and another effect interval;
A computer program that executes

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 102 Music analysis part 104 Music metadata holding part 106 Playback area selection part 108 Effect area setting part 110 Image candidate assignment part 112 Image analysis part 114 Image metadata holding part 116 Effect setting part 118 Image selection part 120 Effect re-playing Setting determination unit 122 Slide show quality determination unit 124 Display range cutout unit 126 Timing adjustment unit 128 Slide show output unit

Claims (14)

An effect section setting section for setting a plurality of effect sections from a predetermined section for continuously displaying image data;
An image assigning unit that assigns the image data to each of the plurality of effect sections set by the effect section setting unit;
An effect setting section for setting an effect for each of the plurality of effect sections set by the effect section setting section;
Whether or not the effect set by the effect setting unit for each of the plurality of effect sections satisfies a condition for realizing each effect is determined using image metadata corresponding to the image data allocated by the image allocation unit. If there is an effect section that does not satisfy the condition, an effect reset determination unit that resets the effect of the effect section to the effect setting unit,
When the effect setting unit resets the effect by the effect reset determining unit, a consistency determining unit that determines the consistency between the effect section in which the effect is reset and another effect section,
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the effect section setting unit sets an effect section using music metadata obtained by analyzing music data reproduced when the image data is displayed.   The image processing apparatus according to claim 2, wherein the effect section setting unit extracts a characteristic section from the music metadata and sets the section as an effect section.   The information processing apparatus according to claim 1, wherein the effect setting unit sets an effect using music metadata obtained by analyzing music data reproduced when the image data is displayed.   The image processing apparatus according to claim 4, wherein the effect setting unit selects an effect that matches the characteristics of the music data reproduced in the effect section set by the effect section setting unit.   2. The consistency determination unit according to claim 1, wherein when the consistency between an effect section in which an effect is reset and another effect section is not consistent, the effect setting unit resets the effect. Image processing apparatus.   7. The consistency determination unit according to claim 6, wherein if the consistency is not obtained in all effect sections even if the effect setting unit resets the effect, the image allocation unit reassigns the image data. Image processing device.   The consistency determination unit causes the effect section setting unit to reset an effect section when consistency cannot be obtained in all effect sections even if the effect setting section resets the effect. Image processing apparatus. A playback section selection unit that selects a playback section of music data that is played back when the image data is displayed;
If the consistency determination unit does not achieve consistency in all effect sections even if the effect setting unit resets the effect, the consistency determination section sets the playback section of the music data to be played back when the image data is displayed. The image processing apparatus according to claim 6, wherein the image processing apparatus is changed by a reproduction section selection unit.   The image processing according to claim 1, further comprising: an image selection unit that selects an image to be displayed in the effect section by the effect set by the effect setting unit from images assigned by the image assignment unit to the effect section. apparatus.   The image processing apparatus according to claim 1, further comprising an image metadata holding unit that holds image metadata obtained by analyzing the image data.   The image processing apparatus according to claim 1, further comprising: a music metadata holding unit that holds music metadata obtained by analyzing music data that is reproduced when the image data is displayed. An effect section setting step for setting a plurality of effect sections from a predetermined section for continuously displaying image data;
An image assignment step for assigning image data to each of the plurality of effect sections set in the effect section setting step;
An effect setting step for setting an effect for each of the plurality of effect sections set in the effect section setting step;
The image metadata corresponding to the image data assigned in the image assignment step is used to determine whether the effect set for each of the plurality of effect sections in the effect setting step satisfies a condition for realizing each effect. If there is an effect section that does not satisfy the condition, an effect reset determination step for resetting the effect in the effect section,
When the effect is reset in the effect setting step by the effect reset determination step, a consistency determination step of determining consistency between the effect interval in which the effect is reset and another effect interval;
An image processing method comprising: On the computer,
An effect section setting step for setting a plurality of effect sections from a predetermined section for continuously displaying image data;
An effect setting step for setting an effect for each of the plurality of effect sections set in the effect section setting step;
An image assignment step for assigning image data to each of the plurality of effect sections set in the effect section setting step;
The image metadata corresponding to the image data assigned in the image assignment step is used to determine whether the effect set for each of the plurality of effect sections in the effect setting step satisfies a condition for realizing each effect. If there is an effect section that does not satisfy the condition, an effect reset determination step for resetting the effect in the effect section,
When the effect is reset in the effect setting step by the effect reset determination step, a consistency determination step of determining consistency between the effect interval in which the effect is reset and another effect interval;
A computer program that executes

Images