JP6179027B2 - Slide show creation server, user terminal, and slide show creation method - Google Patents

Description

  The present invention relates to a slide show creation server, a user terminal, and a slide show creation method for creating a slide show for displaying images according to reproduction of music data.

  2. Description of the Related Art Conventionally, a technique for generating a music slide show using images on a web in accordance with music is known. For example, the portable information terminal described in Patent Document 1 extracts lyrics by voice recognition during music reproduction, extracts keywords from the extracted lyrics based on a predetermined rule, and uses local storage or the Internet based on the keywords. Images are retrieved from the upper WEB page and displayed during music playback.

  Further, the query extraction device described in Patent Literature 2 classifies lyrics into a plurality of parts, determines a priority section in which a keyword group is preferentially selected, and searches for an image that matches the lyrics locally and locally. Yes. The image display device described in Patent Document 3 automatically determines a suitable display time when music lyrics and an image are synchronously reproduced, and prevents the image display from becoming extremely short or extremely long. The display is switched.

JP 2008-8954 A JP 2011-048729 A JP 2011-166386 A

Shintaro Funasawa, Hiromi Ishibe, Keiichiro Hoashi, Yasuhiro Takishima, Jiro Katto: Proposal of Web Image and Music Synchronized Playback System Considering Lyric Features, 8th Information Science and Technology Forum, E-034 (2009). D. A. Shamma, B. Pardo, and K. J. Hammond: MusicStory: a Personalized Music VIDeo Creator, Proceedings of the 13th Annual ACM International Conference on Multimedia, pp.563-566 (2005). R. Cai, L. Zhang, F. Jing, W. Lai, and W.-Y.Ma: Automated Music VIDeo Generation Using Web Image Resource, Proceedings of IEEE International Conference on Acoustic, Speech, and SignalProcessing, 2: pp. 737-740 (2007). Shintaro Funasawa, Hiromi Ishibe, Keiichiro Hoashi, Yasuhiro Takishima, Jiro Katto: Examination of automatic music classification for music search based on the impression of lyrics, 71st National Convention on Information Processing, 5R-2 (2009).

  However, in the existing technology as described above, a slide show that matches music is generated mainly using many images on the WEB, and the selection of individual users cannot be sufficiently reflected. On the other hand, even if the user himself / herself selects an image that matches the music from the beginning and creates a slide show, this is an unfamiliar operation and is difficult.

  The present invention has been made in view of such circumstances, and provides a slide show creation server, a user terminal, and a slide show creation method that allows even an unfamiliar user to easily obtain a unique slide show suitable for music. With the goal.

  (1) In order to achieve the above object, the slide show creation server of the present invention is a slide show creation server that creates a slide show that displays images in accordance with the reproduction of music data, and is an original slide show that identifies a series of image data. Correspondingly, a template management unit that manages the first image feature amount extracted as the image feature amount of the series of image data, and the image feature amount of the image data obtained from the image data source are used as the second image feature. Similarity is determined using the second feature quantity extraction unit that extracts the quantity and the extracted first and second image feature quantities, and the image data specified by the original slide show is similar to this. And a slide show generation unit for generating a slide show replaced with the image data of the image data source.

  In this way, the slide show creation server of the present invention replaces the image data specified in the original slide show with image data from a similar image data source, so that even an unfamiliar user can create a unique slide show that matches the music. Easy to get.

  (2) Further, in the slide show creation server of the present invention, the second feature amount extraction unit uses the image feature amount of the image data stored in the user DB as the image data source as the second image feature amount. It is characterized by extracting. As a result, the user can use the slide show created using the user-specific image data.

  (3) The slide show creation server of the present invention is characterized in that the user DB is permitted to upload image data only to a user identified as a specific user. Thus, the user can create a slide show using the image data uploaded by the individual.

  (4) In the slide show creation server of the present invention, the slide show generation unit selects image data to be replaced from image data stored in the user DB and associated with identification information of a specific user. It is characterized by. As a result, the user can create a unique slide show that is replaced with photographic image data taken by an individual.

  (5) The slide show creation server of the present invention is specified by the original slide show when the slide show generation unit compares the first and second image feature quantities with a similarity greater than a predetermined threshold. It is characterized by generating a slide show in which image data to be replaced is generated. This makes it possible to easily create a unique slide show from an original slide show prepared as an image that matches the characteristics of the music.

  (6) Further, in the slide show creation server of the present invention, the slide show generation unit determines a replaceability parameter set for the image data specified in the original slide show, and the replaceability parameter can be replaced. A slide show in which image data is replaced is generated for designated image data, and a slide show in which image data is replaced is not generated for image data in which the replaceability parameter is designated as non-replaceable.

  As a result, whether or not a specific image in the slide show is replaceable is set, and when the replaceable / unusable parameter designates that the image cannot be replaced, the image can be replaced. Thereby, an image that should not be replaced in the slide show can be left as it is.

  (7) Further, in the slide show creation server of the present invention, the second feature amount extraction unit uses the image feature amount of the online available image data as the image data source as the second image feature amount. It is characterized by extracting. Thereby, an original slide show can be prepared with image data available online.

  (8) Further, the slide show creation server of the present invention generates image data similar to the image data of the image data source based on the similarity determination using the extracted first and second image feature amounts. The music data associated with the slide show to be specified is selected. Thereby, the user can easily find music suitable for the image data.

  (9) The slide show creation server of the present invention is characterized in that the template management unit handles a slide show created by another user as the original slide show. As a result, a slide show can be created using the template based on the slide show provided by another user.

  (10) Also, the slide show creation server of the present invention transmits a slide show derived from the original slide show to a user terminal, and the image data of the replacement candidate or the data corresponding to the image data of the replacement candidate together with the transmitted slide show. A slide show distribution unit for transmitting is further provided. Accordingly, the replacement user image can be displayed in a selectable manner on the terminal of the specific user, and the specific user can select an image to be replaced from the images displayed in the selectable manner.

  (11) Also, in the slide show creation server of the present invention, the slide show generation unit generates a slide show in which the image data specified in the transmitted slide show is replaced with replacement candidate image data instructed from the user terminal. It is characterized by that. Thus, the server can replace the slide show image with an image designated by the specific user and create a slide show unique to the user.

  (12) The slide show creation server of the present invention is further characterized by further comprising a slide show delivery unit that delivers music and a slide show in a streaming manner in accordance with a reproduction instruction from the user terminal. As a result, the server performs streaming distribution of the created slide show according to the user's reproduction instruction. As a result, the slide show can be reproduced without imposing a burden on the terminal.

  (13) Also, the user terminal of the present invention uses the slide show received from the slide show creation server and the image data of the replacement candidate or the data corresponding to the image data of the replacement candidate, and uses the image corresponding to the division of the music. The display is selectable. Thereby, the user can create a slide show corresponding to the music by operating the terminal.

  (14) The slide show creation method of the present invention is a slide show creation method for creating a slide show for displaying an image in accordance with the reproduction of music data. The slide show creation method corresponds to an original slide show for specifying a series of image data, and Preparing a first image feature amount extracted as an image feature amount of the image data, extracting an image feature amount of image data obtained from an image data source as a second image feature amount, and Similarity is determined using the extracted first and second image feature amounts, and a slide show is generated in which the image data specified by the original slide show is replaced with image data of the image data source similar thereto. And a step. Thereby, even an inexperienced user can easily obtain a unique slide show that matches the music.

  According to the present invention, since the image data specified in the original slide show is replaced with image data from a similar image data source, even an inexperienced user can easily obtain a unique slide show that matches the music.

It is a block diagram which shows the slide show creation system of this invention. It is a block diagram which shows a template management part. It is a figure which shows the data structure of music DB. It is a table which shows a template. It is a flowchart which shows the process of whole impression generation. It is a flowchart which shows the image selection process using a lyric and an estimation keyword. It is a flowchart which shows a slide show creation process. It is a figure which shows the example of the photograph share screen in a user terminal. It is a figure which shows the example of an image upload screen. It is a figure which shows the example of an image upload screen. It is a figure which shows the example of the top screen of music selection. It is a figure which shows the example of the screen of a music list which can be reproduced | regenerated. It is a figure which shows the example of a slide show reproduction | regeneration preparation screen. It is a figure which shows the example of a slide show edit screen. It is a figure which shows the example of the screen of candidate image data list. It is a figure which shows the example of the screen of candidate image data list. . It is a figure which shows the example of the uploaded image selection screen. It is a figure which shows the example of the uploaded image selection screen. It is a figure which shows the example of a change confirmation screen. It is a figure which shows the example of a gallery image selection screen. It is a figure which shows the example of a screen of a preservation | save slide show list.

  Next, embodiments of the present invention will be described with reference to the drawings. In order to facilitate understanding of the description, the same reference numerals are given to the same components in the respective drawings, and duplicate descriptions are omitted.

[Slide show creation system]
(System configuration)
FIG. 1 is a block diagram showing a slide show creation system 10. The slide show creation system 10 includes a slide show creation server 100 and a user terminal 200. The slide show creation server 100 creates a slide show for displaying images according to the reproduction of music data. The user terminal 200 enables slide show editing and slide show reproduction.

(Configuration of slide show creation server)
As illustrated in FIG. 1, the slide show creation server 100 includes a template management unit 110, a user DB 120, a second feature amount extraction unit 130, a slide show generation unit 140, and a slide show distribution unit 150.

  The template management unit 110 manages the first image feature amount extracted as the image feature amount of the series of image data in association with the original slide show that specifies the series of image data. Details of the template management unit 110 will be described later.

  The user DB 120 stores image data captured by the user and user terminal information in association with user information. In the user DB 120, only a user identified as a specific user may be permitted to upload image data. Thus, the user can create a slide show using the image data uploaded by the individual.

  The second feature amount extraction unit 130 extracts the image feature amount of the image data obtained from the image data source by the image selection process as the second image feature amount. An image feature amount extracted in advance from the stored image data may be stored in the user DB 120 and used, or the image feature amount may be extracted in real time.

  For example, using OpenCV, the presence / absence of a person (face detection), color information (color moment: average / standard deviation of each color for each local region), edge information (edge direction histogram: edge point extracted by Canny Filter) Image feature quantities such as a luminance gradient histogram, texture information (Local Binary Pattern), and BoVW (feature point shape histogram) can be extracted.

  The second feature amount extraction unit 130 extracts the image feature amount of the image data stored in the user DB 120 corresponding to the specific user as the image data source as the second image feature amount. As a result, the user can use the slide show created using the user-specific image data. The image data source is preferably specific to the user, but may be on the WEB.

  The second feature amount extraction unit 130 may extract an image feature amount of online available image data as an image data source as the second image feature amount. Thereby, an original slide show can be prepared with image data available online.

  The slide show generation unit 140 determines similarity using the extracted first and second image feature amounts, and replaces the image data specified by the original slide show with image data of a similar image data source. Create a slide show. Thereby, even an inexperienced user can easily obtain a unique slide show that matches the music.

  The similarity determination can be performed by comparing the image feature stored in the image template with the image feature of the personal photograph and calculating the similarity. For example, the vector similarity between the image feature vector stored in the user DB 120 and the image feature vector stored in the template management unit 110 is calculated. Then, it is possible to generate a slide show in which the image data specified in the original slide show is replaced with image data whose similarity when the first and second image feature amounts are compared exceeds a predetermined threshold. This makes it possible to easily create a unique slide show from an original slide show prepared as an image that matches the characteristics of the music.

  As the vector similarity, for example, cosine similarity can be used. The similarity threshold value can be set to, for example, 0.70 or the vicinity thereof as a value that can be evaluated to be objectively similar, and images exceeding this value can be used. However, it is preferable that the user can arbitrarily set the similarity value.

  The slide show generation unit 140 may select image data to be replaced from image data stored in the user DB 120 and associated with identification information of a specific user. As a result, the user can create a unique slide show that is replaced with photographic image data taken by an individual.

  In addition, the slide show generation unit 140 determines a replaceability parameter set for the image data specified in the original slide show. The replaceability parameter indicates whether image data can be replaced. The slide show generation unit 140 generates a slide show in which the image data is replaced for the image data designated as replaceable by the replaceability parameter, and replaces the image data for the image data specified in the replaceability parameter as non-replaceable. Does not generate a slide show. As a result, the image can be replaced while leaving the image that should not be replaced in the slide show. In addition, it is preferable that the replacement possibility of the replacement possibility parameter can be set by the user.

  Further, when a specific candidate image is instructed from the user terminal 200, the slide show generation unit 140 replaces the image data specified in the slide show transmitted to the user terminal 200 with the specified replacement candidate image data. Create a slide show. Thereby, the slide show image can be replaced with an image designated by the user's intention, and a user-specific slide show can be created.

  In addition, the slide show generation unit 140 determines the similarity between the feature amount extracted from the image data group stored in the user DB 120 and the feature amount data associated with the music data, and the music data is stored from the stored image data. Data may be selectable. Thereby, the user can select music from a group of image data of photographs taken by individuals. In addition, the slide show generation unit 140 can create a slide show using a template of another user.

  The slide show distribution unit 150 transmits a slide show derived from the original slide show to the user terminal 200. The replacement candidate image data or data corresponding to the replacement candidate image data is also transmitted together with the transmitted slide show. Thereby, in the user terminal 200, the replacement candidate image can be displayed in a selectable manner, and the user can select an image to be replaced from the images displayed in the selectable manner. The slide show derived from the original slide show includes not only the original slide show but also a slide show created from the original slide show and a slide show derived therefrom.

  The slide show distribution unit performs streaming distribution of music and slide shows in accordance with a reproduction instruction from the user terminal 200. As a result, the server performs streaming distribution of the created slide show according to the user's reproduction instruction. As a result, the slide show can be reproduced without imposing a burden on the terminal.

(Template management department)
FIG. 2 is a block diagram showing the template management unit 110. As illustrated in FIG. 2, the template management unit 110 includes a music DB 111, an overall impression generation unit 112, an original slide show generation unit 113, a first feature quantity extraction unit 114, and a template creation unit 115.

  The music DB 111 stores not only music data information but also slide show data and template data. FIG. 3 is a diagram illustrating a data structure of the music DB 111. The music DB 111 describes, for example, a music ID as a unique ID given to each music, a music type indicating the presence / absence of lyrics, a lyrics in which lyrics are described for each line, a start time and an end time of each line of lyrics. Information such as a slide show image describing a synchronization information and a template image displayed for each row can be stored. Among the above, the slide show includes a music ID, synchronization information, and image specifying information.

  The music DB 111 also accumulates slide shows received from other users' terminals. Then, the slide show is stored as an original template, and the created templates are stored as usable templates. In this case, a template created from a slide show received from another user's terminal is shared among the users as a template for the other user.

  The overall impression generating unit 112 assigns the overall impression received from the lyrics as the overall impression label, with the music having the lyrics as learning data. For example, it is realized by the following processing. First, teacher data in which the overall impression label is assigned to the lyrics in advance is prepared. Applies morphological analysis to the lyrics of a song and breaks the lyrics into parts of speech. A representative word is extracted as an element of the lyric feature vector from the decomposed part of speech. Based on the extracted lyric feature vector, each whole impression word is created using a positive and negative two class discriminator, and whether all the whole impression words are correct or not is judged with respect to newly input lyrics. All the overall impression words determined as positive by the discriminator are set as the overall impression of the music.

  For example, Support Vector Machine (SVM) can be used as all the classifiers. Teacher data for parameter creation can be created as follows, for example. First, about 240 songs are prepared as learning data, and these lyrics data are acquired from the Music lyrics database. Then, the questionnaire is conducted with the songs sorted so that about 5 people will answer each song. The overall impression word to be used is determined from the result of the questionnaire, and the overall word impression word for which a majority of responses are obtained is set as the overall word impression of the music. The feature amount of SVM uses a TF IDF value based on the appearance probability of words obtained from all songs.

  Words can be obtained using morphological analysis (POS Tagger, MECAB) and the like. An overall impression word is assigned to a new song lyrics using a discriminator that assigns an overall impression word based on teacher data. The overall process for generating the overall impression will be described later.

  The original slide show generating unit 113 selects image data related to the corresponding category (for example, row) using the lyrics of the input music data or the estimated keyword. Then, a slide show for displaying the selected image in conjunction with the music during reproduction is generated. The image selection process will be described later.

  The first feature amount extraction unit 114 extracts image feature amounts corresponding to each image data from a series of image data specified in the original slide show. The function of extracting the image feature amount from the image data is the same as that of the second feature amount extraction unit 130.

  The template creation unit 115 stores the feature quantity obtained by applying the first feature quantity extraction unit 114 for the image data group obtained by the original slide show generation unit 113 as a template in the music DB 111. The template is created for comparison with the feature amount (second feature amount) of the photograph taken by the user, and image data similar to the feature amount of the template can be automatically extracted. The template creation unit 115 can create a template not only from the original slide show but also from the slide show stored in the music DB 111.

  FIG. 4 is a table showing templates. The line number is the line number of the lyrics, the lyrics are the lyrics, the display time column is the display start and end times, the image ID is the extracted image ID, the image feature column is the image feature, and the interchangeable column is Is stored with a parameter indicating whether or not it can be replaced with a personal photograph. The replaceability parameter can be set arbitrarily. As described above, the template includes the music ID, the synchronization information, and the image feature amount.

(User terminal configuration)
The user terminal 200 uses the slide show and the replacement candidate image data received from the slide show creation server 100 or the data corresponding to the replacement candidate image data, and displays the images corresponding to the music segments in a selectable manner. Thereby, the user can create a slide show corresponding to the music by operating the user terminal 200.

  The user terminal 200 is, for example, a mobile phone or a PC. As illustrated in FIG. 1, the user terminal 200 includes a photographing unit 210, an uploader 220, a slide show editing unit 230, and a playback unit 240. The photographing unit 210 has a function of taking a photograph with a camera and saving it as image data. For example, it can be realized by a camera application that provides a GUI for the user.

  The uploader 220 has a function of selecting one photograph from a photographed photograph and a gallery (past photographed photograph list) and a function of transmitting image data to the slide show creation server 100. Image data to be uploaded to the slide show creation server 100 may be taken by a camera, or a photo that has already been registered as a gallery may be selected. In addition, a sharing function for uploading selected image data and a function for adding tag information or transmitting terminal-specific information at the time of image uploading are also provided.

  The slide show editing unit 230 has a function of editing a slide show on the user terminal 200. For example, using the GUI, it is possible to replace the image data specified in the slide show with the image data of the photograph taken by itself, or to change the order of the image data in the slide show. The slide show created in this way can be used as another user's original slide show, and image feature amounts can be extracted from the slide show and used as templates for other users.

  The playback unit 240 has a function of playing back the distributed slide show. The slide show is played along with the delivered music. For example, a user's operation can be received through the GUI, and music and a slide show can be selected, played, and stopped. The slide show editing and playback operations will be described later.

[Operation of slide show creation server]
Next, the operation of the slide show creation server 100 configured as described above will be described.

(Overall impression generation process)
FIG. 5 is a flowchart showing the overall impression generation process. The slide show creation server 100 acquires a WEB image based on the music and creates an original slide show. First, when a song ID is input (step S1), lyrics data is extracted from the song DB 111 (step S2), and a lyrics feature vector is created by morphological analysis and key word extraction (step S3). Using the discriminator created from the teacher data, the correctness of the overall impression word is output based on the lyrics feature vector, and the overall impression label is finally given to the positive word (step S4). The overall impression word is not limited to Japanese.

(Image selection process)
Next, using the overall impression word thus obtained and the lyrics of the input music or the estimated keyword, image data related to the corresponding line is selected on the WEB. The selected image data is specified to be displayed in conjunction with the music during playback in the slide show. Image selection using lyrics or estimated keywords of the input music can be performed by the following process.

  FIG. 6 is a flowchart illustrating image selection processing using lyrics and estimated keywords. First, in the keyword extraction process, a keyword or an important word is extracted in units of division if the song is accompanied by lyrics as the synchronization data of the song, and if the song is a song without the lyrics (step T1). As the important word, for example, a word having a high TF IDF value can be extracted. In addition, part-of-speech decomposition is applied by a morphological analyzer, and words can be selected for each part-of-speech and used as keywords for image search.

  An image is searched using the image search keyword extracted in this manner (step T2). As an image search target, a WEB service such as FLICKR (registered trademark), a personal photo collection, or the like can be used. An image to be displayed in each row or division unit can be extracted by extracting one image for display from the image group obtained by the search (step T3). In this way, an original slide show and a template can be generated.

(Slide show creation process)
Next, a slide show is created by replacing images using the original slide show and the template. FIG. 7 is a flowchart showing the slide show creation process. First, a first image feature amount extracted as an image feature amount of image data is prepared as a template corresponding to an original slide show for specifying a series of image data (step P1).

  On the other hand, the image feature amount of the image data obtained from the image data source is extracted as the second image feature amount (step P2). The similarity between the first image feature quantity prepared as described above and the extracted second image feature quantity is determined (step P3), and the image data specified by the original slide show is similar to this. The slide show replaced with the image data of the image data source to be generated is generated (step P4). In this way, a user-specific slide show can be created.

[Screen display example]
The operation of the slide show creation system 10 will be described with reference to a display example of the screen 300. FIG. 8 is a diagram illustrating an example of a photo sharing screen on the user terminal 200. In the user terminal 200, a gallery can be selected by a “gallery” button 310 by an operation on the screen 300, and a photograph taken by a “shoot” button 315 can be selected, and image data M10 can be selected from the photograph. It has become.

  9 and 10 are diagrams showing examples of the image upload screen. On the screen 300, in order to upload the image data M10 of one selected photo to the slide show creation server 100, the uploader application can be activated by a “share” button 320. When the uploader application is activated on the user terminal 200, the selected image data M10 is displayed, and the image data can be uploaded to the slide show creation server 100 by pressing the upload button 330. In addition, by placing an input box 335 for assigning tag information at the time of server upload, the user can input tag information. In the input box 335, the user can input text. In this case, when a plurality of tags are set, they can be distinguished by using a half-width comma as a delimiter.

  FIG. 11 is a diagram illustrating an example of a music selection top screen. A screen 300 shown in FIG. 11 is a screen that appears immediately after the music slide show playback application is activated. A “START” button 345 is displayed on the screen 300 together with a logo, background, or cover 341 of the music slide show. For example, when the “START” button 345 is pressed, a transition is made to a “music selection screen”.

  12 and 13 are diagrams showing examples of the music selection screen. In FIG. 12, a list of reproducible music pieces is displayed on the “music selection screen”. For example, a music list acquired from the slide show creation server 100 and a music list stored in the user terminal 200 are displayed.

  On the “music selection screen”, music can be selected by tapping the music name or artist name area. In addition, a search function, a sort function, etc. can be added, and a new button can be installed. Further, when the saved slide show button 348 is pressed, a transition is made to a “saved slide show list screen”. Further, the music genre can be displayed as shown in FIG. By selecting genre information, it is possible to select music from the genre category or to reproduce all music corresponding to the genre.

  FIG. 14 is a diagram illustrating an example of a slide show editing screen. In the screen display shown in FIG. 14, a slide show of the selected music is displayed. After the music (or genre) is selected, the selected music can be played. When the “play” button 350 is pressed, the slide show is played along with the music. Further, when an “image editing” button 355 is pressed, a transition is made to an “image editing screen”.

  FIG. 15 is a diagram illustrating an example of a slide show image editing screen. As shown in FIG. 15, for example, representative image data A to C are displayed for each scene (row of synchronization data). The user can edit the slide show on the screen 300 of the user terminal 200.

  For example, the image data specified using the GUI and the image data specified by the slide show can be exchanged, or the order of the photos can be changed. By selecting one of the images, it is possible to change to the image selection screen and change the target image data. When the edit end button 360 is pressed, the screen changes to a slide show playback screen.

  When a specific image is selected on the image editing screen, a transition is made to the candidate image list screen selected by the slide show creation server 100. Candidate images B2 to B10 are recommended by FLICKR. FIG. 16 is a diagram illustrating an example of a candidate image list screen. Each image data can be displayed as thumbnails by the slide show creation server 100 and resized B2 to B10 (data corresponding to replacement candidate image data). When one of the images is selected, the representative image B (screen being selected) is changed, and a transition is made to the image editing screen.

  Transition to uploaded image and gallery image selection screen is possible. By selecting the previous button 384 or the next button 385, scene advance is possible. Although an image selected from a web service such as FLICKR is recommended as a default, when an “select from upload image” button 370 is pressed, it can also be selected from images uploaded by the user in the past.

  17 and 18 are diagrams showing examples of the image selection screen that has been uploaded. In the screen display example of FIG. 18, it is possible to transition from the gallery or from the received candidate image to the selection screen. When a “select from FLICKR image” button 380 is pressed, selection from an image selected from a web service such as FLICKR becomes possible. Furthermore, scene advance is possible by selecting the previous button 384 or the next button 385.

  FIG. 19 is a diagram illustrating an example of a change confirmation screen. When one image is selected on the screen shown in FIG. 17 or FIG. 18, a confirmation screen as shown in FIG. 19 is displayed, and the image can be replaced. When the “change to this image” button 391 is selected, the image is changed and a transition is made to the image editing screen. If the “delete” button 392 is selected, the image can be deleted from the server (the thumbnail is also deleted at the same time). Further, the “return” button 393 allows the user to return to the original image selection screen by selection.

  FIG. 20 is a diagram illustrating an example of a gallery image selection screen. In the example shown in FIG. 20, a list of photos taken by the terminal is displayed. The image data M40 of the photographed photograph and the image data M50 of the gallery are displayed in parallel on the screen. When one of these images is selected, the currently selected image is changed, and the image editing screen is displayed.

  You can also display a list of slideshows saved after the playback application is closed. FIG. 21 is a diagram illustrating a screen example of a saved slide show list. Playback can be performed by selecting any of the saved slideshows 401 to 403 and pressing a playback button 395. Further, a slide show that is no longer needed can be deleted by a delete button 396. When the title of the slide show is tapped, the title name can be edited.

10 slide show creation system 100 slide show creation server 110 template management unit 111 music DB
112 Overall impression generating unit 113 Original slide show generating unit 114 First feature quantity extracting unit 115 Template creating unit 120 User DB
130 second feature amount extraction unit 140 slide show generation unit 150 slide show distribution unit 200 user terminal 210 photographing unit 220 uploader 230 slide show editing unit 240 playback unit 300 screen

Claims (12)

A slide show creation server for creating a slide show for displaying images according to the reproduction of music data,
An original slide show that specifies a series of image data that matches the characteristics of a specific song is automatically generated from an image on the Web, and the song ID and synchronization information of the specific song are associated with the original slide show, A template management unit for storing a table including an image ID of a series of image data and a first image feature quantity extracted as an image feature quantity of the series of image data as a template;
Only the user identified as a specific user is allowed to upload image data. The image feature quantity of the image data stored in the user DB is used as the image data source, and the image feature quantity of the image data obtained from the image data source is obtained. A second feature amount extraction unit for extracting as a second image feature amount;
Using the first image feature value obtained from the stored template and the extracted second image feature value, image data specified by the original slide show and image data obtained from the image data source A slide show generation server, comprising: a slide show generation unit that determines similarity and generates a slide show in which image data specified by the original slide show is replaced with image data of the image data source similar to the image data . The slide show generation unit, the stored user DB, slide show creation server according to claim 1, wherein the image data associated with the identification information of the specific user selects image data to replace. The slide show generation unit generates a slide show in which the image data specified in the original slide show is replaced when the similarity when the first and second image feature quantities are compared is larger than a predetermined threshold. The slide show creation server according to claim 1 or 2 . The slide show generation unit determines a replaceability parameter set for the image data specified in the original slide show, and replaces the image data for the image data designated as replaceable by the replaceability parameter It generates a slide show, the slide show creation according to claim 3 for the image data to which the replacement permission parameter is specified as not substituted claim 1, characterized in that does not generate a slide show by replacing the image data server. The second feature amount extraction unit extracts an image feature amount of online available image data as the image data source as a second image feature amount. 4. The slide show creation server according to any one of 4 . The slide show generation unit is associated with a slide show that specifies image data similar to the image data of the image data source based on the similarity determination using the extracted first and second image feature amounts. slideshow creation server according to any one of claims 1 to 5, characterized by selecting the music data. The template management unit, slide show creation server according to any one of claims 1 to 6, characterized in that handle slideshow created by another user as the original slide show. A slide show distribution unit is further provided that transmits a slide show derived from the original slide show to a user terminal, and transmits the replacement candidate image data or data corresponding to the replacement candidate image data together with the transmitted slide show. The slide show creation server according to any one of claims 1 to 7 . 9. The slide show creation server according to claim 8 , wherein the slide show generation unit generates a slide show in which the image data specified in the transmitted slide show is replaced with replacement candidate image data instructed from the user terminal. . Wherein according to the reproduction instruction from the user terminal, according to claim 8 or claim 9, wherein the slide show creation server and further comprising a slide show distribution unit for streaming music and a slide show. 10. The slide show received from the slide show creation server according to claim 8 or 9, and the image data of the replacement candidate or the data corresponding to the image data of the replacement candidate are used to selectably display an image corresponding to a music segment. A user terminal characterized by that. A slide show creation method for creating a slide show that displays images according to the playback of music data,
An original slide show that specifies a series of image data that matches the characteristics of a specific song is automatically generated from an image on the Web, and the song ID and synchronization information of the specific song are associated with the original slide show, Storing a table including an image ID of a series of image data and a first image feature quantity extracted as an image feature quantity of the series of image data as a template;
Only the user identified as a specific user is allowed to upload image data. The image feature quantity of the image data stored in the user DB is used as the image data source, and the image feature quantity of the image data obtained from the image data source is obtained. Extracting as a second image feature amount;
Using the first image feature value obtained from the stored template and the extracted second image feature value, image data specified by the original slide show and image data obtained from the image data source Determining the similarity, and generating a slide show in which the image data specified by the original slide show is replaced with the image data of the image data source similar to the image data.

Images