JP2021157007A - Photo movie generation system, photo movie generation device, user terminal, photo movie generation method, and program - Google Patents

Abstract

To provide a photo movie generation system or the like which enables a user to play back the user's favorite music with a photo movie and enjoy watching and listening to the music.SOLUTION: In a photo movie generation system 1, a server 2 includes an acoustic analysis unit 22, an image acquisition unit 23, an image material generation unit 24, a photo movie generation unit 25, and the like. A user terminal 3 includes a music designation unit 31, an image designation unit 32, a transmitting unit 33, a receiving unit 34, and the like. When a music and an image are specified on the user terminal 3 and sound source data (or a music ID) or image data are transmitted to the server 2, the server 2 acquires acoustic feature data 45 of the sound source data, generates a photo movie whose screen changes according to a change in the composition of music which can be detected from the acoustic feature data 45, and transmits the photo movie to the user terminal 3. The user terminal 3 plays back the photo movie in synchronization with the sound source data.SELECTED DRAWING: Figure 4

Description

The present invention relates to a photo movie generation system and the like that generate and provide a photo movie.

In the cloud photo storage service, a service for automatically creating a photo movie using a photo stored in the cloud storage is provided (Non-Patent Document 1). For example, in the cloud photo storage service of Non-Patent Document 1, a user selects a photo or video to be made into a movie from an image saved in the cloud storage, and selects a desired theme from several prepared themes. , The movie is automatically generated on the service side.

In addition, a method of adding a musical piece having an impression to a photographed moving image has been reported (Non-Patent Document 2). In Non-Patent Document 2, the feature amount (video feature amount) of the captured moving image is extracted, the musical feature amount (music feature amount) is extracted, the relationship between the moving image and the impression of the melody / rhythm is calculated, and the user A method of generating a music that matches the impression of is proposed.

On the other hand, Patent Document 1 describes an expression acquisition unit that acquires an acoustic expression representing an impression or timing in arbitrary acoustic data, an effect imparting unit that imparts a visual effect or the like corresponding to the acoustic expression to an image or the like, acoustic data, and the acoustic data. A sound reproduction device including a display for reproducing an image or the like to which an effect is given is described. As a result, it is possible to acquire an acoustic expression of acoustic content such as commercially available music data and add a visual effect or the like according to the acoustic expression.

Japanese Unexamined Patent Publication No. 2013-114088

Google (registered trademark) photos, "Create movie", [online], [Search on March 23, 2nd year of Reiwa], Internet <URL: https://photos.google.com/movies/create> Yurina Shimizu, 4 others, "Automatic selection of video BGM materials based on impression estimation from video features", NICOGRAPH 2016, pp177-184

However, in the movie creation function of Non-Patent Document 1 described above, the user can select the music (BGM) to be added to the photo movie, but the music that can be selected is the default category unit (dramatic, electronic, rockin). ', upbeat, ... etc.) Or DRM (Digital) of the music in the music app on the user's terminal
Only songs that are not protected by Rights Management (digital rights management). Further, the proposed method of Non-Patent Document 2 is a method of generating a musical piece that matches the impression of a moving image. Therefore, neither method can use the music obtained from a commercially available CD or music distribution service for the BGM of the photo movie. On the other hand, the method of Patent Document 1 extracts an acoustic expression from commercially available music data and makes it possible to add a visual effect or the like according to the acoustic expression, but the visual effect set for the expression is provided. Is fixed, so the user may get bored after watching it several times.

The present invention has been made in view of the above problems, and provides a photo movie generation system or the like that enables a user's favorite music to be played with a photo movie and allows the user to enjoy listening to the music. The purpose.

The first invention for solving the above-mentioned problems is a photo movie generation system in which a server and a user terminal are communicated and connected via a network, and the user terminal specifies a music to be used for the photo movie. It is provided with a music designation means for receiving and transmitting sound source data of a designated music or a music ID which is identification information of a music to the server, and a playback means for receiving and playing a photo movie transmitted from the server. The server acquired the acoustic feature data acquisition means for acquiring the acoustic feature data of the music designated on the user terminal, the image acquisition means for acquiring the image used for the photo movie, and the image acquisition means. A photo movie generation characterized by comprising a photo movie generation means for generating a photo movie whose display changes according to a change in music based on the acoustic feature amount data using an image and transmitting the photo movie to the user terminal. It is a system.

According to the first invention, when the server acquires the acoustic feature data of the music specified on the user terminal and acquires the image to be used for the photo movie, the server uses the acquired image and is based on the acoustic feature data. Generates a photo movie whose display changes according to changes in music and sends it to the user terminal. The user terminal receives and generates a photo movie. As a result, for the music specified by the user, a photo movie whose display changes according to the change of the music can be generated and provided to the user terminal, so that the user can watch the favorite music with the photo movie. Therefore, the user can enjoy listening to music more.

In the photo movie generation system of the first invention, the server includes a storage means for storing movie configuration information, which is information constituting the photo movie, in association with the music ID, and the photo movie generation means is designated. The movie composition information stored in association with the stored music ID is referred to, and a photo movie having a configuration different from the movie composition information stored in the storage means is generated. As a result, different movie expressions can be generated for the same music and provided to the user terminal, so that the user can be provided with various enjoyments.

Further, the storage means of the server stores the acoustic feature amount data of the music in association with the music ID, and the acoustic feature amount data acquisition means acquires the acoustic feature amount data stored in the storage means. It is desirable to be characterized by doing so. As a result, acoustic analysis can be omitted for the music once analyzed, so that a photo movie can be provided in a short time. Further, the acoustic feature data acquisition means of the server may acquire the acoustic feature data of the music designated by the user terminal from the music distribution server communicated and connected via the network. As a result, acoustic feature data can be acquired without performing acoustic analysis processing on the server side, and the load on the server can be reduced.

Further, the server further includes an acoustic analysis means for performing an acoustic analysis process of music to obtain acoustic feature amount data, and the acoustic feature amount data acquisition means acquires the acoustic feature amount data obtained by the acoustic analysis means. You may do so. At this time, the server may acquire the sound source data of the music from the music distribution server communicated and connected via the network, or may acquire the sound source data from the user terminal. This makes it possible to generate a photo movie that matches the music desired by the user.

Further, the photo movie generation means of the server generates the photo movie without adding a sound source, associates it with the music ID of the music, and outputs the photo movie, and the playback means of the user terminal associates the photo movie with the photo movie. It is desirable to acquire the sound source data of the music corresponding to the obtained music ID, and to synchronize the acquired sound source data with the photo movie and play it back. As a result, a photo movie having a smaller file size than a photo movie with music can be generated, and the storage capacity for storing the photo movie can be saved.

Further, the server further includes an image material generation means for generating an image material from an image acquired by the image acquisition means, and the photo movie generation means uses the image material generated by the image material generation means. It is desirable to generate the photo movie.
Further, it is desirable that the user terminal further includes an image transmission means for transmitting an image used for the photo movie to the server, and the image acquisition means of the server receives an image transmitted from the user terminal. As a result, a photo movie can be generated using an image specified by the user and an image material generated from the image.

The second invention uses an image acquisition means for acquiring an image, an acoustic feature amount data acquisition means for acquiring the acoustic feature amount data of a music, and an image acquired by the image acquisition means to obtain the acoustic feature amount data. It is a photo movie generation device including a photo movie generation means for generating a photo movie whose display changes according to a change in music based on the above.

According to the second invention, it is possible to generate a photo movie whose display changes according to a change in the music based on the acoustic feature data of the music.

A third invention includes an image acquisition means for acquiring an image, a sound source data acquisition means for acquiring sound source data of music, an acoustic analysis means for analyzing the sound source data and acquiring acoustic feature amount data, and the image acquisition. The photo movie generation device is characterized by comprising a photo movie generation means for generating a photo movie whose display changes according to a change in music based on the acoustic feature amount data using an image acquired by the means. ..

According to the third invention, it is possible to perform acoustic analysis on the acquired sound source data to acquire acoustic feature data, and to generate a photo movie whose display changes according to a change in music based on the acoustic feature data. It becomes.

The fourth invention is a user terminal capable of communicating with a server via a network, accepting a designation of a musical piece, and transmitting the sound source data of the designated musical piece or the musical piece ID which is the identification information of the musical piece to the server. The user terminal is characterized by comprising a music designation means and a reproduction means for receiving and reproducing a photo movie whose display changes according to a change in the music transmitted from the server.

According to the fourth invention, when a music piece is specified on the user terminal and the music sound source data of the music piece or the music ID which is the identification information of the music piece is transmitted to the server, a photo movie whose display changes according to the change of the music piece is produced. It can be received from the server and played back.

A fifth invention is a photo movie generation method in a photo movie generation system in which a server and a user terminal are communicated and connected via a network, and the user terminal accepts and designates a song to be used for the photo movie. A step of transmitting the sound source data of the music or a music ID which is identification information of the music to the server, a step of the server acquiring an image, and an acoustic feature of the music specified by the server on the user terminal. A step of acquiring quantity data, a step of the server using the image, generating a photo movie whose display changes according to a change of music based on the acoustic feature quantity data, and a step of transmitting it to the user terminal. The user terminal is a photo movie generation method including a step of receiving and playing back a photo movie transmitted from the server.

According to the fifth invention, when the server acquires the acoustic feature data of the music specified on the user terminal and acquires the image to be used for the photo movie, the server uses the acquired image and is based on the acoustic feature data. Generates a photo movie whose display changes according to changes in music and sends it to the user terminal. The user terminal receives and generates a photo movie. As a result, for the music specified by the user, a photo movie whose display changes according to the change of the music can be generated and provided to the user terminal, so that the user can watch the favorite music with the photo movie. Therefore, the user can enjoy listening to music more.

A sixth invention uses a computer with an image acquisition means for acquiring an image, an acoustic feature amount data acquisition means for acquiring acoustic feature amount data of music, and an image acquired by the image acquisition means, and the acoustic feature amount data. This is a program for functioning as a photo movie generation means for generating a photo movie whose display changes according to a change in music based on the above.

According to the sixth invention, the computer can function as the photo movie generator of the second invention.

According to a seventh aspect of the present invention, a computer is provided with an image acquisition means for acquiring an image, a sound source data acquisition means for acquiring sound source data of music, an acoustic analysis means for analyzing the sound source data and acquiring acoustic feature amount data, and the image acquisition. This is a program for functioning as a photo movie generation means for generating a photo movie whose display changes according to a change in music based on the acoustic feature amount data using the image acquired by the means.

According to the seventh invention, the computer can function as the photo movie generator of the third invention.

Eighth invention is a music designation means for receiving a music designation by a computer capable of communicating with a server via a network and transmitting the music sound source data of the designated music or the music ID which is the identification information of the music to the server. , A program for functioning as a reproduction means for receiving and reproducing a photo movie whose display changes according to a change in the music transmitted from the server.

The eighth invention makes it possible to make a computer function as a user terminal of the first and fourth inventions.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a photo movie generation system or the like in which a user's favorite music can be played with a photo movie and the user can enjoy listening to the music.

The figure which shows the whole structure of the photo movie generation system 1. The figure which shows the hardware configuration of a server 2. The figure which shows the hardware configuration of a user terminal 3. The figure which shows the functional structure of the photo movie generation system 1. The figure which shows the data composition example of the acoustic feature amount data 45 A flowchart showing the flow of processing executed by the photo movie generation system 1. Flowchart showing the flow of acoustic analysis processing The figure which shows the example of the effect template 221 The figure which shows the example of the effect correspondence table 223 Flowchart showing the flow of photo movie generation processing The figure which shows the example of the photo movie 61, 62, 63 The figure which shows the example of the movie composition information 224 The figure which shows the whole structure of the photo movie generation system 1A The figure which shows the hardware configuration of a music distribution server 4. The figure which shows the functional structure of the photo movie generation system 1A The figure which shows the data composition example of the sound source data 44 Flowchart showing the flow of processing executed by the photo movie generation system 1A

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram showing an overall configuration of a photo movie generation system 1 according to a first embodiment of the present invention. As shown in FIG. 1, in the photo movie generation system 1 according to the present invention, the user terminal 3 and the server 2 are communicated and connected via the network 5. The server 2 is a computer (photo movie generation device) that generates a photo movie in response to a request from the user terminal 3, and is connected to the user terminal 3 by communication.

The user terminal 3 is an electronic device used by the user, and is equipped with a dedicated application program (hereinafter, referred to as a photo movie application) for using the photo movie generation system 1. Alternatively, the user terminal 3 is equipped with a browser that allows the server 2 to browse the WEB site opened on the network 5, and orders and receives the photo movie by performing processing with the server 2 via the WEB site. It may be a thing. The user terminal 3 is composed of, for example, a smartphone, a tablet, a PC, a music player, a game machine, or the like.

FIG. 2 is a diagram showing the configuration of the server 2. As shown in the figure, the server 2 can be realized by a computer configured by connecting, for example, a control unit 201, a storage unit 202, a communication unit 203, and the like by a bus 204 or the like. However, the present invention is not limited to this, and various configurations can be taken as appropriate.

The control unit 201 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU calls and executes a program stored in the storage unit 202, ROM, recording medium, etc. in the work memory area on the RAM, and drives and controls each unit connected via the bus 204. The ROM permanently holds a computer boot program, a program such as a BIOS, data, and the like. The RAM temporarily holds the loaded program and data, and also includes a work area used by the control unit 201 to perform various processes. The control unit 201 functions as each means of the server 2 by reading and executing the above program.

The storage unit 202 is, for example, a storage device such as a hard disk drive, a solid state drive, or a flash memory. The storage unit 202 stores a program executed by the control unit 201, data necessary for program execution, an operating system, and the like. These program codes are read by the control unit 201 as necessary, transferred to the RAM, read by the CPU, and executed. Further, as shown in FIG. 4, the storage unit 202 of the server 2 stores data necessary for generating a photo movie such as the effect template 221 and the general image material 222, the effect compatible table 223, and the movie configuration information 224. NS. Details of these data will be described later.

The communication unit 203 has a communication control device, a communication port, and the like, and controls communication with the network 5 and the like. The network 5 is a LAN (Local Area Network) or a WAN (Wide) connected to a wider area.
Area Network), or public communication lines such as the Internet, base stations, etc. The network 5 may be wired or wireless. The server 2 communicates with the user terminal 3 via the network 5 and can transmit and receive various data.
The bus 204 is a route that mediates the transfer of control signals, data signals, and the like between the devices.

FIG. 3 is a diagram showing the configuration of the user terminal 3. As shown in the figure, the user terminal 3 uses, for example, a control unit 301, a storage unit 302, a communication unit 303, a display unit 304, an input unit 305, a peripheral device I / F unit 306, an audio output unit 307, and the like as a bus 308. It can be realized by a computer or the like configured by connecting with. However, the present invention is not limited to this, and various configurations can be taken as appropriate. The configuration of the control unit 301, the storage unit 302, and the communication unit 303 is the same as the configuration of the control unit 201, the storage unit 202, and the communication unit 203 of the server 2.

The display unit 304 is composed of, for example, a display device such as a liquid crystal panel or a CRT monitor, and a logic circuit (video adapter or the like) for executing display processing in cooperation with the display device, and is input under the control of the control unit 301. Display the display information on the display device. The input unit 305 and the display unit 304 may be a touch panel display in which an input device such as a touch panel is integrally provided on the display screen.

The input unit 305 is, for example, an input device such as a touch panel, a keyboard, a pointing device such as a mouse, or a numeric keypad, and outputs the input data to the control unit 301.

The peripheral device I / F (interface) unit 306 is a port for connecting peripheral devices, and transmits / receives data to / from the peripheral device via the peripheral device I / F unit 306. The peripheral device I / F unit 306 is composed of USB or the like, and usually has a plurality of peripheral device I / Fs. The connection form with peripheral devices may be wired or wireless.

The audio output unit 307 outputs the audio data input from the control unit 301 from the speaker.

An application program (photo movie application) for executing the process (FIG. 6) described later is stored in the storage unit 302 of the user terminal 3, and the control unit 301 of the user terminal 3 performs the process described later according to this application program. Run.

Next, the functional configuration of the photo movie generation system 1 will be described with reference to FIG. As shown in FIG. 4, in the photo movie generation system 1, in addition to the above-mentioned storage unit 202, the server 2 includes a sound source data acquisition unit 21, an acoustic analysis unit 22, an image acquisition unit 23, an image material generation unit 24, and a photo. A movie generation unit 25 and the like are provided. The user terminal 3 includes the above-mentioned storage unit 302, display unit 304, audio output unit 307, music designation unit 31, image designation unit 32, transmission unit 33, reception unit 34, and the like.

The music designation unit 31 of the user terminal 3 receives a music designation from the user. In the first embodiment, the music designation unit 31 designates a music by the user's operation from the sound source data stored in the storage unit 302 of the user terminal 3 or the sound source data read from a CD or the like.

The image designation unit 32 receives from the user the designation of an image to be used as the image material of the photo movie. The designated image may be an image stored in the storage unit 302 of the user terminal 3, or may be designated from the general image material 222 stored in the storage unit 302 of the server 2. When the general image material 222 stored in the storage unit 302 of the server 2 is specified, for example, the server 2 transmits an image selection screen for selecting the general image material 222 to the user terminal 3, and the user selects the image material 222. Accept.

The transmission unit 33 acquires the sound source data of the music designated by the music designation unit 31 and the image data designated by the image designation unit 32 from the storage unit 302 and transmits the data data to the server 2. When the designated image is the general image material 222 of the server 2, the designated image information (image identification information, etc.) is transmitted.

In the storage unit 302, the sound source data of the music is stored in association with the music ID which is the identification information of the music. The sound source data of the music is the sound source data read from the CD, the sound source data downloaded from the server of the music distribution service, or the like. In addition, image data is stored in the storage unit 302. The image data is an image taken by a user with a camera, an image read by a scanner, an image downloaded via a network 5, or the like.

For the music ID, for example, composite information such as album name + artist name + music name + playback time is used. In the music CD database (CDDB; Compact Disc DataBase), the album name, artist name, and song name are Disc.
It is managed in association with the ID. The music application mounted on the user terminal 3 calculates the Disc ID from the information recorded in the area called TOC (Table of Contents) of the music CD, accesses the music CD database via the network 5, and discs. The album name, artist name, and song name corresponding to the ID can be obtained. In the present embodiment, the information obtained by combining the album name, artist name, and song name obtained in this way with the playback time of the song and the like is used as the song ID.

The sound source data acquisition unit 21 of the server 2 acquires the sound source data and the music ID transmitted from the user terminal 3.

The acoustic analysis unit 22 performs an acoustic analysis process on the sound source data acquired by the sound source data acquisition unit 21, and acquires the acoustic feature amount data 45.

Here, the acoustic analysis processing by the acoustic analysis unit 22 and the acoustic feature amount data 45 will be described. FIG. 5 is a diagram showing an example of acoustic feature amount data 45. In the acoustic analysis process, feature quantities such as music composition (A melody, B melody, chorus, etc.), music tempo (BPM; Beats Per Minute), beat (beat), and sounding timing of a specific musical instrument are analyzed. In addition to these, features such as chord progression (flow of music tone), volume change (audible volume change), frequency center of gravity (degree of excitement of music), pitch power separation (complexity of music), etc. May be analyzed.

Acoustic analysis processing is performed by known methods (for example, CRI Middleware Co., Ltd., "Ultra-high-speed, high-precision music analysis middleware BEATWIZ (registered trademark)", [onine], [Search on March 15, 2nd year of Reiwa], Internet. , <URL: https://www.cri-mw.co.jp/product/amusement/beatwiz/index.html>, or Naoaki Horiuchi, co-authored by 5 others, "Song
Surfing: A music playback system that walks through a music library with similar phrases ", PIONEER R & D (Vol.17, No.2 / 2007), etc.) can be used. The acoustic feature data 45 shown in FIG. Is an example, and the present invention is not limited to this. The acoustic analysis unit 22 may also analyze feature quantities other than each item of the acoustic feature quantity data 45 shown in FIG.

The acoustic analysis unit 22 outputs the acoustic feature amount data 45 acquired as a result of the acoustic analysis process to the photo movie generation unit 25, and stores it in the storage unit 202 in association with the music ID. The acoustic analysis unit 22 only acquires the acoustic feature amount data 45 stored in the storage unit 202 for the music that has been subjected to the acoustic analysis processing once. As a result, the load on the server 2 required for the acoustic analysis processing can be reduced, and the processing time can be reduced.

The image acquisition unit 23 acquires the image data transmitted from the user terminal 3 or the designated information of the general image material 222. When the designated information of the general image material 222 is acquired, the general image material 222 is acquired from the storage unit 202 according to the designated information.

The image material generation unit 24 generates an image material from the image data acquired by the image acquisition unit 23. The image material generation unit 24 detects an object such as a person, an animal, or an article from image data such as a photograph, cuts out the detected object, and uses each as an individual image material. The image material generation unit 24 outputs the generated image material to the photo movie generation unit 25.

The photo movie generation unit 25 uses the image acquired by the image acquisition unit 23 or the image material generated by the image material generation unit 24, and responds to changes in the music based on the acoustic feature amount data 45 acquired from the acoustic analysis unit 22. Generate a photo movie whose display changes. The photo movie generation unit 25 refers to the effect correspondence table 223 and the effect template 221 stored in the storage unit 202 to generate a photo movie whose display changes according to a change in the music. Changes in the music are detected from the "music composition" of the acoustic feature data 45, the "pronunciation timing of a specific musical instrument", and the like. The details of the photo movie generation process will be described later.

The photo movie generated by the photo movie generation unit 25 is a photo movie with music or a photo movie without music. The photo movie with music is a photo movie in which the sound source data of the music designated from the user terminal 3 is added as BGM. That is, it is a photo movie including video data and audio data. A photo movie without music is a photo movie containing only video data without adding a sound source (BGM) of music. The photo movie without music is played in synchronization with the sound source data of the music at the time of playback.

When the photo movie is generated, the photo movie generation unit 25 compresses and encodes the data (video data) by the mpeg4 method or the like, and converts the data (video data) into a data format that can be reproduced by the user terminal 3. Then, the movie configuration information 224, which is information representing the configuration of the photo movie, is associated with the music ID and stored in the storage unit 202, and the photo movie data is associated with the music ID and transmitted to the user terminal 3.

The receiving unit 34 of the user terminal 3 receives the photo movie transmitted from the server 2 and sends it to the playback processing unit 35.

The reproduction processing unit 35 reproduces a photo movie. When the photo movie without music is received, the playback processing unit 35 acquires the sound source data of the music corresponding to the music ID associated with the photo movie from the storage unit 302, and the acquired sound source data and the photo movie (no music). Play in sync with the photo movie). The reproduction processing unit 35 decodes the photo movie into video data before encoding, displays it on the display unit 304, and synchronously reproduces the sound source data and outputs it from the audio output unit 307.

When the photo movie with music is received, the playback processing unit 35 decodes the received photo movie data into video data and audio data before encoding, displays the video on the display unit 304, and audio (sound source). ) Is output from the audio output unit 307. The method of playing the photo movie may be streaming playback or download playback.

Next, the flow of processing executed by the photo movie generation system 1 will be described with reference to FIG. In the following description, a photo movie application, which is an application for using the photo movie generation system 1, is installed in the storage unit 302 of the user terminal 3, and the control unit 301 of the user terminal 3 reads out the photo movie application. The procedure to be executed will be described.

When the photo movie application is started on the user terminal 3, the control unit 301 of the user terminal 3 accepts the designation of the music to be used for the photo movie (step S101) and the designation of the image (step S102). In steps S101 to S102, the control unit 301 may display an input screen for selecting and inputting music or images.

In the first embodiment, the music that can be specified is the music (sound source data) stored in the storage unit 302 of the user terminal 3 or the sound source data of the music that can be read by the CD drive. Further, the images that can be specified are the image stored in the storage unit 302 of the user terminal 3 and the general image material 222 stored in the server 2.

When the music and the image are designated, the control unit 301 acquires the designated sound source data and the image data from the storage unit 302 (step S103) and transmits the designated sound source data and the image data to the server 2 (step S104). When the general image material 222 is specified, the control unit 301 transmits the identification information of the designated general image material 222 to the server 2.

When the server 2 receives the sound source data and the image data (identification information of the general image material 222 when the general image material 222 is specified) (step S105), the control unit 201 (acoustic analysis unit 22) receives the data. The acoustic analysis process is executed on the sound source data, and the acoustic feature amount data 45 of the music is acquired (step S106).

FIG. 7 is a flowchart showing the flow of the acoustic analysis process. As shown in FIG. 7, the control unit 201 (acoustic analysis unit 22) acquires the sound source data of the music (step S201) and executes the acoustic analysis process (step S202). In the acoustic analysis process, the tempo (BPM; Beats Per Minute), beat (beat), music composition (A melody, B melody, chorus, etc.) of a musical piece, the sounding timing of a specific musical instrument, and the like are analyzed.

The control unit 201 stores the acoustic analysis result as the acoustic feature amount data 45 in the storage unit 202 in association with the music ID (step S203). By associating the acoustic feature amount data 45 with the music ID and storing it in the server 2, it is not necessary to analyze the music once analyzed from the second time onward.

Further, the control unit 201 (image material generation unit 24) of the server 2 executes an image material generation process as necessary for the image data received in step S105, and cuts out the image material from the image data (step S107). The received image data may be used as it is as an image material.

The control unit 201 (photo movie generation unit 25) of the server 2 uses the image material and the predetermined effect template 221 and changes the screen according to the acoustic feature amount data 45 (for example, a change in the “music composition”). Generate a movie (step S108).

Here, the effect template 221 and the effect corresponding table 223 will be described.
The effect template 221 is a program for applying a predetermined processing (effect) to an image, and as shown in FIG. 8, a plurality of effect templates 221 have effect Nos. And it is stored in association with the number of images used.

The effect template 221 is, for example, "image switching" for switching a plurality of display images in order, "monochrome → color" for changing an image from monochrome to color, "transparency change" for changing the transparency of an image, and an image display position. "Move screen" to move the image on the screen, "Rotate" to rotate the image, "Enlarge" to enlarge the image, "Reduce" to reduce the image, "Change display ratio" to change the display ratio of the image on the display screen. , "Appearance" to make an image appear, "Enlargement move" to move an image while enlarging it, "Enlargement rotation" to rotate an image while enlarging it, "Reduce movement" to move an image while reducing it, "Reduced rotation" to rotate while, "Small image increase" to gradually increase small images, "Multiple images on one screen" to display multiple images on one screen, Multiple images displayed on one screen There is "multiple image single screen display + slide" etc. to slide.

The effect template 221 shown in FIG. 8 is an example, and may include other effects. Further, a plurality of patterns of parameters such as the amount of movement and the speed of change may be prepared for each effect. For example, for the effect "magnification", "magnification 1", "magnification 2", "magnification 3", ... , "Transmittance change 2" ...

The control unit 201 (photo movie generation unit 25) determines which effect to use when generating the photo movie with reference to the effect correspondence table 223. FIG. 9 is a diagram showing an example of the effect corresponding table 223. The effect-compatible table 223 is a table in which music components (intro, verse, B-melody, C-melody, solo, chorus, etc.) are associated with an effect template 221 suitable for them, and as shown in FIG. 9, a plurality of them are associated with each other. Pattern is defined.

In the example of FIG. 9, in the case of the pattern (1), the control unit 201 (photo movie generation unit 25) applies the effect "monochrome → color" in the "intro" of the music, and the effect "A melody" in the music "A melody". Apply "Move 3", apply the effect "Small image increase" with the song "B melody", apply "Multiple image single screen display 1" with the song "C melody", and apply the effect with the song "Solo" Apply "Enlargement 1" and apply the effect "Image switching 1" on the "Sabi" of the song.

Further, in the case of the pattern (2) of FIG. 9, the control unit 201 (photo movie generation unit 25) applies the effect "image switching 1" in the "intro" of the music, and the effect "reduction" in the "verse" of the music. Apply "Rotation 1", apply the effect "Enlargement Rotation 1" with the song "B Melo", apply the effect "Screen Move" with "C Melo", and apply the effect "Enlargement 2" with the song "Solo". Apply and apply the effect "Image switching 2" with the song "Sabi". As described above, the effect application examples of a plurality of patterns are defined in the effect correspondence table 223.

In step S108, the control unit 201 (photo movie generation unit 25) determines which pattern in the effect-corresponding table 223 is used to generate the photo movie, for example, by the process shown in FIG.

As shown in FIG. 10, the control unit 201 (photo movie generation unit 25) of the server 2 first determines whether or not the movie configuration information 224 corresponding to the music ID is stored in the storage unit 202 (step S301). ). The movie configuration information 224 is data in which information for constructing a photo movie, such as image data used for the photo movie, image switching timing, and effects applied to the image, is stored (see FIG. 12). For the photo movie generated by the server 2, the movie configuration information 224 is stored in the storage unit 202 in association with the music ID.

In step S301, if the movie configuration information 224 corresponding to the music ID of the sound source data transmitted from the user terminal 3 is not stored in the storage unit 202 (step S301; No), a photo movie is created for the music. Since there is no history, the control unit 201 of the server 2 reads an arbitrary pattern from the effect correspondence table 223 (step S302), and generates a photo movie according to the effect correspondence table 223 of the read pattern (step S303).

On the other hand, in step S301, when the movie configuration information 224 corresponding to the music ID of the sound source data transmitted from the user terminal 3 is already stored in the storage unit 202 (step S301; Yes), the control unit 201 has an effect. An unused pattern is read from the corresponding table 223 (step S304), and a photo movie is generated according to the effect corresponding table 223 of the read pattern (step S303). In this way, for music that has created a photo movie, a photo movie with a different pattern is created from the second time onward. Therefore, different movie expressions can be generated for the same music, and various enjoyments can be provided to the user.

FIG. 11 is a diagram showing examples of various photo movies 61, 62, 63 generated for music (sound source data 44). The photo movie 61 of FIG. 11A is generated by using the images 71, 72, 73, ... Acquired by the image acquisition unit 23 as they are. The image to be displayed is changed at the timing when the music composition of the music (sound source data 44) changes (timing t1 when the verse changes to B melody, timing t2 when the music changes from B melody to chorus, etc.).

Further, in the photo movie 62 of FIG. 11B, the effect template 221 is applied to the images 71, 72, 73, ... Acquired by the image acquisition unit 23. The image is changed at the timing when the music composition of the music (sound source data 44) changes (timing t1 when changing from verse to B melody, timing t2 when changing from B melody to chorus), and each section (t0 to t1, t1 to 1). A predetermined effect is applied to the image at t2, t2).

In the example of FIG. 11B, the effect “monochrome → color” is applied to the image 71 in the sections t0 to t1 of the “verse”, and the monochrome image 71a is gradually changed to the color image 71b. Next, the image is switched at the time t1 when the verse changes from the verse to the B melody, and the effect is also changed to "reduced rotation 1". In the sections t1 to t2, the image 71 is used as the background and the image 72 is used as the foreground, and an animation is generated in which the foreground image 72 is reduced and rotated from the state of the image 72a to the state of the image 72b. Next, the image is switched at the time t2 when the B melody changes to the chorus, and the effect is also changed to "reduced rotation 2". In the sections t2 and 2, the image 71 is used as the background and the image 73 is used as the foreground, and an animation is generated in which the foreground image 73 is reduced and rotated from the state of the image 73a to the state of the image 73b.

Further, in the photo movie 63 of FIG. 11C, images 74, 76, 79, ... Acquired by the image acquisition unit 23 are used as background images, and image materials 75, 77, 81, 82 (image material generation unit 24) are used in the foreground. It was generated using the image material (image material generated by). The background image is changed at the timing when the music composition of the music (sound source data 44) changes (timing t1 when changing from verse to B melody, timing t2 when changing from B melody to chorus), and each section (t0 to t1, t1). ~ T2, t2 ~) applies a predetermined effect to the foreground image.

In the example of FIG. 11C, the image 74 is displayed as a background in the sections t0 to t1 of the verse, and the effect "screen movement" is applied to the image material 75 in the foreground to move the image 75a → 75b → 75c. .. Next, the background image is switched to the image 76 at the time t1 when the verse changes from the verse to the B melody, and the effect "alternate display" is applied to the image material 77 in the foreground, and the images 77a and 77b are displayed alternately. Animation is generated. Next, the background image is switched to the image 79 and the foreground image is also changed to the image materials 81 and 82 at the time t2 when the B melody changes to the chorus. In the sections t2-, the effect "appearance" is applied to the image materials 81 and 82 in the foreground, and the image materials 81 and 82 are displayed so as to appear in order in accordance with the sounding timing of the bass drum.

When the control unit 201 (photo movie generation unit 25) generates a photo movie, the control unit 201 (photo movie generation unit 25) stores the generated photo movie movie configuration information 224 in the storage unit 202 in association with the music ID. As shown in FIG. 12, the movie configuration information 224 stores the music composition, time, applied effect, number of images used, and image file name used in association with the music ID.

An example of FIG. 12 is movie configuration information 224 of a photo movie in which the music of the music ID “***** (album name + artist name + song name + playback time, etc.)” is used as BGM. The time of the section of the music composition "intro" is "0-15", and the effect of "monochrome → color 1" is applied. The number of images used in this effect is "1", and the file name of the image used is "image15".

Further, the time of the section of the music composition "A melody" is "15-50", and the effect of "enlarged rotation 3" is applied. The number of images used in this effect is "2", and the file name of the image used is "image8" (enlarged and rotated image) in the foreground and "image2" in the background.

Further, the time of the section of the music composition "B melody" is "50-70", and the effect of "reduction rotation 2" is applied. The number of images used in this effect is "2", and the file name of the image used is "image8" (reduced and rotated image) in the foreground and "image17" in the background.

Further, the time of the section of the music composition "rust" is "70-120", and the effect of "image switching 5" is applied. The number of images used in this effect is "5", and the file names of the images used are "image10" (first displayed image), "image11" (second displayed image), and "image12". "(Third displayed image)," image13 "(fourth displayed image)," image14 "(fifth displayed image).

Further, the time (time) of the music composition "bass drum timing" is "70, 110, 111, 112, 113", and the effect of "appearance 5" is applied. The number of images used in this effect is "6", and the file names of the images used are "image15" for the background and "person 1" for the foreground (the image that appears at the first bass drum timing (70)). "Person 2" (image appearing at the second bass drum timing (110)), "Person 3" (image appearing at the third bass drum timing (111)), "Person 4" (fourth bass drum) An image that appears at the timing (112)) and a "person 5" (an image that appears at the fifth bass drum timing (113)).

The items of "music composition" and "time" of the movie composition information 224 correspond to the acoustic feature amount data 45 of the music. The item of "applied effect" is acquired from the effect correspondence table 223 shown in FIG. 9, and the item of "number of images used" is acquired from the effect template 221 shown in FIG. Which effect-corresponding table 223 is used is determined by the control unit 201 (photo movie generation unit 25) executing the process of FIG. Further, the item of "image file name to be used" is arbitrarily determined by the control unit 201 (photo movie generation unit 25) from the image acquired by the image acquisition unit 23 and the image generated by the image material generation unit 24.

Returning to the description of FIG.
When the photo movie is generated by the process of step S108, the server 2 transmits the generated photo movie to the user terminal 3 (step S109).

The user terminal 3 receives the photo movie transmitted from the server 2 (step S110) and reproduces the photo movie (step S111). When the received photo movie is a photo movie with music, the control unit 301 (reproduction processing unit 35) decodes and reproduces the received photo movie to synchronize the music and the video.

When the received photo movie is a photo movie without music, the control unit 301 (reproduction processing unit 35) reads the designated music from the storage unit 302 and reproduces the music in synchronization with the photo movie. The control unit 301 (reproduction processing unit 35) of the user terminal 3 synchronously reproduces the photo movie and the sound source data, so that the photo movie whose screen changes according to the change in the music composition is reproduced.

As described above, in the photo movie generation system 1 of the first embodiment, when music and images are specified on the user terminal 3 and sound source data or image data is transmitted to the server 2, the sound source data is sounded on the server 2 side. The analysis process is performed, and a photo movie is generated based on the acoustic feature amount data 45. The user terminal 3 receives the photo movie generated by the server 2, and in the case of a photo movie without music, synchronizes and reproduces it with the sound source data.

As described above, the photo movie generation system 1 can generate a photo movie whose screen changes according to the change of the music that can be detected from the acoustic feature amount data 45 and provide the user with the photo movie. Further, the server 2 stores and stores the movie configuration information 224 for the music for which the photo movie has been generated before, and generates a photo movie of another pattern, so that the user can be provided with various enjoyments. Further, since the server 2 stores and stores the acoustic feature amount data 45 for the previously acquired sound source data, the acoustic analysis process can be omitted and the photo movie can be generated in a short time. Further, by generating a photo movie without music and playing it in synchronization with the sound source data on the user terminal 3, the file size can be made smaller than that of the photo movie with music, and the storage capacity and communication capacity can be reduced.

[Second Embodiment]
Next, the photo movie generation system 1A of the second embodiment will be described. In the second embodiment, the system configuration when the user terminal 3 does not have the sound source data will be described.

FIG. 13 is a diagram showing the overall configuration of the photo movie generation system 1A according to the second embodiment of the present invention. As shown in the figure, in the photo movie generation system 1A, the user terminal 3 and the server 2 are communicated and connected via the network 5. Further, the user terminal 3 and the server 2 can be communicated and connected to the music distribution server 4 provided with the music database 40 via the network 5. The hardware configuration of the server 2 and the user terminal 3 is the same as that of the first embodiment. The same parts as those in the first embodiment are designated by the same reference numerals, and duplicate description will be omitted.

FIG. 14 is a diagram showing the configuration of the music distribution server 4. As shown in the figure, the music distribution server 4 can be realized by, for example, a computer configured by connecting a control unit 401, a storage unit 402, a communication unit 403, and the like by a bus 404 or the like. However, the present invention is not limited to this, and various configurations can be taken as appropriate. The configuration of the control unit 401, the storage unit 402, and the communication unit 403 is the same as the configuration of the control unit 201, the storage unit 202, and the communication unit 203 of the server 2. The music database 40 of the music distribution server 4 stores music sound source data 44 and acoustic feature data 45.

Next, the functional configuration of the photo movie generation system 1A of the second embodiment will be described with reference to FIG. As shown in the figure, in the photo movie generation system 1A, the server 2 includes a storage unit 202, an acoustic feature data acquisition unit 26, an image acquisition unit 23, an image material generation unit 24, a photo movie generation unit 25, and the like. The user terminal 3 includes a storage unit 302, a display unit 304, an audio output unit 307, a music designation unit 31, an image designation unit 32, a transmission unit 33, a reception unit 34, a sound source data acquisition unit 36, and the like. The music distribution server 4 includes a music database 40, a transmission unit 41, and an acoustic analysis unit 42.

The difference between the photo movie generation system 1A of the second embodiment and the photo movie generation system 1 of the first embodiment is that the server 2 does not have the sound source data acquisition unit 21 and the acoustic analysis unit 22, and the acoustic feature amount data. A point where an acquisition unit 26 is provided, a point where a sound source data acquisition unit 36 is provided without storing sound source data in the storage unit 302 of the user terminal 3, and a music distribution server 4 having a music database 40 and an acoustic analysis unit 42 are provided. This is the point.

The music distribution server 4 stores the sound source data 44 of various songs, the acoustic feature amount data 45, the music lyrics information, the music journal information, and the like in the music database 40 in association with the music ID. As shown in FIG. 16, the sound source data 44 is stored in association with the music ID.

The acoustic analysis unit 42 of the music distribution server 4 performs acoustic analysis processing on the music stored in the music database 40, obtains acoustic feature data 45 as the analysis result, associates it with the music ID, and stores it in the music database 40. do. The acoustic feature amount data 45 is the same as the acoustic feature amount data 45 shown in FIG.

The transmission unit 41 of the music distribution server 4 transmits the sound source data 44 to the user terminal 3 in response to a request from the user terminal 3. Further, the sound source data 44 and the acoustic feature amount data 45 are transmitted to the server 2 in response to the request from the server 2.

When the acoustic feature data acquisition unit 26 of the server 2 receives the music ID of the music designated by the music designation unit 31 of the user terminal 3, the acoustic feature data 45 of the music corresponding to the music ID is transmitted to the music distribution server 4. Request and get.

The sound source data acquisition unit 36 of the user terminal 3 requests and acquires the sound source data 44 of the music designated by the music designation unit 31 from the music distribution server 4. When the music-free photo movie is transmitted from the server 2, the playback processing unit 35 of the user terminal 3 acquires the sound source data from the music distribution server 4 by the sound source data acquisition unit 36, and synchronizes and plays back with the received music-free photo movie. ..

Next, the flow of processing in the photo movie generation system 1A of the second embodiment will be described with reference to FIG.
When the photo movie application is started on the user terminal 3, the control unit 301 of the user terminal 3 accepts the designation of the music to be used for the photo movie (step S401) and the designation of the image (step S402). In steps S401 to S402, the control unit 301 may display an input screen for selecting and inputting music or images.

In the second embodiment, the music that can be specified is the music (sound source data 44) stored in the music database 40 of the music distribution server 4. Further, the images that can be specified are the image stored in the storage unit 302 of the user terminal 3 and the general image material 222 stored in the server 2 as in the first embodiment.

When the music and the image are designated, the control unit 301 acquires the designated image data from the storage unit 302 (step S403) and transmits it to the server 2 together with the music ID of the designated music (step S404). When the general image material 222 is specified, the control unit 301 transmits the identification information of the designated general image material 222 to the server 2.

When the server 2 receives the music ID and the image data (identification information of the general image material 222 when the general image material 222 is specified) (step S405), the control unit 201 (acoustic feature amount data acquisition unit 26) receives the music ID and the image data (identification information of the general image material 222). , Requests the music distribution server 4 for the acoustic feature amount data 45 corresponding to the received music ID (step S406).

When the music distribution server 4 receives the request for the acoustic feature amount data 45 corresponding to the music ID from the server 2 (step S407), the music distribution server 4 reads the acoustic feature amount data 45 corresponding to the requested music ID from the music database 40 and reads the sound feature amount data 45 corresponding to the requested music ID from the music database 40. (Step S408). The server 2 receives the acoustic feature amount data 45 corresponding to the music ID (step S409).

In step S408, when the acoustic feature amount data 45 corresponding to the music database 40 of the music distribution server 4 is not stored, the music distribution server 4 acoustically analyzes the music of the requested music ID by the acoustic analysis unit 42. Is executed, and the acoustic feature amount data 45 is obtained. The music distribution server 4 associates the acoustic feature amount data 45 with the music ID, stores it in the music database 40, and transmits it to the requesting server 2.

Further, the control unit 201 (image material generation unit 24) of the server 2 executes an image material generation process as necessary for the image data received in step S405, and cuts out the image material from the image data (step S410). The received image data may be used as it is as an image material.

The control unit 201 (photo movie generation unit 25) of the server 2 uses the image material and the predetermined effect template 221 and changes the screen according to the acoustic feature amount data 45 (for example, a change in the “music composition”). Generate a movie (photo movie without music) (step S411).

The method of generating the photo movie is the same as that of the first embodiment. That is, as shown in FIG. 10, the control unit 201 (photo movie generation unit 25) determines whether or not the movie configuration information 224 corresponding to the music ID designated from the user terminal 3 is already stored in the storage unit 202. If it is determined and not stored in the storage unit 202, a photo movie is generated from the effect correspondence table 223 using the effect correspondence table 223 of an arbitrary pattern.

Further, when the movie configuration information 224 corresponding to the music ID designated from the user terminal 3 is already stored in the storage unit 202, the control unit 201 changes from the effect correspondence table 223 to the effect correspondence table 223 of an unused pattern. (Step S304), and a photo movie is generated according to the effect correspondence table 223 of the read pattern.

When the control unit 201 (photo movie generation unit 25) generates a photo movie, the control unit 201 (photo movie generation unit 25) stores the generated photo movie movie configuration information 224 in the storage unit 202 in association with the music ID. As shown in FIG. 12, the movie configuration information 224 stores the music composition, time, applied effect, number of images used, and image file name used in association with the music ID. Further, the server 2 transmits the generated photo movie to the user terminal 3 (step S412).

When the user terminal 3 receives the photo movie from the server 2 (step S413), the user terminal 3 performs a playback process. Since the received photo movie is a photo movie without music, the control unit 301 (reproduction processing unit 35) requests the music distribution server 4 for the sound source data 44 of the music ID of the music specified in step S401 (step S414).

When the music distribution server 4 receives the request for the sound source data 44 of the music ID from the user terminal 3 (step S415), the music distribution server 4 reads the sound source data 44 of the requested music ID from the music database 40 and transmits it to the requesting user terminal 3. (Step S416).

When the user terminal 3 receives the sound source data 44 from the music distribution server 4 (step S417), the control unit 301 (reproduction processing unit 35) of the user terminal 3 receives the music-free photo movie and the sound source data 44 in step S413. Are synchronized and reproduced (step S418). This makes it possible to play a photo movie whose screen changes according to changes in the composition of the music.

In the above example, the procedure for generating a photo movie without music has been described. However, when requesting the server 2 to transmit the acoustic feature amount data 45 to the music distribution server 4 in step S406, the sound source data It is also possible to generate a photo movie with music by requesting the transmission of 44 and acquiring the sound source data 44 of the corresponding music. When the photo movie with music is generated by the server 2, the user terminal 3 receives the photo movie with music, and the control unit 301 (playback processing unit 35) decodes and plays the received photo movie to produce music and video. Played at the same time.

As described above, in the second embodiment, the photo movie generation system 1A in which the user terminal 3 does not have sound source data and the server 2 acquires the acoustic feature amount data 45 from the music distribution server 4 to generate a photo movie. explained. According to the photo movie generation system 1A of the second embodiment, the server 2 does not perform the acoustic analysis process, but acquires the already analyzed acoustic feature data 45 from the music distribution server 4 and uses it for generating the photo movie. Therefore, the processing load on the server 2 is reduced. Further, since the sound source data 44 is not transmitted / received between the user terminal 3 and the server 2, the amount of communication is reduced and the time required for communication is shortened.

Although preferred embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to such examples. For example, the music distribution server 4 does not have the acoustic analysis unit 42 or the acoustic feature amount data 45, but the server 2 includes the acoustic analysis unit 42, performs acoustic analysis processing on the music received from the music distribution server 4, and performs acoustic analysis processing on each music. The acoustic feature amount data 45 may be stored and stored.

Further, in the above-described embodiment, the server 2 mainly generates an example of generating a photo movie whose display changes according to a change in the music composition, but the present invention is not limited to this, and changes other than the music composition For example, a photo movie whose display changes according to a change in the tempo of a musical piece, a change in a beat (beat), or the like may be generated. In addition, it is clear that a person skilled in the art can come up with various modified examples or modified examples within the scope of the technical idea disclosed in the present application, and these also naturally belong to the technical scope of the present invention. It is understood that it is.

1 ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ Photo movie generation system 2 ・ ・ ・ ・ ・ ・ ・ ・ Server 3 ・ ・ ・ ・ ・ ・ ・ ・ User terminal 4 ・ ・ ・ ・ ・ ・ ・ ・・ Music distribution server 5 ・ ・ ・ ・ ・ ・ ・ ・ Network 21 ・ ・ ・ ・ ・ ・ ・ ・ ・ Sound source data acquisition unit 22 ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・・ ・ ・ Image acquisition unit 24 ・ ・ ・ ・ ・ ・ ・ ・ ・ Image material generation unit 25 ・ ・ ・ ・ ・ ・ ・ ・ ・ Photo movie generation unit 26 ・ ・ ・ ・ ・ ・ ・ ・ Acoustic feature data acquisition unit 31 ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ Music designation unit 32 ・ ・ ・ ・ ・ ・ ・ ・ ・ Image designation unit 33 ・ ・ ・ ・ ・ ・ ・ ・ Transmission unit 34 ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ Reception unit 35 ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ Playback processing unit 36 ・ ・ ・ ・ ・ ・ ・ ・ ・ Sound source data acquisition unit 40 ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ Music database 41 ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ Transmission Part 42 ・ ・ ・ ・ ・ ・ ・ ・ ・ Acoustic analysis part 44 ・ ・ ・ ・ ・ ・ ・ ・ ・ Sound source data 45 ・ ・ ・ ・ ・ ・ ・ ・ Acoustic feature data 61, 62, 63 ・ ・ ・ Photo movie 221 ・ ・ ・ ・ ・ ・ ・ ・ Effect template 222 ・ ・ ・ ・ ・ ・ ・ ・ General image material 223 ・ ・ ・ ・ ・ ・ ・ ・ Effect compatible table 224 ・ ・ ・ ・ ・ ・ ・ ・ Movie configuration information 201, 301 ·········································································································.・ ・ ・ ・ ・ Peripheral device I / F unit 307 ・ ・ ・ ・ ・ ・ ・ ・ Voice processing unit

Claims (17)

A photo movie generation system in which a server and a user terminal are connected via a network.
The user terminal is
A music designation means that accepts the designation of the music to be used for the photo movie and transmits the sound source data of the designated music or the music ID that is the identification information of the music to the server.
A playback means for receiving and playing back a photo movie transmitted from the server is provided.
The server
An acoustic feature data acquisition means for acquiring acoustic feature data of a music designated on the user terminal, and an acoustic feature data acquisition means.
An image acquisition means for acquiring an image used for the photo movie, and
A photo movie generation means that uses an image acquired by the image acquisition means to generate a photo movie whose display changes according to a change in music based on the acoustic feature amount data, and transmits the photo movie to the user terminal.
A photo movie generation system characterized by being equipped with. The server
A storage means for storing movie composition information, which is information constituting the photo movie, in association with the music ID is provided.
The photo movie generation means is characterized in that it refers to the movie configuration information stored in association with the designated music ID and generates a photo movie having a configuration different from the movie configuration information stored in the storage means. The photo movie generation system according to claim 1. The storage means of the server stores the acoustic feature amount data of the music in association with the music ID.
The photo movie generation device according to claim 2, wherein the acoustic feature data acquisition means acquires acoustic feature data stored in the storage means. The acoustic feature data acquisition means of the server is
The photo movie generation system according to claim 1 or 2, wherein the acoustic feature data of the music designated by the user terminal is acquired from a music distribution server communicatively connected via a network. The server further includes an acoustic analysis means for obtaining acoustic feature data by performing acoustic analysis processing of music.
The photo movie generation system according to claim 1 or 2, wherein the acoustic feature data acquisition means acquires acoustic feature data obtained by the acoustic analysis means. The photo movie generation system according to claim 5, wherein the server acquires sound source data of the music from a music distribution server communicatively connected via a network. The photo movie generation system according to claim 5, wherein the server acquires sound source data of the music from the user terminal. The photo movie generation means of the server generates the photo movie without adding a sound source, and outputs the photo movie in association with the music ID of the music.
A claim characterized in that the playback means of the user terminal acquires sound source data of a music corresponding to a music ID associated with the photo movie, and synchronizes the acquired sound source data with the photo movie to play the music. The photo movie generation system according to any one of claims 1 to 7. The server
An image material generation means for generating an image material from an image acquired by the image acquisition means is further provided.
The photo movie generation system according to any one of claims 1 to 8, wherein the photo movie generation means generates the photo movie by using the image material generated by the image material generation means. .. The user terminal further includes an image transmission means for transmitting an image used for the photo movie to the server.
The photo movie generation system according to any one of claims 1 to 9, wherein the image acquisition means of the server receives an image transmitted from the user terminal. Image acquisition means to acquire images and
Acoustic feature data acquisition means for acquiring acoustic feature data of music,
A photo movie generation means that uses an image acquired by the image acquisition means to generate a photo movie whose display changes according to a change in music based on the acoustic feature amount data.
A photo movie generator characterized by being equipped with. Image acquisition means to acquire images and
Sound source data acquisition means for acquiring music sound source data,
An acoustic analysis means that analyzes the sound source data and acquires acoustic feature data,
A photo movie generation means that uses an image acquired by the image acquisition means to generate a photo movie whose display changes according to a change in music based on the acoustic feature amount data.
A photo movie generator characterized by being equipped with. A user terminal that can communicate with the server via a network.
A music designation means that accepts the designation of a music and transmits the sound source data of the designated music or the music ID that is the identification information of the music to the server.
A playback means for receiving and playing a photo movie whose display changes according to a change in the music transmitted from the server.
A user terminal characterized by being provided with. This is a photo movie generation method in a photo movie generation system in which a server and a user terminal are communicated and connected via a network.
A step in which the user terminal accepts the designation of the music to be used for the photo movie and transmits the sound source data of the designated music or the music ID which is the identification information of the music to the server.
When the server acquires an image,
The step that the server acquires the acoustic feature amount data of the music specified in the user terminal,
A step in which the server uses the image to generate a photo movie whose display changes according to a change in music based on the acoustic feature amount data, and transmits the photo movie to the user terminal.
A step in which the user terminal receives and plays back a photo movie transmitted from the server.
A photo movie generation method characterized by including. Computer,
Image acquisition means to acquire images,
Acoustic feature data acquisition means for acquiring acoustic feature data of music,
A photo movie generation means that uses an image acquired by the image acquisition means to generate a photo movie whose display changes according to a change in music based on the acoustic feature amount data.
A program to function as. Computer,
Image acquisition means to acquire images,
Sound source data acquisition means for acquiring music sound source data,
An acoustic analysis means that analyzes the sound source data and acquires acoustic feature data.
A photo movie generation means that uses an image acquired by the image acquisition means to generate a photo movie whose display changes according to a change in music based on the acoustic feature amount data.
A program to function as. A computer that can communicate with the server via a network
A music designation means that accepts the designation of a music and transmits the sound source data of the designated music or the music ID that is the identification information of the music to the server.
A playback means for receiving and playing a photo movie whose display changes according to a change in the music transmitted from the server.
A program to function as.

Images