JP2006301775A - Moving image with voice producing method, moving image with voice producing system, moving image with voice producing server, and computer readable program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image with voice producing server for producing a moving image with voice which has desired expressive and appeal power by using image information such as still images and the like. <P>SOLUTION: The moving image with voice producing server system 10 comprises a communication controller 11, 12 for controlling communication with a terminal 20; an image information database 14 for storing the image information received from the terminal 20; a moving image editing database 15 in which the image information and voice information for editing the moving image necessary for editing the moving image are stored; an information selecting means 16 for selecting specific information from the image information database 14 and the moving image editing database 15 based on an instruction from the terminal 20; an automatic moving image producing means 17 for automatically producing the moving image with voice from the information selected by the information selecting means based on the editing commands inserted in MIDI information; and a moving image processing means 16 for making the moving image with voice produced based on the commands from the terminal 20 transmittable to the terminal. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a moving image with sound generating method, a moving image with sound generation system, a sound with moving image generation system, and a sound with moving image generation method for generating a moving image with sound based on a still image captured by a terminal device with a photographing unit such as a camera-equipped mobile phone or a digital camera The present invention relates to a video generation server device and a computer-readable program.

  Conventionally, a slide show that plays back a plurality of still images captured by a terminal device with a photographing means such as a camera-equipped mobile phone or a digital camera at regular intervals is generally used for presentations, etc. A slide show with effects is also used. A technique for associating an appropriate BGM or the like with each slide in a presentation such as a slide show is also known (for example, Patent Document 1).

  However, in the conventional technology, the still image switching timing and the video effect insertion timing are not correlated with music, and the expression power and appealing power are not always sufficient.

On the other hand, videos such as commercials and music videos produced by video production professionals express the synergistic effect of video effects and music, and they are sufficiently expressive and appealing. It is difficult for ordinary users to obtain the videos created by.
JP 2004-170936 A

  The present invention has been made in view of such circumstances, and a method for generating a moving image with sound and a sound capable of generating a moving image with sound having desired expressive power and appealing power using image information such as a still image. It is an object to provide an attached moving image generating system, a moving image generating server device with sound, and a computer-readable program.

  In order to solve the above-described problem, in the first aspect of the present invention, an edit inserted in MIDI information from arbitrary image information and audio information including video information for moving image editing and MIDI information necessary for moving image editing. An audio-added moving image generating method is provided that generates an audio-added moving image based on an instruction.

  According to a second aspect of the present invention, there is provided an audio-added moving image generation server device that generates an audio-added moving image based on image information transmitted from a terminal, wherein the communication control unit controls communication with the terminal. An image information database for storing image information received from the terminal; a video editing database for storing video information for video editing necessary for video editing and audio information including MIDI information having an editing instruction function; Information selecting means for selecting specific image information from the image information database based on a command from the terminal, and selecting specific video editing video information and audio information from the video editing database; The editing information inserted in the MIDI information from the image information selected by the information selection means, the video information for moving image editing, and the audio information. An automatic moving image creating means for automatically creating a moving image with sound based on an instruction, and a moving image in a state in which the moving image with sound created by the automatic moving image creating means based on a command from the terminal can be transmitted to the terminal An audio-added moving image generating server device characterized by comprising processing means.

  In a third aspect of the present invention, a moving image generation server with sound for generating a moving image with sound, a terminal for transmitting image information to the moving image generation server with sound, and receiving a moving image with sound from the moving image generation server with sound, , A moving image generating system with sound comprising a network for connecting the moving image generating server with sound and the terminal so that the terminal can communicate with each other, wherein the moving image generating server with sound controls communication with the terminal. Movie editing that stores audio information including MIDI information having a communication control unit, an image information database for storing image information received from the terminal, and video editing video information necessary for movie editing and an editing instruction function Selecting specific image information from the image information database based on a command from the database and the terminal; and From the image editing database, information selection means for selecting specific video editing video information and audio information, image information selected by the information selection means, and video editing video information and audio information are converted into MIDI information. Automatic video creation means for automatically creating a video with audio based on the inserted editing instruction, and video with audio created by the automatic video creation means based on a command from the terminal can be transmitted to the terminal There is provided a moving image generating system with sound characterized by having moving image processing means for setting a state.

  Here, the terminal may be a camera-equipped mobile phone or a personal computer.

  According to a fourth aspect of the present invention, the step of receiving image information transmitted from a terminal, the step of storing the received image information in an image information database, and the image information database based on a command from the terminal Specific video editing video information from the video editing database storing video information for video editing necessary for video editing and audio information including MIDI information having an editing instruction function. And a step of selecting audio information, and a step of automatically creating a moving image with sound based on the editing instruction inserted in the MIDI information from the selected image information, moving image editing video information and audio information, and And a step of making the video with sound created by the automatic video creation means based on a command from the terminal transmittable to the terminal. Providing voice attached video generation method, characterized by.

  According to a fifth aspect of the present invention, there is provided a program for causing a computer to generate a moving image with sound based on image information transmitted from a terminal, the step of receiving image information transmitted from the terminal, Steps for storing image information in an image information database, and selecting information stored in the image information database based on a command from the terminal, and video information for video editing and editing instruction functions required for video editing Selecting a specific video editing video information and audio information from a video editing database storing audio information including MIDI information, and the selected image information, video editing video information and audio Automatically creating a moving image with sound based on the editing instructions inserted into the MIDI information from the information; It provides a computer-readable program characterized by executing the steps of the moving image with sound created by the automatic video creation means based on a command from the end and ready to be sent to the terminal to the computer.

  According to the present invention, various kinds of information created by video production professionals are stored in the moving picture editing information database, and stored in the moving picture editing information database such as image information such as still images from the user. Because it is possible to automatically create a video with sound such as music and narration using the editing function inserted in the MIDI information, the image information selected by the user is very expressive and appealing It can be.

Hereinafter, embodiments of the present invention will be specifically described with reference to the accompanying drawings.
FIG. 1 is a schematic diagram showing the overall configuration of a moving image generating system with sound according to an embodiment of the present invention. This moving image generation system with music includes a moving image generation server 10 that creates a moving image with sound from a still image from a user, and a terminal that transmits image information such as a still image to the moving image generation server 10 (camera-equipped mobile phone 20a and personal computer). A computer 20b + digital camera 30) 20, and a network 40 that connects the moving image generation server 10 and the terminal 20 so that they can communicate with each other.

  The video generation server 10 includes a mail server 11 connected to the network 40 and performing mail communication with the terminal 20, an Internet server (web server) 12 connected to the network 40 and performing Internet communication with the terminal, In addition, a specific mail address is monitored, an e-mail is received, an e-mail processing unit 13 that processes the received e-mail with image information, and a user who registers user information and image information acquired by the e-mail processing unit 13 Information database 14, movie editing database 15 storing video information and audio information necessary for movie editing, and information for creating a movie based on a command from user terminal 20 The information processing unit 16 that processes the processed video and the information processing unit 16 It includes an automatic moving image creation section 17 to automatically create a predetermined video on the basis of the information, and video information database 18 for storing the video that is created by the automatic moving image creating unit 17.

  The e-mail processing unit 13 acquires user information (such as a user e-mail address) and image information from the e-mail with image information sent from the user to the mail server 11 via the terminal 20 and the network 40, and identifies the user e-mail address. As described above, the above-described user information database 14 is created, and image information such as still images and user information are registered in the user information database 14. Further, when the registration of the image information and the user information is completed, the e-mail processing unit 13 sends an image registration completion notification in which a URL for accessing the user information database 14 is described in the user mail address. .

  The user information database 14 includes a management code for identifying the user, an email address registered by the user, and a still image file uploaded by the user.

  As shown in FIG. 2, the moving image editing database 15 includes an audio information file 21 including a WAVE file and a MIDI file, an animation frame information file 22 including a material (still image, moving image) of a video effect “animation frame”, It has an action layer information file 23 containing the material (still image, moving image) of the video effect “action layer” and an overlap color information file 24 containing the material (still image) of the video effect “overlap color”. . An edit instruction is embedded in the MIDI file of the sound information file 21 as described later. These pieces of information include various items produced by video production professionals. As described above, the video effect includes an animation frame, an action layer, and an overlap color. Among these, an animation frame is a relatively long animation effect that is inserted into the entire moving image by theme or target. The action layer is a relatively short animation effect inserted in accordance with the rhythm and timing. Furthermore, the overlap color is an image filter that superimposes a color or pattern on the image.

  The information processing unit 16 includes a Web program, and when the user accesses the user information database 14, the screen of the user terminal 20 is a moving image editing screen that displays the contents of the information database 14 and the moving image editing database 15. Select information from the user information database 14 and the video editing database 15 based on the sound information such as sound and narration that the user inputs from the screen, the image information that the user wants to use, the type of the video effect, etc. It has a function of sending the information to the automatic moving image creating unit 17. In addition, the moving image with sound created by the automatic moving image creating unit 17 is received and stored in the moving image information database 18, and a completion notification mail in which the moving image information URL is described is transmitted to the user mail address. Furthermore, when the user accesses the URL of the video information database 18, the information processing unit 16 uses the screen of the user terminal 20 as a video download screen, and can display the created video on the screen of the user terminal 20.

  The automatic video creation unit 17 functions as, for example, a COM server that can be easily used from a web server (such as an ASP), and the voice information and narration voice information and images selected by the information processing unit 16 based on a user command Predetermined moving image data with sound is automatically generated using information and video effects.

  Specifically, as shown in FIG. 3, the automatic moving image creating unit 17 selects the sound data, such as MIDI data 31, WAVE data 32, animation frame, action layer, overlap, etc., selected from the moving image editing database 15. The video effect data 33 is received, the selected still image 34 is received from the user information database 14, and the moving image data 35 is generated from them according to the editing instruction embedded in the MIDI data.

  Next, with reference to the block diagram showing the information flow of FIG. 4 and the flowchart of FIG. First, the user attaches image information captured by the camera-equipped mobile phone 20a or the digital camera 30 to an e-mail and transmits it to the e-mail address of the moving image generation server 10 (step 1). The e-mail processing unit 13 periodically monitors reception of a predetermined e-mail address and receives the e-mail (step 2).

  The e-mail processing unit 13 acquires user information (such as a user e-mail address) and image information from the received e-mail (step 3), and creates a user information database 14 using the user e-mail address as an identifier (step 4). . Then, image information such as a still image and user information are registered in the user information database 14 (step 5).

  As described above, when the registration of the image information and the user information is completed, the e-mail processing unit 13 sends an image registration completion notification in which the URL for accessing the user information database 14 is described to the user mail address. (Step 6).

  When the user who has received the image registration completion notification accesses the URL of the user information database 14 (step 7), the screen of the user terminal 20 becomes a moving image editing screen that displays the contents of the information database 14 and the moving image editing database 15. On this screen, sound information such as sound and narration to be used, image information to be used, video effect type, and message (character information) are input (step 8).

  Based on the input of the predetermined information, the information processing unit 16 receives image information, audio information such as sound and narration, video effect type, message (character information) from the user information database 14 and the moving image editing database 15. Is selected, and the moving image with sound is generated by the automatic moving image generating unit 17 based on the information (step 9).

  In this moving image creation, as shown below, image synthesis processing and editing instructions (editing events) received by MIDI are the center.

(1) Compositing process A frame image is generated by combining an image corresponding to each video effect (animation frame, action layer, overlap color) and a base still image (photograph). Each effect is processed independently, but the order of superposition is as shown in FIG.

(2) Editing instruction (editing event)
A base image and a video effect image are determined from an editing instruction (editing event) embedded in the MIDI data and its time code, and the combined image is used as a frame image. The editing event is embedded in the MIDI data. In the editing event, it is possible to insert an arbitrary data byte between [sub-ID # 2] of the real-time universal system exclusive message having the following basic form and the end of exclusive F7H. For example, a 2-byte event code can be defined as the arbitrary data.
Basic type: F0H [ID number] [device ID] [sub-ID # 1] [sub-ID # 2]… F7H
For example:
F0H 7FH 7FH 01H 01H mmH nnH F7H (mm is the event code. Nn is the parameter.)

The following six editing events are embedded in the MIDI data.
(1) Performance start event (2) Scene change event (3) Animation frame event (4) Action layer event (5) Overlap color event (6) Message event

Next, the above editing event will be specifically described.
(1) Performance start event Event code: 00H
Parameter: 00H
There is always one event in the MIDI data that represents the beginning of the video. The time code is reset to 00:00:00 00 at the position where this event is embedded.

(2) Scene change event Event code: 01H
Parameter: 00H / 01H
This event indicates the timing for switching to the next still image. When the parameter is 00H, simple cut switching is performed, and when the parameter is 01H, switching is performed by the cross dissolve effect. As for the effect at the time of switching, other things such as wipe and slide are possible. The switching duration when there is an effect is about 1 second.

(3) Animation frame event Event code: 02H
Parameter: 00H / 01H
This event indicates the start / end of an animation frame. As shown in FIG. 7, the parameter 00H represents an In point, and 01H represents an Out point. If there are not enough animations between In / Out points, repeat from the beginning of the animation.

(4) Action layer event Event code: 03H
Parameter: 00H
As shown in FIG. 8, this is an event indicating the start of an action layer, and ends when a specified number of animations are output. If an action layer event occurs during the action layer event processing, the animation is interrupted and the animation is started again from the beginning as shown in FIG.

(5) Overlap color event Event code: 04H
Parameter: 00H
This event indicates the start of the overlap color. As shown in FIG. 10, this is applied only to the still image at the time of this event occurrence. This event is cleared by a scene change event.

(6) Message event Event code: 05H
Parameter: 00H / 01H
This event indicates the timing for displaying a message to be inserted at the end of the moving image, and represents the end of the moving image. The duration of the message display screen is 1.5 seconds. When the parameter is 00H, simple cut switching is performed, and when the parameter is 01H, switching is performed by the cross dissolve effect. The switching duration when there is an effect is about 1 second. Note that video effects (animation frames, action layers, etc.) are not added to the message image.

Next, an example of the file format of the automatic moving image creation unit 17 is shown below.
(1) Animation information file Uncompressed AVI format moving image file or a group of BITMAP format image files (2) Action layer information file Uncompressed AVI format moving image file or a plurality of BITMAP format image files ( 3) Overlapping color information file BITMAP format image file group (4) Sound information file (i) Audio file Uncompressed (PCM) WAVE format audio file (ii) MIDI file Music data file according to standard MIDI standard (5) ) Movie information file Uncompressed AVI file, 3GPP2 format file

  After creating the moving image, the information processing unit 16 receives the created moving image with sound, stores it in the moving image information database 18 (step 10), and sends a completion notification mail describing the moving image information URL to the user mail address. (Step 11).

  When the user who has received the completion notification mail accesses the URL of the video information database 18 (step 12), the screen of the user terminal 20 becomes a download screen, and the created video is displayed on this screen and downloaded as necessary. (Step 13).

  The functions and processes described above can be executed by a computer as a program. This program may be stored in a hard disk or semiconductor memory in the computer, or may be stored in a portable storage medium such as a CDROM or DVD.

  As mentioned above, audio information such as various music and narration created by video production professionals and information necessary for video editing are stored in the video editing information database, and the user's still image and such video editing are stored. Since the moving image with sound can be automatically created based on the information stored in the information database, the information selected by the user can be highly expressed and appealing.

  The present invention can be variously modified without being limited to the above embodiment. For example, in the above-described embodiment, the case of creating a moving image with sound using a web is shown, but the present invention is not limited to this. Further, the image information transmitted by the user is not necessarily a still image.

  According to the present invention, it is possible to create a moving image with sound having high expressive power and appealing power based on image information such as a still image desired by the user.

Schematic which shows the whole structure of the moving image production | generation system with an audio | voice which concerns on one Embodiment of this invention. The figure which shows the moving image edit database in the moving image production | generation system with an audio | voice of FIG. The figure which shows the data structure at the time of the data preparation in an automatic moving image preparation part. The figure for demonstrating transfer of the information in the moving image production | generation system with an audio | voice of FIG. The flowchart for demonstrating the processing flow in the moving image production | generation system with an audio | voice which concerns on one Embodiment of this invention. The figure for demonstrating the order of the superimposition at the time of the production | generation of a frame image. The figure for demonstrating an animation frame event. The figure for demonstrating an action layer event. The figure for demonstrating a process when an action layer event generate | occur | produces during an action layer event process. The figure for demonstrating an overlap color event.

Explanation of symbols

10; Movie generation server 11; Mail server (communication control unit)
12; Internet server (communication control unit)
13; E-mail processing unit 14; User information database (image information database)
15; moving image editing database 16; information processing unit (information selecting means, moving image processing means)
17: Automatic video creation unit (automatic video creation means)
18; Video information database 20; Terminal 20a; Mobile phone with camera 20b; Personal computer 21; Audio information file 22; Animation frame information file 23; Action layer information file 24; Overlap color information file 30; Digital camera 31: MIDI Data 32: WAVE data 33: Video effect data 34: Still image 35: Movie data 40; Network

Claims (7)

  A video with audio is generated from arbitrary image information and audio information including video information for video editing necessary for video editing and MIDI information, based on an editing instruction inserted in the MIDI information. Movie generation method. An audio-added video generation server device that generates an audio-added video based on image information transmitted from a terminal,
A communication control unit for controlling communication with the terminal;
An image information database for storing image information received from the terminal;
A video editing database in which audio information including video information for video editing necessary for video editing and MIDI information having an editing instruction function is stored;
Information selection means for selecting specific image information from the image information database based on a command from the terminal, and selecting specific video editing video information and audio information from the video editing database;
Automatic video creation means for automatically creating a video with audio from the image information selected by the information selection means, video information for video editing and audio information based on an editing instruction inserted into MIDI information;
A moving image generating server with sound, comprising: moving image processing means for making a moving image with sound created by the automatic moving image creating means based on a command from the terminal ready to be transmitted to the terminal. A video generation server with audio for generating a video with audio;
Sending the image information to the moving image generation server with sound, a terminal that receives the moving image with sound from the moving image generation server with sound,
A sound-added moving image generation system comprising a network that connects the sound-added moving image generation server and the terminal in a communicable manner,
The video generation server with sound is
A communication control unit for controlling communication with the terminal;
An image information database for storing image information received from the terminal;
A video editing database in which audio information including video information for video editing necessary for video editing and MIDI information having an editing instruction function is stored;
Information selecting means for selecting specific image information from the image information database based on a command from the terminal, and selecting specific video editing video information and audio information from the video editing database;
Automatic moving image creating means for automatically creating a moving image with sound based on the editing instruction inserted in the MIDI information from the image information selected by the information selecting means and the moving image editing video information and audio information;
A moving image generating system with sound, comprising moving image processing means for making the moving image with sound created by the automatic moving image creating means based on a command from the terminal transmittable to the terminal.   4. The moving image generating system with sound according to claim 3, wherein the terminal is a camera-equipped mobile phone.   The moving image generating system with sound according to claim 3, wherein the terminal is a personal computer. Receiving image information transmitted from the terminal;
Storing received image information in an image information database;
On the basis of a command from the terminal, information stored in the image information database is selected, and audio information including video information for moving image editing necessary for moving image editing and MIDI information having an editing instruction function is stored. Selecting specific video editing video information and audio information from the video editing database;
Automatically creating a moving image with sound based on the editing instruction inserted into the MIDI information from the selected image information, moving image editing video information and sound information;
A method of generating a moving image with sound, comprising: a step of making a moving image with sound created by the automatic moving image creating means based on a command from the terminal ready to be transmitted to the terminal. A program for causing a computer to generate a video with sound based on image information transmitted from a terminal,
Receiving image information transmitted from the terminal;
Storing received image information in an image information database;
Based on a command from the terminal, information stored in the image information database is selected, and audio information including video information for video editing necessary for video editing and MIDI information having an editing instruction function is stored. Selecting specific video editing video information and audio information from the video editing database;
Automatically creating a moving image with sound based on the editing instruction inserted into the MIDI information from the selected image information, moving image editing video information and sound information;
A computer-readable program that causes a computer to execute a step of making a moving image with sound created by the automatic moving image creating means based on a command from the terminal ready to be transmitted to the terminal.

Images