JP5551403B2 - Movie creating apparatus, computer program, and storage medium - Google Patents

Description

  The present invention relates to a moving image creating apparatus, a computer program, and a storage medium, and more particularly to a technique suitable for use in generating a moving image while detecting a beat of music and switching a photo at the beat timing.

  In recent years, some personal computers and the like are equipped with a function for editing a still image or a moving image taken by a user by himself or creating a photo album (electronic album).

  For example, in Patent Document 1, when a user designates an image stored in a hard disk or the like, a predetermined editing process (effect process) is performed on the designated image, and the image is matched with music (BGM) prepared in advance. An image editing process for creating sequentially displayed contents is disclosed. This image editing process has a feature that different contents can be created even if the same image is designated because the editing process to be performed on the image is determined at random.

  Further, in Patent Document 2, a user selects a song prepared in advance and selects a photo album in which a still image to be played is stored. A technique for creating a slide show content (play list) that sequentially displays images together with music is disclosed.

  In Patent Document 3, a beat characteristic component is set by extracting a beat time at which a chord change of music is extracted, and a music excitement component is set by extracting a portion where the music is excited. Then, N paths having a high correlation value between the beat feature component and the key frame component are selected from the selectable paths on the motion graph, and the music excitement component and the motion excitement component are selected from the selected N paths. The path with the maximum correlation value with is set as the execution path and displayed on the display. In this way, motion that matches the music can be created in real time.

JP 2001-285784 A JP 2006-86621 A JP 2007-18388 A

  In the prior art, still images and music are combined using only an effect track (video effect track) for arranging still image material and video effects to be added on the timeline and a BGM track for arranging music material. I was creating a slideshow.

  In the prior art, “effects” such as zoom, panning, and color change to produce a photo (still image) itself, and “transitions” to produce a chronologically adjacent still image such as crossfade and wipe. ”Are prepared, and an“ effect ”is selected and assigned to each still image, and a“ transition ”is selected and assigned between the still images so that a slide show can be produced.

  However, with such a method, it is necessary to prepare various types of “effects” and “transitions” in order to be able to create a wide variety of slide shows. It was necessary. In addition, since the number of “effects” and “transitions” becomes enormous, it is impossible to freely and easily edit video effects synchronized with music.

Moreover, what is performed in Patent Document 3 is simply to switch the photos at beat timing. For this reason, it is impossible to create an attractive movie that is rich in change, and strictly speaking, there is a problem that it is difficult to say that it is synchronized with music.
In view of the above-described problems, a first object of the present invention is to realize a variety of presentation effects rich in change while being synchronized with the music being reproduced.
A second object is to enable anyone to easily create a moving image synchronized with music by a simple operation without being conscious of music synchronization timing at all.

The moving picture creating apparatus of the present invention is a moving picture creating apparatus that creates a moving picture using a plurality of images, and a music track that arranges music material data along a timeline, and a photo track that arranges photo material And a video creation screen display means for displaying on the display a video creation screen independently having a video effect track for applying a video effect to the photo material, and music corresponding to the music material arranged on the music track Synchronous information detecting means for detecting synchronous information, and adding a video effect arranged on the video effect track to the photographic material, and performing a video effect process synchronized with the synchronous information of music corresponding to the music material, and video creation means for creating a moving image which has been subjected to variety of stage effects to be synchronized photo to music, display to display the synchronization timing on the movie-making track The display control means automatically arranges video effects at the nearest synchronization timing when the grid, which is the music synchronization information corresponding to the music material, is on, and when the grid is off, the operator The video effect is automatically arranged at the timing indicated by .

A computer program according to the present invention is a computer program that causes a computer to execute a process of creating a moving image using a plurality of images, including a music track that arranges music material data along a timeline, and a photo material. A moving picture creation screen display step for displaying a moving picture creation screen on a display having a picture track to be arranged and a video effect track for applying a video effect to the photographic material; and a music material arranged on the music track. A synchronization information detecting step for detecting synchronization information of the corresponding music, and a video effect arranged in the video effect track on the photo material, and a video effect synchronized with the music synchronization information corresponding to the music material A video creation process that creates a video with various effects on the photo material in sync with the music And a display control step of displaying the synchronization timing on the video creation track, wherein the display control step, an image effect to the nearest sync timing when the grid is a synchronous information of the music corresponding to the music material on The image effect is automatically arranged, and when the grid is off, the video effect is automatically arranged at a timing instructed by the operator .

According to the present invention, when the grid, which is the music synchronization information corresponding to the music material, is on, the video effect is automatically arranged at the nearest synchronization timing, and when the grid is off, the video effect is displayed at the timing indicated by the operator. Since it is automatically placed, it is possible to automatically place photos and video effects that are input by editing after entering the music at the synchronization timing, and synchronized to the music by a simple operation without being aware of the music synchronization timing at all. Anyone can easily create videos.

1 is a block diagram illustrating an example of the configuration of a moving image creation apparatus that executes an image creation method according to an embodiment of the present invention. It is a flowchart which shows an example of a moving image preparation procedure. It is a figure which shows an example of the screen which performs animation creation and animation editing. It is a figure which shows an example in the state as which "music tab" was selected. It is a figure which shows an example in the state where the "photo tab" was selected. It is a figure which shows an example in the state where the "switching tab" was selected. It is a figure which shows an example in the state in which the "focus tab" was selected. It is a figure which shows an example in the state in which "Camera tab" was selected. It is a figure which shows an example in the state where the "lighting tab" was selected. It is a figure which shows an example in the state where the "background tab" was selected. It is a figure which shows an example in the state where the "text tab" was selected. It is a figure which shows an example in the state where the "screen tab" was selected. It is a flowchart which shows an example of the process sequence which detects the synchronous information of the music corresponding to a music material. It is a figure explaining the specific example of animation creation. It is a figure explaining a mode that a video effect is given to a still picture and a animation is generated.

FIG. 1 is a block diagram illustrating a configuration example of a moving image creation apparatus according to the present embodiment.
As shown in FIG. 1, a series of processes executed by a computer program (moving picture creation program) performed in the moving picture creation apparatus 100 of the present embodiment is controlled and executed by a computer system including CPUs 101 and 102 and a RAM 103. .

  In the moving image creating apparatus 100, a central processing unit (CPU) 101, a read only memory (ROM) 102, and a random access memory (RAM) 103 are connected to each other via a bus 104.

  An input / output interface 105 is further connected to the bus 104. The input / output interface 105 includes an input unit 106, an output unit 107, a storage unit 108 including a hard disk or a nonvolatile memory, a communication unit 109 including a network interface, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory. A drive 110 for driving the removable media 111 is connected.

  In addition, a keyboard 112, a mouse 113, a microphone 114, and the like are connected to the input unit 106. In addition, a display 115, a speaker 116, and the like are connected to the output unit 107.

  In the moving image creating apparatus 100 configured as described above, the CPU 101 loads, for example, the moving image creating program stored in the storage unit 108 to the RAM 103 via the input / output interface 105 and the bus 104 and executes the program. Thus, the moving image creation process is performed.

  In the moving image creating apparatus 100 of this embodiment, the moving image creating program executed by the CPU 101 is, for example, a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc)). Etc.), recorded on a removable medium 111 which is a package medium composed of a magneto-optical disk or a semiconductor memory, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. .

  Specifically, the removable medium 111 can be installed in the storage unit 108 via the input / output interface 105 by attaching the removable medium 111 to the drive 110. In addition, a moving image creation program transmitted via a wired or wireless transmission medium can be received by the communication unit 109 and installed in the storage unit 108. In addition, the moving image creation program can be installed in the ROM 102 or the storage unit 108 in advance.

  Note that the movie creation program executed by the movie creation device 100 may be a movie creation program that is processed in time series according to the order described in this specification, or in parallel or called. It may be a moving image creation program in which processing is performed at a necessary timing.

  When the above-described moving image creation process is executed by software, a series of processes may be realized by one application program, or may be realized by a plurality of functionally divided application programs.

Next, an example of the moving image creation processing procedure of the present embodiment will be described with reference to the flowchart of FIG.
When the process is started, a moving image creation screen display process for displaying the moving image creation screen on the display 115 is performed in step S21. As shown in FIG. 3, the movie creation screen used in the movie creation method of the present embodiment has a music track for arranging music material, a photo track for arranging photo material (still image material), and a still image. A plurality of effect tracks (video effect tracks) for arranging video effects to be added to the image material are provided, and these tracks can be operated independently.

  Further, the video effect track is composed of “section track” and “instant track”. The video effect processing by the video object arranged on the “section track” is continued until the next video object position arranged on the timeline. On the other hand, the video effect processing by the video object arranged on the “instant track” is performed only for a short time before and after the arrangement timing.

  As shown in FIG. 3, in this embodiment, a switching track, a frame track, a focus track, a camera track, a lighting track, a background track, a text track, an illustration track, a screen track, and the like are provided as effect tracks. . Each video track is further provided with a section video track and an instantaneous video track. Furthermore, a plurality of video styles are prepared corresponding to each video track, and a video object having a plurality of patterns is prepared for each style, and a video effect level can be set for each video object.

  A plurality of tabs corresponding to these tracks are provided, and a desired material can be selected by clicking on the tab corresponding to the input track. In the example of FIG. 3, “Frame tab” is selected, and “Rectangle”, “Handwriting style”, “Frame”, “Glass”, “Stamp”, etc. are displayed on the selection screen in a selectable manner. ing. The “image effect” of the frame is data for displaying a photograph in a shape such as a square shape or a heart shape.

FIG. 4 shows an example of a state where the “music tab” is selected. In this example, “part 1” to “part 3” are displayed, and “part 3” is selected.
FIG. 5 shows an example of a state in which the “photo tab” is selected, and photos such as “photo A”, “photo B”, “photo C”, “photo D”,. Has been.

  FIG. 6 shows an example of a state where the “switching tab” is selected. The “video effect” of switching is data for determining how to switch and display each photo while moving it. In this example, video effects such as “twist”, “reverse”, and “inside cube” are displayed in a style name window so as to be selectable. In the pattern name window, “normal”, “left turn”, “right turn” and the like are selectable. Furthermore, a slider for freely selecting the effect level from small to large is displayed.

  For example, if the effect level is a video object having a pattern of “turn left”, the effect level may be turned slowly as the value of the effect level is small, and may be turned quickly as the value of the effect level is small. You may make it turn at a large angle, so that it turns. In other words, the effect level is prepared for adjusting the level of adjustment for expressing subtle differences in video even for video objects having the same pattern. The present invention is not limited to the above example, and any video adjustment can be used as long as it adjusts the difference in the video expression for the video effect having the same pattern. This applies not only to the effect level for “switching” in this section, but also to the effect levels for the next and subsequent sections.

  FIG. 7 shows an example of a state where the “focus tab” is selected. The focus video effect is for applying an effect such as focusing on a photograph to be displayed. In this example, video effects such as “zoom”, “panning”, “jump”, and “unstable” are displayed in the style name window in a selectable manner. In the pattern name window, “horizontal direction”, “vertical direction”, “diagonal direction” and the like are displayed so as to be selectable. Furthermore, a slider for freely selecting the effect level from small to large is displayed.

  FIG. 8 shows an example of a state where the “camera tab” is selected. In this example, “Zoom”, “Panning”, “Unstable”, “Beat zoom”, “Beat panning”, and the like are displayed in a style name window. In the pattern name window, “Zoom In (Constant Speed)”, “Zoom In (Acceleration)”, “Zoom In (Deceleration)”, “Zoom Out (Constant Speed)”, and the like are displayed. Furthermore, a slider for freely selecting the effect level from small to large is displayed.

  FIG. 9 shows an example of a state where the “lighting tab” is selected. In this example, “panning”, “rotation”, “torch”, “beat zoom”, “beat panning”, and the like are displayed in the style name window. Further, “clockwise (normal)”, “counterclockwise (normal)”, “clockwise (reverse)”, “counterclockwise (reverse)”, and the like are displayed in a pattern name window. Furthermore, a slider for freely selecting the effect level from small to large is displayed. A button for selecting a color is also displayed.

  FIG. 10 shows an example of a state where the “background tab” is selected. The video effect of “background” is data for allowing the background of the displayed photo to be arbitrarily selected. In this example, “color”, “noise”, “move”, “unstable”, “star”, and the like are displayed in the style name window so as to be selectable. In the pattern name window, “vertical direction only”, “horizontal direction only”, “omni direction”, and the like are selectable. In addition, there is a window for selecting a photo to be used, with “focus photo” selected from the “focus photo”, “previous photo”, “next photo”, and “random” selection items. It is displayed. Furthermore, a slider for freely selecting the effect level from small to large is displayed.

  FIG. 11 shows an example of a state where the “text tab” is selected. “Text” is effect data for text to be displayed overlaid on the photo being displayed. In this example, “None”, “Normal”, “Zoom”, “Stretch”, “Spin”, etc. are displayed in the style name window in a selectable manner. In the pattern name window, “all characters”, “one character”, “all characters” and the like are displayed so as to be selectable. Furthermore, a slider for freely selecting the effect level from small to large is displayed. When the “edit” button is clicked, a window for setting a text string, font type, color, text display position, etc. is opened.

  FIG. 12 shows an example of a state where the “screen tab” is selected. “Screen” is a video effect for the entire movie, and is data that darkens or blurs the entire screen. In this example, “normal”, “fade in”, “fade out”, “bright”, “color”, and the like are displayed in a style name window. In addition, “Fixed” is displayed in the window of the pattern name so that it can be selected. Furthermore, a slider for freely selecting the effect level from small to large is displayed.

In the present embodiment, an example in which a plurality of effect tracks are provided in the frame track to the text track is shown (two rows in the example of FIG. 3).
That is, in the upper part of the frame track, “Rectangle, Round Large” at “Time 0:02”, “Frame, Real” at “Time 0:14”, “Rectangle, Round Large” at “Time 0:28”, “User effect, rotation type” “video effect” is arranged at “time 0:30”. In the lower row, “image effect” of “handwriting style, rectangle” is arranged at “time 0:33”.

  In the upper part of the focus track, “normal, rectangular” at “time 0:02” and “image effect” of “jump” at “time 0:25” are arranged. In the upper part of the background track, “color” at “time 0:02”, “noise, period change” at “time 0:17”, “color, selected color” at “time 0:30” "Effect" is arranged. In the lower row, “noise, cycle change” at “time 0:21” and “video effect” of “noise, cycle change” at “time 0:37” are arranged.

  The moving image creation process is executed by the user operating the mouse 113 on the “moving image creation screen” as described above. That is, in step S22, it is determined whether or not the music selection has been completed by the user. If the result of this determination is that music selection has not been completed, a standby state is entered. As a result of the determination in step S22, if the music selection is completed, the process proceeds to step S23. In the present embodiment, as described with reference to FIG. 4, since “No. 3” is selected, the process proceeds from step S22 to step S23. In step S23, music synchronization information is detected. Details of the processing will be described later with reference to the flowchart of FIG. When the music synchronization information detection process is completed, the process proceeds to step S24.

In step S24, it is determined whether or not photo selection has been completed. If the result of this determination is that photo selection has not been completed, a standby state is entered. If it is determined in step S24 that photo selection has been completed and a photo has been placed on the photo track, the process proceeds to step S25.
In step S25, in order to perform a predetermined editing process (effect process) on the image arranged in the photo track in step S24, each of the switching track to the screen track has been described with reference to FIGS. It waits for the “image effect” to be input. When “video effect” is input to each effect track, the process proceeds to step S26. Each process of S22-S25 mentioned above becomes an information input process.

  In step S26, in each process of step S22 to step S25, a moving image creation process is performed based on information input on the moving image creation screen. In the moving image creation process in the present embodiment, first, the synchronization information of music corresponding to the music material is detected, and the synchronization information is created. This synchronization information can be generated by analyzing from the timing and sound pressure level obtained by a known method for frequency analysis of music, and is used as various feature information.

  In the present embodiment, an example is shown in which “grid information”, “drum information”, “swell information”, and “intro / ending information” are used as feature information. Here, “grid information” is an information array composed of periodic sound pressure peak timing, power, and reliability. “Drum information” is an information array of a section in which a sound pressure peak in the low frequency range appears periodically.

  “Exciting information” is an information array of sections in which high sound pressure peaks frequently occur in a wide frequency band. Specifically, (a) the pitch of the sound is large. (B) The volume is high. (C) There is a lot of sound (musical instruments, number of sounds). (D) A section where the change in height is large is called a rising section. “Intro / Ending Information” is an information array such as fade-in / fade-out, a start / end of a loud sound, a start / end without a drum, a section where a long sound is output, and the like. An example of a processing procedure for acquiring the feature information will be described later with reference to FIG. Using the above feature information, the following music-synchronized video effect processing is performed.

  A process according to the “video effect” is individually executed for each image arranged on the photo track. As a result, it is possible to perform multiple video processing on a single “still image”, and a wide variety of effects can be achieved without preparing a large number of types of video effect processing as in the past. It is possible to generate a given video. When the moving image creation process ends, the process proceeds to step S27.

  In step S27, the reproduction process of the moving image created in step S26 is performed. During the period during which the moving image reproduction process is performed, the process proceeds to step S28 at a predetermined timing set in advance, and it is determined whether or not the user (operator) performs an interruption operation of the moving image reproduction process. As a result of the determination, if the user performs an interruption operation, the process returns to step S22 even while the moving image is being played back, and an editing screen for editing the moving image created by the moving image creating process of steps S22 to S25 is displayed. Display control to display is performed. Thereby, it is possible to perform an editing process in which a desired effect is added or changed in steps S22 to S25. Then, based on the editing result, a moving image creating process is performed in step S26 (in this case, the moving image editing process is performed). Thereby, editing work can be freely performed while confirming the moving image creation result. On the other hand, if it is determined in step S28 that the user has not performed an interruption operation, the process proceeds to step S29.

  In step S29, it is determined whether or not the moving image playback process has ended. If the result of this determination is that the moving image playback process has not ended, processing returns to step S22. In this case, since the interruption operation by the user is not performed, Steps S22 to S26 are “through”, and the moving image reproduction process in Step S27 is continued. If the result of determination in step S29 is that the moving image playback process has ended, end processing is performed.

  In the present embodiment, by creating a moving image as described above, it is possible to generate a moving image with a variety of presentation effects rich in variety without preparing a large number of types of video effect processing.

Next, an example of a processing procedure for detecting feature information will be described with reference to the flowchart of FIG.
When the process is started, grid information is detected in step S231, drum information in step S232, excitement information in step S233, intro / ending information in step S234, and other feature information in step S235. In the above steps S231 to S235, the synchronization information detection process for detecting the music synchronization information corresponding to the music material arranged on the music track is performed.

Next, an example of video effect processing synchronized with the feature information detected in steps S231 to S235 will be described.
As an example of the video effect processing synchronized with the grid information, (a) the appearance and exit time of the photo are made to coincide with the synchronization timing. (B) Pan the focus, camera, and lighting to the beat. (C) The lighting and background are flashed and the color is changed at a timing with high power and reliability. Here, power is information corresponding to a sound pressure peak at a grid position (timing such as beat or subbeat), and reliability is information indicating the reliability of the grid position.

  As an example of video effect processing synchronized with drum information, (a) In the drum section, the photographic motion acceleration is increased, and conversely, in the non-drum section, the photographic motion acceleration is decreased and the appearance / exit time of the photograph is intentionally shifted from the grid timing. . (B) In the drumless section, video processing synchronized with the grid should be as conservative as possible. (C) In the no-drum section, the photo switching frequency is set lower than that of the drum section.

As an example of the video effect processing synchronized with the climax information, (a) In the climax section, the photo switching frequency is set higher than that in the normal section. (B) In the climax section, the effect level value of various video effects is increased and exaggerated. (C) In the excitement section, increase the number of photos appearing at once, or arrange a lot of the same photos to create a lively atmosphere.
As an example of the video effect processing synchronized with the intro / ending information, (a) In this section, the photograph is not switched as much as possible. (B) In the intro, the photograph is zoomed in or faded in (the ending is the reverse), and so on. (C) For songs whose intro or ending begins or ends with a loud sound, the screen is instantly flashed, the photo is zoomed in, or the color is changed. (D) In songs where intros and endings begin and end with long sounds, do not switch photos during long sound intervals.

  As an effect obtained by using the characteristic information as described above, for example, in a cappella where the beat without a drum is unclear, it is possible to produce a relaxed performance, and on the contrary, the drum sounds and the beat is clear. In the part where it is, it is possible to produce a good ride with sharpness. In addition, it can be directed to the feeling of the beginning and end of the song. In addition, it is possible to obtain a remarkable effect such as a production that makes the mood rise by the excitement of the music.

Next, a specific example of moving image creation will be described with reference to FIGS. 14 and 15.
As shown in FIG. 14, in this example, the switching track has a “reverse” “video effect” near “time 0:12”, and a “parade” “parade” near “time 0:16”. In the lower part, “image effect” of “left back → right front (synchronized)” is arranged. In the vicinity of “Time 0:24”, “Video effect” of “Merging” is arranged in the upper row, and “Video effect” of “Right on the front and left on the back (x-axis rotation)” is arranged on the lower row. Yes.

  In the frame track, a “video effect” of “rectangular, large round” is arranged near “time 0:09”. In the focus track, “jump” “video effect” is arranged at “time 0:12”. In the camera track, a “video effect” of “beat zoom 3 beats” is arranged near “time 0:09”.

  Each photo arranged on the photo track is displayed on the display 115 by fading in, displaying all, and fading out as shown by a trapezoid in FIG. More specifically, Photo 1 starts to fade in near “time 0:08”. Then, near the “time 0:09”, the fade-in is finished and the entire display is completed. Thereafter, the fade-out is started in the vicinity of “time 0:10” slightly, and the fade-out is ended in the vicinity of “time 0:12”.

  Also, the photo 2 starts to fade in near “time 0:10” when the photo 1 starts to fade out. Then, the fade-in is completed at the time when the photograph 1 has finished the fade-out, and all the images are displayed. Therefore, during the period from the start of fade-out of photo 1 to the end of fade-out, photo 1 and photo 2 are displayed on display 115 so as to overlap.

  For Photo 2, the fade-out starts near “time 0:15” and the fade-out ends near “time 0:16”. The photo 3 starts to fade in near “time 0:15” when the photo 2 starts to fade out. Then, the fade-in is finished at the time when the photograph 2 finishes the fade-out, and all the images are displayed. Thereafter, the fade-out starts near “time 0:25” and the fade-out ends near “time 0:25”. In addition, the photo 4 starts to fade in near “time 0:25” when the photo 3 starts to fade out. Then, the fade-in is completed at the time when the photograph 3 has finished the fade-out, and all the images are displayed.

  By arranging the “video effects” as described above on each effect track, the photos arranged on the photo track are processed as shown in FIGS. 15 (1) to (8) to create a moving image. . That is, in FIG. 15A, the photograph 2 displayed with the focus “jump” (beat clear section) is given the “jump effect” as shown in FIG. 15B. In this case, the beat is panned left and right.

  Next, as shown in FIG. 15 (3), a twisting process is performed to rotate the photograph greatly immediately before the synchronization time. Next, as shown in FIG. 15 (4), a process of instantaneously returning the rotation at the synchronization time is performed to give a punch feeling. If the power and reliability of the synchronization time in the grid information are high, as shown in this example, adding a large change to the video instantaneously according to the synchronization time will produce a very effective performance that matches the music. .

  Next, as shown in FIG. 15 (5), “image effect” of “parade” is started, and the climax section is started. In FIG. 15 (6), “image effect” of “parade” is continued, producing a climax effect. In this example, a lot of the same photos appear to produce a sense of excitement, but it is also effective to produce a variety of photos that appear at the same time. Thereafter, FIG. 15 (7) shows “switching (merging) is started and the climax effect is finished. Then, you can make it suitable for music by not making it lively.

  As a method for generating a moving image in the present invention, a photographic image texture is pasted on polygon data having three-dimensional coordinates to animate, and image processing such as a lighting effect and a color tone change is performed thereon, and a camera work-like coordinate conversion is performed. It is possible to apply well-known techniques such as generating an image projected onto two-dimensional coordinates by rendering.

  As described above, in the present embodiment, each effect has a plurality of “style” data and a plurality of “pattern” data. Therefore, an effect that can be given to a photo is “video track” × The number of “styles” × “patterns” can be obtained. Therefore, it is possible to realize a vast variety of production effects without possessing many types of video effects.

(Second Embodiment)
In the present embodiment, music synchronization information (synchronization timing information) corresponding to the music material is included. This synchronization information is feature detection information such as beat timing and sound pressure level obtained by a well-known method for frequency analysis of music, but the user himself taps while listening to the whole music or a part of the music. It may be information obtained by manual input. The music synchronization information is information shown in the vertical direction at the timing of the music synchronization information corresponding to the music pasted on the music track, separately from the timing of the time information of the movie in FIGS. is there. In the above example, the music synchronization information indicates periodic timing information such as beats, subbeats, and the beginning of measures, but is not limited to this. Corresponding segmental information may be indicated.

  As in this embodiment, by having the music synchronization information, the object that is input later to the other tracks by the music material arranged on the music track on the timeline and the corresponding synchronization information is the music Based on the music synchronization information corresponding to, the display control can be performed so that the mark corresponding to the synchronization time is automatically arranged on the moving image creation screen.

  In addition, a mode for turning on / off the synchronization grid is provided, and when the grid is on, it is automatically arranged at the nearest synchronization timing, and when the grid is off, it can be automatically arranged at the timing designated by the pointing device or the like. This also applies to editing such as changing the fade-in / fade-out time of the photo material on the timeline after placement and changing the video object switching timing time.

  When the grid is on, the grid position is displayed so that it can be identified on the timeline. The above-mentioned video effects such as “frame”, “focus”, “camera”, “lighting”, “background”, “text”, “screen” use video materials such as photographs (still images), text, illustrations, etc. Then, by performing video processing in real time based on the synchronization information, a movie synchronized with the music can be created and reproduced.

  It is not only the timing at which the video is synchronized with the music but at the time of switching from one video style to the next. For example, in the video effect styles such as “Focus” and “Lighting”, there is a video style called “beat panning”. If a video object belonging to this style is placed on the video track, the music is synchronized. Image processing such as panning in accordance with the beat position corresponding to the information is performed.

  In addition, some video effect styles such as “frame”, “text”, and “camera” have a “beat zoom” video style. Video processing that zooms in time with is performed.

  Not limited to this, the video styles corresponding to each video track include those related to video processing synchronized with the synchronization information. When video objects of these video styles are arranged, the next video style Video processing such as periodic or instantaneous video expression is performed until the time. Further, although not in the present embodiment, a video track “illustration” may be prepared, and video processing corresponding to the selected video style may be performed on the selected illustration material.

  If an input operation for the selected video object is performed while the movie data being edited is being reproduced during movie editing, the movie data is automatically arranged at a position corresponding to the movie playback pointer according to the grid mode.

(Other embodiments of the present invention)
A program code of software for realizing the functions of the embodiment is provided to an apparatus connected to the various devices or a computer in the system so that each program code is operated to realize the functions of the embodiment. What is implemented by operating the various devices in accordance with a moving image creating program supplied and stored in a computer (CPU or MPU) of the system or apparatus is also included in the scope of the present invention.

  In this case, the program code of the software itself realizes the functions of the above-described embodiments, and the program code itself and means for supplying the program code to the computer, for example, the program code are stored. This storage medium constitutes the present invention. As a storage medium for recording the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code supplied by the computer, not only the functions of the above-described embodiments are realized, but also the OS (operating system) or other application software in which the program code is running on the computer, etc. The program code is also included in the embodiment of the present invention even when the functions of the above-described embodiment are realized in cooperation with the above.

  Further, after the program code supplied from the computer-readable storage medium is stored in the memory of the function expansion board of the computer or the function expansion unit connected to the computer, the function expansion board is based on the instruction of the program code. The present invention also includes the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

S21 video creation screen display process, S22 music selection process, S23 photo selection process, S24 video effect input process, S25 video creation process, S26 video playback process, S27 editing operation acceptance process, S28 end determination process

Claims (15)

A video creation device that creates a video using a plurality of images,
Displays a movie creation screen that independently has a music track for arranging music material data, a photo track for arranging photo material, and a video effect track for applying a video effect to the photo material along the timeline Video creation screen display means to be displayed on,
Synchronization information detecting means for detecting synchronization information of music corresponding to the music material arranged in the music track;
The video effect placed on the video effect track is added to the photo material, and the video effect processing synchronized with the music synchronization information corresponding to the music material is performed, and various effects are produced on the photo material in synchronization with the music. A video creation means for creating a video with effects,
Display control means for displaying the synchronization timing on the video creation track,
The display control means includes
When the grid, which is music synchronization information corresponding to the music material, is on, the video effect is automatically arranged at the nearest synchronization timing, and when the grid is off, the video effect is automatically arranged at the timing instructed by the operator. A featured video creation device.   The moving image creating apparatus according to claim 1, further comprising a moving image reproducing unit that reproduces the moving image created by the moving image creating unit.   As the video effect track, any two or more of a switching track, a frame track, a focus track, a camera track, a lighting track, a background track, a text track, an illustration track, and a screen track are provided. The moving image creating apparatus according to claim 1.   The moving image creation apparatus according to any one of claims 1 to 3, further comprising a section video track and an instantaneous video track for each video effect track.   The music synchronization information corresponding to the music material is one or more of “grid information”, “drum information”, “swelling information”, and “intro / ending information”. The moving image creation apparatus according to any one of claims 1 to 4.   5. The display control unit according to claim 1, further comprising: a display control unit configured to display a mark indicating a synchronization time on the moving image creation screen based on synchronization information corresponding to the music arranged on the timeline. The moving image creating apparatus according to item 1.   The display control means interrupts the moving picture reproduced by the moving picture reproducing means according to an instruction of an operator, and displays an editing screen on which the moving picture created by the moving picture creating means can be edited is displayed on the display. The moving image creating apparatus according to claim 2. A computer program that causes a computer to execute a process of creating a moving image using a plurality of images,
Displays a movie creation screen that independently has a music track for arranging music material data, a photo track for arranging photo material, and a video effect track for applying a video effect to the photo material along the timeline Video creation screen display process to be displayed on
A synchronization information detecting step of detecting synchronization information of music corresponding to the music material arranged in the music track;
The video effect placed on the video effect track is added to the photo material, and the video effect processing synchronized with the music synchronization information corresponding to the music material is performed, and various effects are produced on the photo material in synchronization with the music. A video creation process to create a video with effects,
A display control process for displaying the synchronization timing on the video creation track,
The display control step includes
When the grid, which is music synchronization information corresponding to the music material, is on, the video effect is automatically arranged at the nearest synchronization timing, and when the grid is off, the video effect is automatically arranged at the timing instructed by the operator. A featured computer program.   The computer program according to claim 8, further comprising a moving image reproduction step of reproducing the moving image created by the moving image creation step.   As the video effect track, any two or more of a switching track, a frame track, a focus track, a camera track, a lighting track, a background track, a text track, an illustration track, and a screen track are provided. The computer program according to claim 8 or 9.   The computer program according to any one of claims 8 to 10, further comprising a section video track and an instantaneous video track for each video effect track.   The music synchronization information corresponding to the music material is one or more of “grid information”, “drum information”, “swelling information”, and “intro / ending information”. The computer program according to any one of claims 8 to 11.   13. The display control step of displaying a mark indicating a synchronization time on the moving image creation screen based on synchronization information corresponding to the music arranged on the timeline. 13. The computer program according to item 1. The display control step suspends the video being played back by the video playback step according to an operator's instruction, and displays an editing screen on which the video created by the video creation step can be edited is displayed on the display. The computer program according to claim 9 .   A computer-readable storage medium storing the computer program according to any one of claims 8 to 14.

Images