JP2014150355A - Image editing apparatus and moving image display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily deal with an obtained moving image file.SOLUTION: The image editing apparatus comprises: an image reading unit which reads plural moving images; a scene classification unit which aligns plural moving images read by the image reading unit and classifies the aligned moving images by each scene included in the moving images; an image acquisition unit which respectively acquires representative images in plural scenes classified by the scene classification unit; and a display unit which displays a slider representing the plural moving images classified on scene-by-scene basis and displays a time line in which representative images in the scenes classified by the moving images are displayed in the vicinity of the scenes of the slider.

Description

  The present invention relates to an image editing apparatus and a moving image display method.

  When editing a moving image file using an image editing apparatus such as a personal computer, a time composed of a slider indicating an elapsed time from the start of reproduction (recording) of the moving image file and a time code indicating an elapsed time from the start of shooting. It is common to display using lines. On the other hand, a moving image file obtained at the time of moving image shooting conforms to a file format based on the imaging device. Depending on the file format, the file size is limited, and a moving image file acquired at the time of moving image shooting may be divided into a plurality of moving image files and recorded. In addition, when the remaining capacity of the storage medium attached to the imaging device is small at the time of moving image shooting, it may be divided into a plurality of moving image files, and the divided moving image files may be stored in a plurality of storage media, respectively. Recently, it has been devised to combine a plurality of moving image files that are divided and stored in this way into one moving image file and to import them into an image editing apparatus.

JP 2011-223327 A

  However, even when capturing a plurality of moving image files as a single moving image file or when capturing a plurality of moving image files as they are, the image editing process is performed using the timeline display described above. Is inconvenient because the user confirms the configuration of each file.

  It is an object of the present invention to provide an image editing apparatus and a moving image display method that can easily handle the obtained moving image file.

  In order to solve the above-described problem, an image editing apparatus according to the present invention includes an image reading unit that reads a plurality of moving images, and a plurality of moving images that are arranged by arranging the plurality of moving images read by the image reading unit. A scene sorting unit that sorts images into scenes included in the plurality of moving images, an image acquisition unit that obtains representative images in a plurality of scenes sorted by the scene sorting unit, and the plurality of sorts for each scene And a display unit for displaying a timeline in which a representative image in a scene sorted by the plurality of moving images is displayed near the scene of the slider. .

  The moving image display method of the present invention includes an image reading step of reading out a plurality of moving images, and arranging the plurality of moving images read out in the image reading step, and arranging the arranged plurality of moving images into the plurality of moving images. A scene sorting process for sorting scenes included in a moving image, an image acquisition process for acquiring representative images in a plurality of scenes sorted by the scene sorting process, and the plurality of moving images sorted for each scene are shown. A display step of displaying a slider and performing a timeline display in which a representative image in a scene sorted by the plurality of moving images is displayed in the vicinity of the scene of the slider.

  According to the present invention, it is possible to easily handle the obtained moving image file.

It is a figure which shows the relationship between an imaging device and a personal computer (PC). It is a functional block diagram which shows the structure of PC. It is a functional block diagram which shows the structure of CPU. It is a figure which shows an example of an image edit screen. It is a figure which shows an example of the image edit screen in the case of displaying the image list in a folder. It is a figure which shows an example of the image edit screen in the case of displaying the timeline based on a moving image file. It is a figure which shows an example of the image edit screen in the case of displaying the timeline based on a scene. It is a flowchart which shows the flow of a process at the time of displaying the timeline based on a scene using a some moving image file. It is a flowchart which shows the flow of a process at the time of displaying the timeline based on a scene using a some moving image file. It is a figure which shows the case where the moving image file in the one part scene of the timeline displayed is memorize | stored. (A) is a figure which shows an example of the display of a timeline when the granularity of a scene display is set to "thin", (b) is an example of the display of a timeline when the granularity of a scene display is set to "coarse" FIG. It is a figure which shows the case where the moving image file in the one part scene of the timeline displayed is memorize | stored when the granularity of a scene display is set to "rough". It is a flowchart which shows the flow of a process at the time of displaying the timeline based on a scene using the some moving image file containing the moving image file to which the marking process was performed. It is a figure which shows an example of the reproduction speed at the time of reproducing | regenerating the moving image based on a scene.

  Hereinafter, a personal computer (PC) will be described as an example of an image editing apparatus to which the present invention is applied.

  As shown in FIG. 1, the PC 11 can perform data communication with an imaging device 12 such as a digital camera 12a or a video camera 12b. Data communication between the PC 11 and the imaging device 12 includes data communication using wireless communication in addition to wired data communication such as a USB cable. In addition, the PC 11 attaches the storage medium 34 removed from the imaging device 12 to read out image data stored in the attached storage medium 34 and write image data to the storage medium 34. be able to.

  As shown in FIG. 2, the PC 11 includes a CPU 21, ROM 22, RAM 23, display control circuit 24, display device 25, buffer memory 26, image memory 27, media controller 28, connection I / F 29, communication unit 30, keyboard 31, It consists of a pointing device 32. Here, the CPU 21, ROM 22, RAM 23, display control circuit 24, buffer memory 26, image memory 27, media controller 28, connection I / F 29, and communication unit 30 are electrically connected via a bus 33.

  The CPU 21 controls each part of the PC when the control program or application program stored in the ROM 22 is executed. An example of the application program is an image editing program. The CPU 21 executes control according to operation signals from the keyboard 31 and the pointing device 32. Here, examples of the pointing device 32 include a mouse, a trackball, and a pen tablet.

  The ROM 22 stores data such as an operator used when executing these programs in addition to a control program and application programs executed by the PC 11. The RAM 23 temporarily stores data generated when the CPU 21 executes the above-described program.

  The display control circuit 24 performs display control in the display device 25. Examples of the display device 25 include a liquid crystal display (LCD) and an organic EL display. The buffer memory 26 temporarily stores image data and the like. The image memory 27 stores an image file that is captured when the image editing program is executed.

  The media controller 28 electrically connects the PC 11 and the storage medium 34 by mounting the storage medium 34. Here, examples of the storage medium 34 include a memory card, an optical disk, and a magnetic disk.

  The connection I / F 29 includes, for example, a USB port or an IEEE port. A connector provided at one end of the cable is attached to the connection I / F 29. A connector provided at the other end of the cable is connected to the imaging device 12. Thereby, the imaging device 12 and the PC 11 are electrically connected.

  The communication unit 30 is a communication interface configured with a communication device or the like for connecting to a network. Examples of the communication unit 30 include a communication unit compatible with wired communication in addition to a communication unit compatible with wireless LAN (Local Area Network) and a communication unit compatible with wireless USB.

  As illustrated in FIG. 3, the CPU 11 executes the functions of the moving image management unit 41, the timeline generation unit 42, the scene analysis unit 43, and the representative image acquisition unit 44 when executing the image editing program. The moving image management unit 41 manages image files (moving image files and still image files) imported to the PC 11.

  The timeline generation unit 42 generates information necessary for performing timeline display on a moving image file. For example, although details will be described later, the timeline display includes a timeline display based on a plurality of sorted moving image files and a timeline display based on scenes included in the sorted moving image files. For example, when performing timeline display based on a plurality of sorted moving image files, information necessary for performing timeline display includes information on the arrangement of sorted moving image files and sorted moving images. And time code information for each frame image in the image file.

  In this case, the timeline generating unit 42 first sorts a plurality of moving image files stored in the target folder among the folders provided in the image memory 27 in order of oldest recording date and time. Thereby, the information regarding the arrangement | sequence of a moving image file is acquired.

  Next, the timeline generation unit 42 reads out the time code attached to each frame image data of the plurality of sorted moving image files. As is well known, in moving image shooting using the imaging device 12, a frame rate is determined in advance. That is, each frame image constituting the moving image file is assigned a time code indicating a temporal position with the frame image as a minimum unit. Therefore, the timeline generation unit 42 newly sets the time code attached to the frame image constituting each moving image file with reference to the time code of the frame image of the first frame of the sorted moving image file. Set. Thereby, time code information for each frame image of the plurality of sorted moving image files is acquired.

  On the other hand, when performing timeline display based on scenes included in the sorted moving image file, information necessary for performing timeline display is set based on the determination result in the scene analysis unit 43. Information on the position of the scene change flag is given.

  When performing timeline display based on a scene, the timeline generation unit 42 reads two frame images that are temporally forward and backward, and determines whether or not the scene in the frame image has changed. Note that the two frames that move forward and backward in time may be a predetermined number of frames before and a predetermined number of frames with respect to an arbitrary frame. The user may arbitrarily determine a predetermined number of previous frames and a predetermined number of subsequent frames. Moreover, the frame immediately before and one frame after an arbitrary frame may be used. In this determination, when it is determined that the scene has changed, a scene change flag is assigned to a frame image positioned later in time among the two frame images moving back and forth in time. This process is executed for all the read moving image files. Thereby, the scene change flag is set, and information on the position of the set scene change flag is obtained.

  The scene analysis unit 43 performs scene analysis of scenes included in the sorted moving image file. Examples of the scene analysis include a method of analyzing a scene by generating a saliency map, in addition to a method of analyzing a scene using region division processing for each frame image.

  The representative image acquisition unit 44 acquires a representative image in the moving image file and a representative image in each scene. For example, as a representative image of a moving image file, for example, a thumbnail image based on the frame image of the first frame is acquired as a representative image from a plurality of frame images constituting the moving image file. As a representative image in the scene, a thumbnail image based on the frame image based on the analysis result of the scene analysis unit 43 is acquired as a representative image from the frame images constituting the scene. Then, the representative image acquisition unit 44 outputs the acquired thumbnail image to the display control circuit 24.

  FIG. 4 shows an example of an image editing screen displayed on the display device 25. When the image editing program is executed, an image editing screen 51 is displayed on the display device 25. Here, the image editing screen 51 may be displayed on the entire display device 25 or may be displayed using a part of the display device 25. The image editing screen 51 is composed of a list of executable functions (function menu in FIG. 4) and contents based on the selected function menu. Here, the user can execute a process based on the selected function by selecting a function item displayed on the function menu. Here, the function item “Easy Backup” is selected when all the image files obtained by the imaging device 12 are backed up from the imaging device 12 to the PC 11. In this case, all the image files to be backed up are stored in the image memory 27 of the PC 11. Also, the item “selected image capture” is selected when an image file to be backed up by the user is stored in the PC 11 among the image files obtained by the imaging device 12. The item “one-touch disk” is selected when an image file stored in the PC 11 is stored in a storage medium such as an optical disk.

  Next, a case where image editing processing is performed on the image editing screen 51 will be described. In this case, it is necessary to select a moving image file for performing image editing processing. When a moving image file to be subjected to image editing processing is selected, a directory structure and a folder structure are displayed on the image editing screen 51. For example, when a folder for image editing processing is selected, a list of moving image files included in the selected folder is displayed (see FIG. 5). Here, a case is shown in which three moving image files “a.mov”, “b.mov”, and “c.mov” are stored in the selected folder.

  Here, the display related to the list of moving image files included in the folder includes detailed display of each moving image file (display of shooting time, file type, etc.) and reduced display of moving image file (thumbnail of the first frame). In addition to (image display), a timeline display described later is included. As described above, the timeline display includes a timeline display based on the arrangement of the sorted moving image files and a timeline display based on the scenes included in the sorted moving image files.

  FIG. 6 shows an example of a timeline display based on the arrangement of sorted moving image files. Here, in the timeline display, a slider 52 indicating the structure of a moving image based on the sorted moving image file and a representative image of each moving image file are displayed. Here, the moving image file “a.mov”, the moving image file “b.mov”, and the moving image file “c.mov” are the moving image file “a.mov”, the moving image file “b.mov”, and the moving image. If the images are taken in the order of the image file “c.mov”, the moving image file “a.mov”, the moving image file “b.mov”, and the moving image file “c.mov” are displayed in the timeline display. Are displayed in this order. Here, in the slider 52, the file name and each moving image file are displayed in different colors. In FIG. 6, the color classification of the moving image file is shown by changing the type of hatching. Corresponding to the position indicating each moving image file on the slider 52, the representative image in each moving image file is displayed using a balloon.

  In FIG. 6, the slider 52 is displayed using three sliders. However, the number of the sliders 52 is not limited to this, and the sliders displayed when the timeline is displayed. Can be displayed using two sliders or four or more sliders. In FIG. 6, marks “及 び” and “Δ” are attached to the boundary of the moving image file.

FIG. 7 shows an example of a timeline display based on a scene included in a moving image based on the sorted moving image file. In the case of the timeline display based on the scene included in the moving image based on the sorted moving image file, similarly to the timeline display indicating the structure of the moving image based on the sorted moving image file, a plurality of sliders, It consists of a representative image. In FIG. 7, “scene1”, “scene2”, “scene3”, “scene4”, “scene5” are a moving image file “a.mov”, a moving image file “b.mov”, and a moving image file “c.mov”. It is a scene obtained from In this timeline display, marks “▽” and “Δ” are attached not to the moving image file but to the boundary of the scene included in the moving image file.
In this case, a scene name is displayed on the slider. In this timeline display, the timeline display is based on the scene included in the moving image, but in the slider, the moving image file is color-coded so that the user can know the association between the moving image file and the scene. Show. In FIG. 7, the color coding of the moving image file is shown by changing the type of hatching. In addition, the representative image in each scene is displayed using a balloon so as to correspond to the position indicating each scene on the slider.

Hereinafter, the flow of processing when performing timeline display will be described. Hereinafter, a case where a plurality of moving image files included in a selected folder are processed without being combined will be described as a first embodiment.
<First Embodiment>
Hereinafter, the flow of processing during timeline display will be described with reference to the flowchart of FIG.

  Step S101 is processing for designating a folder. When the user operates the keyboard 31 or the pointing device 32 and designates a folder intended by the user, the CPU 21 reads out an image file included in the designated folder.

  Step S102 is processing for determining whether or not there is a moving image file. If there is a moving image file in the read image file, the CPU 21 sets the determination result in step S102 to Yes. In this case, the process proceeds to step S103. On the other hand, if there is no moving image file in the read image file, the determination result in step S102 is No. In this case, the process of this flowchart ends.

  Step S103 is processing for sorting moving image files. The CPU 21 reads information on the shooting date and time attached to each of the read moving image files. Then, the CPU 21 rearranges the read moving image files in order of shooting date and time. Then, the CPU 21 outputs to the display control circuit 24 information on the order of moving image files for timeline display and time code. Further, the CPU 21 acquires representative image data of each moving image file, and outputs the acquired representative image data to the display control circuit 24.

  Step S104 is a process for performing timeline display based on the arrangement of the sorted moving image files. The display control circuit 24 performs display control in the display device 25 based on the information and representative image data output from the CPU 21 in step S103. Thereby, the timeline based on the arrangement of the sorted image files is displayed on the image editing screen 51 (see FIG. 6).

  Step S105 is processing for determining a scene change. The CPU 21 reads out two frame image data that are temporally changed in the sorted moving image file. Then, it is determined whether or not the scene has changed in the two frame image data that are temporally changed.

  Step S106 is processing for determining whether or not there is a scene change. In the process of step S105, when the scene changes in two frame image data that are temporally changed, the CPU 21 sets the determination result of step S106 to Yes. In this case, the process proceeds to step S107. On the other hand, when the scenes in the two frame image data that change in time are not changed, the CPU 21 sets the determination result in step S106 to No. In this case, the process proceeds to step S108.

  Step S107 is processing for setting a scene change flag. This is executed when it is determined in step S106 that there is a scene change. The CPU 21 assigns a scene change flag to the frame image data that is later in time among the two frame image data that are temporally changed.

  Step S108 is processing for determining whether or not the end of the moving image file. In step S105, when the scene change is determined using the final frame image in one of the sorted moving image files, the CPU 21 sets the determination process in step S108 to Yes. In this case, the process proceeds to step S109.

  On the other hand, when the scene change is determined without using the final frame image in any one of the sorted moving image files in step S105, the CPU 21 sets the determination process in step S108 to No. . In this case, the process returns to step S105. In this case, the processing from step S105 to step S107 is repeated until the scene change determination using the final frame image of the moving image file is performed.

  Step S109 is processing for determining whether or not there is a next moving image file. The scene change determination for any one of the plurality of sorted moving image files has been completed. Therefore, if there is a moving image file positioned next to the sorted moving image file, the CPU 21 sets the determination result in step S109 to Yes. In this case, the process proceeds to step S110. In this case, the CPU 21 outputs information on the position of the frame image with the scene change flag to the display control circuit 24.

  On the other hand, if there is no moving image file positioned next to the sorted moving image file, the CPU 21 sets the determination result in step S109 to No. In this case, the process proceeds to step S114.

  Step S110 is a process of determining whether or not the final frame image of the moving image file for which the scene change has been determined and the frame image that is the first frame of the next moving image file are the same scene. . The CPU 21 reads the final frame image of the moving image file for which the scene change has been determined and the frame image that becomes the first frame of the next moving image file. Then, a scene change determination process in these frame images is performed.

  Step S111 is processing for determining whether or not there is a scene change. If the scenes in the two frame images have changed, the CPU 21 determines Yes in step S111. In this case, the process proceeds to step S113. On the other hand, when the scenes in the two frame images have not changed, the CPU 21 sets the determination result in step S111 to No. In this case, the process proceeds to step S112. In this case, a scene change flag is added to the frame image that becomes the first frame of the moving image file arranged later.

  Step S112 is processing for deleting a boundary between moving image files. The CPU 21 deletes the boundaries of the plurality of sorted moving image files.

  Step S113 is processing to refer to the next moving image file. When this process is performed, the process returns to step S105, and a process for determining a scene change and a process for setting a scene change flag are executed using each frame image of the next moving image file.

  On the other hand, if the determination result in step S109 is No, in other words, if there is no next moving image file, the process proceeds to step S114.

  Step S114 is a process of analyzing the scene. CPU21 analyzes a scene using the frame image between each scene change flag.

  Step S115 is processing to acquire a representative image. Based on the result of the scene analysis executed in step S114, the CPU 21 acquires a frame image representing the scene between the scene change flags, and acquires a thumbnail image based on the frame image as a representative image. The CPU 21 outputs the generated second timeline information to the display control circuit 24.

  Step S116 is a timeline display based on the scene. The display control circuit 24 performs display control of the display device 25 based on the input information. Thereby, the timeline display is switched from the timeline display based on the sorted moving image file to the timeline display based on the scene included in the sorted moving image file (see FIG. 7).

  In this way, when a plurality of moving image files are stored in a folder, when a timeline display is performed, a plurality of moving image files sorted in order of shooting date and time are pseudo-moved as one moving image file. The assumed timeline is displayed. At that time, a representative image of each moving image file is displayed for each moving image file, and the moving image file is displayed in a separated manner.

  Thereafter, the timeline display when the plurality of sorted moving image files are assumed to be one moving image file is switched to the timeline display based on the scenes included in the plurality of moving image files. In the timeline display based on the sorted scenes, in addition to the scenes included in the moving image file, the association between each scene and the moving image file can be recognized. I can grasp it. In addition, since a representative image representing the scene is displayed for each scene, the user can recognize the contents of each scene.

  In the first embodiment, the case where the timeline display based on a plurality of sorted moving image files is performed and then the timeline display based on the scene is switched and displayed is described. However, the plurality of sorted moving images are described. It is also possible to perform only the timeline display based on the scenes included in the plurality of sorted moving image files without performing the timeline display based on the file arrangement.

In the first embodiment, a case has been described in which a timeline display is performed without combining a plurality of moving image files included in a selected folder. However, the present invention is not limited to this, and a plurality of sorted files are not limited. It is also possible to display a timeline based on the scenes included in the moving image file after combining the moving image files. The process in this case will be described as a second embodiment.
Second Embodiment
As described above, in the second embodiment, after combining a plurality of sorted moving image files, timeline display based on the scenes included in the moving image files is performed. Here, the timeline generation unit 42 described above sorts a plurality of moving image files, and then executes a process of combining the plurality of sorted moving image files.

  Hereinafter, the flow of processing in the timeline display will be described using the flowchart of FIG.

  Step S201 is processing to specify a folder. Since the process of this step S201 is the same as that of step S101, it is omitted here.

  Step S202 is processing to determine whether there is a moving image file. The processing in step S202 is the same as that in step S102. That is, if there is a moving image file in the designated folder, the CPU 21 sets the determination result in step S202 to Yes. In this case, the process proceeds to step S203. On the other hand, if there is no moving image file in the designated folder, the CPU 21 sets the determination result in step S202 to No. In this case, the process of this flowchart ends.

  Step S203 is processing for sorting moving image files. The CPU 21 reads shooting date / time information from the plurality of read moving image files. Then, the CPU 21 arranges a plurality of moving image files in the order of the shooting date and time using the read shooting date and time information.

  Step S204 is processing for combining the sorted moving image files. The CPU 21 combines the moving image files sorted in step S203 to generate one moving image file.

  Step S205 is processing for determining a scene change. Since the process of step S205 is the same as that of step S105, it is omitted here.

  Step S206 is processing for determining whether or not there is a scene change. In the process of step S205, when the scenes in the two frame images that change in time are changing, the CPU 21 sets the determination result of step S206 to Yes. In this case, the process proceeds to step S207. On the other hand, when the scenes in the two frame images that change in time are not changed, the CPU 21 sets the determination result in step S206 to No. In this case, the process proceeds to step S208.

  Step S207 is processing for setting a scene change flag. The CPU 21 sets a scene change flag for a frame image that is later in time among the two frame images that are temporally changed.

  Step S208 is processing for determining whether or not the end of the original moving image file has been reached. In step S205, when the scene change is determined using the final frame image in any one of the moving image files before combining, the CPU 21 sets the determination process in step S208 to Yes. In this case, the process proceeds to step S209. On the other hand, when the scene change determination is not performed using the final frame image in any one of the moving image files before combining, the CPU 21 sets the determination result in step 208 to No. In this case, the process returns to step S205. Therefore, the processing from step S205 to step S207 is repeatedly executed until the end of the original moving image file is reached.

  Step S209 is processing for determining whether or not there is a moving image file arranged next to the current moving image file in the plurality of sorted moving image files. The CPU 21 determines whether or not the next moving image file is combined after the moving image file including the frame image subjected to the scene change. When the next moving image file is combined, the CPU 21 determines Yes in step S209. In this case, the process proceeds to step S210. On the other hand, when the next moving image file is not combined, the CPU 21 sets the determination result of step S209 to No. In the case of this determination result, the CPU 21 outputs information on the position of the scene change flag to the display control circuit 24. Then, the process proceeds to step S213.

  Step S210 is a process of determining whether or not the frame image that is the final frame of the moving image file subjected to the scene determination and the frame image that is the first frame of the next moving image file are the same scene. . The CPU 21 reads the frame image that is the final frame of the moving image file that has been subjected to the scene change determination, and the frame image that is the first frame of the moving image file that is to be subjected to a new scene determination. Then, it is determined whether or not the scene in these frame images has changed.

  Step S211 is processing for determining whether or not there is a scene change. This process is the same process as step S206. If the scenes in the two frame image data have changed, the CPU 21 sets the determination result in step S211 to Yes. In this case, the process proceeds to step S212. On the other hand, if the scenes in the two frame image data have not changed, the CPU 21 sets the determination result in step S211 to No. In this case, the process proceeds to step S205.

  Step S212 is processing for setting a scene change flag. The CPU 21 adds a scene change flag to a frame image that is later in time among the two frame images. After this process, the process returns to step S205.

  If the determination process in step S209 described above is No, the process proceeds to step S213.

  Step S213 is processing for performing scene analysis. The process of step S213 is the same process as step S114.

  Step S214 is processing to acquire a representative image. The process of step S214 is the same process as step S115. CPU21 acquires the thumbnail image based on a typical frame image among the frame images which comprise a scene as a representative image.

  Step S215 is processing for determining whether or not the processing in steps S213 and S214 has been performed for all scenes. When the processes of step S213 and step S214 are performed on all scenes, the CPU 21 sets the determination result of step S215 to Yes. The CPU 21 outputs the acquired representative image to the display control circuit 24. In this case, the process proceeds to step S216. On the other hand, when the processes in steps S213 and S214 are not performed on all scenes, the CPU 21 sets the determination result in step S215 to No. In this case, the process proceeds to step S213.

  Step S216 is processing for performing timeline display based on the scene. The display control circuit 24 performs display control on the display device 25 using the input information and the representative image. Thereby, the timeline based on the scene is displayed.

  In the second embodiment, a plurality of moving image files stored in a folder are combined to generate one moving image file, and then a timeline is displayed based on the scene. Also in the second embodiment, as in the first embodiment, since the timeline based on the scene is displayed on the display device 25, it is possible to grasp the connection of the scenes and the scene contents in a plurality of moving image files. it can.

  In the first embodiment and the second embodiment, it is described that the timeline display based on the scene is performed. The timeline display based on this scene is a list display of moving image files, a moving image file display, and the like. It is also possible to switch and display between reduced display and timeline display based on a plurality of sorted moving image files. In the preferred embodiment, the switching display is performed based on an operation of the keyboard 31 or the pointing device 32 by the user.

  In the first embodiment and the second embodiment, an object is to perform timeline display based on a scene. As described above, the timeline display based on the scene is a timeline display when all the moving image files stored in the folder are represented by one continuous moving image file. Therefore, when the timeline based on the scene is displayed, the following processing can be performed.

  For example, when there is a scene intended by the user in a timeline display based on a scene, there is a case where it is desired to generate a moving image file based on the scene. In such a case, the corresponding scene or representative image is moved to the target folder by a click-and-drop operation using a pointer. FIG. 10 shows a case where the user generates a moving image file for “scene2”. In this case, the pointer 55 is set to the position of “scene2” of the slider 52 and, for example, the mouse is clicked and dropped to move to the corresponding folder 56. In response to this operation being performed, the CPU 21 generates a moving image file based on a target scene from a plurality of moving image files. Then, the CPU 21 stores the generated moving image file in a folder provided in the image memory 27. During this process, the CPU 21 duplicates the target frame image from the corresponding moving image file using the reset time code. When the selected scene is straddling a plurality of moving image files, the CPU 21 may perform a process of duplicating the target frame image from the corresponding moving image file and synthesizing the duplicated frame images.

  In the first embodiment and the second embodiment, a scene change flag is set for a portion where there is a scene change in the moving image file. For this reason, many scenes are generated. For this reason, some users may feel that there are too many scenes. In order to cope with such a case, for example, a set scene change flag can be given priority. When priority (importance) is given to such a scene change flag, a slider 57 for changing the fineness (granularity) of the scene display is provided during timeline display, and this slider 57 is operated. Thus, the granularity of the scene display can be changed. As shown in FIG. 11A, in the case of a user who wants to confirm the scene in the timeline display in detail, the slider 57 indicating the granularity of the scene display is moved to “fine”. In this case, each scene in the timeline display is displayed using all set scene change flags.

  For a user who wants to roughly check the scene in the timeline display, the slider 57 indicating the granularity of the scene display is moved to “rough”. In this case, the scene in the timeline display is displayed using only the scene change flag having a higher priority among all the set scene change flags.

  As shown in FIG. 11B, the priority of the scene change flag indicating the boundary between “scene2” and “scene3” and the scene change flag indicating the boundary between “scene4” and “scene5” Priority is set higher than the scene change flag (scene change flag indicating the boundary between “scene1” and “scene2” and the scene change flag indicating the boundary between “scene3” and “scene4”). The priority of the scene change flag indicates the importance of the scene change, and may be set by the user or automatically set from the content of the moving image.

  If the granularity of scene display can be changed, the following processing can be performed. For example, when the granularity of scene display is set, in some of the displayed scenes, the scene may not be divided at a position corresponding to a scene change flag with a low priority. In this case, even if the same scene is set, the scene is subdivided. FIG. 12 shows a case where “scene1” is subdivided into three scenes “scene1-1”, “scene1-2”, and “scene1-3”. As described above, when generating a moving image file based on a scene intended by the user, click and drag using a pointer. When an operation of moving to a target folder is performed, a moving image file based on the scene is generated. When the moving image file is generated, a folder based on the scene is generated without generating the moving image file of the entire selected scene, and then a plurality of moving image files based on the subdivided scenes are included in the folder. Remember.

  In the first embodiment and the second embodiment described above, the PC 11 performs a process of determining a scene. For example, there may be a case where a marking process is performed to indicate an important location for the user by operating an operation member provided in the imaging device 12 during moving image shooting in the imaging device 12. In such a case, a moving image file that has undergone marking processing is included in the plurality of read moving image files. Therefore, if a moving image file that has been subjected to marking processing is included, it is first determined whether or not the frame image data has been subjected to marking processing, and is a moving image file that has not been subjected to marking processing. When the determination is made, in addition to determining whether or not there is a change in the scene between two frame images that are temporally changed, it is also possible to set a scene change flag.

Hereinafter, a case where the same image file subjected to the marking process in the imaging device 12 is included in the plurality of moving image files will be described as a third embodiment.
<Third Embodiment>
Hereinafter, processing based on the timeline display will be described with reference to the flowchart of FIG.

  Step S301 is processing to specify a folder. The process of step S301 is the same process as step S101.

  Step S302 is processing for determining whether or not there is a moving image file. The process of step S302 is the same process as step S102.

  Step S303 is processing for sorting moving image files. The process in step S303 is the same as that in step S103.

  Step S304 is processing for performing timeline display based on the sorted moving image file. The process of step S304 is the same process as step S104.

  Step S305 is processing for determining whether or not marking processing has been performed. For example, if the marking process is not performed on the frame image data read from the moving image file, the CPU 21 sets the determination result in step S305 to No. In this case, the process proceeds to step S306. On the other hand, when the marking process is not performed on the frame image data, the CPU 21 sets the determination result of step S305 to No. In this case, the process proceeds to step S308.

  Step S306 is processing for determining a scene change. The CPU 21 reads out two frame image data that are temporally changed in the sorted moving image file. Then, it is determined whether or not the scenes in the two frame image data that change in time are changing.

  Step S307 is processing for determining whether or not there is a scene change. When it is determined in the process of step S306 that the scene in the two frame image data before and after has changed, the CPU 21 sets the determination result of step S307 to Yes. In this case, the process proceeds to step S308. On the other hand, when the scenes in the two frame image data that change in time are not changed, the CPU 21 sets the determination result in step S307 to No. In this case, the process proceeds to step S309.

  Step S308 is processing for setting a scene change flag. The process of step S308 is the same process as step S107.

  In addition, the process of step S309-step S314 is the same process as the process of step S108-step SS113. When the process of S314 is completed, the process returns to step S305, and the scene change flag is set for the frame image data of each moving image file. Thereafter, after scene change flags are set for all moving image files, scene analysis processing (S-315) and processing for obtaining representative images (S-316) are performed. Then, a timeline based on the scene is displayed (S-317).

  As described above, when a moving image file on which marking processing has been performed is used, if there is a frame image on which marking processing has been performed, a scene change flag can be set for the position of the frame image. In other words, the location where the marking process is performed indicates, for example, a location where an important scene for the user starts or a location where the important scene ends. By setting the flag, it is possible to perform display that takes into account the user's intention in the timeline display based on the scene.

  In the third embodiment, when the marking process has not been performed, it is determined whether or not the scene has changed. However, for a moving image file that has been subjected to the marking process, the marking process is performed. It is also possible to simply determine whether or not the scene has been applied and to omit the determination process whether or not the scene has changed. In this case, a determination process for determining whether or not the scene has changed may or may not be performed on a moving image file that has not been subjected to the marking process.

  In the case of the third embodiment, when moving image shooting is performed by the imaging apparatus, a folder for storing a moving image file is automatically generated on the storage medium, and then the moving image file is stored in the folder. The For this reason, it is also possible to previously determine the continuity of moving image files obtained by moving image shooting in the imaging device. By performing this determination, when performing timeline display using these moving image files, it is possible to omit the determination process of whether or not the scene has changed.

  The first to third embodiments described above are intended to perform timeline display based on a scene. In such a timeline display, a user may reproduce a moving image corresponding to an intended scene and check the content of the moving image. However, in a scene, it may be a moving image that is long in time, and it takes time and effort to confirm all the contents of the scene. For this reason, when reproducing the intended scene, it is possible to change the speed at which the moving image is reproduced.

  As shown in FIG. For example, playback is performed at normal speed for 5 seconds from the beginning of the scene. When 5 seconds have elapsed from the beginning of the scene, playback is performed at a speed that is, for example, 30 times the normal speed. By performing such reproduction, it is possible to reduce the reproduction time when each scene is individually reproduced. For this reason, the user can grasp the contents of each scene in a short time.

  In addition, it is also possible to obtain the similarity between adjacent scenes and change the playback speed according to the similarity between adjacent scenes. In this case, for example, when the similarity between adjacent scenes is higher than a threshold value, the playback in that scene is skipped, or the playback speed is doubled compared to normal playback. On the other hand, when the similarity between adjacent scenes is lower than the threshold value, playback is performed at a normal speed. Also in this case, the contents of the scene can be grasped in a short time. In this case, it is possible to adjust the playback speed according to the similarity between frames, not the scene.

  In the above-described embodiment, the thumbnail image is acquired for the purpose of displaying an image in the timeline display. However, these thumbnail images can be stored together as a still image.

  In the first to third embodiments described above, the image editing apparatus is described as an example. However, the image management program for executing the functions shown in FIG. 3 and the flowcharts of FIGS. It may be. In this case, the image editing program is preferably stored in a storage medium that can be read by a PC, such as a memory card, an optical disk, or a magnetic disk.

  DESCRIPTION OF SYMBOLS 11 ... PC, 12 ... Imaging device, 21 ... CPU, 24 ... Display control circuit, 25 ... Display device, 27 ... Image memory, 41 ... Moving image management part, 42 ... Timeline generation part, 43 ... Scene analysis part, 44 ... Representative image acquisition unit

Claims (6)

An image reading unit for reading a plurality of moving images;
A plurality of moving images read by the image reading unit, a scene sorting unit that sorts the arranged moving images for each scene included in the plurality of moving images;
An image acquisition unit for acquiring representative images in a plurality of scenes sorted by the scene sorting unit;
A display unit that displays a slider indicating the plurality of moving images sorted for each scene, and performs a timeline display that displays a representative image in a scene sorted by the plurality of moving images in the vicinity of the scene of the slider;
An image editing apparatus comprising: The image editing apparatus according to claim 1,
The scene extracting unit arranges a plurality of moving images read by the image reading unit based on shooting date and time, and then combines these moving images into one moving image, and then includes them in the combined moving image An image editing apparatus characterized by extracting a scene to be recorded. The image editing apparatus according to claim 1,
The scene extracting unit classifies the plurality of moving images into a plurality of scenes by determining a change between two frame images that are temporally changed among a plurality of frame images constituting the moving image. An image editing device. The image editing apparatus according to claim 1,
A setting unit for setting a priority for each of the scenes included in the plurality of moving images;
A display control unit that changes the fineness of the scene in the timeline display according to the set priority;
An image editing apparatus comprising: The image editing apparatus according to claim 1,
A selection unit for selecting a scene to be displayed in the timeline display;
An image editing apparatus, comprising: an image generation unit that generates a moving image based on the scene selected by the selection unit using the plurality of moving images. An image reading step of reading a plurality of moving images;
A scene sorting step of arranging the plurality of moving images read in the image reading step and sorting the arranged moving images for each scene included in the plurality of moving images;
An image acquisition step of acquiring representative images in a plurality of scenes sorted by the scene sorting step;
A display step of displaying a slider indicating the plurality of moving images sorted for each scene, and performing a timeline display in which a representative image in the scene sorted by the plurality of moving images is displayed near the scene of the slider;
A method for displaying a moving image, comprising:

Images