JP2013055409A - Video editing device and control method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make an edited video be a natural video in performing video editing for inserting another video into a section where an unnecessary section of a video is deleted.SOLUTION: A video editing device extracts a video in an unnecessary section from a video to be edited (203), determines a video to be inserted into the unnecessary section on the basis of an image corresponding to the extracted video in the unnecessary section (206), and inserts the determined video into the unnecessary section (207).

Description

  The present invention relates to a technique for editing video.

2. Description of the Related Art Conventionally, when a person who is not skilled in shooting performs shooting, many failure videos such as camera shake are recorded. Therefore, during video editing, editing that cuts or corrects a failed video portion is often performed. As processing for deleting unnecessary sections, processing for not recording or deleting moving images in which the amount of camera shake is greater than a predetermined value is performed. (See Patent Document 1)
However, when editing is performed by cutting the failed video portion and connecting them as they are, there may be a sense of incongruity such as a jump cut that makes it seem that the object has moved instantaneously. For this reason, in order to connect discontinuous images without any sense of incongruity in video editing, a method of inserting a video different from the previous and next video called “insert cut” is generally performed. As a process of inserting a part of the video, a process of determining the video to be inserted from the video selected by the user is performed. (See Patent Document 2)

JP 2008-160637 A JP 2008-22519 A

However, in Patent Document 2, depending on the video selected by the user, the connection with the video before and after the unnecessary section may become unnatural. For example, if there is an object that exists before and after the unnecessary section in the inserted video, the object appears to move instantaneously at the joint of the video, which is unnatural.
Therefore, the present invention aims to solve the above-described problem and to make the edited video a natural video when performing video editing in which another video is inserted into a section in which an unnecessary section of the video is deleted.

In order to achieve the above object, an information processing apparatus according to an aspect of the present invention includes the following arrangement. That is,
Extraction means for extracting video of an unnecessary section from the editing target video; and determination means for determining a video to be inserted into the unnecessary section from the editing target video based on an image corresponding to the extracted video of the unnecessary section; And inserting means for inserting the determined video into the unnecessary section.

  According to the present invention, when video editing is performed in which another video is inserted into a section in which an unnecessary section of the video is deleted, it is possible to obtain an effect that enables editing so that the edited video becomes a natural video.

Hardware configuration diagram of video editing device Functional block diagram showing the outline of the video editing device Table showing an example of object information Table showing an example of background information The figure explaining the background similarity determination process The figure explaining the outline of the video editing processing of the video editing device Flow chart explaining details of a series of video editing processing Flowchart for explaining details of insertion video determination process The figure explaining the outline of the video editing processing of the video editing device Flowchart for explaining in detail a series of processing of inserted video determination processing

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(Embodiment 1)
Embodiment 1 is demonstrated based on drawing. FIG. 1 is a hardware configuration example of a video editing apparatus according to the present embodiment. The hard disk drive 101 stores data such as video data, object information, and background information. The ROM 102 is used for storing programs. A flow processing program to be described later is also stored here. The RAM 103 is used when executing a program. The CPU 104 is a control unit that controls the video editing apparatus. Processing of a flow to be described later is also controlled by the CPU 104. The input device 105 is used when the video editing apparatus accepts input from the user. The display 106 is used to display the output result from the video editing device. A bus 107 is a data line that connects the hard disk drive 101, ROM 102, RAM 103, CPU 104, input device 105, and display 106.

  Hereinafter, a moving image is referred to as a video, and a still image is referred to as an image. In the middle, object information and background information will be described with reference to FIGS. 3 and 4, and background similarity determination processing in the present embodiment will be described with reference to FIG.

  FIG. 2 is a functional block diagram schematically showing the video editing apparatus according to the present embodiment. The storage medium 201 includes a hard disk 101 that stores data, and stores video data, object information, background information, and the like.

  FIG. 3 shows an example of object information. ID 301 is information for specifying the video in which the object is detected. The time stamp 302 is time information of the frame in which the object is detected, and its unit is milliseconds. 303 and 304 are the x-coordinate and y-coordinate of the position of the object on the screen, and the unit is a pixel. Reference numeral 305 denotes the area of the detected object area, which is represented by the number of pixels.

  Reference numeral 306 denotes information on the hue of the object. For example, RGB color distribution information of the object area is raised. The edge information 307 is image data indicating information about the edge of the object. For example, the image 307 is image data obtained by binarizing and thinning the object area of the frame image. However, the edge information may be acquired by other methods as long as the information indicates the edge information.

  FIG. 4 shows an example of background information. ID 401 is information for specifying a video whose background has been detected. The time stamp 402 is time information of a frame in which the background is detected, and its unit is milliseconds. Reference numeral 403 denotes background hue information. The edge information 404 is edge information similar to the edge information 307. As long as this also indicates the edge information, the edge information may be acquired by various methods.

  The outline of the video editing apparatus according to this embodiment will be described with reference to FIG. The video editing device 202 deletes an unnecessary section of the video and performs video editing by inserting other data into the deleted section. The unnecessary section extraction unit 203 extracts an unnecessary section of the editing target video. The unnecessary section extraction unit 203 extracts a section with a lot of camera shake, a section that is too bright, a section that is too dark, or the like as an unnecessary section. For example, a section where camera shake is severe is detected from the average of the motion vectors of each frame and the variance value of the frequency distribution. Further, a section that is too bright and a section that is too dark is determined based on whether the average value of the luminance of the frame image is greater than or equal to a predetermined upper limit value or less than a lower limit value. Further, the user may extract unnecessary sections. In addition, these are examples and are not limited to these.

  The object determination unit 204 determines whether or not the same object exists in the video immediately before and immediately after the unnecessary section extracted by the unnecessary section extraction unit 203. The discrimination of the object is performed by pattern matching the hue information 306 or the edge information 307 of the object. For example, the object region is divided into a predetermined number of blocks, and it is determined whether the color distributions match for each of the divided blocks. If the matching blocks are equal to or higher than a predetermined value, they are determined to be the same object. This is an example of the object discrimination method and is not limited to this.

  Note that as the video object immediately before and after the unnecessary section, only an object existing in one frame immediately before and after the unnecessary section may be used, or the number of frames before and after the specified number of frames from the frames immediately before and after the unnecessary section. An object that exists later may be used. Further, if an object can be specified even in an unnecessary section, the object in the unnecessary section may be used, and the object may be any object corresponding to the object in the unnecessary section.

  The background determination unit 205 determines whether the background image is similar to the background of the image immediately before or immediately after the unnecessary section extracted by the unnecessary section extraction unit 203. Background discrimination is performed by pattern matching of background hue information 403 or edge information 404. For example, the background region is divided into a predetermined number of blocks, and it is determined whether the color distributions match for each of the divided blocks. If the matching blocks are equal to or higher than a predetermined value, they are determined to be the same object. This is an example of a background discrimination method and is not limited to this.

  Note that only the background existing in one frame immediately before and after the unnecessary section may be used as the background of the video immediately before and immediately after the unnecessary section, or a predetermined number of frames before the frame immediately before and after the unnecessary section. Alternatively, the background after the number of frames may be used. Further, if the background can be specified even in the unnecessary section, an object in the unnecessary section may be used, and the background may be any background corresponding to the background of the unnecessary section.

  The background similarity determination process will be described with reference to FIG. Frame images 502 and 503 are images obtained by zooming in a little from the frame image 501 and shooting. A region 504 indicates an overlapping portion of the frame image 502 and the frame image 503. When partial pattern matching is performed using the frame image 502 and the frame image 503, the overlapping portions 504 are matched. If the matched area is equal to or greater than a predetermined value, the frame image 502 and the frame image 503 are determined as similar backgrounds.

  The frame image 511 is a frame image taken by zooming in on the frame image 502. Either one of the frame image 502 and the frame image 511 is enlarged or reduced to perform pattern matching. If the enlargement rate or reduction rate when they match is less than a predetermined value, the frame image 502 and the frame image 511 are determined to have similar background images.

  The inserted video determination unit 206 includes a video that is inserted by the object determination unit 204 and the background determination unit 205 into a section in which the same object does not exist and the background is determined not to be similar to the section from which the unnecessary video is deleted. decide. Note that only one of the object determination unit 204 and the background determination unit 205 may be configured, and the determination may be made by one determination.

  Note that the video to be inserted may be determined in units of video files, or may be determined in video sections obtained by temporally dividing the video.

Further, the range of the material video that determines the video to be inserted may be limited. For example, you may limit to the image | video in the imaging | photography time made to move back and forth by the predetermined defined value from the imaging | photography time of the edit object image | video. By limiting in this way, it becomes possible to insert a video that is close in time to the video immediately before and immediately after the unnecessary section and that is inserted from a highly related video. Consideration Further, without determining the background, it may be limited to a video with the same object as the object existing immediately before or immediately after the unnecessary section, a video with a similar background, or an image without the same object. In addition, it may be combined with background judgment.

  For example, it is effective to limit the images to the same background and no object. Thereby, it is possible to select an image of only a landscape.

  Further, the object may be recognized, the object of the image to be inserted may be other than a person, and the background may be limited to the same image.

  Here, it is effective to prevent the above-mentioned uncomfortable feeling by making the video to be inserted a video that is not similar to the video in the unnecessary section.

  By limiting in this way, it is possible to determine the video to be inserted from the related video even if it is temporally separated from the video immediately before and immediately after the unnecessary section.

  In addition, it is good also as the raw material image | video of not only the image | video image | photographed with the single apparatus but the image | video reproduced | regenerated with the several apparatus, and the image | video on a network.

  The outline of the video editing apparatus according to this embodiment will be described with reference to FIG. The video deletion / insertion unit 207 deletes the unnecessary section extracted by the unnecessary section extraction unit 203 and performs editing to insert the video determined by the insertion video determination unit 206 into the deleted section. The display unit 208 displays the video data edited by the video deletion / insertion unit 207.

  A processing example in this embodiment will be described with reference to FIG.

  Screen shots of the video before and after video editing using this case are shown. An object A601 is an object detected in the video. Reference numerals 602 to 604 are screen shots of video before video editing. Reference numeral 605 denotes a frame immediately before the unnecessary section. Reference numeral 606 denotes a frame immediately after the unnecessary section. Reference numeral 607 denotes a screen shot of the inserted video determined by the inserted video determining unit 206. Object B608 is an object detected in the video. The unnecessary section extracting unit 203 extracts the portion of the screen shot 603 as an unnecessary section due to severe camera shake. The object determination unit 204 detects an object immediately before the unnecessary section from the frame 605 and an object immediately after the unnecessary section from the frame 606, and determines whether the same object exists in the video 607.

  Although the objects immediately before and immediately after the unnecessary section are detected from the frame 605 and the frame 606, this is an example, and the present invention is not limited to this. For example, the object immediately before the unnecessary section may be detected from a plurality of frames between the frame 605 and a frame that is a predetermined number of frames back.

  The background determination unit 205 detects the background immediately before the unnecessary section from the frame 605 and the background immediately after the unnecessary section from the frame 606, and determines whether the video 607 is a similar background video. The background immediately before and immediately after the unnecessary section is detected from the frame 605 and the frame 606. However, this is an example, and the present invention is not limited to this. For example, the background immediately before the unnecessary section may be detected from a plurality of frames between the frame 605 and a frame that is a predetermined number of frames back.

  Based on the determination results of the object determination unit 204 and the background determination unit 205, the insertion video determination unit 206 determines a video 607 different from the object and background of the video 602 and the video 604 as a video to be inserted. The video deletion / insertion unit 207 deletes the unnecessary section 603 and performs video editing to insert the video 607 in the deleted section.

  By determining and inserting the video to be inserted in this way, it is possible to obtain a naturally connected video after editing.

  FIG. 7 shows a flowchart of the video editing process of the video editing apparatus 202 in the present embodiment.

  When the video editing process is started, the video to be edited is read from the recording medium 201, and a section with severe camera shake is extracted as an unnecessary section (S701). In the present embodiment, a section with severe camera shake is extracted as an unnecessary section, but a section that is too bright or a section that is too dark may be extracted. Also, the user may extract it.

  The video deletion / insertion unit 207 sets the range of the material video used as the video to be inserted as the material video within the shooting time that is around the specified time from the shooting time of the editing target video (S702). In this embodiment, the video to be inserted is the material video within the shooting time that is around the specified time from the shooting time of the video to be edited. However, a material video that contains the same object or a material video of a similar background is inserted. It is good also as an image to do. Also, the user may decide.

  The shooting insertion video determination unit 206 performs an insertion video determination process for determining a video to be inserted from within the determined range of material video (S703). Details of the insertion video determination process will be described later with reference to FIG.

  The video deletion / insertion unit 207 deletes the unnecessary section, performs video editing for inserting the video determined by the inserted video determination unit 206 into the section from which the unnecessary section has been deleted (S704), and ends the video editing process. FIG. 8 is a flowchart of the insertion video determination process according to the first embodiment. The insertion video determination process is started. Of the material videos within the range determined in step S702, a video whose shooting time is closest to the video to be edited is set as a candidate video (S801).

  The object determination unit 204 determines whether the object existing in the video immediately before and immediately after the unnecessary section extracted by the unnecessary section extraction unit 203 is the same as the object existing in the candidate video (S802).

  If the same object exists in step S802, it is determined whether there is a material video whose shooting time is closest to the editing target next to the current candidate video in the range determined in step S702 (S803). If there is a material video whose shooting time is closest to the editing target next (S803), the candidate video is changed to the material video whose shooting time is close to the shooting time of the editing target next to the current candidate video (S804), and the process proceeds to step S802. Transition.

  If there is no material video whose shooting time is closest to the editing target next (S803), a material video having a background that is most similar to the background of the editing target in the range determined in step S702 is selected (S805). Then, the insertion video determination process is terminated.

  The video to be inserted here is a material video within the range determined in step S702 and has a background that is not most similar to the background to be edited. However, this is only an example, and the video is limited to this. is not. For example, the material time may be a material image closest to the editing target.

  When the same object does not exist (S802), the background determination unit 205 determines whether the background of the video immediately before and immediately after the unnecessary section extracted by the unnecessary section extraction unit 203 is similar to the background of the candidate video ( S806). If the background is similar (S806), the process proceeds to step S803. If the background is not similar (S806), it is determined that the candidate video is to be inserted (S807), and the inserted video determination process is terminated.

  Note that although the material video is selected as the candidate video from the next closest to the editing video, the method for selecting the candidate video is not limited to this. For example, the shooting time may be selected as a candidate video from a new material video.

  As described above, according to the first embodiment, when performing video editing in which another video is inserted into a section in which an unnecessary section of the video is deleted, editing can be performed so that the edited video becomes a natural video. The effect to do.

(Embodiment 2)
Embodiment 2 is demonstrated based on drawing. The basic configuration is the same as that of the first embodiment described with reference to FIGS. Hereinafter, the description according to the first embodiment is omitted.

  In the second embodiment, when the video to be inserted in the inserted video determination process of the first embodiment does not exist in the material video, a process of generating an inserted video by performing a processing process of spatially cutting the frame of the video is performed. Do.

  An example of video processing will be described with reference to FIG.

  A video 901 is a video before processing the video to be inserted. The background 902 is a background in the vicinity of the object existing in the video before and after the unnecessary section. A region 903 is a cutout range. A screen shot 904 is a screen shot of the video after processing the video 901.

  The inserted video determination unit 206 determines the video to be inserted as a video obtained by cutting out a portion of the region 903 that is not the background in the vicinity of the object existing in the video before and after the unnecessary section in the background of the video 901. The video deletion / insertion unit 207 generates and inserts a video 703 to be inserted by performing a process of cutting out and expanding the cutout range 903 determined by the insertion video determination unit 206 from the video 901.

  By determining the video to be inserted in this way and inserting the video, it is possible to obtain a video having a natural connection after editing.

  This process is an example. The video to be inserted after processing is not limited to this example as long as no object exists immediately before and after the unnecessary section and the background is not similar.

  FIG. 10 shows a flowchart of the inserted video determination process in the second embodiment.

  In the second embodiment, when it is determined that there is no material video whose shooting time is next closest to the editing target (S803), the material video within the range determined in step S702 is most similar to the background immediately before and immediately after the unnecessary section. A material video having a background that has not been processed is set as a processing target video (S1001).

  Here, the processing target video is a material video having a background that is not most similar to the editing target background, but this is only an example, and the present invention is not limited to this. For example, a material video whose shooting time is closest to the editing target may be used as the processing target video.

  The inserted video determination process is terminated by setting a portion of the background of the processing target video that is not the background in the vicinity of the object existing in the video immediately before and immediately after the unnecessary section and inserting the enlarged portion (S1002).

  In the present embodiment, the processing for cutting out and enlarging the video area is performed as the processing, but this is an example, and the present invention is not limited to this.

  As described above in the second embodiment, by configuring and processing in this way, by editing and inserting another video into a section in which an unnecessary section of the video is deleted, the edited video becomes a natural video. The effect becomes.

  As described above, according to the first and second embodiments, when performing video editing in which another video is inserted into a section in which an unnecessary section of the video is deleted, an effect that enables editing so that the edited video becomes a natural video. Can be obtained.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (11)

Extraction means for extracting video of unnecessary sections from the video to be edited;
Determining means for determining from the editing target video a video to be inserted into the unnecessary section, based on an image corresponding to the extracted video of the unnecessary section;
An image editing apparatus comprising: an inserting unit that inserts the determined image into the unnecessary section.   The video editing apparatus according to claim 1, wherein the determination unit determines the video to be inserted as a video that is not similar to the video in the unnecessary section. An object determination means for determining whether an object exists in an image corresponding to the video of the unnecessary section;
A background determination means for determining whether the image has a background similar to the background of the image corresponding to the video of the unnecessary section;
The video editing apparatus according to claim 1, wherein the determination unit determines a video based on a determination result by at least one of the object determination unit and the background determination unit.   The determining means determines that there is no object present in the image corresponding to the video in the unnecessary section by the object determining means, and is similar to the background of the image corresponding to the video in the unnecessary section by the background determining means. The video editing apparatus according to claim 3, wherein the video editing apparatus determines that the video to be inserted is a video that is determined not to be a video having a background.   5. The video editing apparatus according to claim 1, wherein the determination unit determines, as the video to be inserted, a video that is temporally close to a video in the unnecessary section among the video to be edited.   6. The video editing apparatus according to claim 1, wherein the image corresponding to the video in the unnecessary section is a frame image immediately before and after the video in the unnecessary section of the editing target video.   The video editing apparatus according to claim 1, wherein the video in the unnecessary section is at least one of a section where camera shake is intense, a section that is too bright, and a section that is too dark. It further has a processing means for cutting out a part of the image,
8. The video editing apparatus according to claim 1, wherein the determination unit determines the video processed by the processing unit as a video to be inserted. An extraction process for extracting images of unnecessary sections from the images;
A determination step for determining a video to be inserted into the unnecessary section from the editing target video based on an image corresponding to the extracted video of the unnecessary section;
An image editing apparatus control method comprising: an inserting step of inserting the determined image into the unnecessary section.   The program for making a computer perform each process of the control method of the video editing apparatus of Claim 9.   A computer-readable storage medium storing the computer program according to claim 10.

Images