JP2010232814A - Video editing program, and video editing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine whether video is appropriate as an object to be edited before editing the video. <P>SOLUTION: Image data of a predesignated person or object is compared with a frame image constituting video to find the person or object in the video (S4). A state with the person or object reflected in the video is detected based on the search result (S7). Based on the state with the person or object reflected in the video, propriety of editing the video with the person or object centered is determined (S7). <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a video editing program and a video editing apparatus.

  In order to shoot video suitable for various themes and events, a series of shooting instruction commands according to the theme and event type is given to the camera, and the photographer is allowed to perform shooting according to the shooting commands. Digital video systems for sequence synthesis are known.

Special table 2007-529951 gazette

According to the above-described system, it is possible to shoot a good-looking image as if a professional photographer shoots along the script for each event, but it must shoot according to a series of shooting instruction commands. In addition, it is very cumbersome and troublesome, and in some cases, the flow of the event itself may have to be matched with a series of shooting instruction commands, which may hinder the excitement of the event.
Therefore, it is conceivable to identify an event from the captured video and automatically edit the video according to a scenario corresponding to the event. However, the edited video is not always enjoyed by the viewer. Viewers are interested in videos where people close to viewers appear for a long time, such as videos where the children of the viewers are active as protagonists in athletic events, but quickly get bored with videos where many unknown children appear. End up. Therefore, it is necessary to edit an image in which people or objects interested in the viewer frequently appear.

(1) According to the first aspect of the present invention, a person search procedure for searching for a person or an object from the image by comparing image data of a person or an object specified in advance with a frame image constituting the image, and a person search procedure Based on the search result of, the state detection procedure that detects the state in which a person or an object is reflected in the video, and editing the video centering on the person or the object based on the state detected by the state detection procedure This is a video editing program that causes a computer to execute an editing suitability determination procedure for determining whether or not to perform the editing.
(2) In the video editing program according to claim 1, in the video editing program according to claim 1, in the status detection procedure, a time during which a person or an object is reflected in the video is detected as a status, and in the editing suitability determination procedure, When the ratio of the time to the total playback time of the video is less than the predetermined ratio, it is determined that the video is not suitable for editing mainly on a person or an object.
(3) According to a third aspect of the present invention, in the video editing program according to the first or second aspect, in the state detection procedure, the size of the frame image of the person or thing searched in the person search procedure is set as the state. In the detection and editing suitability determination procedure, if the size of the person or the object in the frame image is less than a predetermined size, it is determined that the video is not suitable for editing centering on the person or the object.
(4) According to a fourth aspect of the present invention, in the video editing program according to any one of the first to third aspects, in the state detection procedure, blurring of a person or an object in the frame image searched in the person search procedure is performed. The condition is detected as a state, and in the editing suitability determination procedure, if the blur condition of the person or object in the frame image is larger than the allowable value, the video is not suitable for editing centering on the person or object. judge.
(5) In the video editing program according to any one of claims 1 to 4, the invention of claim 5 analyzes the content of the video and identifies an event reflected in the video; The computer further executes a scenario selection procedure for selecting a scenario corresponding to the event identified by the event identification procedure from among a plurality of scenarios, and in the state detection procedure, a specific scene defined in the scenario in the video is processed. Whether or not a person or an object is reflected is detected as a state, and in the editing suitability determination procedure, if the state detection procedure does not detect a state where a person or an object is reflected in a specific scene, It is determined that it is not suitable for editing centering on an object.
(6) In the video editing program according to claim 5, in the video editing program according to claim 5, in the status detection procedure, the playback time for each scene defined in the scenario in the video is detected as the status, and the editing suitability determination procedure In the state detection procedure, if the playback time for each scene in the video is less than the playback time for each scene specified in the scenario, the video is not suitable for editing with a focus on people or objects. judge.
(7) The invention of claim 7 includes a storage device for storing video, a computer for executing the video editing program according to any one of claims 1 to 6, and a display device for displaying video. It is a video editing device.

  According to the present invention, before editing a video, it is possible to know whether or not the video is suitable for generating a new video centered on a predetermined person or object, and a wasteful editing process. Can be prevented in advance.

The block diagram which shows the structure of the video editing apparatus of one embodiment The flowchart which shows the edit operation | movement of the video editing apparatus of one embodiment FIG. 2 is a flowchart illustrating the editing operation of the video editing apparatus according to the embodiment, following FIG. Diagram showing how to set the hero Diagram showing how to set the hero Timing chart showing the analysis result of the main editing target video 1 frame image showing video editing process 1 frame image showing video editing process 1 frame image showing video editing process 1 frame image showing video editing process 1 frame image showing video editing process 1 frame image showing video editing process 1 frame image showing video editing process 1 frame image showing video editing process 1 frame image showing video editing process 1 frame image showing video editing process 1 frame image showing video editing process 1 frame image showing video editing process The figure which shows the frame image in the image | video produced | generated by the edit of one embodiment Figure showing an example of a title image The figure which shows an example of the still picture extracted from the video file of the athletic meet taken after the competition

  FIG. 1 is a block diagram showing a configuration of a video editing apparatus 1 according to an embodiment. The storage device 2 has a slot into which various recording media 2a can be inserted, and stores video files including video (moving image) data and audio data read from the recording media 2a, and is generated by editing. Write the video file to the recording medium 2a. Note that the video to be edited by the video editing apparatus 1 includes a series of still images captured by a digital camera as well as a moving image captured by a video camera or a digital still camera. In the video editing apparatus 1 according to the embodiment, not only video but also audio recorded together with video is edited in accordance with video editing. Furthermore, the video file may be read into the video editing apparatus 1 via a network (not shown). In this case, the video editing apparatus 1 does not necessarily include the recording device 2.

  The microcomputer 3 includes a control unit 3a, an image analysis unit 3b, and an image processing unit 3c in software form, and performs video and audio editing processing. The control unit 3a controls a series of editing processes. The image analysis unit 3b analyzes the video data and audio data included in the video file to be edited, and the content of the video such as what kind of “event” is taken or what subject is shown. Identify. The image processing unit 3c selects a “scenario” corresponding to the video content of the analysis result by the image analysis unit 3b, edits the video data and audio data in the video file according to the scenario, and generates a new video file.

  In this specification, “event” refers to various events and events that are captured by a video camera or a digital still camera. In this embodiment, an athletic meet, competition, a birthday party, etc. will be described as examples of events. Some events are developed hierarchically, and for example, there are various events such as competition and ball placement in athletic meet events. There are various scenes in the video of the event. For example, there are a start scene, a running scene, a goal scene, etc. in an athletic meet competition event.

  Further, in this specification, “scenario” means a script or a storyboard that defines a method and procedure for editing video and audio. As described above, various scenes are included in the event video, and in the scenario, selection (cut) for each scene with respect to these scenes, priority for each scene, playback time for each scene, scene switching, The playback order for each scene is defined, and by editing according to the scenario corresponding to the event, a video like an event video captured and edited by a professional cameraman can be obtained.

  The operation unit 4 has various operation members, such as selecting a video file to be edited from a plurality of video files read from the recording medium 2a, and selecting a hero in the video to be edited. And input user operation instructions for voice editing. The display device 5 displays images before and after editing, a selection screen for the main character, and the like. The speaker 6 reproduces the sound recorded together with the video before and after editing.

  In this embodiment, the video editing apparatus 1 that performs editing by inputting video captured by an imaging apparatus such as a video camera will be described as an example. A configuration may be adopted in which all or part of the apparatus 1 is built in and imaging to editing are executed by the imaging apparatus. In such an imaging apparatus incorporating the whole or a part of the video editing apparatus 1 according to the embodiment, it is also possible to capture and edit video captured by another imaging apparatus.

  By the way, usually, when capturing images of various events with an imaging device such as a video camera, zoom in (close-up) to capture a large image of the hero's face, pan the imaging device chasing the hero, Alternatively, the user tries to capture a good-looking image centered on the main character by using camera work, such as cutting an unnecessary scene (scene), that is, not capturing the image. However, it is difficult for ordinary users to capture images that look good like those photographed by professional photographers, and even if you edit images captured using camera work, you can zoom in and zoom out from the expanded image captured. A video that is intended by the user is generated because it is not possible to acquire a reduced video that has been acquired, or to acquire a video of the imaging range before panning after panning the imaging device, or to acquire a video of the cut period It ’s difficult.

  Therefore, in this embodiment, when an image is picked up by the image pickup device, the image is picked up at a wide angle so that all the events occurring in a wide range can be acquired, and the image picked up at the wide angle is set as an editing target. . Then, a part of the frame image composing the image captured at the wide angle is cut out by trimming and edited as if it was zoomed in or zoomed out at the time of imaging, or the trimming range was changed as if at the time of imaging. Edit as if the imaging device was panned, or delete as part of the same scene and edit as if a cut was made during imaging. By performing such editing, it is possible to realize a pseudo camera work as if it were captured using the camera work at the time of imaging, and obtain the same video effect as when the camera work was actually performed at the time of imaging. be able to.

  Note that an image captured by the user using camera work such as zooming, panning, and cutting during imaging may be an object to be edited. An image captured at a wide angle by the first imaging device and a camera captured by the second imaging device. Editing may be performed in combination with images captured using the work.

  On the other hand, when generating a video composed of an image obtained by trimming and editing a part of the original image by trimming, it is difficult to see if the number of pixels on the display screen of the display device 5 changes every time the cropping region changes. Therefore, the pixel number conversion process is performed by the image processing unit 3c in accordance with, for example, the effective display pixel number of the display device 5 so that the pixel number of the image generated by trimming from the original image is always constant. . Note that the larger the enlargement magnification by pixel number conversion, that is, the smaller the number of pixels in the trimming area, the greater the degradation of image quality, so the size of the trimming area is limited so that it does not fall below the size that satisfies the predetermined image quality standard. Is provided. In addition, when a part of the frame image captured at a wide angle is cut out by trimming to generate a new frame image, and a new video is created by connecting them, there is camera shake during imaging and the video is blurred The camera shake correction is performed during trimming.

  Moreover, in this embodiment, not only the video imaged by one imaging device is set as an editing target, but also a plurality of videos imaged by a plurality of imaging devices are set as editing targets. Although details will be described later, in this embodiment, an example is shown in which an image of an athletic meet is captured by two imaging devices X and Y.

  2 and 3 are flowcharts showing an editing operation of the video editing apparatus 1 according to the embodiment. FIGS. 4 and 5 are diagrams showing the setting method of the main character, FIG. 6 is a timing chart showing the analysis result of the main editing target video, and FIGS. 7 to 18 are images of one frame showing the video editing process. . The video editing operation of the embodiment will be described with reference to these drawings. Here, an example will be described in which an image obtained by capturing a competition image of an athletic meet in which a fictional kindergarten child “Nao-chan” participated is edited into a video having Nao-chan as the main character. The main character is not particularly limited to the above example, and may be anyone (for example, a dog) or an object such as a moving car. In step 1, a video file designated by the user through the operation unit 4 is set as a main editing target file. It is assumed that a plurality of video files are read from the recording medium 2a and stored in the storage device 2 prior to setting the main editing target file.

  In step 2, the main character is set in the video of the main editing target file, and the appearance time of the main character is calculated. In this embodiment, a hero is set by either or both of two main character setting methods. In the first setting method, the video of the above-described main editing target file is displayed on the display device 5, and the user uses the mouse of the operation unit 4 in a state where one frame in which the main character is reflected is frozen. In this method, the subject as the main character is pointed and the main character area on the frame image is extracted and set. The second setting method is a method in which a main character candidate list registered and stored in the storage device 2 in advance is displayed on the display device 5 and the main character is selected and set by the user using the mouse of the operation unit 4.

  In the first main character setting method, as shown in FIG. 4, image data of a set main character region (for example, image data in a region surrounded by a broken line in the figure) is stored in the storage device 2 as a template image, The main character is tracked while performing template matching with each frame image in the video. According to the tracking of the main character by this template matching, the position of the main character can be estimated even if the main character is lost on the way. On the other hand, in the second hero setting method, as shown in FIG. 5, personal authentication is performed on each frame image in the video based on the detailed data of the hero selected from the hero candidate list to track the hero. According to the second main character setting method, personal information such as the name and address of the main character can be automatically acquired, and the personal information can be used for title subtitles and narration of a new video.

  By setting the main character, it is possible to detect the existence, position, size, etc. of the main character for each frame in the video, thereby calculating the time the main character is reflected in the total video playback time, that is, the appearance time of the main character can do. If the main character candidate list does not store the character as the main character and the main character can be set only by the first main character setting method, personal information such as the name of the main character cannot be obtained. 5 and the speaker 6 prompt the user to input personal information of the hero through the operation unit 4.

  In step 3, a video file close to the imaging time of the main editing target file designated by the user (within a predetermined time) is extracted from the video files stored in the storage device 2. In addition, since the imaging position is recorded in the video file captured by the imaging device with a built-in GPS receiver, a video file close to the imaging position of the main editing target file (within a predetermined distance) may be extracted. Alternatively, a video file close to the imaging time of the main editing target file (within a predetermined time) and close to the imaging position of the main editing target file (within a predetermined distance) may be extracted. The video file extracted in this way is considered to have captured the same event as the main editing target file.

  In step 4, the main character is searched from the main editing target file and each video file extracted as being close to the imaging time (or position) of the main editing target file, and the appearance time of the main character is calculated. In the subsequent step 5, the video data recorded in each video file is analyzed, and the event of the video shown in each video file is determined. The discrimination of the event is realized by performing recognition processing of the movement of the main subject such as the main character shown in the video, extraction of clothing features such as the main character, or recognition processing. For example, as disclosed in Japanese Patent Application Laid-Open No. 5-46583, the motion recognition process divides an image into meshes, and uses feature quantities (such as an optical flow) calculated inside each mesh as vectors. A method of recognizing a time-series change of a vector by a probabilistic state transition model or a recognition process when recognizing an action as disclosed in Japanese Patent Application Laid-Open No. 2005-215927 is a first step and a second step. The motion element extraction unit in the first step recognizes a certain motion pattern performed in a short time based on the time-series change of the motion information detected by processing the image data, and the second step. In the action recognition unit, a method for recognizing a motion over a long period of time based on a ratio of a certain motion pattern included in a period to be recognized, The like can be used how to recognize operation using higher order local autocorrelation features as disclosed in 40267 JP. Also, feature extraction such as clothes can be realized by extracting color information, edge information, and texture information of a lower region of the recognized face position, for example. Furthermore, based on the extracted information, it is possible to recognize clothes peculiar to events such as gym clothes and wedding dresses. Note that more accurate event determination may be performed by referring to audio data recorded in a video file when determining an event. Further, the event may be determined by recognizing an event-specific subject (for example, a national flag for an “athletic event” event). Such event-specific subject types are recorded in advance in the control unit 3a in association with each event together with the recognition method.

  Here, an event determination method will be described. For example, the main editing target file A was captured on October 10, 2008 from 11: 00 to 11:20. From the image, (a1) a plurality of people wearing gym clothes (here, kindergarten children ) Running in the same direction, (a2) there are people with the tape of the goal in the running direction, (a3) multiple kindergarteners who ran towards the goal moved to the number flag If a scene such as crouching, (a4) many people watching kindergarten children running, (a5) the national flag fluttering in the wind, etc. The video of the target file A is identified as “Sport competition” of “Athletic meet”. The main character set in the main editing target file A is “***” from the video of the video file B (captured on October 10, 2008, 14: 00 to 14:05) whose imaging time is close to the main editing target file A. If a scene posing with a medal in hand is recognized beside the signboard of “Kindergarten Sports Day”, the video of the main editing target file B is identified as “Sport Day”. Furthermore, wearing the same gym suit as the main character set in the main editing target file A from the video file C (captured at 09:00 to 09:15 on October 10, 2008) whose imaging time is close to the main editing target file A When a scene where kindergarten children are doing gymnastics is recognized, the video of the video file C is identified as a “athletic meet”.

  On the other hand, from the video file D (captured at 11: 00 to 11:05 on November 25, 2008) whose image capture time is close to the main editing target file A, (d1) Children sing a song together, (d2) Indoor, (d3) Cake is shown, (d4) A candle is in the dark room, (d5) One child blows out the candle, (d6) Then the room becomes bright When the scene such as, is recognized, the video of the video file D is identified as “birthday party”. From the analysis results of the video scenes of the video files A to D, the event of the files A to C is identified as an athletic meet, and the event of the file D is identified as a birthday party. In this embodiment, a video file D whose event type is clearly different from the main editing target file A is excluded from the editing target.

  An example of automatically deleting a video file in which a type different from the event of the main editing target file is recorded among the video files extracted as having a shooting time (or position) close to that of the main editing target file is shown. Further, it may be instructed by the user via the operation unit 4 whether to manually exclude from the editing target. In addition, a configuration may be adopted in which video files in which different types of events are recorded are simultaneously edited, and new video files are generated respectively.

  In step 6, an editing event is set based on the event determination result of each video file. All of the video files A to C are event videos of “athletic meet”, but from the above-described analysis result of the video scene, video file A is “competition” in “sports meet”, video file B is “medal acquisition”, Video file C is an event video for each “preparation exercise”. In this embodiment, the event of the main editing target file first designated by the user is set as an editing event, and a message to that effect is displayed on the display device 5. When the user selects an event other than the main editing target file as an editing event, the operation unit 4 causes the video file of the desired editing event to be designated as the main editing target file from the video file. Here, a case where “competition” of the main editing target file A is set as an editing event will be described as an example. Note that even when “Adult Competition” is set as an editing event, the “Medal Acquisition” video file B and the “Preparatory Gymnastics” video file C are also subject to editing, and a new video file for the competition is generated. Part of the video recorded in the video files B and C can be used.

  In step 7 after the editing event is set, the suitability of editing for the editing target video is determined based on the appearance time of the main character. In this embodiment, the appearance times of the main characters in the video files A to C related to the “competition” are added together, and the ratio of the main character appearance time in the total reproduction time of the video files A to C is less than a predetermined ratio. If it is, the process proceeds to step 8 where a warning is given by the display device 5 and the speaker 6 that the user does not satisfy the content even if a new video recording the main character is created, and the process is terminated. In this case, the user can reset the hero through the operation unit 4 and start editing again from the beginning.

  In addition, when calculating the appearance time of the hero, it is necessary to consider the following points. As described above, in this embodiment, an image captured at a wide angle may be trimmed, and a trimming area may be enlarged to generate a new image. In this case, the size of the main character in the captured image is small. And the enlargement magnification increases and the image quality decreases. Therefore, when the size of the main character reflected in the frame image is smaller than a predetermined size, even if the main character is detected, the scene is not included in the appearance time of the main character. Alternatively, if the size of the main character reflected in the frame image is smaller than a predetermined size, it may be determined that the video to be edited is not suitable for editing centering on the main character. .

  Further, when calculating the appearance time of the main character, the degree of blur of the main character in the captured image is taken into consideration. The position of the main character in each frame image in the video can be specified by subject tracking processing by the first main character setting method or personal authentication processing by the second main character setting method. Therefore, it is possible to determine the degree of blur in the main character area by analyzing the frequency component of the image in a predetermined size area including the main character. When the ratio of the high frequency component in the main character area is lower than a predetermined value, it is determined that the main character has a large degree of blur, and even if the main character is detected in the frame image, the scene is not included in the appearance time of the main character. Or, if the blur of the main character reflected in the frame image is larger than a predetermined tolerance (the blur is severe), the video to be edited is not suitable for editing with the main character at the center. You may make it determine.

  In an imaging apparatus having a configuration in which focus detection pixels are arranged in a two-dimensional array of imaging pixels and a focus adjustment state of the imaging optical system, that is, a defocus amount is detected at a plurality of positions on the imaging screen, Since the defocus amount data at a plurality of positions on the shooting screen is recorded together with the image data of the frame image, it is determined that the main character is not in focus when the defocus amount of the main character area is a predetermined value or more, Even if the hero is detected, the scene can be excluded from the appearance time of the hero. Alternatively, when the defocus amount of the main character area is equal to or greater than a predetermined value, it may be determined that the video to be edited is not suitable for editing with the main character at the center.

  Furthermore, when a new video is generated from a plurality of videos obtained by capturing the same event by a plurality of imaging devices, even if videos of the main characters appearing on the plurality of imaging devices at the same time, these main characters are captured. You may not add up the appearance time. In addition, although the example which edits an image | video centering on the main character previously designated by the user here is shown, the thing designated beforehand by the user, for example, the favorite car designated by the user in the image which captured the car race, etc. The video can be edited at the center.

  In addition to the ratio of the hero's appearance time to the total playback time of the captured video to be edited, it was set based on whether or not a climax scene predetermined for each event exists in the edited video. It may be configured to determine whether or not the main character generates a new edited video. If there is no climax scene in the captured video to be edited, a warning is given by the display device 5 and the speaker 6 that the user will not be satisfied even if a new edited video by the set main character is created, and the processing is performed. finish. Also, even if there is a predetermined climax scene in the captured video to be edited, but the main character is not reflected in that climax scene, content that satisfies the user even if a new video that records the main character is created If not, a warning is given by the display device 5 and the speaker 6 and the process is terminated.

  Furthermore, the time of the captured image of a predetermined scene in the editing event, for example, in the case of competition, start, running, goal, etc., is longer than the playback time set for each scene. If it is short, a warning may be given by the display device 5 and the speaker 6 to end the process, if a new video that records the hero does not satisfy the user's content.

  In this way, prior to editing, a protagonist is set to determine whether or not to generate a new video, so that a professional knows in advance whether or not a good-looking video like a video using camerawork can be generated. Editing operations that are likely to be wasted can be stopped. If it is determined that it is not suitable for generating a new video, a hero designation screen may be displayed on the display device 5 and a red frame surrounding the hero may blink to warn. In addition, instead of the main character, it may be determined whether or not editing is centered on an object designated in advance, for example, a favorite sports car in a car race.

  If it is determined that the imaged video of the editing event is suitable for generating a new video centered on the main character, the process proceeds to step 9 where a plurality of scenarios (screenplays) stored in advance in the storage device 2 are stored. To select a scenario (screenplay) that matches the editing event. A scenario may be selected and downloaded from among a plurality of scenarios stored in advance on a server on the Internet that matches the editing event.

  The following is an example of a scenario that matches the editing event for “Self Competition”. Here, (1) Insert the title image, (2) Start the competition without changing the size of the cropping area from the scene characterizing the athletic meet, for example, the scene of the athletic meet flag against the blue sky After changing the cut to the starting point, a scene to zoom in to the starting point, (3) a scene showing a starter with a pistol and a kindergarten aligned on the starting line, and (4) all kindergarten children competing by switching the pistol to a depression scene Zooming to the start of the scene, (5) Zooming in to the main character as the competing kindergarten approaches the goal, (6) Scene following the main character moving to the place of the flag of the landing position, and sitting (7) The kindergarten It consists of a series of scenes that exit from the exit gate and (8) fade-in after the fade-out.

  For reference, an example of a scenario corresponding to a “birthday party” event is shown. In this scenario, (1) Insert the title image, (2) The main character is up (starting to sing Happy Birthday song), (3) The panning is carried out so that everyone can be seen around the main character until the end of the song. Zoom out (scenes you are happy with), (4) lit up the cake with the candles off, (5) pan out to show the main character and cake, zoom out, and (6) the main character erase the candle, It consists of a series of scenes of (7) lighted and (8) zoomed out so that everyone is applauding.

  In step 10, the edited video playback time (video broadcast time) is set, priorities are assigned to the scenes of the scenario according to the playback time, and the time of each scene is set. The playback time can be specified by the user to an arbitrary length by the operation unit 4, but here, an example in which three types of preset playback time, one minute, three minutes, and five minutes of video are generated simultaneously. Show. The one-minute video is a so-called digest version, in which a high priority is set for the most important scene in the event so that an overview of the event can be shown in a short playback time. The three-minute video is a short version in which high priority is set for important scenes in the event so that the entire event can be seen in an appropriate playback time. The longest video of 5 minutes is a long version, and shows the detailed contents of the event by setting a high priority to the scenes necessary for composing the scenario in the event. If these three types of versions of video are generated at the same time, it is possible to select and view versions of video according to the playback purpose. The time of each scene in the scenario is automatically set to an appropriate length according to the playback time, that is, the playback version and the priority order.

  An example in which the priority and time of each scene of the above-described scenario of competition is set will be shown. For example, in the short version with a playback time of 3 minutes, (1) title image insertion (~ 3 seconds, priority 4), (2) a scene characterizing an athletic meet, for example, an athletic meet flag against a blue sky flutters in the wind After changing the cut to the starting point of the competition without changing the size of the trimming area from the scene, the scene zooming in to the starting point (-10 seconds, priority 5), (3) Kindergarten children aligned with the starting line A scene showing a starter with a pistol (~ 15 seconds, priority 6), (4) A start scene (~ 19 seconds, priority 1) with all the kindergarten children competing by switching the pistol to a depression scene ), (5) Zoom in to the hero as the competing children approach the goal (~ 35 seconds, priority 2), (6) Follow the hero moves to the flag of the landing order and sits down (~ 40 seconds, priority 3), (7) Scene where the child leaves the exit gate (~ 50 seconds, priority 7), (8) After fade out, fade-in to the end screen (~ 5 seconds, priority) Order 8) is set.

  In step 11, a title image of a video generated from information such as an editing event, an imaging date / time, and a hero is created. In this embodiment, since the protagonist Nao-chan created a video of the first in a competition at a kindergarten athletic meet imaged on October 10, “Nao-chan athletic meet-Ketkko edition-2008 10 The title caption “Month 10th” is determined. Also, as a background scene of the title caption, a scene characteristic of an event in which a main subject including the main character is not captured, for example, a scene in which a national flag flutters in the wind against a blue sky in the case of an athletic meet is selected.

  In the subsequent step 12, the main editing target video is analyzed, and the playback time of each scene when no editing is performed is calculated. When the video data of the main editing target file is analyzed, it is divided into a series of scenes as shown in FIG. At time t0, four players line up on the start line as shown in FIG. 7, and at time t1, four players start with a pistol as a signal as shown in FIG. The main character, Nao-chan, is a player who runs the second course from the front. At time t2, four players are running the courses as shown in FIG. 10, but at time t3, one player is off the course as shown in FIG. 11, and again at time t4 as shown in FIG. I'm running back to the course.

  While a player is running, a person enters in front of the imaging device X of the two imaging devices at time t5, and the running player can no longer be seen in the image by the imaging device X as shown in FIG. As shown in FIG. 14, the video of another imaging device Y captures a running player. At time t6, as shown in FIG. 15, the main character Nao-chan finishes in the first place, and at time t7, the players move to the flags of the respective rankings as shown in FIGS. . At time t8, as shown in FIG. 18, the main character Nao-chan is waiting at the first flag. The playback time when no editing is performed for each of these scenes is calculated. Of the frame images shown in FIGS. 7 to 18, only FIG. 14 is a frame image taken by the imaging device Y (indicated by “Y” at the right end of FIG. 7), and all other frames are taken by the imaging device X. It is a frame image.

  Next, in step 13, the control unit 3a performs pseudo camera work, that is, a pseudo camera such as cut, pan, zoom in, and zoom out, based on the playback time of each scene when not edited and the playback time of a new video. Determine the workpiece. For example, in the frame image of the video obtained by capturing the athletic meet competition shown in FIG. 7 at a wide angle, as shown in FIG. 8, after the portion (a) is cut out by trimming and displayed in the background of the title image, the portion (c) is Cut out and show the scene where players enter from the entrance gate, that is, make a pseudo cut. After that, the trimming area is moved while being gradually reduced from the (c) part to the (d) part, and the starter with the player and the pistol aligned on the start line is closed up, that is, pseudo panning and zoom-in are performed. The video edited in this way can be viewed as if it had been subjected to camera work such as cutting, panning, zooming in at the time of imaging.

  In step 14, the video and audio of the editing target file are automatically edited based on the event scenario, the priority and time of each scene in the scenario, the main character, the title character, the pseudo camera work, and the like. Details of this automatic editing will be described later. In step 15, the edited new video file is stored in the storage device 2 and recorded on the recording medium 2 a in accordance with a user instruction from the operation unit 4.

《Video and audio automatic editing processing》
FIG. 19 shows a frame image in a video generated by editing according to an embodiment. An embodiment executed by the control unit 3a, the image analysis unit 3b, and the image processing unit 3c (see FIG. 1) of the computer 3 while comparing with a series of frame images of the video before editing shown in FIGS. The automatic video and audio editing process will be described. In this example, a new version of a short version of 3 minutes is generated by video editing, and editing is automatically performed according to the priority of each scene and its playback time specified in the short version scenario. For example, in the 1-minute digest version and the 5-minute long version, automatic editing is performed in accordance with the priority order of scenes defined in a scenario corresponding to those versions and the playback time thereof.

  Here, the following video editing process will be described on the assumption that a new image is generated by trimming the frame image shown in FIGS. 7 to 18, but the frame image trimming process shown in FIGS. 7 to 18 is convenient. It is an explanation, and in practice, all the frame images of the scene to be edited or a frame image of the necessary range in the scene to be edited are subjected to the same trimming process, and the series of frame images are trimmed. A new video is generated from the image group.

  First, create a title image for the edit event of the competition. FIG. 20 shows an example of the title image. As described above, the title characters “Nao-chan athletic meet-Kakekko edition-October 10, 2008” are determined based on information such as an editing event, an imaging date and time, and a hero. The language selected by the user in advance using the operation unit 4 is applied. When the video editing device 1 has a built-in GPS receiver, the language of the detected current country is applied. The date is generated based on the date data recorded in the imaging data of the main editing target file. For the background scene of the subtitles, select a scene that does not show the main subject including the main character and is unique to the editing event.For example, in the case of an athletic meet, a scene where the national flag flutters in the wind against the blue sky. Is done. In this embodiment, a part of the frame image shown in FIG. 8 (part (a) in FIG. 8) is trimmed and used as a background scene for subtitles. As shown in FIG. 19A, the scene of this title image is continued for a predetermined time, for example, 3 seconds.

  Next, the subtitles in the title image are gradually faded out, and as shown in FIG. 19B, the scene of the national flag of the blue sky background is continued for a predetermined time, for example, 3 seconds. In addition, when the 6-second title scene shown in FIGS. 19A and 19B is generated, if these images are not included in the original video for 6 seconds, another series shown in FIGS. You may acquire from the image of.

  From the title scene (see Fig. 19 (b)) where the national flag is fluttering in the wind against the blue sky, the scene where the competitors enter from the entrance gate and line up on the start line (see Fig. 19 (c)) In order to generate, the part (c) is cut out from the image shown in FIG. 8 without changing the size of the trimming area. That is, the scene is changed by performing a pseudo cut by trimming. Editing is performed so that this scene continues for a predetermined time, for example, 2 seconds. Thereafter, in order to generate a scene (see FIG. 19D) in which a starter having a pistol with the four players lined up on the start line enters the screen (see FIG. 19D), (c) in the image shown in FIG. The trimming area is gradually made smaller while changing the position of the trimming area from part (d) to part (d). In other words, pseudo-panning and zoom-in are performed to close up the players and starters on the start line.

  In addition, when the playback time of the edited result is sufficient for the playback time of each version, a scene that introduces an event from a scene of a national flag with a blue sky background, for example, a scene where the whole course of competition is on the screen After performing a pseudo-scene change and continuing for a predetermined time, editing may be performed so as to perform pseudo-panning and zoom-in on a scene where players and starters aligned on the start line enter the screen. By inserting a scene of a wide-angle image explaining the surrounding situation of the event starting from the beginning of the event video, the viewer can easily understand the event situation. In addition, the image in which the entire course of competing competition enters the screen is adopted as an image suitable for introducing an event of competing competition by analyzing the moving range of the main character in the captured video.

  Next, in order to generate a scene for starting the competition (see FIG. 19E), the part (e) is cut out from the image shown in FIG. Thereafter, in order to generate a scene in which four players are running (see FIG. 19F), the part (f) is cut out from the image shown in FIG. In other words, pseudo panning is performed as if the running players panned the imaging device so that they fit on the screen. At this time, pseudo zoom-in may be performed gradually so that the main character is projected in the approximate center of the screen.

  As described above, each event scenario is assigned a priority for each scene. For example, in a competition event, the start and goal scenes that are indispensable to the competition are given the first priority, the running scene is the second priority, and the scenes that are lined up with the 1st to 3rd flags are the third priority. The scene that received the medal is set as the fourth priority, and the scene moving from the goal to the flag is set as the fifth priority. Here, for example, the running scene of the second priority order is an important scene, but if the running time is long like 200 m running, a part of the video may be cut off. This prevents the viewer from getting bored because the same composition continues for a long time. However, scenes in which the hero overtakes other players are preferentially incorporated into new videos. Such a scene can be detected by a method of personal authentication and subject tracking.

  In order to incorporate a happening scene (see FIGS. 19 (g) and (h)) that returns to the course after one of the athletes has left the course in the middle of a race, the (g) part is taken from the image shown in FIG. The (h) portion is cut out from the image shown in FIG. 12, and pseudo panning, zooming out, and zooming in are performed. Such a happening (a player's course out or a fall in a competitive race) is recognized at the time of analyzing the main editing target video (analysis of the subject movement trajectory by the method of personal authentication and subject tracking), and (f) in FIG. A series of changes in the trimming area from the part to the part (h) in FIG. 12 through the part (g) in FIG. 11 are gradually enlarged, reduced or moved so as not to cause a sudden and unnatural change in the image. In the scene where the player is running, the trimming area is set so that the portion in front of the player (the direction in which the player runs) is projected larger than the portion in the rear of the player (the direction in which the player ran). .

  In this embodiment, the pixel number conversion (resolution conversion) is performed so that the number of pixels of the cut-out image is always constant regardless of the size of the trimming region. Therefore, by gradually reducing the size of the trimming area, it is possible to obtain the same effect as when zooming in on the newly generated video, and conversely, by gradually increasing the size of the trimming area, The same effect as when zooming out is obtained on a newly generated image.

  Next, since a person enters in front of the imaging device that is imaging in the middle of a running race, as shown in FIG. Such a video failure can be detected by analyzing the video to be edited that is performed before editing. For example, in the case where players including the main character tracking on the image are suddenly lost and appear again after a while, it can be recognized that some kind of video failure has occurred. When there is such a video failure, the video captured at the failure occurrence time is searched by an imaging device other than the imaging device that captured the video with the failure. At the time of occurrence of the video failure, a video obtained by imaging the competition shown in FIG. 14 at a wide angle is captured by another imaging device Y (shown at the left end of FIG. 13). This video has also been analyzed prior to editing, and it has already been detected that the main character Nao is running.

  However, since the imaging device X images the competition from the left side of the course, the imaging device Y images the competition from the right side of the course, so that the direction in which the player is running is reversed between the imaging devices X and Y. In two images captured by such different imaging devices X and Y, the difference in the moving direction of the same subject can be easily recognized by analyzing the images. If these images are simply connected to form a new video, the viewer will feel unnatural. Therefore, in this embodiment, when switching from the video of the imaging device X in which the video failure has occurred to another imaging device Y, a cut video of a completely different scene is inserted to reduce the viewer's uncomfortable feeling.

  In one embodiment, when switching from the image of the image pickup apparatus X shown in FIG. 13 to the image of the image pickup apparatus Y shown in FIG. 14, the part (i) in FIG. 13 is cut out and inserted by trimming. Thereby, competition between the image shown in FIG. 19H by the image pickup device X and the image shown in FIG. 19J by the image pickup device Y occurs in the image pickup device Y as shown in FIG. The image of the person who is showing and the person who is supporting the competition of the people around is inserted. Note that the insertion time of such a cut video is preferably a predetermined time (for example, 3 seconds). Even when the time when the image generated by the imaging device X has been in trouble exceeds the predetermined time, the viewer is particularly required to insert the video into an urgent scene as described above. It is desirable that the insertion time of the cut video is set to the predetermined time (for example, 3 seconds) in order not to interrupt the feeling of tension. Further, a cut video is searched in a predetermined order from among a plurality of cut videos corresponding to an event determined in advance in a scenario, and the cut video searched first is used. The cut image is not particularly limited to the image obtained by imaging the surrounding environment as described above, and an image obtained by imaging the main character from different viewpoints, for example, an image showing the player from the goal direction is used. Furthermore, the cut video to be inserted is not necessarily limited to the video captured at the time of the video failure occurrence, and a predetermined cut video is searched from all the videos to be edited.

  When switching from the video of the imaging device X to the video of the imaging device Y, the following may be performed instead of inserting a cut scene. That is, trimming is performed so that the main subject such as the main character is framed out from the right side of the screen in the image of the imaging device X, and then trimming is performed so that the main subject is framed in from the right side of the screen in the image of the imaging device Y. Switch to the image of the imaging device Y. Alternatively, the image of the imaging device Y is trimmed so that the main subject is framed out from the left of the screen, and then the image of the imaging device X is trimmed so that the main subject is framed in from the left of the screen. The video may be switched from the video of the imaging device Y to the video of the imaging device Y, or from the video of the imaging device Y to the video of the imaging device X.

  The image (see FIG. 19 (j)) by the imaging device Y switched due to the occurrence of the video failure is obtained from the image of the imaging device Y shown in FIG. 14 with the size of the trimming area of the image of the imaging device X before the video failure occurrence. It is desirable to generate by cutting out the (j) part having the same size. When returning from the video of the imaging device Y to the video of the imaging device X again, a cut video is inserted in order to reduce the viewer's discomfort due to the discontinuity of the video. That is, when returning from the image of the imaging device Y shown in FIG. 19 (j) to the image of the imaging device X shown in FIG. 19 (l), a cut image as shown in FIG. 19 (k) is inserted. This cut video may be generated by the same method as the cut video shown in FIG. Here, an example is shown in which an image obtained by capturing images of people who support a player from images of competition is adopted as a cut image.

  On the other hand, the editing apparatus 1 according to the embodiment also performs audio editing together with video editing. In audio editing, the directivity and size of the audio in the playback state are changed to be optimal according to the position and size of the video trimming area. For example, when the video zoomed in on the audience supporting the competition is a new video, the playback volume is increased. That is, the sound is edited so that the playback volume increases as the trimming area decreases. Further, when a region close to the right end of the captured image is set as a trimming region, the amplification degree of the right channel audio signal among the left and right channel audio signals is increased. That is, the left and right volume balance is changed, and the right volume is increased from the left. The edited audio may be written into the video file together with the video data as audio data, or the video as the control data of the audio output circuit of the playback device, that is, the amplification ratio of the amplifier and the mixing ratio (directivity) of the plurality of microphones. It may be written to a file.

  For the scene shown by the players shown in FIG. 19 (l), the (l) portion of the image obtained by the imaging apparatus X shown in FIG. 15 is cut out and trimmed. After that, in order to generate the scene shown in FIG. 19 (m) in which the main character who is wearing first place is close-up, the part (m) in FIG. 15 is cut out and trimmed. That is, pseudo panning and zoom-in are performed. In order to generate the close-up scenes 19 (n), (o), and (p) in which the main character has continued to move to the first flag, tracking processing of the main character by template matching is performed. (N) part, (o) part of FIG. 17, and (p) of FIG. 18 are cut out in this order for trimming. That is, pseudo panning and zoom-in are performed.

  If it takes a long time for the hero to move to the first flag after finishing the goal, a part of the video in the middle of the movement is cut and the previous and next videos are overlapped. The overlap is realized by gradually changing the mixing ratio of the image before and after the cut portion. Due to this overlap, skipping on the time axis is accepted by the viewer without a sense of incongruity.

  A scene in which the main character at the flag of the first place is shown in close-up FIG. 19 (p) is continued for a predetermined time, for example, 4 seconds, and then the main character shown in FIG. 19G) is continued for a predetermined time, for example, 5 seconds. Note that the scene shown in FIG. 21 is a still image or a moving image extracted from the video file of the athletic meet taken after the competition. Thereafter, fade-out is performed and the “END” scene FIG. 19 (r) is inserted, and the automatic editing process ends.

  In the embodiment described above, the viewer gets bored when the same composition continues in the video for a long time. This degree differs depending on the relationship between the viewer of the edited video and the main character. In other words, when the main character and viewer are in a close relationship, a scene with a long composition can be tolerated, but the longer the scene, the more tired the scene with a long composition. As is clear from news programs on television broadcasting, it is appropriate that one composition scene is about 15 to 30 seconds. Among the digest (1 minute), short (3 minutes), and long (5 minutes) playback time versions, the long version makes a composition scene longer to some extent on the assumption that viewers who are close to the main character watch it. At the same time, priorities such as goal scenes and medal scenes are increased. On the other hand, the digest version and the short version are intended to be viewed by viewers who are relatively distant from the main character, so that the length of one composition scene does not increase, and the goal scene and medal scene do not stand out. To do.

  When the playback time of the same scene is a predetermined time, for example, 20 seconds or more, the composition is changed by pseudo-camera work such as zooming in and zooming out so that the viewer does not get bored, for example, in a scene where a student race is running Let Alternatively, a part of the running image may be replaced with an image of the surrounding environment, such as a support scene, and the running scene may be shortened. For example, if there is a running scene for 20 seconds, insert a scene that the audience supports in the middle of the running scene for 3 seconds and cut the running scene with the start and goal scenes left for 10 seconds. Playback time can be reduced by 7 seconds. If you cut the middle of the running scene for 10 seconds and simply join the images before and after, the viewer will feel unnatural, but by inserting a cheering landscape in the middle, without feeling unnatural Playback time can be shortened. Conversely, when extending the playback time, it is effective to insert a video of the surrounding environment in the middle.

  In the above-described embodiment, an example is shown in which a plurality of imaging devices capture an athletic competition and use the image of the other imaging device when a video failure occurs in one imaging device. When the same subject is imaged at a wide angle by a plurality of imaging devices, an image with a short shooting distance to the subject in the trimming area of the two images may be used. For example, in the scene where the main character is pleased with the flag of the first place, the main character is larger when the shooting distance between the main character in the trimming area and the shooting distance to the flag is shorter. New images can be generated with high image quality. Alternatively, an image in which the size of the main subject in the trimming area is a predetermined value or more may be employed. As a result, the image quality deterioration due to the pixel number conversion described above can be reduced. In any case, it is a matter of course that an image in which the face of the main character faces the front is desirable.

  When a climax scene is defined for each event, the climax scene may be edited so as to be in slow motion. For example, in a competition event, the goal scene is one climax scene, and the goal scene detected by the recognition of the main character's motion is re-edited into a slow motion scene. When the climax scene is edited in slow motion, the playback time of the slow motion scene is set to fall within the determined playback time.

  In the above-described embodiment, when an image in the middle of a scene in which movement continues (a scene in which competition is running) is cut and the previous and subsequent images are connected, an image of the surrounding environment is inserted between them, or In the example shown, the front and back images are overlapped and connected. However, when inserting other images without overlapping, the size of the trimming area of the front and back images can be made different. Is desirable. If the trimming areas have the same size, the viewer will feel uncomfortable. Further, in this case, it is desirable to cut and connect the main subject so that the state and movement of the main subject, for example, the facial expression of the main character and the movement of the limbs are continuous. For example, when the left foot is in front in the previous video, the cut is made so that the right foot does not go out in the video after being connected.

  Note that voice or caption narration may be automatically inserted based on the image analysis result of the original video. For example, in the above-mentioned competition event, voice or caption narration such as “Going in with Nao first” is automatically inserted in the goal scene shown in FIG. When subtitles are inserted, the trimming area is determined in consideration of the subtitle insertion portion. When voice is inserted, voice such as a cheer recorded along with the video of the goal scene is automatically controlled so that the volume is reduced, so that voice narration can be heard easily.

  In the above-described embodiment, an example in which a scenario corresponding to an editing event is selected and automatic editing is performed according to the scenario has been described. However, a plurality of scenarios corresponding to each event may be stored. Or you may make it acquire the some scenario according to an edit event via the internet. In such a configuration, the application condition of the video to be edited is defined for each scenario, and whether or not the video to be edited satisfies the application condition by the control unit 3a and the image analysis unit 3b of the video editing apparatus 1. A scenario that satisfies the application condition is selected.

  For example, when the editing event is the competition described above, a scenario that satisfies the application condition of the video to be edited is selected from the following scenarios. In scenario A, (A1) The image of the athletic meet flag is shown for 10 seconds continuously, (A2) The main character is shown from 5 seconds before the start line alignment, (A3) The running image is 10 to 15 It is shown continuously for 2 seconds, (A4) the main character wins from 1st to 3rd, (A5) there is a video of moving to the flag after the goal, (A6) from the flag The application condition that there is a video up to the exit gate is set. In scenario B, (B1) The image of the athletic meet flag is shown for 10 seconds continuously, (B2) The main character is shown from 5 seconds before the start line alignment, (B3) The image of running is 10 Applicable conditions are set such that it is reflected for ˜15 seconds, and (B4) the hero has scored a goal in 4th or less.

  Furthermore, in scenario C, (C1) The image of the athletic meet flag is shown for 10 seconds continuously, (C2) The main character is shown 5 seconds before the start line alignment, (C3) The image of running is 15 (C4) The main character wins from 1st to 3rd, (C5) There is a video of moving to the flag after the goal, (C6) Flag The application condition is that the video from the exit to the exit gate is shown. And in scenario D, (D1) The image of the athletic meet flag is shown for 10 seconds, (D2) The main character is shown 5 seconds before the start line alignment, (D3) The running image is continuous for 20 seconds or more (D4) Applicable conditions are set that the hero has scored a goal in 4th or less.

  In the application conditions of scenarios A to D, the images running in scenarios A and B are 10 to 15 seconds, while the images running in scenarios C and D are continuous for 20 seconds or more. Has been. When viewing the same scene for a long time, the viewer gets bored. Therefore, in scenarios C and D, a surrounding environment image is inserted for 4 seconds, for example, in the middle of the image running by the above-described method. In Scenarios A and C, the main character wins 1st to 3rd, and in Scenarios B and D, the main character wins 4th or less. Especially in the case of scenarios A and C, “after the goal, the video until moving to the place of the flag is shown” and “the image from the place of the flag until leaving the exit gate” is also shown. As a condition. According to such an application condition, it is possible to accurately edit how the hero is pleased or disappointed depending on the result of the competition.

  The control unit 3a of the video editing device 1 compares the analysis result of the video to be edited by the image analysis unit 3b with the application conditions of the above-described scenarios A to D, and determines which application condition the video to be edited satisfies. To do. After searching for a scenario that satisfies the application condition of the editing target video, the editing target video is automatically edited according to the scenario. In this way, by preparing a plurality of scenarios for one event and selecting a scenario that is optimal for the editing target video from among them, it is possible to perform editing that is optimal for the editing target video.

  In the above-described modification, an example is shown in which a scenario according to the analysis content of the video such as the duration of the running video and the order of arrival of the goals is selected. It is different from the video expression. For example, a scenario in which application conditions such as “movie director XY style” and “documentary style” are set may be prepared and automatically or manually selected from them according to the video to be edited. Thereby, a user's favorite picture can be generated.

  In the above-described embodiments and their modifications, all combinations of the embodiments and the modifications are possible.

  According to the above-described embodiment and its modification, the content of the video is analyzed to identify the event reflected in the video, and the scenario corresponding to the event of the identification result is selected from a plurality of scenarios. A new video is generated from the video by performing a pseudo camera work process on the multiple frame images that make up the video according to the selected scenario. Therefore, it is possible to easily obtain a video of the same level as the edited video.

  In one embodiment and its modification, pseudo camera work processing is performed in which a part of a region is cut out from a frame image of a video captured at a wide angle to generate a new frame image. It is difficult for ordinary users to capture images that look good like those photographed by professional photographers, and even if you edit images captured using camera work, you can zoom in and zoom out from the expanded image It is not possible to acquire a video, acquire a video of the imaging range before panning after panning the imaging device, or acquire a video of the cut period, so it is not possible to generate a video intended by the user difficult. According to the embodiment and the modification thereof, it is possible to acquire all the events occurring in a wide range, and to prevent an important scene from being missed due to a camera work error at the time of imaging.

  According to one embodiment and its modification, pseudo camera work processing is performed in which a new frame image is generated by changing the size and position of a region cut out from each of consecutive frame images constituting a video. In other words, a part of the frame image composing a wide-angle image is cut out by trimming and edited as if zoomed in or out during shooting, or the trimming range was changed as if shooting. Edit as if the imaging device was panned, or delete as part of the same scene and edit as if a cut was made during imaging. By performing such editing, it is possible to realize a pseudo camera work as if it were captured using the camera work at the time of imaging, and obtain the same video effect as when the camera work was actually performed at the time of imaging. be able to.

  According to one embodiment and the modification thereof, the contents of the analyzed video and the application conditions for each scenario are collated from a plurality of scenarios having different application conditions for the same event, and a scenario in which the application conditions match is determined. Since the selection is made, the optimum editing can be performed on the editing target video.

  According to one embodiment and its modification, a scenario records a technique and a procedure for editing a person or thing specified in advance in a video, and a plurality of frame images constituting the video. While searching for a person or object specified in advance, the pseudo camera work processing is applied to the multiple frame images that make up the image according to the scenario. Can be generated.

  According to one embodiment and its modification, the scenario defines the priority order for each of the scenes included in the event video, the playback time for each scene, the switching of scenes, and the playback order for each scene. Therefore, it is possible to perform optimum editing for the editing target video.

  According to one embodiment and its modification, the contents of a plurality of videos whose imaging time is within a predetermined time or whose imaging location is within a predetermined distance are analyzed to identify an event shown in the plurality of videos, and Selecting a scenario corresponding to the event from a plurality of selected images, selecting a scenario corresponding to the event, and configuring a plurality of images that include the same event according to the selected scenario Since the pseudo-camera work process is applied to the frame image, the same level of video as the image captured and edited by the professional from the many event images captured by the general user from the many images captured from the same event. Easy to obtain.

  According to the embodiment and the modification thereof, since the sound volume and the left and right sound volume balance are changed according to the pseudo camera work process, an effective sound suitable for the video can be easily obtained.

  According to one embodiment and its modification, the image data of a person or an object designated in advance is collated with a frame image constituting the image, and the person or the object is searched from the image, and the search result is Based on the detected state of a person or object in the video, and based on the detected state, it is determined whether or not it is appropriate to edit the video around the person or object. Before editing, it is possible to know whether or not the video is suitable for generating a new video centered on a predetermined person or object, and it is possible to prevent unnecessary editing operations. .

  According to one embodiment and its modification, the time during which a person or object specified in the video is reflected in the video is detected, and the ratio of the time during which the human or object is reflected in the total playback time of the video. Is less than a predetermined ratio, it is determined that the video is not suitable for editing with a pre-specified person or object, so that the video is specified in advance before editing the video. Therefore, it is possible to know whether or not it is suitable for generating a new video centered on a person or an object, and it is possible to prevent an unnecessary editing operation.

  According to one embodiment and its modification, if the size of the frame image of the person or object specified in advance is less than a predetermined size, the video to be edited is the person or object specified in advance. Since it is determined that the image is not suitable for editing at the center, it is possible to prevent the generation of a new image in which the main character is small.

  According to one embodiment and its modification, if the blurring degree of a predesignated person or object in the frame image is larger than the allowable value, the video to be edited is centered on the predesignated person or object. Thus, since it is determined that it is not suitable for editing, it is possible to prevent the generation of a new video in which the focused main character is reflected.

  According to one embodiment and its modification, the video content is analyzed to identify an event reflected in the video, and a scenario corresponding to the event of the identification result is selected from a plurality of scenarios, and the video is selected. If the person or object specified in advance is not reflected in the specific scene specified in the scenario, the video to be edited is not suitable for editing around the person or object specified in advance. Therefore, it is possible to prevent the generation of a new video in which the main character is not shown in a specific scene such as a climax scene.

  According to one embodiment and its modification, when the playback time for each scene in the video is less than the playback time for each scene specified in the scenario, the video to be edited is a person designated in advance or Since it is determined that the object is not suitable for editing, it is possible to prevent the generation of a new video in which the main character is not reflected in each scene of the event.

1; video editing device, 2; storage device, 3; computer, 4; operation unit, 5; display device, 6;

Claims (7)

A person search procedure for collating image data of a person or object designated in advance with a frame image constituting the image and searching for the person or object from the image,
A state detection procedure for detecting a state in which the person or object is reflected in the video based on a search result of the person search procedure;
An image editing program for causing a computer to execute an editing suitability determination procedure for determining whether or not to edit the image centered on the person or an object based on the state detected by the state detection procedure. The video editing program according to claim 1,
In the state detection procedure, the time during which the person or object is reflected in the video is detected as the state,
In the editing suitability determination procedure, if the ratio of the time to the total playback time of the video is less than a predetermined ratio, the video is determined not to be suitable for editing centering on the person or thing. A video editing program featuring: In the video editing program according to claim 1 or 2,
In the state detection procedure, the size of the person or the object searched in the person search procedure in the frame image is detected as the state,
In the editing suitability determination procedure, if the size of the person or object in the frame image is less than a predetermined size, it is determined that the video is not suitable for editing centering on the person or object. A video editing program characterized by that. In the video editing program according to any one of claims 1 to 3,
In the state detection procedure, the degree of blur of the person or object in the frame image searched in the person search procedure is detected as the state,
In the editing suitability determination procedure, when the blurring degree of the person or object in the frame image is larger than an allowable value, it is determined that the video is not suitable for editing around the person or object. A video editing program featuring: In the video editing program according to any one of claims 1 to 4,
An event identification procedure for analyzing the content of the video and identifying the event shown in the video;
Causing the computer to further execute a scenario selection procedure for selecting a scenario corresponding to the event identified by the event identification procedure from a plurality of scenarios,
In the state detection procedure, it is detected as the state whether or not the person or object is reflected in a specific scene defined in the scenario in the video,
In the editing suitability determination procedure, when the state detection procedure does not detect the state in which the person or object is reflected in the specific scene, the video is suitable for editing centering on the person or object. A video editing program characterized by determining that it is not. The video editing program according to claim 5, wherein
In the state detection procedure, the playback time for each scene defined in the scenario in the video is detected as the state,
In the editing suitability determination procedure, if the playback time for each scene in the video is less than the playback time for each scene specified in the scenario in the state detection procedure, the video is centered on the person or thing. A video editing program characterized in that the video editing program is determined not to be suitable for editing. A storage device for storing images;
A computer that executes the video editing program according to any one of claims 1 to 6;
A video editing apparatus comprising a display device for displaying video.

Images