JP2019213160A - Video editing apparatus, video editing method, and video editing program - Google Patents

Abstract

To provide a video editing apparatus that enables editing that reflects intentions of a video creator while reducing burden of video editing, a video editing method, and a video editing program.SOLUTION: A video editing apparatus 1 includes: a video recognition unit 22 for calculating a video recognition result with a time section from video data included in moving image data; a voice recognition unit 23 for calculating a voice recognition result with a time section from voice data included in the moving image data; a recognition result combining unit 24 for calculating a composite recognition result obtained by combining the video recognition result with time section and the voice recognition result with time section at each playback time; and an edit information allocating unit 25 for determining edit information to be applied on the basis of the composite recognition result.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a moving image editing apparatus, a moving image editing method, and a moving image editing program.

  2. Description of the Related Art Conventionally, when creating a moving image, the moving image editing such as unnecessary part cut, telop addition, effect addition, moving image content addition, audio content addition, etc. may be performed after shooting the moving image as the editing material. In order to perform such video editing, it is necessary to use a high-performance computer and dedicated software for engineers, and advanced techniques and expertise are required to operate these computers. In recent years, software that allows easy editing of moving images has been developed due to the increased functionality of personal computers and the spread of video cameras, etc., and it has become possible to easily edit moving images even for general users without specialized technology or expertise. Yes.

  For example, Patent Document 1 discloses a video editing support apparatus that generates editing information by learning an edited sample moving image and performs editing support for a user based on the generated editing information. By using such an apparatus, a general user can easily perform moving image editing simulating unnecessary partial cut, telop addition, effect addition, moving image content addition, audio content addition, and the like.

JP2013-080989A

  However, there is a technology that automatically detects scenes close to the sample video and automatically performs the same editing as the sample video, but a sufficient amount including close scenes is necessary to perform editing that reflects the intention of the video creator. Sample videos are required. In addition, a technique for automatically cutting a silent part and a technique for automatically inserting a telop are also known, but editing that reflects the intention of the movie creator cannot be performed.

  Therefore, in view of the above-described problems, the present invention provides a moving image editing apparatus, a moving image editing method, and a moving image editing program capable of reducing the burden of moving image editing while enabling editing reflecting the intention of the moving image creator. For the purpose.

  The problem to be solved by the present invention is as described above. Next, means for solving the problem will be described.

  That is, in the present invention, a moving image recognition result storage unit that stores a moving image recognition result with a time interval calculated from moving image data included in the moving image data, and a time calculated from the audio data included in the moving image data. A speech recognition result storage unit that stores a speech recognition result with a section; a recognition result combining unit that calculates a combined recognition result obtained by combining the moving image recognition result with the time section and the speech recognition result with the time section at each reproduction time; An editing information allocating unit that determines editing information with a time interval to be applied based on the composite recognition result.

  Further, in the recognition result combining unit, the moving image recognition result with time interval and the speech recognition result with time interval are combined by collating with a predetermined combination pattern, and the predetermined combination pattern can be registered. is there.

  Further, in the editing information allocating unit, the editing information with time section to be applied is determined by comparing the composite recognition result and a predetermined combination pattern of the composite recognition result and the editing information with time section, The predetermined combination pattern can be registered.

  The editing information with a time section determined by the editing information allocating unit is output as an editable data group.

  As effects of the present invention, the following effects can be obtained.

In the present invention, a complex event that can be recognized from moving image data can be linked to a specific editing content. For example, a telop generated from the utterance content of a video performer by voice recognition is recognized by moving image recognition. Add to the position corresponding to the gesture of the recognized video performer, or add the moving image content according to the keyword recognized by voice recognition around the face position of the video performer detected by moving image recognition , Adding audio content according to the gesture of the video performer recognized by moving image recognition at a timing that does not overlap with the utterance interval detected by voice recognition, or moving image recognition among non-speech intervals detected by voice recognition Cut only before the time when the specific object recognized by
Can be automatically edited.
In addition, the video creator can specify the intended editing content for any video playback time during video recording using a complex event linked to specific editing content, While enabling editing that reflects the intention of the video creator, it is possible to reduce the burden on video editing.

1 is a block diagram showing a moving image editing apparatus according to a first embodiment of the present invention. The flowchart figure which shows the recognition result acquisition method among the moving image editing methods which concern on 1st embodiment of this invention. The flowchart figure which shows the edit information acquisition method among the moving image edit methods which concern on 1st embodiment of this invention. The flowchart figure which shows the moving image automatic editing method among the moving image editing methods which concern on 1st embodiment of this invention. The flowchart figure which shows animation automatic cut edit among the animation editing methods concerning 1st embodiment of this invention.

  Next, embodiments of the invention will be described.

<First embodiment>
A moving image editing apparatus 1 according to an embodiment of the present invention will be described with reference to FIG.
The moving image editing apparatus 1 is an apparatus for editing captured moving image data. The moving image editing apparatus 1 includes a communication unit 11 that communicates with an external moving image capturing device, a storage unit 12 that stores captured moving image data, edited moving image data, and editing information, and a control unit 13 that edits the captured moving image data. And.
The communication unit 11 is a part that communicates with the external video shooting device 2 through a wired or wireless communication line.
The storage unit 12 is a part that stores captured moving image data, edited moving image data, and editing information, and includes, for example, a RAM or a ROM.
The control unit 13 is a part that edits the captured moving image data, and is composed of, for example, an arithmetic device.
The control unit 13 also includes a moving image input unit 21 that inputs moving image data, and a moving image as a moving image recognition result storage unit that stores a moving image recognition result with a time interval calculated from moving image data included in the moving image data. A recognition unit 22, a speech recognition unit 23 as a speech recognition result storage unit for storing a speech recognition result with time interval calculated from speech data included in the moving image data, a moving image recognition result with time interval, and a sound with time interval. A recognition result combining unit 24 that calculates a combined recognition result obtained by combining the recognition results at each reproduction time, and an editing information allocating unit 25 that determines editing information with a time section to be applied based on the combined recognition result. The moving image input unit 21, the moving image recognition unit 22, the voice recognition unit 23, the recognition result combination unit 24, and the editing information allocation unit 25 are realized by, for example, a general information processing apparatus in which general web browser software is installed. It is.

  The moving image input unit 21 is a portion to which moving image data is input from the external moving image capturing apparatus 2 via the communication unit 11. The moving image data is a data group including at least moving image data, audio data, and time data. In addition to these data, the moving image data may further include subtitle data, multiplexed audio data, subtitle data / chapter data / metadata (tag), and the like.

  The moving image recognition unit 22 is a part that calculates a moving image recognition result with a time section from moving image data and time data included in the moving image data. The moving image recognition result with time interval is, for example, a result of recognizing a moving image by decomposing moving image data into frame images and performing recognition for each frame. Note that the moving image recognition method is not limited to frame decomposition for each frame, and for example, a moving image recognition method based on multi-frame recognition can be adopted. Note that the present invention is not limited to this embodiment, and for moving image recognition, services such as an external tool or web API can be used. That is, the moving image editing apparatus 1 only needs to include at least a portion for storing the calculated moving image recognition result with a time section.

The moving image recognition unit 22 can also use object recognition and motion recognition.
Object recognition is a recognition method in which frame image data is used as input information, and at least output information includes a region where a pre-registered object appears and the type of object. There may be a plurality of areas where the objects appear. Further, the output information may include the recognition reliability.

  Further, a specific movement of the object may be targeted by adopting multi-frame recognition. For example, when a specific movement of a person is targeted, the output includes at least an area in which the person appears, a specific gesture type of the person, and a time interval in which the person is recognized.

  The moving image recognition unit 22 includes a frame image extraction unit 31, a second frame number extraction unit 32, and a reproduction time conversion unit 33. The frame image extraction unit 31 is a part that extracts a frame image with a frame index from moving image data. The second frame number extraction unit 32 is a part that extracts the second frame number from the moving image data. The reproduction time conversion unit 33 is a part that calculates the reproduction time from the frame image with frame index extracted by the frame image extraction unit 31 and the number of frames per second extracted by the frame number extraction unit 32 per second.

  As described above, the moving image recognition with the time interval starting from the reproduction time is obtained from the indexed frame image and the reproduction time acquired by the frame image extraction unit 31, the second frame number extraction unit 32, and the reproduction time conversion unit 33. Get the result.

  As an embodiment of moving image recognition, for example, gesture recognition that detects a specific gesture using a moving image recognition result with a time interval, position recognition that detects a position or movement (difference) of a specific object, and a video performer It includes facial expression recognition that recognizes facial expressions.

  The voice recognition unit 23 is a part that calculates a voice recognition result with a time section from voice data and time data included in the moving image data. The speech recognition result with time section indicates, for example, the result of recognizing the text of the utterance content, the specific keyword, the non-speech section, and the like when the speech recognition is used. Note that the speech recognition method is not limited to spoken speech recognition, but includes sound waveform recognition that recognizes specific speech other than silence intervals and speech, acoustic signal recognition such as music, and word string recognition by multipath search. They can also be used in combination. Note that the present invention is not limited to this embodiment, and for voice recognition, an external tool or a service such as a web API can be used. That is, the moving image editing apparatus 1 only needs to include at least a portion for storing the calculated speech recognition result with time interval.

  The voice recognition unit 23 calculates a voice recognition result with a time section using speech voice recognition for a section starting from the reproduction time from the voice data and time data included in the moving image data. The voice recognition unit 23 may further include a voice volume, a reliability of recognition, a speaker identification result, and the like from the voice data and time data included in the moving image data.

  The recognition result combining unit 24 calculates a composite recognition result from the moving image recognition result with time interval and the speech recognition result with time interval. The composite recognition result is composite data of a moving image recognition result with time interval and a speech recognition result with time interval grouped by reproduction time. Note that it may be data obtained by adding a reproduction time to metadata (tag), which is other data, and further combining it.

In the recognition result combining unit 24, the moving image recognition result with time interval and the speech recognition result with time interval are combined by collating with a predetermined combination pattern. The elements constituting the combination pattern are an arbitrary element (for example, an object) recognized by a moving image recognition method that can be used by the moving image editing apparatus 1 and a voice recognition method that can be used by the moving image editing apparatus 1. It is an arbitrary element (for example, keyword) recognized by A video creator who is a user of the video editing device 1 can arbitrarily select an arbitrary element recognized by the moving image recognition method and an arbitrary element recognized by the voice recognition method, and can designate a combination pattern thereof.
In the moving image recognition method that can be used in the moving image editing apparatus 1, for example, when using the face of a specific person, the moving image creator uses the moving image recognition unit to learn a learned model that recognizes the face of the person in advance. It is registered so that it can be used, and the learned model is used for recognizing an arbitrary element by the moving image recognition unit.
Further, in the voice recognition method that can be used in the video editing device 1, for example, when using a specific keyword, the video creator registers the keyword in advance so that it can be used from the voice recognition unit. The keyword is used for recognizing an arbitrary element by the voice recognition unit.
The recognition result combining unit 24 includes a combination pattern input unit 35. The input unit 35 is an input means for the movie creator to specify a combination pattern, and the movie creator can specify a combination pattern from the input unit 35.

  The editing information allocation unit 25 is a part that allocates editing information to the composite recognition result and acquires editing information with a time section. The editing method combination pattern that links the relationship between the combined recognition result and the editing information with time section is registered in the storage unit 12 in advance. For example, editing method combination patterns as shown in Table 1 are registered in the storage unit 12.

The editing method combination patterns shown in Table 1 can be specified by the video creator. The elements constituting the editing method combination pattern are an arbitrary combination pattern specified by the recognition result combining unit 24, and editing information with time interval and moving image data output by the moving image editing apparatus 1 and actually editing the moving image. This is an arbitrary editing method that can be used by another moving image editing apparatus to be performed or the moving image editing apparatus itself. The video creator can select an arbitrary combination pattern and an arbitrary editing method specified by the recognition result combining unit 24, and can specify the editing method combination pattern.
The editing information allocation unit 25 includes an editing method combination pattern input unit 36. The input unit 36 is an input means for the moving image creator to specify the editing method combination pattern, and the moving image creator can specify the editing method combination pattern from the input unit 36.

  The editing information with time section is output as an irreversible container file. Note that the editing information with time section is not limited to being output as a container file, and can be output as, for example, an editable data group.

  Moreover, the moving image editing apparatus 1 may further include a moving image automatic editing unit 26 that performs automatic moving image editing according to editing information with a time section. In this case, the automatic video editing unit 26 further includes an image effect addition editing unit 51, a telop addition editing unit 52, a moving image content addition editing unit 53, an audio effect addition editing unit 54, and an audio content addition editing unit 55. And comprising.

  First, the playback time conversion unit 33 is used to acquire a frame index from the playback time and the number of frames per second given to the editing information with time section, and select an indexed frame image to which the frame index is given.

  The image effect addition editing unit 51 determines an image effect based on the editing information with time section and the playback time, and adds the image effect to the indexed frame image. The image effect is a method for digitally processing a screen, and includes, for example, image processing such as black and white gradation, particle processing, and the like.

  Further, the telop addition editing unit 52 determines a telop composed of characters or symbols based on the editing information with time section and the reproduction time, and adds it to the frame image with index. A telop is a layer describing characters or symbols to be superimposed on the screen.

  The moving image content addition editing unit 53 determines moving image content based on the editing information with time section and the reproduction time, and adds the determined moving image content to the indexed frame image. The moving image content is another moving image that is further added to the frame image.

  Further, the audio effect addition editing unit 54 determines an audio effect from the editing information with time interval and adds it to the audio data. The sound effect is a method for digitally processing sound, and includes high-pass processing and echo processing.

  The audio content addition editing unit 55 determines audio content from the editing information with time interval and adds it to the audio data. The audio content is audio data such as sound effects and music added to the audio data.

  The moving image editing apparatus 1 can also calculate a cut section from the editing information with a time section. In this case, the control unit 13 of the moving image editing apparatus 1 further includes a cut section calculation unit 27.

When the cut section calculating unit 27 is provided, the edit information with time section is configured to include cut edit information.
When the cut edit information is included, the cut section calculation unit 27 calculates a cut section from the edit information with time section.

  The cut section calculation unit 27 includes a moving image cut editing unit 61 and an audio cut editing unit 62.

  First, the playback time conversion unit 33 is used to acquire a frame index from the playback time and the number of frames per second given to the editing information with time section, and select an indexed frame image to which the frame index is given.

  The moving image cut editing unit 61 deletes the indexed frame image included in the cut section. In addition, the voice cut editing unit 62 deletes the voice data of the cut section from the voice data.

  The moving image combining unit 63 generates edited moving image data. The moving image combining unit 63 generates edited moving image data from the edited indexed frame image and the edited audio data.

Next, a moving image editing method according to an embodiment of the present invention will be described with reference to FIGS.
First, of the moving image editing methods, a method for obtaining a recognition result will be described with reference to FIG.

  First, moving image data is read using the moving image input unit 21 (step S10). The moving image data read in step S10 is transmitted to the control unit 13.

  Next, a first step of calculating a time-partitioned moving image recognition result from moving image data included in the moving image data will be described.

  In the first step, first, the frame image extraction unit 31 is used to obtain a frame image with a frame index from the moving image data (step S20).

  Next, the number of frames per second is obtained from the moving image data using the second frame number extraction unit 32 (step S30).

  Next, the playback time is acquired from the frame image with the frame index acquired by the frame image extraction unit 31 and the number of frames per second acquired by the frame number extraction unit 32 using the playback time conversion unit 33 ( Step S40).

  Next, from the reproduction time, indexed frame image, and reproduction time acquired by the frame image extraction unit 31, the second frame number extraction unit 32, and the reproduction time conversion unit 33, the interval starting from the reproduction time is obtained. Then, the moving image recognition result with time interval is acquired (step S50).

  In addition, a second process of calculating a voice recognition result with a time section from audio data included in the moving image data will be described. The first process and the second process are processed in parallel.

  In the second step, first, voice data is acquired from the moving image data using the voice recognition unit 23 (step S60).

  Next, using the voice recognition unit 23, a voice recognition result with a time section is acquired for the section starting from the reproduction time from the voice data and the time data included in the moving image data (step S70).

Next, a third step of calculating a composite recognition result obtained by combining the moving image recognition result with time interval and the speech recognition result with time interval at each reproduction time will be described with reference to FIG.
Using the recognition result combining unit 24, a composite recognition result is calculated from the moving image recognition result with the time interval to which the time interval including the reproduction time is given and the speech recognition result with the time interval (step S110).

Next, a fourth process for determining editing information to be applied based on the composite recognition result will be described.
Using the edit information allocation unit 25, the edit information with time section is acquired from the composite recognition result (step S120). The relationship between the combined recognition result and the editing information is recorded in the storage unit 12 as a table shown in Table 1. When the combined recognition result is input, the editing information assigning unit 25 generates editing information with a time section based on the table. The

Next, a fifth process for automatically editing a moving image according to editing information with a time section will be described with reference to FIG.
In moving image automatic editing, editing of moving image data and editing of audio data are performed in parallel.
In editing moving image data, the playback time conversion unit 33 is used to acquire a frame index from the playback time and the number of frames per second given to the editing information with time interval, and to obtain the frame image with index to which the frame index has been assigned. select. (Step S150)

Next, the image effect addition editing unit 51 is used to determine an image effect from the editing information with time interval and the reproduction time, and add the frame effect to the indexed frame image (step S160).
Next, using the telop addition editing unit 52, a telop image composed of characters or symbols is generated based on the editing information with time section and the reproduction time, and is added to the frame image with index (step S170).
Next, using the moving image content addition editing unit 53, the moving image content is determined based on the editing information with time section and the reproduction time, and added to the frame image with index (step S180).

In editing audio data, first, the audio effect addition editing unit 54 is used to determine the audio effect from the editing information with time interval and add it to the audio data (step S220).
Next, the audio content adding / editing unit 55 is used to determine audio content from the editing information with time interval and add it to the audio data (step S230).

  With this configuration, an image effect, a telop image, a moving image content, an audio effect for audio data, and an audio content can be added to the moving image.

Further, a sixth process for performing cut editing will be described with reference to FIG.
First, it is determined whether or not cut editing information is included in the editing information with time section (step S250). If the edit information with time section is not included in the edit information, it is not necessary to perform the cut edit, so the cut edit is terminated.
If it is determined in step S250 that the cut edit information is included, the cut section is calculated from the edit information with time section using the cut section calculation unit (step S260).

  In cut editing, moving image cut editing and audio data cut editing are performed in parallel.

  First, at the playback time included in the cut section, the playback time conversion unit 33 is used to acquire the frame index from the playback time and the number of frames per second added to the editing information with the time section, and with the index to which the frame index is added. A frame image is selected (step S270).

  Next, the selected frame image with index is deleted using the moving image cut editing unit 61 (step S280). Using the voice cut editing unit 62, the voice data of the cut section is deleted from the voice data (step S290).

  With this configuration, cut editing for cutting unnecessary portions from moving image data is performed.

  After the first to sixth steps are performed, edited moving image data is generated using the moving image combining unit 63 (step S310). Next, the edited moving image data is output (step S320).

Next, a specific example of a moving image editing method will be described.
In the first step, a specific gesture (hand sign) recognized by object recognition is included as a specific example of the moving image recognition result with time interval calculated from the moving image data included in the moving image data. A learned model for recognizing a specific gesture is registered in advance. Object recognition is executed using frame image data extracted from moving image data as input information, and an area in the frame image data where a gesture appears is detected as a recognition result. These recognition results are associated with time intervals.
In the second step, as a specific example of the speech recognition result with time interval calculated from the audio data included in the video data, text information based on the utterance content of the video performer, whether or not a specific keyword is uttered Information, and information related to a non-voice section. Keywords included in the utterance content sentence output by utterance voice recognition are registered in advance. Voice recognition is executed using voice data extracted from moving image data as input information, and for example, it is detected that a specific keyword is uttered, and the result is recognized. These recognition results are linked to the time interval.

  In the recognition result combining unit 24, a composite recognition result is acquired from the moving image recognition result with time interval and the speech recognition result with time interval associated with the time interval. The composite recognition result includes a composite of the recognition information. For example, when the utterance time interval acquired as a result of speech recognition and the time interval acquired as a result of object recognition are combined temporally and a common interval is included, all of the corresponding utterance speech recognition or object recognition Alternatively, a time interval including at least one of them is calculated and used as a combined recognition result associated with the time interval.

  The editing information allocating unit 25 acquires editing information with a time section from the combined recognition result acquired by the recognition result combining unit 24 based on the editing method combination pattern shown in Table 1. For example, the earliest or latest time of the time interval is calculated as the edit point. Instead of registering edit points, it is also possible to associate edit contents such as screen effects and sound effects as time-section edit information.

  An editing method combination pattern can be designated for the editing information allocation unit 25 using the input unit 36. The editing method combination pattern is a combination pattern that associates the combination pattern specified in the recognition result combining unit 24 with any available editing method, and the editing method is registered in advance by the video creator. The editing method can be expressed by, for example, pre-registered gestures and keywords, a combination pattern that combines them in time, and edit information with a time section corresponding to the combination pattern. In addition, these editing methods may be selected from preset procedures, or a plurality of preset procedures may be collectively selected.

As another specific example of the moving image editing method, a method of transcribing the utterance content combined with speaker identification will be described.
The movie creator registers in advance a learned model for recognizing the face of each performer recognized by object recognition. A learned model for recognizing these components is registered in advance for the facial components recognized by object recognition. The facial component is, for example, the mouth.

  The voice recognition unit 23 outputs a voice recognition result with a time section from voice data and time data included in the moving image data. The moving image recognition unit 22 performs object recognition using the frame image data extracted from the moving image data as input information, detects frame image data in which a face for each performer appears, and acquires a face area at each time. .

  In addition, object recognition is executed using frame image data extracted from moving image data as input information, frame image data in which a mouth that is a face component appears is detected, and a mouth region at each time is acquired.

  Also, motion recognition is performed using frame image data extracted from moving image data as input information, and motion vectors at each time and each region are acquired.

  Combine the results of object recognition and motion recognition in time and area. That is, when each performer's face area and mouth area have a common part, this mouth area is associated with the performer, and if there is movement based on the motion vector in the mouth area, It is determined that an utterance is being performed, and an utterance time interval by moving image recognition is acquired.

  When the utterance time section by utterance voice recognition and the utterance time section by moving picture recognition are combined in time, and there is a common section, the utterer by moving picture recognition is combined with the utterance content sentence by utterance voice recognition and edited with time section Calculate as information.

  The editing method includes utterance content sentences, faces for each performer registered in advance as motion vectors, face components, combination patterns that combine them temporally and regionally, and editing information with corresponding time intervals, The editing method is registered in advance by the video creator.

In addition, a method for performing cut editing will be described as another specific example of the moving image editing method.
The movie creator registers in advance a learned model for recognizing the performer's face recognized by object recognition.

  The voice recognition unit 23 calculates a voice recognition result with a time section from voice data and time data included in the moving image data. The moving image recognition unit 22 performs object recognition using the frame image data extracted from the moving image data as input information, detects the frame image data in which the performer's face appears, and acquires the time interval based on the frame. To do.

  The time-recognized speech recognition result acquired as a result of utterance speech recognition and the time interval acquired as a result of object recognition are temporally combined, and a time interval that is not included as a segment that is deleted by cut editing Calculated as editing information with time interval.

  The editing method can be expressed by an utterance content sentence and a pre-registered performer's face, a combination pattern that combines them in time, and corresponding editing information with a time interval. The method is registered in advance by the video creator.

  Edited moving image data is generated by combining moving image data and audio data that have been edited by means already described using the editing information with time interval. The edited moving image data is output as an irreversible container file. Note that the editing information with time section is not limited to being output as a container file, and can be output as, for example, an editable data group. When the data is generated as an editable data group, the editor can perform further manual editing using general moving image editing software.

As described above, the moving image editing apparatus 1 includes the moving image recognition unit 22 as a moving image recognition result storage unit that stores a moving image recognition result with a time interval calculated from moving image data included in the moving image data, and the moving image data. A voice recognition unit 23 serving as a voice recognition result storage unit for storing a voice recognition result with a time interval calculated from voice data included in the voice data, a moving image recognition result with a time interval, and the voice recognition result with a time interval at each reproduction time. A recognition result combining unit 24 that calculates a combined recognition result combined in FIG. 5 and an editing information allocating unit 25 that determines editing information with a time interval to be applied based on the combined recognition result.
By configuring in this way, it is possible to link a complex event that can be recognized from moving image data and specific editing contents in moving image editing. For example, a telop generated from the utterance content of a video performer by voice recognition can be added to a position corresponding to the gesture of the video performer recognized by moving image recognition. In addition, moving image content corresponding to the keyword recognized by voice recognition can be added around the position of the face of the video performer detected by moving image recognition.
Moreover, the audio content according to the gesture of the video performer recognized by the moving image recognition can be added at a timing that does not overlap with the utterance section detected by the voice recognition. In addition, it is possible to cut only the part before the time at which a specific object recognized by moving image recognition starts to move in the silent section detected by voice recognition.

  By configuring in this way, the video creator can specify the intended editing content for any video playback time during video recording using a complex event linked to the specific editing content. As a result, it is possible to reduce the burden of moving image editing while enabling editing that reflects the intention of the moving image creator.

  The above-described embodiments are merely representative examples, and can be modified and implemented without departing from the scope of one embodiment. It goes without saying that the present invention can be embodied in various forms, and the scope of the present invention is indicated by the description of the claims, and the equivalent meanings described in the claims, and all the scopes within the scope of the claims. Includes changes.

DESCRIPTION OF SYMBOLS 1 Movie editing device 2 Movie shooting device 11 Communication unit 12 Storage unit 13 Control unit 21 Movie input unit 22 Moving image recognition unit 23 Speech recognition unit 24 Recognition result combination unit 25 Edit information allocation unit 26 Movie automatic editing unit 27 Cut section calculation unit 31 frame image extraction unit 32 second frame number extraction unit 33 playback time conversion unit

Claims (12)

A moving image recognition result storage unit for storing a moving image recognition result with a time interval calculated from moving image data included in the moving image data;
A voice recognition result storage unit that stores a voice recognition result with a time interval calculated from voice data included in the moving image data;
A recognition result combining unit that calculates a combined recognition result obtained by combining the moving image recognition result with time interval and the speech recognition result with time interval at each reproduction time;
An editing information allocating unit for determining editing information with a time interval to be applied based on the composite recognition result;
A video editing apparatus comprising: In the recognition result combining unit, the moving image recognition result with time interval and the speech recognition result with time interval are combined by collating with a predetermined combination pattern, and the predetermined combination pattern can be registered. The moving image editing apparatus according to claim 1. In the editing information allocating unit, the editing information with time section to be applied is determined by comparing the composite recognition result with a predetermined combination pattern of the composite recognition result and the editing information with time section. The moving image editing apparatus according to claim 1, wherein the combination pattern can be registered. 4. The moving image editing apparatus according to claim 1, wherein the editing information with a time section determined by the editing information allocation unit is output as an editable data group. 5. A first step of storing a moving image recognition result with a time interval calculated from moving image data included in the moving image data;
A second step of storing a voice recognition result with a time interval calculated from voice data included in the video data;
A third step of calculating a combined recognition result obtained by combining the moving image recognition result with time interval and the speech recognition result with time interval at each reproduction time;
A moving image editing method comprising: a fourth step of determining editing information with a time interval to be applied based on the composite recognition result. In the second step, the moving image recognition result with time interval and the speech recognition result with time interval are combined by collating with a predetermined combination pattern, and the predetermined combination pattern can be registered. The moving image editing method according to claim 5. In the third step, the editing information with time section to be applied is determined by comparing the composite recognition result with a predetermined combination pattern of the composite recognition result and the editing information with time section, The moving image editing method according to claim 5 or 6, wherein the combination pattern can be registered. 8. The moving image editing method according to claim 5, wherein the editing information with a time section determined by the editing information allocating unit is output as an editable data group. 9. In a video editing program that causes an information processing device to function as a video editing device,
A moving image recognition step for storing a moving image recognition result with a time interval calculated from moving image data included in the moving image data;
A voice recognition step of storing a voice recognition result with a time interval calculated from voice data included in the video data;
A recognition result combining step of calculating a combined recognition result obtained by combining the moving image recognition result with time interval and the speech recognition result with time interval at each reproduction time;
A moving image editing program that causes the information processing apparatus to execute an editing information allocation step of determining editing information with a time interval to be applied based on the composite recognition result. In the recognition result combining step, the moving image recognition result with time interval and the speech recognition result with time interval are combined by collating with a predetermined combination pattern, and the predetermined combination pattern can be registered. The moving image editing program according to claim 9. In the editing information allocating step, the editing information with time section to be applied is determined by comparing the composite recognition result with a predetermined combination pattern of the composite recognition result and the editing information with time section, The moving image editing program according to claim 9 or 10, wherein the combination pattern can be registered. The moving image editing program according to any one of claims 9 to 11, wherein the editing information with a time section determined in the editing information allocation step is output as an editable data group.

Images