JP3537753B2 - Editing processing device and storage medium storing editing processing program - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an editing processing device and a storage medium storing an editing processing program, and more particularly to a video editing processing device capable of attaching a thumbnail appropriately representing the content. And a storage medium storing an editing processing program.

[0002]

2. Description of the Related Art Conventionally, in an editing process by an editing apparatus and an editing program of a storage medium storing the editing program, image data of a group of moving images divided for each content or for a predetermined time is used. It is common to attach thumbnails such as still images. The break for each moving image content is detected as a scene break. In this scene break, as shown in FIG. 7A, when a black frame (blackout) not including an image is inserted, as shown in FIG. When the scene transitions, as shown in FIG. 7C, the scene may occasionally transition gradually and gradually depending on the special effects or the imaging device side such as a camera. In a conventional editing process, a series of scenes is identified as a group of moving images by detecting these scene breaks.

[0003] The image data of the temporally first image data among the group of moving image data and the group of moving image data separated by time is specified as thumbnail image data. Has adopted. The editor refers to the thumbnails and performs operations such as continuation of the image data of each group and change of the order.

[0004]

However, when the first image of a group of moving images is used as a thumbnail as described above, the thumbnail does not necessarily appropriately represent the contents of the group of moving images. May not be
When editing, it is difficult to identify the contents, and there is a possibility that the processing operation may lack smoothness. For example, in a news broadcast, an announcer or a reporter is often shown at the beginning of each topic, and after reading a manuscript, an image or background related to the topic is displayed. Therefore, when each topic is grouped, an image of an announcer or a reporter is assigned to each group of thumbnails regardless of the content. Even if it is not a news broadcast, the part of the most important content may not be a thumbnail. Furthermore, in smooth scene transitions such as fade-in, wipe, camera zoom and tilt, scene breaks may not be automatically recognized properly, and the image in transition may become a thumbnail or the last image of the previous scene. May become a thumbnail.

[0005] As a method of using an image whose content is appropriately represented as a thumbnail, it is conceivable to display a list of each frame of a group of images and let the operator manually select a thumbnail from the list. However, there are many cases where the number of frames in each group is large, and there is a problem that the work of listing those frames and selecting from among them is troublesome.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and has a video editing processing apparatus and an editing processing program capable of easily attaching a thumbnail appropriately representing the content. It is intended to provide a storage medium.

[0007]

In order to achieve the above object, the present invention provides an image data acquiring means for acquiring a set of image data of a moving image, and a device for converting audio data of sound associated with the moving image. a document data obtaining means for obtaining document data, the document data acquired by the document data acquiring means, a dividing means for dividing a predetermined unit, before
Form of the document data obtained by the document data obtaining means
Perform elementary analysis, and the importance calculation means for calculating the importance of each segment from the key word included in said section, the document data of said calculated by the importance calculation <br/> means the high importance divided ,
Important section obtaining means for obtaining the important section, and associating the voice data with the document data of the important section
Important image data obtaining means for obtaining predetermined image data as important image data from the corresponding image data obtained;
By providing an editing processing device (first configuration) including a correspondence unit that associates the important image data acquired by the important image data acquisition unit with a set of the image data acquired by the image data acquisition unit. This achieves the above object.

The image data acquisition means communicates image data acquired by an external imaging device from the imaging device or another computer via a public line or a dedicated line such as the Internet, a cable connection, or the like. It can be obtained from a storage medium via an interface. Further, the image data acquisition means may include an image imaging unit (camera), and acquire the image data by imaging. The document data obtaining means converts the sound data of the sound accompanying the moving image, which is recognized by an external sound recognition device and converted into document data, from another computer via a public line such as the Internet or a dedicated line, or a cable. By communication such as connection,
In addition, it can be obtained through various storage media. Further, the document data acquisition unit includes a voice recognition unit inside the document data acquisition unit itself, and converts the moving image from a recording device incorporated in an external imaging device or a recording device used with an external imaging device. Acquisition of the accompanying audio data via a public line or a dedicated line such as the Internet, communication via a cable connection, or via various storage media, and conversion of the obtained audio data by voice recognition as a conversion result. May be obtained. Further, a voice input unit (microphone) may be provided in addition to the voice recognition means, the voice accompanying the moving image may be provided, and the voice data may be converted into document data by the voice recognition means to obtain the document data. The “unit” of the image data of a group of moving images is a unit for creating a thumbnail. This unit can be automatically obtained by dividing the unit into units having uniform contents, for example, by detecting a scene break by analyzing image data. In addition, it is also possible to form a unit automatically divided according to a predetermined time or time. Further, the unit may be a group separated by an operator manually. The unit of the division into which the document data corresponding to the set of image data is further divided by the division unit is one or a predetermined number of sentences, phrases, phrases, words, and the like. It is a unit of size that can specify importance. Although the size of this section depends on the size of a group of documents before the section, etc., when extracting not the entire image data corresponding to the important section but a part of the image data as an important image, It is preferable that the unit is as small as possible within a range in which the importance can be specified for each section. This is because, when extracting a part of the image data from the entire image data corresponding to the important section, the possibility that the image data appropriately representing the document of the important section is extracted as the important image becomes extremely high. The important category acquisition means,
Even if the category with the highest importance is acquired, a predetermined number of the categories with the highest importance are acquired from the top, and those with the importance equal to or higher than the predetermined height are acquired regardless of the number of the categories Good. The important image acquiring means may acquire the entire corresponding image data as important image data from the corresponding image data corresponding to the important section. Also,
From the corresponding image data, for example, image data of a moving image for a predetermined period of time can be obtained, such as obtaining image data for one second that is temporally centered. Further, instead of the image data forming a series of moving images as described above, image data for one still image may be obtained. Further, similarly to the image data, audio data corresponding to the important image and document data converted from the audio data may be associated with the group of image data together with the important image data.

In the above-described editing processing device of the present invention, a section having a high importance is determined based on a sound corresponding to an image.
The image data of the high importance section is associated with the entire group of images as important image data. Therefore, by making the data of the important image a thumbnail of a group of images, the image data of the image that is presumed to be important in content corresponds to the audio that is important in content, and the thumbnail is used as the thumbnail. Can be attached.

In the editing processing apparatus according to the present invention, in the editing processing apparatus having the first configuration, the important image data obtaining means obtains one still image data from the corresponding image data as the important image data. An editing processing device (second configuration) that is still image data acquisition means can be provided. In the editing processing device having the second configuration, a still image can be obtained as a thumbnail. In the editing processing apparatus according to the present invention, in the first or second editing processing apparatus, the important section obtaining unit obtains a plurality of the important sections, and the important image data obtaining unit includes the important section obtaining unit. Candidate image data acquisition means for acquiring candidate image data from corresponding image data corresponding to each of the plurality of acquired important sections, and candidate image output means for outputting an image of the candidate image data acquired by the candidate image data acquisition means, Selecting and obtaining means for obtaining one selected from the candidate images output by the candidate image output means, and converting the candidate image data of the selected candidate image obtained by the selecting and obtaining means into the important image data. (Third configuration). In the editing processing device of the third configuration, the operator can select one of a plurality of important image candidates, and by adding the judgment of the operator, the content of a group of images can be more reliably determined. An image that is appropriately represented and easy for the operator to understand can be specified as an important image. In addition, since the image data corresponding to the important section is automatically selected in advance as a candidate for image data of an image representing the contents of a group of images, the operator can select one important image with little effort. Is possible. The editing processing apparatus of the present invention, in the first to third editing processing apparatuses, further includes an image data dividing unit that divides image data of a moving image including a plurality of scenes for each scene. According to a fourth aspect of the present invention, there is provided an editing apparatus (fourth configuration) for acquiring image data of each scene as image data of a group of moving images according to the division by the image data dividing unit. In the editing processing device having the fourth configuration, the acquired image data is divided into a plurality of units by the image data dividing unit, and important image data is associated with each of the units. The image data dividing means may include a scene break detecting means and divide the image data for each scene break. Further, the image data may be divided at predetermined time intervals. Further, the image data dividing unit analyzes the contents of the document from the document data acquired by the document data acquiring unit, divides the document of the document data into a plurality of documents such as paragraphs and chapters according to the meaning and content, and handles the document data. May be used to divide the image data.

According to the present invention, there is provided an image data obtaining function for obtaining a set of moving image data, a document data obtaining function for obtaining document data obtained by converting sound data of sound associated with the moving image, A classification function for dividing the document data acquired by the data acquisition function into predetermined units, and a document data acquisition function
A morphological analysis of the data, and an importance calculation function for calculating the importance of each of the categories from the important words included in the category, and a document of the category with a high importance calculated by the importance calculation function An important category acquisition function that acquires data as important categories,
From the corresponding image data which is correlated to the audio data corresponding to the document data of the important segment, the predetermined image data, and important image data acquisition function of acquiring the key image data, acquired by the important image data acquisition function A storage medium storing a computer-readable editing processing program for causing a computer to realize a function of associating the important image data with the set of image data acquired by the image data acquisition function (fifth configuration) The above-mentioned object is achieved by providing). Further, the image data acquiring function, the document data acquiring function, the sorting function, the importance acquiring function, the important category acquiring function, the important image data acquiring function, and the corresponding function are realized. Processing program, editing processing program transmission medium, editing processing program carrier wave, editing processing program signal, or program product. Here, the program product includes a storage medium, a server system computer, a computer system, and the like for realizing each of the functions according to the editing processing program. The image data acquisition function is to acquire image data acquired by an external imaging device by communication via a public line or a dedicated line such as the Internet, by a cable connection, or the like, and via various storage media. be able to. Further, the image data acquisition function may include an image imaging unit (camera) and acquire image data by imaging. The document data acquisition function communicates the sound data of the sound accompanying the moving image, which is recognized by an external sound recognition device and converted into document data, via a public line or a dedicated line such as the Internet, or by a cable connection or the like. In addition, it can be obtained through various storage media. The document data acquisition function includes a voice recognition function, and externally communicates voice data of voice accompanying the moving image via a public line or a dedicated line such as the Internet, a cable connection, or the like, and various types of storage. It may be obtained via a medium. Furthermore, in addition to the voice recognition function, voice data may be acquired from the voice accompanying the moving image from a voice input unit (microphone) and converted into document data by the voice recognition function. The “unit” of the image data of a group of moving images is a unit for creating a thumbnail. This unit can be automatically obtained by dividing the unit into units having uniform contents, for example, by detecting a scene break by analyzing image data. In addition, it is also possible for the operator to manually set the unit. Further, the unit may be a unit automatically separated by a predetermined time or time. The unit of the division into which the document data corresponding to the group of image data is further divided by the division function is one or a predetermined number of sentences, phrases, phrases, words, etc. It is a unit of size that can specify importance. Although the size of this section depends on the size of a group of documents before the section, etc., when extracting not the entire image data corresponding to the important section but a part of the image data as an important image, It is preferable that the unit is as small as possible within a range in which the importance can be specified for each section. This is because, when extracting a part of the image data from the entire image data corresponding to the important section, the possibility that the image data appropriately representing the document of the important section is extracted as the important image becomes extremely high. The important category acquisition function, even if the category with the highest importance is acquired, even if a predetermined number of categories with high importance are acquired from the top, regardless of the number of categories, the importance is equal to or higher than the predetermined height. It may be acquired. The important image acquisition function may acquire the entire image data in the important section as image data corresponding to the important section. Also, from the image data in the important section, it is possible to acquire predetermined image data such as, for example, acquiring image data for one second which is in the middle in time. Further, instead of the image data forming a series of moving images as described above, image data for one still image may be obtained. Also, like the image data, the audio data in the important section and the document data converted from the audio data, together with the image data corresponding to the important section,
The image data may be associated with the set of image data. According to the storage medium storing the editing processing program of the present invention, and the editing processing program, the editing processing program transmission medium, the editing processing program carrier wave, the editing processing program signal, or the program product, the program is mounted on various computers. By the first
It is possible to realize an editing processing device having the above configuration.

[0012] In the storage medium storing the editing program according to the fifth aspect, the important image obtaining function obtains one still image data from the image data in the important section as the image data of the important image. (A sixth configuration).
In the storage medium storing the editing processing program according to the fifth configuration and the sixth configuration, the important section acquisition function acquires the plurality of important sections, and the important image data acquisition function acquires the important section acquisition. A candidate image data acquisition function for acquiring candidate image data from corresponding image data corresponding to each of the plurality of important sections acquired by the function, and a candidate image output function for outputting the candidate image data acquired by the candidate image data acquisition function to an image And a selection obtaining function for obtaining one selected from the candidate images output by the candidate image output function, wherein the candidate image data of the selected candidate image obtained by the selection obtaining function is A storage medium storing an editing processing program to be image data (the seventh storage medium)
Configuration). Further, the editing processing program, the editing processing program transmission medium, the editing processing program carrier wave, the editing processing program signal, or the program product may be used. Any one of the fifth to seventh aspects
The storage medium storing the editing processing program having the configuration of
The image data of a moving image including a plurality of scenes is provided with an image data dividing function of dividing the image data for each scene.The image data acquiring function separates the image data of each scene according to the division by the image data dividing function. It may be a storage medium (eighth configuration) in which an editing processing program to be obtained as a group of moving image image data is stored. Further, the editing processing program, the editing processing program transmission medium, the editing processing program carrier wave, the editing processing program signal, or the program product may be used.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an edit processing apparatus and a storage medium storing an edit processing program according to the present invention will be described below in detail with reference to FIGS. FIG. 1 is a conceptual diagram showing the configuration of an embodiment of an editing processing apparatus according to the present invention, and is a conceptual diagram showing the configuration of a computer that has read the editing processing program of the present invention from a storage medium storing the program. is there. As shown in the conceptual block diagram, the editing processing device (computer) includes an input unit 1, an image data obtaining unit 2, a document data obtaining unit 3,
Classification means 4, importance acquisition means 5, important division acquisition means 6,
An important image data acquisition unit 8, a correspondence unit 9, and an output unit 10 are provided.

The input means 1 is used by a user to input commands for various processes to be performed by the editing processing apparatus and to select data. The input means 1 includes a keyboard, a mouse, a microphone, and voice recognition of voice from the microphone. Equipment etc. are included. The input means 1 is for inputting image data to be edited, audio data accompanying the image data, document data obtained by converting the audio data by voice recognition, and the like, and for specifying image data to be edited. The input unit 1 is connected to a video camera equipped with a microphone, an external computer, an auxiliary storage device such as a CD-ROM or a DVD, or another external device directly or via a line network such as the Internet. Thus, the image data to be edited, the voice data accompanying the image data, and the document data obtained by converting the voice data by voice recognition are obtained using the communication unit.

The image data obtaining means 2 performs image data obtaining processing for obtaining image data of a group of moving images.
In the present embodiment, the image data acquisition unit 2 includes an image data division unit 21 that performs image data division processing of dividing image data of a moving image including a plurality of scenes for each scene. This image data dividing unit 21
By performing image analysis on the acquired image data designated as an editing target, blackouts and scene transitions are detected. When a blackout or a transition of a scene is detected, it is assumed that the acquired image data includes a plurality of scenes, and the detected blackout or a transition of the scene is divided for each scene as a boundary. It is set as image data of a group of moving images. As a result, the image data obtaining means 2 sequentially obtains and outputs the image data designated to be edited by the input means 1 for each divided unit (scene). This image data is
It is accompanied by time data by a timer which is common to the audio data.

The document data obtaining means 3 performs a document data obtaining process of obtaining document data obtained by converting voice data of voice accompanying the moving image by voice recognition in correspondence with image data. When the image data obtaining means 2 obtains the image data for each group, the document data obtaining means 3 refers to the time data attached to the image data, and outputs the sound corresponding to the image data for each group. Get the document data of the recognition result. This document data is accompanied by time data by a timer common to the image data. The document data acquisition unit 3 includes a speech recognition unit 31 that converts speech data into document data by speech recognition. If the input unit 1 acquires speech data corresponding to image data, the speech data Is converted into document data by voice recognition, and the image data acquisition means 2 acquires document data corresponding to the group of image data. Input means 1
In the case where the document data is obtained as a result of the voice recognition, the document data corresponding to the group of the image data by the image data obtaining means 2 is extracted and obtained from the document data. The voice recognition unit 31 converts voice data such as a voice waveform into a voice recognition dictionary and converts the voice recognition result into document data, or converts them into kana-kanji characters. This kana-kanji conversion can be performed by adding an analysis based on natural language processing. Note that even if the input means 1 has acquired the document data after voice recognition, the document data acquiring means 3
May be re-converted by unique analysis to obtain new document data.

The classifying means 4 performs a classifying process for classifying the document data obtained in the document data obtaining process into predetermined units. In the present embodiment, the predetermined unit is a sentence, and the dividing unit 4 divides the document data into units of each sentence obtained by the document data obtaining unit 3 (each scene). The importance obtaining unit 5 performs an importance obtaining process for obtaining the importance for each of the sections divided by the dividing unit 4. In the present embodiment, the sections classified by the classifying unit 4 are each sentence, and the importance acquiring unit 5 acquires the importance of each sentence. The importance acquiring unit 5 performs a morphological analysis on each sentence included in a set of document data, extracts candidate words (phrases) including independent words, noun phrases, compound noun phrases, and the like, and extracts the extracted candidate words. Each candidate word (phrase) importance f (x) is determined from the appearance frequency of the word (phrase) in a group and the evaluation function. Here, as the evaluation function, for example, when a predetermined important word is specified in advance, weighting for the important word, weighting according to the type of a candidate word (phrase) such as a word, a noun phrase, a compound noun phrase, and the like are used. used. Then, the importance level f (x) of each sentence is added by adding the importance level f (x) of the candidate word (phrase) appearing in each sentence.

The important category acquiring means 6 performs an important category acquiring process for acquiring the important category of high importance acquired in the importance acquiring process among the document data categories. In the present embodiment, the important category is an important sentence, and the important category acquisition unit 6
Is the importance F of each sentence acquired by the importance acquisition means 5.
A sentence with a high (x) is acquired as an important sentence. In the present embodiment, a predetermined number of sections are acquired in descending order of importance. The important image data obtaining means 8 performs an important image data obtaining process of obtaining predetermined image data as important image data from the corresponding image data corresponding to the important section. In the present embodiment, the important image data acquisition means 8
Acquires the image data of one still image from the corresponding image data corresponding to the important sentence as important image data. The important image data acquisition means 8 includes a corresponding image data acquisition unit 7. The corresponding image data acquisition unit 7 performs a corresponding image data acquisition process of acquiring image data (corresponding image data) corresponding to document data of an important category from the image data acquired in the image data acquisition process. In the present embodiment, the division is a sentence, and the important division acquisition unit 6
A predetermined portion of the image data acquired by the image data acquiring means 2 is made to correspond to each sentence which is regarded as an important sentence. The document data obtained by the document data obtaining means 3 and the image data obtained by the image data obtaining means 2 have time recorded by a common timer. Then, the correspondence acquisition unit 7 divides the image data into image data having the same start time and end time based on the start time and end time of each section of the document data, and associates them. For example,
For the document data "Initially, today's main items."
If the time is seconds and the end point is 1.5 seconds, the image data of the image from 0.3 seconds to 1.5 seconds by the same timer among the image data corresponds.

In the present embodiment, the important image data obtaining means 8 includes a candidate image data obtaining unit 81, a candidate image output unit 82, and a selection obtaining unit 83. The candidate image data obtaining unit 81 obtains image data corresponding to each of the plurality of important sections obtained by the important section obtaining unit 6 as a processing result of the corresponding image data obtaining process, and obtains the corresponding image data corresponding to each important section. A candidate image data acquisition process for acquiring image data of still images one by one as image data of a candidate image of an important image is performed. As image data of one still image, of image data from the start time to the end time of each important section,
The image data of the still image at the intermediate time point is selected. For example,
In the case of the image data in which the start time by the common timer is 0.3 seconds and the end time is 1.5 seconds, this timer corresponds to the above-mentioned document data "First, today's main item." The image data at the time point of 0.9 seconds in is set as the image data of the candidate image of the important image.

The candidate image output unit 82 performs a candidate image output process for outputting image data of the candidate image acquired in the candidate image acquisition process from the output unit 10. Selection acquisition unit 83
Performs a selection acquisition process of acquiring one selection from among the candidate images output to the output unit 10. When a candidate image is output from the output unit 10, the operator selects one image as an important image and inputs the selection result from the input unit 1. The selection acquisition unit 83 acquires which image is selected as an important image from the input result, and determines the selected image as an important image.

The correspondence unit 9 performs a correspondence process of associating the image data of the important image acquired in the important image acquisition process with a set of image data acquired in the image data acquisition process. In the present embodiment, the correspondence unit 9 further outputs the correspondence between the important image and the set of image data from which the important image has been extracted so that the correspondence can be recognized.
Display at 0. For example, a file name of a group of image data and an important image are displayed side by side. Accordingly, the still image functions as a thumbnail of the image data of a group of moving images, and the operator can clearly know the contents of the group of image data from the still image.

FIG. 2 shows a specific system configuration of the editing processing apparatus configured as shown in FIG. As shown in FIG. 2, the editing processing device is configured by a personal computer or a computer system including a personal computer. The editing processing apparatus includes a control unit 11 for controlling the entire system as shown in FIG. The control unit 11 includes a keyboard 12 and a mouse 13 as input means 1, a display device 14 as output means, a printing device 15, a storage device 16, and a storage medium driving device 17 via bus lines such as a data bus. , A communication control device 18, an input / output I / F 19, and a voice recognition device 20.

The control unit 11 includes a CPU 111, a ROM 11
2. A RAM 113 is provided. The CPU 111 controls various devices according to programs to perform calculations. ROM1
Reference numeral 12 denotes a read-only memory in which programs to be executed when the computer is started are stored in advance. RA
The M113 is used as a working memory in which programs and data for the CPU to perform various controls and calculations are stored.

When acquiring image data of a moving image to be edited, audio data attached to the image data, and document data, the keyboard 12 specifies a moving image to be edited,
The input unit 1 is configured to select and designate an important image from candidate images as a part of the selection acquisition unit. Keyboard 12
Contains kana keys and numeric keys for entering kana characters,
Various keys such as a function key and a cursor key for executing various functions are arranged. The mouse 13 is a pointing device.
By left-clicking a key, an icon, or the like displayed on the display device 14, a moving image to be edited is specified or an important image is selected. The display device 14 is, for example, a CRT
And a liquid crystal display are used. On this display device, a moving image that can be edited is displayed by an icon or a file name, and an image to be edited is selected by the keyboard 12 or the mouse 13. In addition, an image of an important image candidate is displayed, and the important image is selected by the keyboard 12 and the mouse 13. Furthermore, important images are displayed as thumbnails of a group of moving images from which the important images have been extracted. The printing device 15 constitutes the output unit 10 and prints an image or the like displayed on the display device 14. This printing device includes laser printers, dot printers, inkjet printers, page printers, thermal printers,
Various printing devices such as a thermal transfer printer are used.

The storage device 16 comprises a readable and writable storage medium and a drive device for reading and writing various information such as programs and data on the storage medium.
As a storage medium used for the storage device 16, a hard disk is mainly used, but a readable / writable storage medium among various storage media used in a storage medium drive device 17 described later may be used. Good. The storage device 16 stores the kana-kanji conversion dictionary 161, the program storage unit 1
62, a data storage unit 163, a speech recognition dictionary 164, and other storage units (not shown) (for example, the storage device 16).
Storage unit for backing up programs, data, and the like stored therein. The program storage unit 162 includes, in the editing processing according to the present embodiment, an image data acquisition processing program for acquiring image data of a group of moving images, as a program for performing a thumbnail process for giving a thumbnail to each group of moving images;
A document data acquisition processing program for acquiring document data obtained by converting audio data of audio associated with the moving image,
A division processing program that divides the document data acquired in the document data acquisition processing by the document data acquisition processing program into predetermined units, and a degree of importance that acquires the importance of the document data for each division divided by the division processing An acquisition processing program, an important category acquisition processing program for acquiring, as important categories, the document data of the high importance categories acquired in the importance acquisition processing by the importance acquisition program, and important category acquisition by the important category acquisition processing program From the corresponding image data corresponding to the important section obtained in the processing, predetermined image data is obtained by an important image data obtaining processing program for obtaining as important image data, an important image data obtaining processing by the important image data obtaining processing program. Acquire the important image data in the image data acquisition process Corresponding processing program is stored associating the a collection of the image data. Also,
In addition to the thumbnail processing, a video editing processing program (for example, a program that uses the result of thumbnail processing, such as a program for rearranging a group of videos while referring to thumbnails attached by thumbnail processing, And kana-kanji conversion dictionary 161
And various programs, such as a kana-kanji conversion program for converting a kana character string input by using.

In the data storage section 163, the storage medium drive 17 and the communication control
From the moving image data, the audio data acquired together with the moving image data, the document data as the sound recognition result of the audio data, and the thumbnails of the respective moving image data extracted from the moving image data by the editing processing according to the present embodiment. Is stored as image data of a still image. The speech recognition dictionary storage unit 164 stores a speech recognition dictionary corresponding to speech data and words in units of phonemes, single syllables, words, morphemes, phrases, and the like (hereinafter, morphemes and the like). In the present embodiment, a morpheme dictionary corresponding to a speech pattern for each morpheme and a morpheme is stored. This voice recognition dictionary 1
64 finds a corresponding word from a voice data pattern of voice or a voice data pattern created by another device and read from the storage medium driving device 17 or the communication control device 18, and the input voice is used as a word. Used for recognition.

The storage medium drive 17 is a drive for the CPU 111 to read a computer program or data including a document from an external storage medium. Computer programs and the like stored in the storage medium include various processing programs such as image information search processing, subjective evaluation expression dictionary calibration processing, and subjective evaluation information dictionary calibration processing executed by the editing processing apparatus of the present embodiment, and The dictionary, data, etc. used there are also included. Here, the storage medium refers to a storage medium in which a computer program, data, and the like are stored, and specifically, a magnetic storage medium such as a floppy disk, a hard disk, and a magnetic tape, and a semiconductor storage medium such as a memory chip and an IC card. , CD-ROM or M
O, PD (phase-change rewritable optical disk) and other storage media from which information can be read optically, storage media using paper such as paper cards and paper tapes (and media having functions equivalent to paper), and various other types A storage medium on which a computer program or the like is stored in the method is included. As a storage medium used in the editing processing device of the present embodiment, a storage medium such as a CD-ROM or a floppy (registered trademark) disk is mainly used. The storage medium driving device 17 can read data stored in the RAM 113 and the storage device 16 into a writable storage medium such as a floppy disk in addition to reading a computer program from these various storage media. It is.

When the moving image data, audio data, and document data stored in a floppy disk, a memory chip, an IC card, or the like are read through the storage medium driving device 17 and the editing processing according to the present embodiment is performed, The storage medium driving device 17 functions as the image data obtaining unit 2 and the document data obtaining unit 3.

In the editing processing apparatus of the present embodiment, the control unit 1
The first CPU 111 reads a computer program from an external storage medium set in the storage medium drive 17 and stores (installs) it in each unit of the storage 16. When executing various processes such as the image information search process according to the present embodiment, the corresponding program is read from the storage device 16 into the RAM 113 and executed. However, instead of the storage device 16, the storage medium driving device 17 directly outputs the RAM 113 from an external storage medium.
It is also possible to read and execute the program.
Further, depending on the editing processing device, various programs and the like for the editing processing of the present embodiment may be stored in the ROM 112 in advance, and the CPU 111 may execute the programs. Furthermore, various programs and data for the editing process according to the present embodiment may be downloaded from another storage medium via the communication control device 18 and executed.

The communication control device 18 is a control device for making a network connection between the editing processing device and various external electronic devices such as other personal computers. It is possible to acquire image data of a moving image to be edited and audio data or document data accompanying the image data. The communication control device 18 in this case constitutes the image data obtaining means 2 and the document data obtaining means 3. Further, the communication control device 18 configures the output unit 10 and the selection acquisition unit 83, and outputs the image data of the important image candidate extracted in the editing processing program to the external electronic device via the communication control device 18, and outputs the important image The selection of the important image from the candidates can be obtained from the external electronic device via the communication control device 18.

The input / output I / F 19 is an interface for connecting various devices such as a video camera, and constitutes the image data obtaining means 2 and the document data obtaining means 3.
Image data of a moving image to be edited and audio data accompanying the image data can be obtained from an external electronic device. The voice recognition device 20 constitutes the document data acquisition means 3, from a video camera connected via the input / output I / F 19, from an external electronic device via the communication control device 18, and further from a storage medium driving device The voice data specified by the input means 1 and read from the storage medium via the storage means 16 and from the data storage section 163 of the storage means 16 are recognized using the voice recognition dictionary 164 and converted into document data.

The editing apparatus according to the present embodiment is not only constituted by a computer system including a personal computer, a word processor, etc., but also by a LAN (local area network) server, a computer (personal computer) communication host and the Internet. It is also possible to configure with a connected computer system or the like. Further, it is also possible to distribute the functions to the devices on the network and configure the editing processing apparatus on the entire network. FIG. 3 shows a system configuration diagram when the editing processing device is configured by such a network. As shown in FIG. 3, the editing processing device includes a host device 30 that performs thumbnail processing and the like, a plurality of client PCs 50 that transmit moving image data, audio data, and document data to the host device 30. A network 40 connects the client PC 50 to the network. Although the network 40 is mainly connected to the Internet, a LAN (local area
Network) and various networks 40 such as a computer network. A client P such as a personal computer is connected to the network 40.
The C50 is appropriately connected, and a plurality of client PCs 50 can access the host device 30 at appropriate times.

The client PC 50 is constituted by a so-called personal computer system such as a personal computer, is connected to a network 40 (Internet) by dial-up software or the like, and is connected to a WWW (World).
Data from Wide Web is browsed by browsers
er) It can be browsed by software. On the other hand, the client PC 50 includes a control unit, a display unit, an input unit, an output unit, a communication control unit, a storage unit, and other devices.
The control unit of the client PC 50 processes and controls the entire apparatus according to a predetermined program, and transmits image data, audio data, and document data of a moving image input from the input unit via the communication control unit and the network 40. In addition to transmitting to the host device 30, the thumbnail given by the thumbnail processing in the host device 30 is received and displayed together with the moving image data on the display unit, stored in the storage unit, or printed out from the output unit. I have.

On the other hand, the host device 30 includes a control unit 31. The control unit 31 has an input / output unit 32, a display unit 34, a storage unit 36, a communication control unit 38, and a bus line such as a data bus. No other equipment is connected. The basic configuration of each of the units 31 to 38 is substantially the same as that of the editing processing apparatus shown in FIG. 2, and the control unit 31 functions as a WWW server.
The control unit 11 of the editing processing apparatus shown in FIG.
High-speed processing is possible as compared with the control unit 11 of the C50, and parallel processing is possible to cope with access from a plurality of client PCs 50. Similarly, the communication control unit 38 can be connected to a plurality of ISDN lines and can perform processing at a higher speed than that of the client PC 50. The host device 30 is connected to the input unit 1
Under the control of the communication control unit 38 constituting the output unit 10, image data and audio data of moving images are received from the client PC 50 via the network 40, and the document data of the voice recognition result of the audio data is received and acquired, and the candidate image of the important image is Output to the client PC 50 and output to the client PC 5
From 0, the selection of the important image is obtained. The correspondence between the important image and the moving image from which the important image is extracted is stored in the data storage unit 363 together with the image data, audio data, and document data of the moving image, or transmitted to the client PC 50 via the network 40. .

Next, the operation of the thumbnail processing performed by the editing device configured as described above will be described. FIG.
FIG. 5 is a flowchart showing the operation of the thumbnail processing performed in the editing processing by the editing processing apparatus.
FIG. 6 is an explanatory diagram conceptually showing processing in each step of the thumbnail processing. The thumbnail processing by the editing processing device is performed when a user inputs a command to display a list of predetermined moving image data in the thumbnail display mode in the editing processing from the input unit 1 and detects moving image data to which no thumbnail is assigned. This is performed on the moving image data to which the thumbnail is not provided.

In the thumbnail processing according to the present embodiment, the image data A of the moving picture to which the thumbnail is not assigned among the moving pictures displayed in the thumbnail display mode by the user is determined by the image data acquiring means 2 as the thumbnail processing target. (Image data acquisition processing) (step 11) (FIG. 5A). The image data A obtained by the image data obtaining unit 2 is checked by detecting a scene break by a scene break detecting unit provided in the image data obtaining unit 2 to determine whether the image data includes a plurality of scenes. (Step 13). If the image data includes a plurality of scenes, it is determined that the image data includes a plurality of scenes. (Image data a, image data b, image data c,...)
.) (Image data division processing) (step 1)
5) (FIG. 5 (b)). When the image data obtained by the image data obtaining means 2 is one scene (step 13;
N) and after the image data of a plurality of scenes are divided into a group of image data for each scene (after step 15),
The document data obtaining means 3 generates the document data (the document data a ′ corresponding to the image data a, the document data b ′ corresponding to the image data b, and the image data c) as the voice recognition result of the voice recorded together with the moving image of the image data. ) Corresponding to each set of image data (document data acquisition processing) (step 1).
7) (FIG. 5 (c)). At this time, the document data acquisition means 3
When the voice data of the voice corresponding to the image data obtained by the image data obtaining means 2 is not recognized, the voice data is recognized by the voice recognition unit 31 to obtain the document data. If the speech recognition result already exists, the document data of the speech recognition result is obtained as it is. The audio data and the document data corresponding to the group of the image data are obtained by referring to the time data attached to the image data and determining the audio data and the document data for the same time data.

Next, the classifying means 4 sets the document data (document) corresponding to the first group of image data (the N-th group of image data when N = 1, ie, the first group of image data a). Data a ′) is obtained, and this document data is divided into sentences (sentence 1, sentence 2, sentence 3,...) (Segmentation processing) (steps 19 to 23) (FIG. 5).
(D)). The importance acquiring means 5 performs morphological analysis on each sentence 1, sentence 2, sentence 3,... To extract important words (phrases) including independent words, noun phrases, compound noun phrases, and the like. The importance f of each important word (phrase) is determined from the frequency of appearance of the extracted important word (phrase) in the first group and the evaluation function. The evaluation function is, for example, weighting for important words (phrases), important words (phrases) such as words, noun phrases, compound noun phrases, etc.
Is a function representing the weighting or the like depending on the type of. In the present embodiment, the important words (phrases), the weights of the important words (phrases), and the weights for the types of the important words (phrases) are determined in a predetermined storage unit (the program storage unit 162 or the data storage unit 163 of the storage device 16). Etc.) are stored in advance. Then, the importance F of each sentence 1, sentence 2, sentence 3,... Is determined by accumulating the importance f of the important word (phrase) appearing in each sentence. (Importance acquisition processing) (Step 25) (FIG. 5)
(E)).

When the importance F of each sentence is determined, referring to the importance F, the important division acquiring means 6 determines the number of all sentences in the document data in descending order of importance F. A sentence of a predetermined ratio is selected and identified as an important sentence (important sentence 1, important sentence 2, ...) (important category acquisition processing) (step 27).
(FIG. 5 (f)). Next, the corresponding image data obtaining unit 7
Acquire image data (corresponding image data 1, corresponding image data 2,...) Corresponding to important sentences 1, 2, and so on (corresponding image data acquisition processing). The image data corresponding to the important sentence is referred to the time data attached to each important sentence, and the image data for the same time data is determined and obtained (corresponding image data obtaining processing) (step 29) (FIG. 6).
(G)). Then, by the candidate image data acquisition unit 81,
For each of the corresponding image data 1, the corresponding image data 2,... Corresponding to each of the important sentences 1, 2,..., The image data of the still image at the time between the start point and the end point is extracted. Are image data of important image candidates (candidate image data 1, candidate image data 2,...)
(Important image candidate acquisition processing) (FIG. 6)
(H)). Candidate image data 1, Candidate image data 2, ...
Is output from the output unit 10 by the candidate image output unit 82 (candidate image output processing) (step 31).
(FIG. 6 (i)). The operator looks at the candidate images based on the candidate image data output from the output unit 10, selects a suitable thumbnail as the content of the set of image data, and inputs the thumbnail from the input unit 1.

The selection by the operator is performed by the selection
It is acquired based on the input result from the input means 1 (step 33), and the selected candidate image is set as an important image (image for thumbnail) (important image acquisition processing). Then, the image data of the important image is provided by the correspondence unit 9 in step 21.
A set of image data corresponding to the document data acquired in (1) is associated as image data for thumbnails (corresponding processing) (FIG. 6 (j)) and stored in a predetermined storage unit (step 35). After the image data and the image data of the important image are associated with each other for the first group, the second group, the third group, and so on, are used thereafter.
Then, the processing from step 21 is repeated, and the image data of the important image is similarly associated and stored for all the divided groups of the moving image. When the image data of the important image is associated with all the groups, the thumbnail processing ends. Thereafter, when an instruction for displaying a list of moving image data in the thumbnail display mode is input in the editing process, the set of image data a, image data b, image data c,. A thumbnail image based on the given thumbnail image data is displayed.

As described above, in the present embodiment, a document in which voices corresponding to images of a group of scenes are recognized by voice is acquired, and a sentence having a high importance (important sentence) in the document is specified. The image corresponding to the important sentence (the image of the scene where the important sentence is output as sound) is a still image included in the moving image corresponding to the important sentence, assuming that the content of the set of scenes is well reflected. Are output as thumbnail image candidates (candidate images). Then, the operator's selection is obtained from the output candidate images, the selected candidate image is specified as a thumbnail, and the selected candidate image is associated with the entire group of scenes.

Therefore, according to the present embodiment, the image data of a scene displaying important contents is considered as a thumbnail candidate in consideration of the contents of a group of scenes, so that a thumbnail appropriately representing the group of contents is generated. It is possible to attach. According to the present embodiment, a still image of a scene displaying important contents is output as a thumbnail candidate image, and an appropriate image is selected by the operator.
An image that appropriately represents the contents of a group of scenes and that is easy for the operator to understand is specified as an important image. At this time, since the image of the scene displaying the important content is automatically selected as the thumbnail, the operator can select one thumbnail image with little effort. According to the present embodiment, a moving image including a plurality of scenes is automatically divided by a scene break, and a thumbnail is assigned to each scene.
No two thumbnails are given.

While one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the invention described in the claims. It is possible. For example, in the above embodiment,
Although a computer is used as the editing processing device, the invention is not limited to the computer, and may be a dedicated machine for editing processing. In the above embodiment, still image data is extracted as an important image candidate from image data (corresponding image data) corresponding to an important sentence, but moving image data for a predetermined time may be extracted. In this case, the entire image data corresponding to the important sentence can be set as an important image candidate. In the above-described embodiment, from the image data corresponding to the important sentence, the still image data positioned in the middle in time is regarded as the important image candidate. However, the image data of the important image candidate is obtained from the image data corresponding to the important sentence. It is sufficient that the image data is extracted, and the image data at the start of the important sentence or the image data at a predetermined time after the start can be used. In the above-described embodiment, the document data corresponding to a set of image data is classified in sentence units and an important sentence is acquired as an important division. However, the unit of division is not limited to a sentence unit. A plurality of sentences may be set as one unit, or a phrase may be set as a unit. For example, in the case of the document data a 'shown in FIG. 5, it is divided into phrases such as "at the beginning", "today", "news", "tell me", "today", "around 3am", and so on. The important phrases “earthquake”, “of seismic intensity 6”, and “damage status”, which are important categories, are extracted from the importance of the words included in each phrase, and the image corresponding to each of these important phrases may be used as the corresponding image. As in the above-described embodiment, a plurality of important sections can be extracted even if the section is not a sentence unit. In addition, regardless of whether the unit of the important division is sentence, phrase, or any other, if a plurality of divisions with the same importance are detected, the time data among them is determined based on predetermined conditions such as the first and intermediate time data. One can be selected, or both can be set as important categories.

In the above-described embodiment, a plurality of important image candidates are proposed to the operator for a set of image data, and the important image is determined by the operator's selection. Select only important sentences,
A still image extracted from the important sentence may be automatically determined as a thumbnail as an important image. In the above-described embodiment, each scene is grouped by detecting a scene break of image data, but a group of scenes is detected based on document data of the corresponding voice data after voice recognition. Is also good. When detecting a group of scenes based on document data, a conventionally known technique such as the technique described in JP-A-11-45278 can be used. That is, each sentence of the document data is divided into provisional paragraphs, and an index such as a document vector representing the content of each provisional paragraph is created. Tentative paragraphs are grouped together. In this case, as the document vector, a keyword appearing in the document by the morphological analysis is used as a parameter, and for each keyword, each importance f is determined from an appearance frequency and an evaluation function in the provisional paragraph, and the importance is determined by the value of each parameter Can be adopted
The similarity in this case can be obtained by a cosine depending on the angle between the document vectors. That is, assuming that the angle between the document vectors bn and bn + 1 is q, the inner product of both document vectors is bn · bn + 1, and the magnitudes of both document vectors are | bn | and | bn + 1 | s is obtained by the following equation 1.

[0044]

[Formula 1] Similarity s = COS (q) = (bn · bn +
1) / (| bn | × | bn + 1 |)

The value of the similarity s takes a value up to −1 ≦ s ≦ 1, and as the value is closer to 1, the document vectors of the two provisional paragraphs are closer to each other and the two provisional paragraphs are considered to be similar. be able to.

Further, the order of each processing can be changed as appropriate. For example, in the above-described embodiment, for each unit of image data (for each scene), from acquisition of corresponding document data to identification of an important sentence and determination of an important image, this is repeated for other units. However, the processing for each group may be performed for all the groups, and then the next processing may be performed. That is, after obtaining document data (division of document data) for each unit for all units, an important sentence is specified for all units, and then output of image candidates and selection of important images for each unit. The acquisition of
The important image may be associated with each set of image data.

In the above embodiment and each of the modifications,
Although the input voice is in Japanese, it is possible to obtain voice data and perform editing processing in any language. In this case, it is only necessary to change a portion that does not affect the configuration of the present invention, such as using a morphological analysis algorithm for the target language.

The above modifications can be applied by selecting a plurality of them as appropriate and combining them. For example, a modification in which image data of a moving image is extracted as an important image candidate based on an important sentence is combined with a modification in which only one important sentence is determined for a set of image data to determine an important image. Only one important sentence is determined for a group of image data, and the image data of a moving image can be determined as an important image based on this important sentence.

[0049]

As described above, according to the present invention,
It is possible to attach a thumbnail appropriately representing the content to the image data of the moving image.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a computer, which is an embodiment of an editing processing apparatus of the present invention and which reads an editing processing program of the present invention from an embodiment of a storage medium storing the program. is there.

FIG. 2 is a specific system configuration diagram of the editing processing device (computer).

FIG. 3 is a system configuration diagram when the editing processing device is configured by a network;

FIG. 4 is a flowchart showing a flow of a thumbnail process in an editing process performed by the editing processing device (computer).

FIG. 5 is an explanatory view conceptually showing processing in each step of the thumbnail processing in FIG. 4;

FIG. 6 is an explanatory view conceptually showing processing in each step subsequent to FIG. 5 in the thumbnail processing of FIG. 4;

FIG. 7 is an explanatory diagram showing a form of scene transition in a moving image including a plurality of scenes in the present invention and the related art.

[Explanation of symbols]

1 Input means 2 Image data acquisition means 21 Image Data Division 3 Document data acquisition means 31 Voice Recognition Unit 4 Classification means 5 Importance acquisition means 6. Means of acquiring important categories 7 Compatible image data acquisition unit 8 Important image data acquisition means 81 Candidate image data acquisition unit 82 candidate image output unit 83 Selection acquisition unit 9 Corresponding means 10 Output means

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ identification symbol FI H04N 5/781 H04N 5/781 510F 5/85 5/91 R (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N 5/91-5/956 G11B 27/02

Claims (5)

(57) [Claims] An image data acquiring unit that acquires a group of moving image data; a document data acquiring unit that acquires document data obtained by converting audio data of a sound associated with the moving image; The document data obtained by the means,
A classifying unit for classifying the document data in a predetermined unit, and a form of the document data acquired by the document data acquiring unit
Importance calculating means for performing a morphological analysis and calculating the importance of each of the categories from the important words included in the category, and converting the document data of the category having a high importance calculated by the importance calculating means into important categories and important segment acquiring means for acquiring a, from the corresponding image data which is correlated to the audio data corresponding to the document data of the key segment, and importance image data acquisition means for acquiring predetermined image data, as a key image data, An editing processing apparatus comprising: a correspondence unit that associates the important image data acquired by the important image data acquisition unit with a group of the image data acquired by the image data acquisition unit. 2. The important image data acquiring means according to claim 1, wherein the important image data acquiring means acquires one still image data from the corresponding image data as the important image data. Edit processing device. 3. The important section acquiring means acquires a plurality of important sections, and the important image data acquiring means acquires corresponding image data corresponding to each of the plurality of important sections acquired by the important section acquiring means. Candidate image data obtaining means for obtaining candidate image data, candidate image output means for outputting an image of the candidate image data obtained by the candidate image data obtaining means,
From the candidate images output by the candidate image output means.
3. A selection obtaining means for obtaining a selected one , wherein candidate image data of a selected candidate image obtained by the selection obtaining means is used as the important image data. 3. The editing processing device according to 1. 4. An image data dividing unit that divides image data of a moving image including a plurality of scenes for each scene, wherein the image data obtaining unit is configured to divide the image data of each scene according to the division by the image data dividing unit. 4. The method according to claim 1, wherein data is acquired as image data of the group of moving images.
The editing processing device according to claim 1. 5. An image data obtaining function for obtaining a group of moving image data, a document data obtaining function for obtaining document data obtained by converting audio data of sound associated with the moving image, and the document data obtaining. The document data obtained by the function,
A classification function for classifying in a predetermined unit, and a form of the document data acquired by the document data acquisition function
An importance calculation function for performing a morphological analysis and calculating the importance of each section from the important words included in the section, and document data of the high importance section calculated by the importance calculation function , and importance classification obtaining function for obtaining a, from the corresponding image data which is correlated to the audio data corresponding to the document data of the important segment, the predetermined image data, and important image data acquisition function of acquiring the key image data, A computer-readable editing processing program for causing a computer to implement a function of associating the important image data acquired by the important image data acquisition function with the set of image data acquired by the image data acquisition function is stored. Storage media.