JPH06337898A - Moving picture editor - Google Patents

Abstract

PURPOSE:To efficiently edit moving picture information. CONSTITUTION:This moving picture editor is provided with a reproducing part 11 for reproducing moving picture information for which a specific image is synthesized and is recorded by optional timing, an A/D conversion part 12 for quantizing the moving picture information reproduced by the reproducing part 11 at every frame picture and a character/graphic detecting part 14 for analyzing a frame picture obtained by the conversion part 12 and for detecting a frame picture to which a specific image is added and characteristically constituted so as to detect a representative frame picture being the characteristic frame picture of the moving picture information based upon the frame picture detected by the detecting part 14.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture editing apparatus for efficiently editing moving picture information taken by a video camera or the like.

[0002]

2. Description of the Related Art In recent years, with the widespread use of video cameras, anyone can easily collect moving picture information in various situations. When an amateur shoots, he does not make a detailed shooting plan such as storyboard or time schedule,
I often shoot scenes that I am interested in as soon as I have a chance. When editing video information shot in this way,
Since many redundant scenes are included in the moving image information, it is necessary to perform editing work to extract only necessary scenes. In editing moving image information, first of all, grasping the outline content of the entire moving image information contributes to a great improvement in editing work efficiency.

Conventionally, in order to grasp the contents of moving image information, fast-forward reproduction, that is, a so-called high-speed search function has been used to continuously watch a moving image, but it still takes too much time. It was

Therefore, when switching scenes in a moving image,
A method has been proposed in which camera work such as zoom, pan, tilt, and fade-in / out is automatically detected, these characteristic scenes are displayed as a still image, and the content of a moving image is grasped.

[0005]

As described above, in the prior art,
In order to edit the moving image information efficiently, the high-speed search function was used to sequentially view the images from the beginning to grasp the general contents of the moving image information. However, even if the high-speed search function is used, it takes a very long time to check all the moving image contents.

[0006] Further, a method has been proposed in which scenes in a moving picture are automatically detected, camerawork is automatically detected, and these characteristic scenes are displayed as a still picture to grasp the outline content of the moving picture information. The scenes of switching and camera work may not necessarily correspond to the scenes that one needs. Therefore, in the case of long-time moving image information, the number of even the above-mentioned characteristic scenes becomes considerable, and there is a problem that the unnecessary mixture of representative scenes becomes a factor to reduce the editing work efficiency. The present invention has been made in view of the above points, and an object of the present invention is to provide a moving image editing apparatus capable of efficiently editing moving image information.

[0007]

According to the present invention, there is provided reproducing means for reproducing moving picture information in which a specific image is combined with a characteristic scene at an arbitrary timing and recorded, and the reproducing means. Quantizing means for quantizing moving image information for each frame image, detecting means for analyzing the frame image obtained by the quantizing means, and detecting the frame image to which the specific image is added, and the detecting means. A representative frame image that is a characteristic frame image of the moving image information is detected based on the detected frame image.

Further, it is characterized by further comprising display means for displaying, as the representative frame image, a frame image separated by a predetermined frame from the frame image detected by the detecting means.

Further, the specific image is generated by a specific color, and the detecting means detects the frame image by extracting the specific color.

Further, the specific image is obtained by painting the entire screen with a specific color, and the detecting means detects the frame image by color histogram analysis.

Further, the specific image is characterized by representing time. Further, the detecting means has a recognizing means for recognizing the specific image representing the time, and the information representing the time obtained by this recognition is attached to the detected frame image.

[0012]

According to this structure, by superimposing a specific character or symbol on the scene which is considered to be characteristic of the photographer and recording the moving image information, the time of editing can be improved. By analyzing the video information recorded in, detecting the frame image to which specific characters or symbols are added, and detecting and displaying the characteristic representative frame image based on this frame image, the photographer's It is possible to easily understand the outline of the moving image content as intended. Therefore, it solves the problem that it takes too much time when using high-speed search, and it is different from the point originally intended by the photographer, such as when scene switching or camera work scene is used as a representative image. Since the problem that the scenes are mixed is solved and the outline content of the moving image information can be accurately grasped, the work efficiency of the moving image editing can be greatly improved.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. The moving image editing apparatus of the present embodiment detects a frame to which specific characters or symbols are added at the time of editing, with respect to moving image information in which specific characters or symbols are superimposed and recorded in advance at arbitrary timing. The display is performed so that the outline of the moving image content can be grasped based on the frame.

First, the moving picture processing apparatus 1 for recording a moving picture to be processed in the moving picture editing apparatus will be described. In the present embodiment, the moving image recording apparatus 1 will be described using a video camera as an example. The video camera is for home use, for example, and only the basic functions are described here.

As shown in FIG. 1, the moving image processing apparatus 1 includes a camera section 2, a superimposing section 3, a recording section 4, and an A / D.
It is composed of a conversion unit 5, a binarization unit 6, an image memory 7, a time information generation unit 8 and a time information generation unit 9.

The most basic function is to capture a moving image and record it on a recording medium such as a video tape. The camera unit 2 converts an image such as a landscape input via an optical system into an electric signal. The superimposing unit 3 superimposes an image (title, specific characters, symbols, etc.) stored in the image memory 7 on the moving image information captured by the camera unit 2 according to an arbitrary instruction. . The recording unit 4 records the moving image information captured by the camera unit 2 on a recording medium such as a video tape.

The A / D conversion section 5 quantizes the image signal input from the camera section 2. The binarization unit 6 binarizes the signal quantized by the A / D conversion unit 5 and stores it in the image memory 7 as binary image data. The image stored in the image memory 7 is used by the superimposing unit 3 to superimpose the moving image information. (Specific creation of images for superimposition will be described later). The image memory 7 can hold a plurality of images. Based on the image, the superimposing unit 3 can superimpose on the image being picked up with a designated color among several colors.

The time information generator 8 generates time information used to access an arbitrary frame image when reproducing the image. The time information generated by the time information generating unit 8 is recorded on the recording medium by the recording unit 4 along with the moving image information.

The time information generating section 9 generates an image (time information) representing the time. The time information is superimposed by, for example, the lower right corner in the captured image by the superimposing unit 3.

Next, the structure of the moving image editing apparatus 10 will be described. FIG. 2 is a block diagram showing the configuration of a moving image editing apparatus according to an embodiment of the present invention. The video editing device 10
As shown in FIG. 2, the reproduction unit 11, the A / D conversion unit 12, the image memory 13, the character / graphics detection unit 14, and the CPU 1
5, a memory 16, an affine conversion unit 17, an image memory 18, an image memory 19, a D / A conversion unit 20, and a display unit 21.

The moving picture editing apparatus 10 reproduces / edits the moving picture information recorded by the moving picture recording apparatus 1, analyzes the moving picture information based on specific superimposed characters or symbols, and requires it. Automatically detect a predetermined frame.
Further, a frame immediately after being detected, or a frame image before or after n frames is set as a representative frame image,
Display to understand the video content.

The reproducing section 11 reproduces the moving image information recorded on a recording medium such as a video tape and outputs it as a video signal. Also, the time information recorded together with the moving image information is output.

The A / D conversion unit 12 quantizes the video signal output from the reproduction unit 11 in frame units, for example, 1
The pixel is converted into a digital image represented by 8 bits and stored in the image memory 13.

The character / graphic detection unit 14 analyzes the image stored in the image memory 13 and detects a predetermined character or symbol to indicate a specific frame image.
The CPU 15 controls the operation of the moving image editing apparatus 2 as a whole. The memory 16 is a buffer memory that is handled by the CPU 15 and temporarily stores information.

The affine conversion unit 17 is an A / D conversion unit 12
The quantized image is reduced to a predetermined size. The image memory 18 is an image memory for temporarily storing the image reduced by the affine transformation unit 8. The image memory 18 stores a plurality of reduced images.

The image memory 19 is the image memory 1
A display image memory for temporarily storing a display image in which a plurality of images stored in 8 are laid out for one screen.

The D / A converter 20 includes an image memory 19
The display image stored in is converted into a video signal by D / A conversion. The display unit 21 is changed to D by the D / A conversion unit 20.
It is a monitor that displays an image according to the A / A converted video signal.

Next, the operation of the moving picture editing apparatus according to this embodiment, which efficiently realizes the understanding of the moving picture contents, will be described. As an example, here we take various animals in the zoo with a video recording device 1 (video camera) and classify and edit the videos obtained by this shooting into mammal scenes, bird scenes, and reptile scenes. explain.

The moving image recording apparatus 1 is a general home video camera as described above. In the video recording device 1,
An image (for a title, etc.) including a specific symbol for superimposing the photographed image is usually created by the procedure described below.

First, a title or the like written in black magic or the like on white paper is photographed by the camera unit 2, and a title input instruction is input from a switch or the like (not shown). The image input from the camera unit 2 is quantized by the A / D conversion unit 5, further binarized by a predetermined threshold value by the binarization unit 6, and stored in the image memory 7.

When the image (title) stored in the image memory 7 is superimposed on the moving image information, for example, a region corresponding to black in the image input when the title is input can be displayed in a predetermined color. Here, it is possible to select from three colors of red, green, and blue. Usually, a plurality of such titles can be stored in the image memory 7.

In this embodiment, a specific symbol as shown in FIG. 3A is used as a specific character or symbol to be superimposed for searching a specific frame image. A plurality of symbols are scattered in the screen to ensure recognition. The larger the number, the more reliably the detection is possible and the higher the recognition accuracy. Also, the influence of the background can be reduced. Of course, if all the backgrounds have the same color as the display color of the symbol, detection is impossible, and in that case, the display color of the title is changed.

To nullify the influence of the background at all, the title of full screen filling is used. In that case, as a scene detection method described later, a color histogram of the entire screen may be analyzed. For example, when the entire screen is red, the histograms for the three primary color components, red, green, and blue, have only the red component protruding and are characteristic histograms, which can be easily differentiated.

When an image of an animal as shown in FIG. 2 (b) is taken by the moving picture recording apparatus 1, a specific symbol stored in advance in the image memory 7 is displayed in FIG.
Superimpose as shown in. The scene in which the specific symbol is superimposed is automatically detected in the editing work in the moving image editing apparatus 10.

In this embodiment, one type of symbol is used to mark the characteristic scene, but it is also possible to use a plurality of types of symbols and assign them to mammals, birds and reptiles, respectively. In that case, the type of symbol may have a meaning, or the color may have a meaning. In any case, it is sufficient for the photographer to give appropriate meaning to the symbols and colors.

Next, the operation of the moving picture editing apparatus which analyzes the moving picture information photographed as described above and detects the frame image in which the specific symbol is superimposed will be described.

First, a recording medium such as a video tape on which moving image information is recorded is reproduced by the reproducing section 11 and output as a video signal. The reproduced video signal is quantized by the A / D conversion unit 12 in units of one frame, and is fixed as a digital image of, for example, 8 bits of pixels and stored in the image memory 13.

The character / graphic detection unit 14 analyzes the image stored in the image memory 13 and detects a predetermined character or symbol. Detecting specific characters and symbols can be done, for example, by
It is realized by a specific color extraction technology.

The character / graphic detection unit 14 detects a specific character or symbol by the following procedure. The color components can be represented by the three primary colors of red, green and blue. It is generally called R, G, B3 primary colors and expressed in a color space as shown in FIG.

In the R, G, B color space, the title (character or symbol) colors red, green, and blue exist in the spatial regions A, B, and C in FIG. 4, respectively. The spatial areas A, B, and C are defined within the range of the distance D from the points a, b, and c in FIG. 4, respectively.
For example, when only red color is desired to be extracted, pixels having color information existing in the spatial area A are converted into 1, and pixels having color information existing outside the spatial area A are converted into 0.

For example, when the specific symbol as shown in FIG. 3 is superimposed in red, the binarized image as shown in FIG. 4 can be obtained. Usually, it is sufficiently conceivable that a specific color is mixed in the image. In that case,
As shown in, the binarized image contains noise.

The character / graphic detection unit 14 further analyzes the connected area and detects a candidate area for the specific symbol. In the analysis of the connected area, the size of the connected area is measured by, for example, a labeling process, and the area whose size information falls within a predetermined range is detected as a candidate area for the specific symbol.

In the example shown in FIG. 4, areas 1, 2, 3 and 4 are determined to have appropriate size information, and are detected as candidate areas for the specific symbol. On the other hand, the area 5 is not detected as a candidate area because the size information no longer matches due to the influence of noise. Further, although the area 6 is noise, it is determined as a candidate area only by the size information.

As described above, noise may be detected as a candidate area only with the size information. Therefore,
The character / graphic detection unit 14 may recognize the cut-out connected region by a pattern recognition technique such as pattern matching to confirm that it is a specific symbol. In this case, dictionary data for pattern recognition is prepared in advance in the character / figure detecting section 14, or dictionary data is prepared in the memory 16 and loaded into the character / graphic detecting section 14 when necessary.

The character / figure detecting unit 14 detects the connected region cut out in order to absorb the fluctuation of the specific symbol when the superimposing image is created in the moving image recording apparatus 1 when performing the pattern recognition. Make corrections such as normalization.

The dictionary data prepared in advance on the moving image editing apparatus 10 side is associated with the specific symbol superimposed on the moving image recording apparatus 1. Video recording device 1
In the case of creating an image including a specific symbol, for example, a paper or the like on which an image corresponding to dictionary data prepared in advance is printed is used.

When it is determined that the recognition result is the specific symbol, the CPU 15 reads the time information of the frame. In this case, since a plurality of specific symbols are present in one screen, a plurality of recognition results can be obtained. For example, if a common recognition result has a predetermined probability or more, it is assumed that the specific symbol exists in the frame.

In the example shown in FIG. 5, detection of one of the five specific symbols has failed, but if the recognition results of four match, assuming that the threshold value is 70%, this frame is It can be a specific symbol frame.

Since the specific symbol is superimposed for about 1 second by the operation of the photographer, it is not recorded in only one frame but continuously in several tens of frames. Usually, it is thought that the actual shooting will start immediately after the superimpose is started, and immediately after it ends, investigate from when the specific symbol frame is detected until it is no longer detected,
The frame immediately after being no longer detected is detected as the representative frame image.

In this case, in consideration of detection error of the final frame of the specific symbol frame, even if a frame image several frames (for example, 2 to 3 frames) after it is no longer detected is detected as a representative frame image. good.

The CPU 15 determines the representative frame image based on the detection information indicating that the specific symbol frame from the character / graphics detecting section 14 is detected, and records the corresponding time information in the memory 16.

The automatic detection of the representative frame image as described above is performed once for the target moving image information, all the representative frame images in the moving image information are detected, and the corresponding time information is stored in the memory 16. Record.

In this case, in the case where the kind or color of the symbol is given meaning as described above, information such as a character code is attached to the time information for each color information or symbol for convenience, and the memory 16 is used. To record.

The CPU 15 refers to the time information recorded in the memory 16 and causes the reproducing section 11 to reproduce the corresponding representative frame image. The A / D conversion unit 12 quantizes the reproduced image into, for example, an image of 512 × 512 pixels 8 bits, and further reduces the representative frame image in the affine conversion unit 17 into, for example, an image of 128 × 128 pixels 8 bits. The reduced representative frame image is stored in the image memory 1
It is temporarily stored in 8.

The representative frame image stored in the image memory 18 is laid out as an image for display as shown in FIG. 6, and the image memory 1 is an image memory for display.
It is temporarily stored in 9.

The D / A conversion section 20 includes an image memory 19
The image stored in is converted into D / A and displayed on the display unit 21 which is a display monitor. The editing operator can efficiently grasp the content of the moving image information by looking at the list of the representative frame images.

Further, the CPU 15 can edit the moving image information based on the frame image detected by the character / graphics detecting section 14 in response to an editing instruction for the moving image information given from the outside.

When the type and color of the symbol for superimposing are to be given meanings as described above, each color information and information such as a character code attached to the symbol for the sake of convenience are the time. If you record it along with the information,
The representative frame image can be searched and displayed only for a specified color or symbol. For example, in the case of the example described in the present embodiment, classification display such as only mammals, only birds, and only reptiles is possible. With such a display, it is possible to further improve the editing efficiency.

In the above-described embodiment, the method of superimposing a specific character or symbol by using the title function and automatically detecting the character or symbol has been described. In a video camera, time information is information that a photographer can easily superimpose. This time information may be superimposed instead of the specific character or symbol described above. In this case, if the time information is recognized and is recorded in the image memory 7 along with the time information of the representative frame image, editing such as "editing only moving image information captured at what time" can be performed. More flexible editing is possible.

Character detection / recognition in such an image is similar to detection of a specific symbol or character as described above, a specific color detection technology such as color clustering, a connected area analysis technology such as labeling, pattern matching, etc. It is realized by the recognition technology.

In the present embodiment, the moving picture recording apparatus 1 is described as a home video camera, but a recording medium such as an optical disk or a magnetic disk is also conceivable, and video information on such a recording medium is used. Video signals from TVs, VCRs, etc. are also conceivable. An arbitrary title (image) can be superimposed on such a video signal in the same manner as described above.

Further, although the present embodiment has been described by using the specific symbols shown in FIG. 3, it goes without saying that the present invention is not limited to this. Further, the detection of the specific symbol is not limited to the labeling process, and there is also a method of analyzing the projection information on the X axis and the Y axis. As for the recognition of the specific symbol, not only the pattern matching but also the structure analysis can be considered.

In this way, the specific characters superimposed on the scene which the photographer thinks are characteristic,
Alternatively, a frame to which a symbol is added is automatically detected and displayed as a representative frame image, so that the outline of the moving image content can be grasped as intended by the photographer.

In recent years, even a home video camera has a superimposing function, and editing information can be attached to moving image information without requiring special equipment or standards.

This solves the problem that it takes too much time, such as high-speed search, and the point originally intended by the photographer, such as when scene switching or camerawork scenes are used as representative frame images, Since the problem that different scenes are mixed can be solved and the outline content of the moving image information can be grasped easily and accurately, the working efficiency of the moving image editing can be greatly improved.

[0066]

As described above, according to the present invention, the characteristic representative frame image in the moving image information can be detected and displayed as intended by the photographer. We solved the problem of taking too much time and solved the problem that scenes different from the point originally intended by the photographer were mixed, such as when switching scenes and scenes of camera work as representative images, Since the outline content of the moving image information can be accurately grasped, the work efficiency of moving image editing can be greatly improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of a moving image recording apparatus 1 according to an embodiment of the present invention.

FIG. 2 is a block diagram showing the configuration of a moving image editing apparatus 10 according to an embodiment of the present invention.

FIG. 3 is a diagram for explaining a process of superimposing a specific symbol example on moving image information.

FIG. 4 is a diagram for explaining a process of extracting a specific color in the R, G, B color space and the color space.

FIG. 5 is a diagram showing an example of a binarized image and an example of detecting a candidate area.

FIG. 6 is a diagram showing a display example of a representative frame image.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Video recording device, 2 ... Camera part, 3 ... Superimposing part, 4 ... Recording part, 5 ... A / D conversion part, 6 ... Binarization part, 7 ... Image memory, 8 ... Time information generation part, 9 ... Time information generation unit, 10 ... Video editing device, 11 ... Playback unit, 1
2 ... A / D converter, 13 ... Image memory, 14 ... Character / figure detector, 15 ... CPU, 16 ... Memory, 17 ... Affine converter, 18 ... Image memory, 19 ... Image memory, 20 ... D / A Conversion unit, 21 ... Display unit.

Claims (6)

[Claims] 1. A reproducing unit for reproducing moving image information in which a specific image is combined with a characteristic scene at an arbitrary timing and recorded, and moving image information reproduced by the reproducing unit is quantized for each frame image. Based on the frame image detected by the detecting unit, which detects the frame image added with the specific image by analyzing the frame image obtained by the quantizing unit A moving image editing apparatus, which detects a representative frame image that is a characteristic frame image of the moving image information. 2. The moving image editing apparatus according to claim 1, further comprising display means for displaying, as the representative frame image, a frame image separated from the frame image detected by the detecting means by a predetermined frame. . 3. The moving image editing apparatus according to claim 1, wherein the specific image is generated by a specific color, and the detecting unit detects the frame image by extracting the specific color. . 4. The moving image editing apparatus according to claim 1, wherein the specific image is obtained by painting the entire screen with a specific color, and the detecting unit detects a frame image by color histogram analysis. . 5. The moving image editing apparatus according to claim 1, wherein the specific image represents time. 6. The detecting means has a recognizing means for recognizing the specific image representing the time, and the information representing the time obtained by this recognition is attached to the detected frame image. The moving image editing apparatus according to item 1.