JP3321402B2 - Video data editing method and device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture data editing technique, and more particularly to a technique which is effective when applied to the synthesis of a specific image among a plurality of moving picture data.

[0002]

2. Description of the Related Art In the process of editing image data, it may be necessary to perform a combining operation such as cutting out one specific image of two pieces of image data and replacing it with a specific image of the other image.

[0003]

With still image data, the amount of data is small, so that the user can easily extract the data, and a specific image on the screen can be manually synthesized from a plurality of material image data. However, in the case of moving image data, since the image itself has a motion, the data amount is large (for example, 30 frames / second), and it is practically impossible to perform the sequential composition manually.

[0004] Although image synthesis at the still image level is performed in some systems, synthesis at the moving image level involves only analog data processing, and there are few synthesis methods using digitized moving image data. .

[0005] For this reason, despite the high demand for working on the moving image data itself, most of the currently used moving images themselves are used at present, and re-editing once converted to moving image data is a fact. Not done on.

For example, according to the technique disclosed in Japanese Patent Application Laid-Open No. 9-35082, a predetermined number of pixels are demarcated at the boundary portion of an image in which a plurality of objects are present so that the image is not deformed as a whole. Although a technique for performing the processing is disclosed, there is no reference to cutout of a specific image from moving image data.

In the technique disclosed in Japanese Patent Application Laid-Open No. 5-110983, in the editing of moving picture data with sound, the audio information is Fourier-transformed and the spectrum is displayed for each frame, so that the moving picture data can be manually edited for each frame. Only technology that enables audio data change processing is disclosed,
There is no reference to the extraction or synthesis of specific image data between a plurality of moving image data.

An object of the present invention is to provide a moving image data editing technique capable of efficiently synthesizing a plurality of moving image data having an arbitrary length.

Another object of the present invention is to provide a moving image data editing technique capable of efficiently and accurately synthesizing a plurality of moving image data having different spatial and temporal specifications.

Another object of the present invention is to provide a moving image data editing technique capable of generating various moving image data usable for any purpose from a plurality of moving image data having an arbitrary length.

[0011]

According to the present invention, a still image is generated from a synthesis range of a designated arbitrary length of each of a plurality of moving image data, and the first one of the still images specifies a synthesis target. Designation of an image is performed based on color boundary data and the like. After automatically discriminating a specific image in all the still images thereafter, if necessary, the number of still images of the synthesis source is thinned out in the time axis direction or copied to match the number of still images of the synthesis destination. Adjust in the time axis direction. Further, a compositing process for generating a new still image in which the specific image of the compositing source is replaced by the specific image of the compositing source between the composing source / compositing still images corresponding in the time axis direction is performed. The still image is converted into moving image data of a desired format and compression method.

When replacing a specific image,
When the size of the specific image at the combining destination is different, an operation of adjusting the size of the specific image at the combining destination is performed by enlarging / reducing the specific image at the combining source.

The present invention also provides a moving picture data editing apparatus including a medium to which moving picture data is input / output, a processing unit for processing the moving picture data, and a user interface for allowing a user to control the processing unit. The processing unit is provided with control logic and a program for performing the above-described series of operations for synthesizing moving image data.

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a flowchart showing an example of the operation of a moving image data editing method and apparatus according to an embodiment of the present invention. FIG. 2 shows an example of the configuration of the moving image data editing apparatus according to the embodiment. FIG. FIGS. 3, 4, 5, 6, 7, 8, and 9 are conceptual diagrams illustrating an example of the operation of the moving image data editing method and apparatus according to the present embodiment.

First, an example of the configuration of the moving image data editing apparatus according to the present embodiment will be described with reference to FIG.

A moving image data editing apparatus 10 according to the present embodiment
A central processing unit 12 for controlling the entire system, a main memory 13 for storing a program for controlling the central processing unit 12, a storage device 14, and an operation I / F unit 1 via a system bus 11.
5. A computer system to which the data output interface 16 and the like are connected.

The storage device 14 is, for example, a magnetic disk device (HDD), a magneto-optical disk device (MO), an optical disk device (OD), a digital video disk (DVD).
Etc., or a DRAM, SRAM, F
This is a storage device provided as a storage medium with a memory card or the like made of an arbitrary semiconductor memory such as a RAM (ferroelectric memory) and a batch erase EEPROM. Note that the storage device 14 is not limited to these storage media, but may be a communication medium through which moving image data is input and output.

The storage device 14 of the present embodiment includes, for example, a desired standard such as MPEG, motion JPEG, and AVI, an industry standard, a specific moving image data format used by specific moving image data processing software, and the like. The moving image data before and after the editing, such as the combining source moving image data file 21, the combining destination moving image data file 22, and the combined moving image data file 23, which are configured in an arbitrary moving image data format, are stored.

The operation I / F unit 15 comprises an arbitrary information input device such as a keyboard and a mouse. The data output interface 16 includes a display 16a, a speaker 16b, and the like for visualizing and audible information such as moving image data before and after editing as necessary and presenting the information to the user.

The main memory 13 includes a resident operating system 13a (OS), a moving image editing program 13b which operates under the operating system 13a, and executes a moving image data editing process described later, and a moving image editing program 13b.
Buffer 13c and buffer 13d used in
Is provided to control the central processing unit 12 to perform editing work of moving image data as described later.

Hereinafter, an example of the operation of the moving picture data editing method and apparatus of the present embodiment will be described with reference to the flowchart of FIG.

First, the moving picture editing program 13b is started, and the moving picture data file 2
1. Read out the synthesis destination moving image data file 22, output it to the display 16a, and present it to the user, and specify the synthesis range for each. In this operation, for example, during reproduction at normal speed, fast forward, or the like, the user sets the start timing and end timing of the target synthesis range by operating the I / F.
This is performed by designating from the unit 15.

Next, the moving image data in the specified synthesis range (playback time range) in each of the source video data file 21 and the destination video data file 22 is converted into a plurality of still images 21a and a plurality of still images 22a. Then, the data is developed in the buffers 13c and 13d. As a result, in each of the buffers 13c and 13d, as shown in FIG. 3, a plurality of still images 21a and a plurality of still images 22a are displayed during the playback time from the start time t to tn of the designated synthesis range. Are developed in chronological order by the number corresponding to the length of.

Next, the still image 21a (still image 22a) (frame X) at the start time t of the synthesis source (synthesis destination) is
The still image 21a (still image 22a) is output to the display 16a, and is provided to the user by a method such as color boundary designation which will be described later.
The specific image 31 (specific image 32) is specified (step 101, step 102).

After that, the moving picture editing program 13b takes out the frames X + i of t + i following the start time t one after another, and until the frame Xn at the synthesizing end time tn.
In 1a (still image 22a), the specific image 31 (specific image 32) is automatically specified using the specified color boundary data (step 103, step 104).

At this time, as illustrated in FIG. 4, in the still image 21a of the synthesis source, the background portion 31a other than the specified specific image 31 has colors (unused colors) that are not used in the moving image data. If there is no color, use the color C with the lowest appearance frequency.
Is converted to an approximate color) and used as the background color.

In addition, in the still image 22a to be synthesized, the internal region (the portion other than the background portion 32a) of the specified specific image 32 is set to a color that is not used in the moving image data (if there is no unused color). , Use the color C that has the lowest appearance frequency. In this case, however, C in the original data is converted to an approximate color).

Here, an example of the above-described color boundary designation by the user will be described. In natural moving images, the border between the image and the background often has an extremely different color. A method of using this to specify the boundary of an object by color is used.

Normally, still image data extracted from moving image data often uses full color (16 million colors). Therefore, as illustrated in FIG. 5, the color of the extracted still image data 40 is reduced to a certain number of colors or less to flatten the color, and a specific object stands out from the background. For example, in the example of FIG. 5, when a dark color object image 40a and a light color object image 40b exist in the background 40c of the still image data 40, the color reduction operation performs Object image 4 with dark color inside
Still image data 41 in which only 0a is visible can be obtained.

The flattened still image data 41
Is presented to the user, and the color boundary (inner color A, outer color B) around the dark color object image 40a is specified to specify the image of the target object image 40a.

However, since the color around one image is rarely one color, a plurality of color boundaries exist. In this case, the combination of the color boundaries is usually plural. For example, in the example of FIG. 6, the background 52 and the background 5 having different colors from the object image 51 of the still image data 50 are displayed.
The color boundary of No. 3 is a combination of (internal A, external B) and (internal A, external C).

On the other hand, when the length (reproduction time) of the data in the synthesis range is different between the moving image data of the synthesis source and the synthesis destination, it is necessary to match on the time axis. Therefore, in the case of the present embodiment, when the lengths of the data of the combination source and the combination destination are different, copying or thinning of the still image 21a (frame) generated from the combination source moving image data file 21 in the time axis direction is performed. By the method described above, the matching in the time axis direction is realized.

In this case, the multiple Mt of the frame is obtained by (the number of frames to be combined / the number of frames to be combined). For example, according to the type of image and the request from the user, for example, copying in the time axis direction is performed. And thinning out all frames equally according to the multiple Mt,
There are two types of cases where the magnification of copying or thinning is increased in a time range where movement is small, and the magnification of copying or thinning is reduced in a time range where movement is large. Normally, the influence of frame copying or thinning is more noticeable in the time range in which the movement is greater, and in the latter case, there is an effect that the change in the display state due to copying or thinning can be reduced.

For example, in the example of FIG. 7, the synthesis source data is shorter than the synthesis destination data. In this case, the number of frames of the synthesis source data is synthesized by copying (interpolating) in the time axis direction. The number of frames (reproduction time) of the preceding data is matched (step 105 to step 10).
8).

On the other hand, similarly to the time axis direction, it is generally necessary to adjust the size in the space axis direction. That is, even if the size of the image itself is the same, the specific image 31
It is rare that the image 32 and the specific image 32 to be combined have the same size. In this case, it is necessary to adjust the size of the specific image 32 of the synthesis destination by enlarging / reducing the specific image 31 of the synthesis source.

This enlargement / reduction processing is performed for all the still images 21a in the synthesis range of the synthesis source moving image data file 21 developed in the buffer 13c (steps 109 and 110).

In this case, after all of the moving image data of the combining source and the combining destination are converted into still images as described above, still images on the same time axis are compared, and portions other than the background color are replaced with the X-axis (or Y-axis).
Enlargement / reduction is performed for each line (pixel column) of (axis).

In this case, (Y size of the image of the synthesis destination data on an arbitrary X axis) / (arbitrary X of the image of the synthesis source data)
(Y size on the axis), the enlargement / reduction magnification Ms can be obtained. Also in this case, depending on the type of image and the request of the user, a method of multiplying all points (pixels) in an arbitrary X axis (Y axis) by the magnification Ms and a method in a place close to the color boundary in the object A method is conceivable in which the magnification Ms is reduced and is increased at a position distant from the color boundary.

For example, in the example of FIG. 8, the specific image 31 in the still image 21a as the synthesis source data is smaller than the specific image 32 in the still image 22a as the synthesis destination data. Accordingly, the dimension Yf in the Y-axis direction at the point X in the X-axis direction of the specific image 31 is enlarged so as to match the dimension Yt in the Y-axis direction at the point X in the X-axis direction of the specific image 32.

After the temporal / spatial size matching as described above is performed, the moving image data is synthesized.

At this point, the plurality of still images 21a and still images 22a of the combining source and the combining destination developed in each of the buffers 13c and 13d are:
In the former, the background other than the specific image 31 and in the latter, the inside of the specific image 32 is filled with the specific color C that is not used elsewhere, as described above, so that they are at the same position in the time axis direction. By comparing the still image 21a and the still image 22a, a new still image 23a can be created from a portion other than the background color (specific color C).

At this time, in most cases, the positions of the specific image 31 and the specific image 32 in the data of the synthesis source and the synthesis destination are shifted from each other in the screen. Since the size has already been made the same by the above-described enlargement / reduction processing, processing can be performed only by aligning the two at the time of synthesis.

This operation is performed for all the still images 21 in the time axis direction.
a and a still image 22a (step 1
11, step 112).

Then, the plurality of new time-series still images 23a thus obtained are converted (compressed) again into moving image data of a desired format and compression method, and then output to the synthesized moving image data file 23 (step 113). ).

If necessary, the synthesized moving image data file 23 is directly output to the display 16a to check the editing result, or transmitted to another system via a desired communication medium such as a communication line. It is also possible.

As described above, according to the moving image data editing method and apparatus of the present embodiment, moving image data is obtained by combining specific images in a plurality of moving image data having an arbitrary length to obtain new moving image data. Can be accurately performed in a short time with a minimum of user intervention, and a plurality of moving image data having an arbitrary length can be efficiently synthesized.

Further, even if the two moving image data have different temporal and spatial sizes, the matching is automatically performed. Therefore, a plurality of moving image data having different spatial and temporal specifications can be efficiently and accurately recorded. Can be synthesized.

As a result, it is possible to easily generate moving image data in which a moving specific image in two moving image data having an arbitrary length is replaced, and to use various moving image data that can be used for any purpose from a plurality of moving image data. Can be generated.

Although the invention made by the present inventor has been specifically described based on the embodiments, the present invention is not limited to the above-described embodiments, and can be variously modified without departing from the gist thereof. Needless to say, there is.

For example, the device configuration illustrated in the above embodiment is an example, and any configuration can be adopted as long as the moving image data editing method of the present invention can be realized.

[0052]

According to the moving picture data editing method and apparatus of the present invention, it is possible to efficiently combine a plurality of moving picture data having an arbitrary length.

According to the moving picture data editing method and apparatus of the present invention, a plurality of moving picture data having different spatial and temporal specifications can be efficiently and accurately synthesized.
The effect is obtained.

Further, according to the moving image data editing method and apparatus of the present invention, it is possible to generate various moving image data usable for any purpose from a plurality of moving image data having an arbitrary length. .

[Brief description of the drawings]

FIG. 1 is a flowchart illustrating an example of the operation of a moving image data editing method and apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating an example of a configuration of a moving image data editing device according to an embodiment of the present invention.

FIG. 3 is a conceptual diagram showing an example of the operation of a moving image data editing method and apparatus according to an embodiment of the present invention.

FIG. 4 is a conceptual diagram showing an example of the operation of a moving image data editing method and apparatus according to an embodiment of the present invention.

FIG. 5 is a conceptual diagram showing an example of the operation of a moving image data editing method and apparatus according to an embodiment of the present invention.

FIG. 6 is a conceptual diagram showing an example of the operation of a moving image data editing method and apparatus according to an embodiment of the present invention.

FIG. 7 is a conceptual diagram showing an example of the operation of a moving image data editing method and apparatus according to an embodiment of the present invention.

FIG. 8 is a conceptual diagram showing an example of the operation of a moving image data editing method and apparatus according to an embodiment of the present invention.

FIG. 9 is a conceptual diagram showing an example of the operation of a moving image data editing method and apparatus according to an embodiment of the present invention.

[Explanation of symbols]

10 moving image data editing device, 11 system bus, 12
... Central processing unit, 13 ... Main memory, 13a ... Operating system, 13b ... Video editing program, 13c ...
Buffer, 13d buffer, 14 storage device, 15 ...
Operation I / F section, 16 data output interface, 1
6a: display, 16b: speaker, 21: source moving image data file (first moving image data), 21a: still image (first still image), 22: destination moving image data file (second moving image data), 22a: Still image (second still image), 23: Combined moving image data file, 23a: Still image (third still image), 31: Specific image (first specific image), 31a: Background portion, 32 ... Specific image (second specific image), 32a ... Background part, 40 ... Still image data, 40
a: object image, 40b: object image, 40c: background, 41
... still image data, 50 ... still image data, 51 ... object image, 52 ... background, 53 ... background.

Claims (7)

(57) [Claims] 1. A step of specifying a temporal synthesis range in first and second moving image data of a synthesis source and a synthesis destination; and a first and a second at the start of synthesis of each of the first and second moving image data A step of designating the first and second specific images to be combined in the second still image by a desired identification method; and in each of the first and second moving image data, Automatically generating the first and second still images and specifying the first and second specific images by the identification method; and In the first specific image in the first still image, the second
Generating a plurality of new third still images by replacing the second specific image in each of the second still images of the moving image data; and Generating moving image data in a format or a compression method , wherein the synthesis range in the first and second moving image data is included.
When the time lengths of the first and second moving images differ,
A plurality of the first static images generated from the synthesis range of data.
By thinning or copying in the time axis direction of the still image,
Generated from the synthesis range of the second moving image data of the synthesis destination
Moving image data editing method, wherein the moving image data is matched with the number of second still images to be processed. 2. The moving image data editing method according to claim 1 , wherein in the thinning-out or copying of the first still image in the time axis direction, (the number of the second still images / the first first still images) A method of calculating the magnification obtained by the number of still images) and performing thinning or copying of the first still image at a constant magnification, or increasing the magnification for a portion of the first moving image data with little motion. A method of editing moving image data, characterized by using any one of the following: a method of reducing a magnification for a portion having a lot of movement. 3. The moving image data editing method according to claim 1, wherein the size of the first specific image in the first still image as the synthesis source is changed to the size in the second still image as the synthesis destination. after enlarged or reduced to fit the size of the second specific image, expansion of the second conversion example placed a specific image, the synthesis source of the first of said first specific image in the still image
A large or reduction ratio is defined between the outline area of the first specific image and the outline area.
A moving image data editing method characterized in that the moving image data is changed depending on a partial area . 4. The moving image data editing method according to claim 1 , wherein in the identification method, the first and second specific images are identified by discriminating a color boundary in the first and second still images. A moving image data editing method characterized by being performed. 5. The moving image data editing method according to claim 4 , wherein, in the first still image to be synthesized, a background area of the first specific image is not used in the first still image. In the second still image to be filled with the color or the least frequently used color and synthesized, the internal area of the second specific image is set to a color not used in the second still image, or A moving image data editing method characterized by filling with a color with the lowest frequency. 6. The moving image data editing method according to claim 4 , wherein, when the first and second specific images are designated, the color data of the first and second still images is reduced to an arbitrary number of colors or less. A moving image data editing method characterized by performing color flattening. 7. A moving image data editing apparatus including a medium to which moving image data is input / output, a processing unit for processing the moving image data, and a user interface for a user to control the processing unit, The processing unit includes a first synthesis source and a first synthesis destination obtained from the input / output medium.
Specifying a temporal synthesis range in the first and second moving image data; and a first and second still image at the start of synthesis of each of the first and second moving image data via the user interface. Specifying the first and second specific images to be combined by a desired identification method; and in each of the first and second moving image data, the first and second moving images after the synthesis start time Automatically generating the still image and specifying the first and second specific images by the identification method; and the first still image of the first moving image data in the synthesis range. In the first specific image in the image, the second in the second still image of the second moving image data
Generating a plurality of new third still images by replacing the specific image of the above, generating moving image data of an arbitrary format or compression method from the plurality of third still images and outputting the moving image data to the input / output medium Performing the synthesizing range in the first and second moving image data.
When the time lengths of the first and second moving images differ,
A plurality of the first static images generated from the synthesis range of data.
By thinning or copying in the time axis direction of the still image,
Generated from the synthesis range of the second moving image data of the synthesis destination
And a control logic for executing an operation of matching the number of second still images to be performed.