JPH06176149A - Moving image processor - Google Patents

Abstract

PURPOSE:To improve image editing efficiency by effectively retrieving in image. CONSTITUTION:This processor is provided with an image input part converting moving image information to the digital image of a frame unit, an image compressing part 12 compressing the digital image obtained by the image input part 10. an image recording part 13 for timewise-continuously and successively recording the image compressed by the image compressing part 12, an image extension part 14 extending the image recorded by the image recording part 13, a mobile object extracting part 15 extracting a mobile object in the image through the use of plural images recorded by the image recording part 13, a mobile object attribute adding part 16 adding attribute information concerning the image from which the mobile object is extracted by the mobile object extracting part 15 and a moving image retrieving control part 18 retrieving a pertient image by referring to attribute information added by the mobile object attribute adding part 16 in accordance with retrieving information specifying the image to be the object of retrieving.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image processing apparatus for image processing in a multimedia system, and particularly for editing including moving image retrieval.

[0002]

2. Description of the Related Art In recent years, image processing systems have begun to use still images such as color natural images for processing from simple images handled in documents. Furthermore, recently, real-time images (moving images) recorded on recording media such as videotapes and CDs (Compact Disks).
Conventionally, the editing work has been performed by an expert whose use in a broadcasting station is limited, but a system that can be handled by general users has been commercialized.

This is because the integration of personal computers and video equipment has progressed.
(Video Tape Recoder), CD player, LD (Laser
This is because products such as disk players can be purchased at a low price and can be handled with a simple operation.

Further, a technique for compressing / decompressing a still image and a moving image is JPEG (Joint Photographic Experts).
Group), MPEG (Motion Picture Experts Group)
The standardization is progressing in such a form, and a large amount of data such as a moving image can be handled by a personal computer.

In an editing system that handles such moving images,
There is a need for a technology that can search for a desired image or scene from a large volume of moving images with good operability. Conventionally, in the case of such a simple system using a personal computer, an operator reproduces a moving image and manually searches for the required image or scene while watching the moving image.

[0006]

As described above, in the conventional editing system for handling a moving image, since a video tape or the like is used as a recording medium, what is once recorded as image information is re-recorded. When attempting to use it, there are many cases in which a desired image or scene is searched while playing back and displaying a video tape or the like. This is a great labor for the operator, and if such a moving image is manually searched, it will hinder the reuse of the image information recorded on the video tape or the like.

As a means for automatically performing a moving image search to some extent in response to such a problem, scene switching in a moving image is extracted by using a predetermined algorithm, and this scene information is used to perform a search. The process of using it as a keyword has also been attempted, and it is effective depending on the application.

However, in the case of a moving image in which scene switching is frequently extracted, the number of candidates for scene information as a moving image search keyword becomes extremely large,
The effect of the search process is rather reduced.

In addition, in a moving image editing system mainly composed of a VTR, since the image source for editing is recorded on a video tape, it takes a lot of time to access it, and the editing efficiency is high. It will be very bad.

The present invention has been made in view of the above points, and an object thereof is to provide a moving image processing apparatus capable of improving image editing efficiency by effectively performing image retrieval. To do.

[0011]

According to the present invention, image input means for converting moving image information into a digital image in units of frames and digital images obtained by the image input means are sequentially and sequentially recorded. Image recording means, a moving body extracting means for extracting a moving body in the image using a plurality of digital images recorded in the image recording means, and a moving body extracted by the moving body extracting means And setting means for setting the object as a target object, and the processing relating to the moving image is performed by utilizing the setting contents of the setting means.

Further, the setting means is responsive to the attribute adding means for adding attribute information to the image in which the moving body is extracted by the moving body extracting means, and the search information for designating the image to be searched. An image search means for searching for a corresponding image by referring to the attribute information added by the attribute information adding means is provided.

Further, the moving body extraction means obtains a logical product between a plurality of subtracted images obtained by performing subtraction processing between the plurality of images recorded in the image recording means, and determines the logical product to the moving body. Corresponding area is extracted, the area is extracted while changing the time interval value of the image to be subjected to the subtraction process, and when the area of the area becomes maximum, the area moves in the image. The feature is that the moving body is extracted by discriminating the original shape of the body.

[0014]

According to this structure, the moving image to be processed is once recorded as a temporally continuous digital image in the image recording means, and then the time interval value t between the continuous images is recorded.
By variably controlling the moving object, which is a feature in the moving image, the moving object can be surely extracted, and this moving object is used as search information for moving image search, and attribute information is added to the extracted moving object. In addition, it is possible to perform a meaningful search that matches the feeling of the searcher and to effectively issue a search instruction.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the arrangement of a moving image processing apparatus according to this embodiment. As shown in FIG. 1, the moving image processing apparatus includes an image input unit 10, an image compression unit 12, an image recording unit 13, an image decompression unit 14, a moving body extraction unit 15, a moving body attribute addition unit 16, and an attribute input unit 17. , Video image search control unit 1
8, a search input unit 19, and an information display unit 20.

The image input unit 10 is intended for real-time video images (moving images) from TV cameras, video disc players, and VTRs. The image input unit 10 converts a moving image into digital image data in frame units.

The image compression section 12 sequentially compresses the temporally continuous frame images from the image input section 10 to an original image size of about 1/30, and records and saves them in the image recording section 14. As an image compression method, for example, JPEG (Joint
Photographic Experts Group) method, MPEG (Motion)
Picture Experts Group) method is used. The compression rate is controlled according to the input image.

The image recording section 13 is for recording the image compressed by the image compression section 12. The image recording unit 13 is composed of, for example, a hard disk device (hereinafter abbreviated as HD). As a hard disk device, a product having a large capacity of several hundred megabytes has recently come out, and when this is used, it is possible to record an image recorded at a video rate for 30 minutes or more. In addition, H
Not limited to D, it is of course possible to use another external storage device such as a magneto-optical disk device or a semiconductor storage device. The images input from the image input unit 10 and compressed by the image compression unit 12 are sequentially recorded in the image recording unit 13. Therefore,
The image recording unit 13 records a plurality of temporally consecutive images. The image decompression unit 14 decompresses the compressed image recorded in the image recording unit 13 and restores the image to the original size.

The moving body extracting unit 15 extracts a moving body in the image by subtraction processing between the images using the plurality of temporally continuous images expanded by the image expanding unit 14. Further, by extracting the moving body, it is possible to simultaneously extract the portion other than the moving body, that is, the background. At this time, the moving body extraction unit 15 can control the capturing time interval between the input continuous images.

The moving body attribute adding unit 16 is used by the moving body extraction unit 1
Attribute information is added to the image, the scene, or the background determined to have a moving object by 5, and the attribute information is recorded in the image recording unit 13. The moving body extraction unit 16 can automatically add attribute information to a predetermined attribute prepared in advance, or can add an attribute arbitrarily designated by the user. As attributes when setting attribute information automatically,
For example, there are screen position information, color information, background information, etc. of the moving body.

The attribute input unit 17 is a mobile unit attribute addition unit 16
The attribute information for the moving object or the background added to the image or the scene is input according to the designation from the user.

The moving image search control unit 18 searches the image recorded in the image recording unit 13 for the target image or scene based on the search information described later, and displays it on the information display unit 20 together with the attribute information. Let

The search input unit 19 is a moving image search control unit 18.
The search information which is the basis of the search target of the image or scene in is input according to the designation from the user. As the search information, for example, an instruction to specify a search target such as execution of a search for an image or a scene in which a moving body exists, or execution of a search for an image or a scene in which a blue moving body exists. Alternatively, the background portion in the image can be the search target. The information display unit 20 displays the image or scene searched by the moving image search control unit 18 based on the attribute information. next,
The operation of this embodiment will be described.

First, the image input unit 10 A / D-converts an image signal of a real-time image (moving image) from a TV camera or the like (not shown) and converts it into a digital multi-valued image in units of frames. The image compression unit 12 compresses the frame images sequentially input from the image input unit 10 by a predetermined method and sequentially records them in the image recording unit 13.

In this way, the real-time image is temporarily recorded as a digital image in the image recording unit 13, so that the moving image editing system searches for the necessary image and scene for the material moving image data. In this case, it becomes possible to freely handle the image later.

The compressed image data recorded in the image recording unit 13 is restored to the original image size by the image expansion unit 14 and output to the moving body extraction unit 15. The moving body extraction unit 15
Uses a plurality of temporally consecutive images recorded in the image recording unit 13 and performs a subtraction process between the images to extract the moving body. At this time, the time interval value t between images can be changed to an integral multiple by utilizing the fact that a temporally continuous image group is already recorded. That is, in the process of extracting the original moving body, it is possible to control the capturing time interval between the input continuous images. Details will be described later. The moving body extraction unit 15 outputs information of the image or scene from which the moving body is extracted, for example, the image number (corresponding to the frame number) to the moving body attribute adding unit 16.

The moving body attribute adding section 16 adds attribute information such as screen position information of the moving body in association with the image number of the image or scene in which the moving body exists. Here, the user can arbitrarily set the attribute information for the image or scene in which the moving body is present. When the user inputs a designation of attribute information to be added via the attribute input unit 17, the mobile unit attribute addition unit 16 adds the attribute information according to the designation. Therefore, it is possible to set the attribute information that matches the feeling of the search user at a fine level. In this case, the search user inputs the designation of the attribute information whenever the image or the scene including the moving body is extracted. The attribute information added in this way is rewritten in the image recording unit 13 together with the compressed image.

On the other hand, the moving image search control unit 18 wants to search only the search information from the search input unit 19, for example, only the image or the scene in which the moving body exists, or the image or the scene in which the blue moving body exists. By giving such an instruction, the image recording unit 13 searches for an image or scene that matches the instruction based on the attribute information.
When the matching image or scene can be searched, the moving image search control unit 18 displays the corresponding image or scene as a still image or a moving image together with the attribute information on the information display unit 20.

Next, the control of the capturing time interval between consecutive images when the moving body extraction unit 15 performs the subtraction processing will be described. 2 and 3 show how the capturing time interval between consecutive images is controlled by using three consecutive images.

First, FIG. 2 illustrates a subtraction process between images when the time interval value t between consecutive images is set to a small value ts, that is, when images are captured from the image recording unit 13 at short time intervals. ing.

The rectangular frame in FIG. 2 (a) shows the area of the moving body in each of the three continuous images (T-ts), (T), and (T + ts). The position of the moving body region in the image sequentially changes as time passes.

The moving body extraction unit 15 performs a subtraction process between the image (T-ts) and the image T. Further, the image obtained by the subtraction process is binarized with a predetermined threshold value. As a result, the image (T-t
s) -T is obtained. The image (T-ts) -T is an image in which the image areas of the moving body do not overlap between the two images, that is, the portions rs1 and rs2 remain.

Similarly, the moving body extraction unit 15 performs subtraction processing between the image (T) and the image (T + ts),
The obtained image is binarized. As a result,
An image (T)-(T + ts) as shown in (c) is obtained. The image (T)-(T + ts) is an image in which the image areas of the moving body do not overlap between the two images, that is, the portions rs3 and rs4 remain. As in FIG. 2B, the area where the two images overlap is not extracted.

The moving body extraction unit 15 uses the remaining area rs1,
For rs2, rs3, rs4, finally (rs1,
AND operation (AN of rs2) and (rs3, rs4)
D) is performed to extract the area rsa. This area rsa
The area of is rs1.

Next, FIG. 3 shows images when the time interval value t between consecutive images is set to a value tl which is larger than that in the example of FIG. 2, that is, when images are captured from the image recording unit 13 at long time intervals. The subtraction process between is explained.

The rectangular frame in FIG. 3 (a) indicates the region of the moving body in each of the three continuous images (T-tl), (T), and (T + tl). The position of the moving body region in the image sequentially changes as time passes. As shown in FIG. 3A, when the time interval is long, the regions of the moving body do not overlap each other.

The moving body extraction unit 15 performs a subtraction process between the image (T-tl) and the image T. Further, the image obtained by the subtraction process is binarized with a predetermined threshold value. As a result, the image (T-t
l) -T is obtained. The image (T-tl) -T is an image in which the image areas of the moving body do not overlap between the two images, that is, the portions rl1 and rl2 remain. rl
The areas 1 and rl2 are the same as the area of the moving object in the original image.

Similarly, the moving body extraction unit 15 performs subtraction processing between the image (T) and the image (T + tl),
The obtained image is binarized. As a result,
An image (T)-(T + tl) as shown in (c) is obtained. The image (T)-(T + tl) is an image in which the image areas of the moving body do not overlap between the two images, that is, the portions rl3 and rl4 remain. Similar to FIG. 3B, it is the same as the region of the moving body in the original image.

The moving body extraction unit 15 detects the remaining area rl1,
For rl2, rl3, rl4, finally (rl1,
rl2) and (rl3, rl4) AND operation (AN
D) is performed to extract the region rla. This region rla
The area of is rl1.

In the case of the example in FIG. 3, the time interval value tl is sufficiently large and the continuous images do not overlap each other, so that the region rla has the same area as the region of the moving body in the original image. .

Here, by setting the time interval value between consecutive images from the smaller value to the larger value one by one, that is, by setting in the direction of extending the time interval, the area of the finally extracted region is reduced. It can be seen that the number increases from 0 → rs1 → rl1. The area of this region will increase until it reaches the original area of the moving body, corresponding to the extension of the time interval value.

The relationship between the time interval and the area is shown in FIG. As shown in FIG. 4, after reaching the original area of the moving body, that is, after the interval tk, a minute increase / decrease is repeated based on the area H. This increase / decrease is due to the influence of the shooting environment of the input image. Considering this, when the increase rate of the area reaches the point tk where the value becomes 0, it is determined that the moving body is extracted. The image sk1 obtained when the time interval for calling the image from the image storage unit 13 is tk
(The same area as sl1) becomes the shape of the moving body. When the shape of the moving body is specified, the area other than the area corresponding to the moving body becomes the background area.

The moving body attribute adding section 16 adds attribute information to the moving body or the background thus detected. The moving image search control unit 18 responds to the instructed search information by
While referring to the attribute information, the corresponding image, that is, the image in which the moving body exists is searched.

In this way, the moving image is recorded in the image recording unit 13 in a compressed state as a temporally continuous digitized frame image, and the moving body extraction unit 15 is used.
By this, since the moving body is extracted while variably controlling the time interval value t between the images that are temporally consecutive, it is possible to accurately extract the moving body and the background. In addition, since the image to be processed is recorded in the image recording unit 13 in a compressed state, it is not necessary to process it in real time, and a practical recording time can be secured. Therefore, it is based on a personal computer. Even with the system, it is possible to practically process a moving image.

Further, by adding an arbitrary attribute to the moving object or background extracted in this way and using it as search information, a specified image in a moving image having a meaning suitable for the searcher's sense, Alternatively, since a scene or the like can be designated and automatically searched, work such as image editing can be efficiently and effectively performed. The present invention is not limited to the above embodiment.

For example, in the present embodiment, the search processing is performed by a single method, but in combination with a method of automatically extracting a scene change that has been tried conventionally and using this as search information. It is also possible to use. In this case, it is possible to determine that a moving object from scene to scene has no meaning when viewed from the content of the moving image, and perform a masking process or the like to detect the moving object in that portion.
Furthermore, although the present embodiment has been described assuming moving image editing, it is of course applicable to other uses, such as measurement of an extracted moving body.

[0047]

As described above, according to the present invention, a plurality of temporally consecutive images are used to extract the moving body and the background while changing the time interval of the images, and the moving body and the background are extracted. Since an arbitrary attribute is added to the search information and used as search information, a desired image can be searched by designating predetermined search information, so that the image search is effectively performed and the image editing efficiency is improved. It is possible.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of a moving image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram for explaining a state of moving object extraction when a time interval value between temporally consecutive images is small.

FIG. 3 is a diagram for explaining a state of moving body extraction when a time interval value between temporally consecutive images is large.

FIG. 4 is a diagram for explaining a change in a moving body extraction area when a time interval value in an image subtraction process is variable.

[Explanation of symbols]

10 ... Image input section, 12 ... Image compression section, 13 ... Image recording section, 14 ... Image decompression section, 15 ... Moving body extraction section, 16 ... Moving body attribute adding section, 17 ... Attribute input section, 18 ... Moving image search Control unit, 19 ... Search input unit, 20 ... Information display unit.

Claims (5)

[Claims] 1. An image input unit for converting moving image information into a digital image in units of frames; an image recording unit for sequentially and sequentially recording the digital images obtained by the image input unit; Moving body extracting means for extracting a moving body in an image using a plurality of digital images recorded in the image recording means, and setting means for setting the moving body extracted by the moving body extracting means as a specific object. And a moving image processing apparatus, wherein the moving image processing is performed by using the contents set by the setting unit. 2. The setting means, in accordance with the attribute adding means for adding attribute information to the image in which the moving body is extracted by the moving body extracting means, and the search information for designating an image to be searched, The moving image processing apparatus according to claim 1, further comprising: an image search unit that searches for a corresponding image by referring to the attribute information added by the attribute information adding unit. 3. The moving body extracting means obtains a logical product between a plurality of subtracted images obtained by performing subtraction processing between the plurality of images recorded in the image recording means, and determines the logical product to the moving body. Corresponding area is extracted, the area is extracted while changing the time interval value of the image to be subjected to the subtraction process, and when the area of the area becomes maximum, the area moves in the image. The moving image processing apparatus according to claim 1, wherein the moving body is extracted by determining that the body has the original shape. 4. The moving image according to claim 2, wherein the attribute adding unit adds the attribute information according to arbitrarily specified attribute contents when the moving body is extracted by the moving body extracting unit. Image processing device. 5. The attribute adding means adds attribute information of color information indicating a specific color to the moving body extracted by the moving body extracting means.
The moving image processing apparatus described.