JPH06251147A - Video feature processing method - Google Patents

Abstract

PURPOSE:To provide a method for fetching automatically a feature quantity from an image, classifying automatically the image, and also, describing the whole image by the feature quantity fetched from the image. CONSTITUTION:First of all, each frame RCB data of an original image is subjected to color conversion, and set to a value of at least a hue. By measuring a hue histogram from this hue value, and selecting a hue representative value of each frame from its hue histogram, a feature quantity is fetched automatically. In a color feature space in which this hue representative value is the feature quantity, its hue representative value is expanded, and each frame is shown as a position in the space, respectively. Subsequently, by calculating a distance between each frame, clustering each one, and classifying it at every cluster, a lump of each frame, that is, a video scene is classified automatically. Also, the whole image is described by a clustering condition parameter in this case.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for classifying video, and more particularly to a video feature processing method for classifying video using color features.

[0002]

2. Description of the Related Art Video has a large amount of information and is ambiguous in terms of meaning. In order to handle videos conveniently, it is necessary to classify video scenes that take out easy-to-understand features from videos and combine similar videos into one. BACKGROUND ART Research has been conventionally performed to extract one frame from a video and extract the feature amount of one frame to describe the content, or to automatically classify the video based on the described content.

For example, so far, (1) Yamane, GONG, Sato, Sakauchi, "A Study on Video Scene Detection System Using Video Scene Description Language", Fall Society of Shinkai, D-282,1992 (2). Sato, Yamane, GONG, Sakauchi "Motion picture scene classification using scene description language" Fall Society of the SIJ, D-283.
1992 has been announced.

[0004]

However, the extraction of the feature quantity of the image by the above-mentioned conventional technique requires a lot of manpower and it is difficult to describe the entire image. It was

The present invention has been made to solve the above problems, and an object thereof is to provide a processing method for automatically extracting a feature amount from a video and automatically classifying the video. . Another object of the present invention is to provide a processing method for describing the entire video by the feature amount extracted from the video.

[0006]

In order to achieve the above object, in the image feature processing method of the present invention, first, a hue of a video image consisting of one frame or several frames of a color video is Saturation, lightness, or a combination thereof is converted into information, then a histogram is measured from the information, then n extreme values are extracted from the histogram as representative values, and then the By calculating the distance between the image clusters from the cluster position of the image in the n-dimensional feature space having the representative value as the feature quantity, and then clustering the image clusters within the distance within the constant range, the images are clustered. The feature is that the blocks are grouped together into a group of images.

Further, the present invention is characterized in that the entire video is described by the clustering condition parameter of the video cluster obtained as described above.

[0008]

According to the image feature processing method of the present invention, a histogram is measured from information including any one of hue, saturation, and lightness obtained by converting a lump of an image, or a combination thereof.
The extreme value is extracted from the histogram as a representative value and used as a feature amount. As a result, the feature amount can be automatically extracted. In addition, the cluster of images is developed in a feature space having the number of feature quantities as a dimension, the distance between clusters of multiple images is calculated from the position, and clusters of the images within a certain distance are clustered. By doing so, the blocks of those images can be automatically classified. At the same time, the whole image can be described by the clustering condition parameter.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Specific embodiments of the present invention will be described in detail below with reference to the drawings.

In the example described below, the MTM conversion "Miyahara, Yoshida" color data (R, G, B) ⇔ (H, V, C) mathematical conversion method proposed by Nagaoka University of Technology for color conversion is used. , "TV Journal Vol. 43 No. 10 (198
9) ”, but other methods such as HSI, HSV, HSL for converting to hue, saturation, and lightness may be used.
In the example described below, the representative value is determined by using the hue, but the representative value may be determined by using the saturation and the brightness together.

FIG. 1 is a flow chart showing an embodiment of the present invention.

First, the outline of the processing procedure will be described. First, each frame RGB data of the target video (original image) is color-converted to have values of hue, saturation, and brightness (however, it is sufficient to convert the values used below, that is, at least only the hue value in this example). Is). Of these, the hue histogram is measured from the hue value, and the hue representative value of each frame is selected from the hue histogram and determined as the feature amount. The hue representative value is developed in a color feature space having the hue representative value as a feature amount, and each frame is represented as a position in the space. Next, the distance between each frame is calculated, the close ones are clustered, and the clusters, that is, the video scenes are classified by each cluster.

The details of each of the above processes will be described below.

<Color conversion> The image information represented in the computer is RG because of the relation of the frame buffer of the computer.
Often expressed in B. From this RGB information,
Converts color information into information about hue, saturation, and lightness that is close to human perception. Here, the MTM conversion proposed by Nagaoka University of Technology is used as the color conversion method, but as described above, this converts RGB information such as HSI, HSV, and HSL into hue, saturation, and lightness. You may use the method of. Further, when the image information is expressed by hue, saturation, and brightness, it may be used as it is.

<Measurement of Hue Histogram> Next, a hue histogram is calculated for each frame using the converted hue value. An example of the hue histogram is shown in FIG. The horizontal axis represents the hue value (Hue). The hue is expressed in a ring shape as shown in FIG. 3, but here it is expressed by a one-dimensional straight line. The vertical axis represents the number of pixels taking each hue value (Pixel
s). In this embodiment, since the original image of 256 × 240 pixels is used, the total number of pixels is 61440 pixels and is constant.

<Selection of Representative Value> Next, a hue representative value characterizing each frame is obtained from the hue histogram. In this embodiment, the maximum value of the hue histogram is used as the hue representative value of the frame, that is, the feature amount. At this time, an arbitrary number of representative values may be selected. In this embodiment, two of the maximum values having the largest number of pixels are set as the representative value of the frame.
Although the maximum value is used here, the maximum value and the minimum value of the hue histogram can also be used.

A method for obtaining the maximum value of the hue histogram will be described in detail.

An example of the hue histogram is shown in FIG. As it is, the hue histogram has a slight change in each hue value and has many maximum values. Therefore, in order to eliminate the minute change in the hue histogram and detect only the large change (first peak, second peak), the following processing is performed.

First, the hue histogram is converted into a hue cumulative histogram. FIG. 5 shows an example of the cumulative histogram. Next, when a certain hue width d (u) is taken from the hue value u, the hue width d (u) required to have a constant number of pixels T
To calculate. The relational expression is shown below.

[0020]

[Equation 1]

T: constant (constant number of pixels) hist (h): hue histogram d (u): hue width required to reach a certain number of pixels In the present embodiment, the constant number of pixels T is the total number of pixels (61,440 pixels) An example will be described in which measurement is performed at 6144 pixels, which is 10% of. This may be an appropriate value as long as there is no minute change in the extreme value. Next, the relationship between the hue value and the hue width will be described from this relational expression. In the hue histogram, the width d (u) is small when the hue value has a large number of pixels, and when the hue value has a small number of pixels in the hue histogram, the width d (u) is small.
(U) becomes large.

FIG. 6 shows the above hue value (Hue) u and width (W).
The graph of idth) d (u) is shown. The hue values, which are the two maximum values having large changes in FIG. 4, are represented as the minimum values in FIG. In this embodiment, the smallest of the minimum values in FIG. 6 is used as the representative value. This process is performed on the hue histogram of each frame to obtain a representative value. Also,
For this, an arbitrary number of n representative values may be selected depending on the image.

<Expansion into Feature Space> Next, the representative value of each frame is expanded into the color feature space. Here, a case where two hue values are used as representative values will be described. An example of the color feature space is shown in FIG. Of the two hue values used as the representative value, the larger value is plotted on the horizontal axis and the smaller value is plotted on the vertical axis, and they are positioned in the color feature space as the representative value of each frame. In this case, the horizontal axis and the vertical axis are determined regardless of the magnitude of the width d of the representative hue value. That is, the feature amount of each frame is represented as a position only in the shaded portion in FIG. 7. For example, if the hue values of the representative values are "green" and "purple", they are added to the position A in FIG.

In FIG. 8, 158 seconds (4740 frames), 4
The result of developing the representative value of the image consisting of 6 shots in the color feature space is shown. Of course, a four-dimensional space including the width d of each representative hue value may be created and positioned in that space. When n hue values are used as representative values, an n-dimensional space can be created and positioned in the space.
Further, when n hue values and respective widths d are used as representative values, a 2n-dimensional space can be created and positioned in the space.

<Clustering in Feature Space> Next,
By using the positions of the respective frames distributed in the color feature space described above, similar ones in the space are clustered. As a clustering method, a method of calculating mutual distances from positions of images in an n-dimensional color feature space and clustering those having similar distances, n-dimensional K-means clustering, horizontal axis, vertical axis Cut out by projection of
You can use connections such as morphology. FIG. 9 shows the result of classifying the video scenes by clustering from the distribution of the color feature space. This video consists of a ship scene and a seabed diving scene. The part A is mainly a shot of diving under the sea, and some shots on the ship are included. Part B consists of shots on the ship.

As described above, image classification can be performed by clustering each frame using the position in the color feature space. Further, the entire image can be described by the clustering condition parameter. That is,
In the example of FIG. 9, each scene can be described by a combination of hue values of each dimension.
The present invention can be applied in various ways in accordance with its gist and can take various embodiments.

[0027]

As is apparent from the above description, according to the image feature processing method of the present invention, it is possible to automatically classify image scenes etc. by paying attention to the color feature of the image. For example, if similar images can be put together, the storage efficiency can be improved. Also, when considering the reuse of images, several similar images can be managed collectively.

[Brief description of drawings]

FIG. 1 is a flowchart showing an embodiment of the present invention.

FIG. 2 is a diagram showing an example of a hue histogram in the above embodiment.

FIG. 3 is an explanatory diagram of a hue circle for explaining hue values in the above embodiment.

FIG. 4 is an explanatory diagram of an example of a hue histogram in the above embodiment and a maximum value therein.

FIG. 5 is a diagram showing an example of a cumulative histogram of hues used to calculate a hue width in the above embodiment.

FIG. 6 is a graph of a hue value and a hue width in the above embodiment.

FIG. 7 is an explanatory diagram of a two-dimensional color feature space used in the above embodiment.

FIG. 8 is a diagram showing a result of expanding the image feature amount in the color feature space in the above embodiment.

9 is a diagram showing the result of classifying video scenes by performing clustering from the distribution of the color feature space in FIG.

Claims (2)

[Claims] 1. First, for a block of a color image consisting of one frame or several frames, the image is converted into information including any one of hue, saturation, lightness, or a combination thereof, and then the A histogram is measured from the information, then n extreme values are extracted from the histogram as a representative value, and then the image is clustered from the cluster position of the image in an n-dimensional feature space having the representative value as a feature amount. An image feature processing method characterized in that the distances between the images are calculated, and then the clusters of the videos within a certain distance are clustered to collectively collect the clusters of the videos into a batch of the videos. 2. The image feature processing method according to claim 1, wherein the entire image is described by a clustering condition parameter of the obtained image aggregation.