JP2007188176A - Method and device for specifying content information, program, and computer readable recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To focus on the distribution of pixel values, which are included in content, inside a color space for uniquely identifying the content. <P>SOLUTION: An inputted image is divided into a plurality of subsections, and in the divided subsections, color information included in the respective subsections is mapped in the color space. Based on the mapping result, clustering is carried out in the color space. On each cluster inside the color space, representative information is found, a feature quantity of the subsection is found from the representative information of the respective clusters, and an information string for the image as a whole is formed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a content information specifying method and apparatus, a program, and a computer-readable recording medium, and more particularly, a local information for extracting information necessary for uniquely specifying a content such as an image or a video. The present invention relates to a content information specifying method and apparatus using a color space distribution, a program, and a computer-readable recording medium.

  Specifically, it is intended to prevent unauthorized use of content distributed on the Internet or outside, search content, and apply to a reference service using content as a key.

  Conventional techniques for specifying content include a method of embedding information in the content itself using a digital watermark technique and a method of using the feature amount of the content itself.

  First, in the digital watermark technique, identification information such as an ID is embedded in the content as a digital watermark in advance, and the watermark information is detected from the content to be identified, so that it is possible to specify whether the content is predetermined content (for example, Non-Patent Document 1).

On the other hand, in the latter method using the feature amount of the content, the content is divided into small regions of appropriate sizes, and if the vectors having the feature amount (for example, average value) of each small region are similar to each other, This is a technique that considers content to be similar (see, for example, Non-Patent Document 2).
http://www.jpo.go.jp/shiryou/s_sonota/hyoujun_gijutsu/denshi_sukashi/2_b_1.htm Jia Li, James Z Wang, Gio Wiederhold, “IRM: Ingrated Region Matching for Image Retrieval”, Proceedings of the eighth ACM international conference on Multimedia, pp.147-156 (2000)

  However, the above-described conventional methods have the following problems.

  The digital watermark technique has a problem that it takes time and effort because the identification information is superimposed on the content itself, resulting in quality degradation and the work of embedding the watermark in advance.

  On the other hand, the content feature quantity extraction technology is aimed at extracting feature quantities for searching for similar contents, and since there may be a plurality of contents having the same feature quantities, it is not suitable for detecting only the same contents. is there. One of the causes of the above problem is that the feature amount of the content is represented by an average value of a small area. That is, when there are a plurality of types of color areas (colors of clearly different object areas) for one small area, the average value, which is a value representing the small area, is the color of any area included in the small area. Is also not representative. As a solution to this, a means to reduce the number of content divisions can be considered, but object regions having different colors do not always match the shape of the division, so that the amount of information is appropriate. It cannot be said that it represents the characteristics of each small area.

  The present invention has been made in view of the above points, and a content information specifying method and apparatus capable of uniquely identifying a content by paying attention to a distribution in a color space of pixel values included in the content, and An object is to provide a program and a computer-readable recording medium.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention (Claim 1) is a content information specifying method using a local color distribution for extracting information from the image itself for the purpose of specifying the image,
An area dividing step (step 1) in which the image dividing means divides the input image into a plurality of small areas;
A color information mapping step (step 2) in which the color information mapping means maps the color information included in each small area to the color space with respect to the small areas divided in the area dividing step;
A color information clustering step (step 3) in which the color information clustering means performs clustering in the color space based on the result mapped in the color information mapping step;
A cluster representative information extraction means for obtaining representative information for each cluster in the color space obtained in the color information clustering step (step 4);
The information sequence forming means performs an information sequence forming step (step 5) for obtaining the feature amount of the small area from the representative information of each cluster and forming the information sequence of the entire image.

The present invention (Claim 2) is a content information specifying method using local color distribution for extracting information from the video itself for the purpose of specifying the video,
An area dividing step in which the image dividing means divides each frame of the input video into a plurality of small areas;
A color information mapping step in which the color information mapping means maps the color information included in each small area to the color space with respect to the small areas divided in the area dividing step;
A color information clustering step in which the color information clustering means performs clustering in the color space based on the result mapped in the color information mapping step;
A cluster representative information extracting unit is a cluster representative information extracting step for obtaining representative information for each cluster in the color space obtained in the color information clustering step;
An information sequence forming unit performs an information sequence forming step of obtaining a feature amount of a small area from the representative information of each cluster, forming an information sequence for each frame, and arranging the information sequence in the time axis direction.

Further, the present invention (Claim 3) is the area dividing step of Claim 1 or 2,
The step of dividing the entire frame of the image or video into a plurality of types of sizes according to the characteristics of the image is performed.

The present invention (Claim 4) provides the color information clustering step of Claim 1 or 2,
Clustering is performed based on the adjacency relationship in each frame of the image or video.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 5) is a content information specifying device using a local color distribution for extracting information from an image itself for the purpose of specifying the image,
Area dividing means 20 for dividing the input image into a plurality of small areas;
Color information mapping means 30 for mapping the color information contained in each small area to the color space for the small areas divided by the area dividing means 20;
Color information clustering means 40 for clustering in the color space based on the result mapped by the color information mapping means 30;
Cluster representative information extracting means 50 for obtaining representative information for each cluster in the color space obtained by the color information clustering means 40;
Information sequence forming means 60 for obtaining a feature amount of a small area from the representative information of each cluster and forming an information sequence of the entire image.

The present invention (Claim 6) is a content information specifying device using a local color distribution for extracting information from the video itself for the purpose of specifying the video,
Area dividing means 20 for dividing each frame of the inputted video into a plurality of small areas;
Color information mapping means 30 for mapping the color information contained in each small area to the color space for the small areas divided by the area dividing means 20;
Color information clustering means 40 for clustering in the color space based on the result mapped by the color information mapping means 30;
Cluster representative information extracting means 50 for obtaining representative information for each cluster in the color space obtained by the color information clustering means 40;
It has information sequence forming means 60 for obtaining feature amounts of small areas from representative information of each cluster, forming an information sequence for each frame, and arranging the information sequence in the time axis direction.

Further, the present invention (Claim 7) is the area dividing means 20 of Claim 5 or 6,
Means for dividing the entire frame of the image or video into regions of a plurality of types according to the characteristics of the image.

The present invention (Claim 8) provides the color information clustering means 40 according to Claim 5 or 6,
A clustering unit is included based on the adjacent relationship in each frame of the image or video.

  The present invention (Claim 9) is a content information specifying program that causes a computer to function as the content information specifying apparatus according to Claims 5 to 8.

  The present invention (Claim 10) is a computer-readable recording medium storing a program for causing a computer to function as the content information specifying device according to Claims 5 to 8.

  As described above, according to the present invention, each frame of an image or video is divided into small regions, clustering in a color space is performed for each small region, and color regions included in each frame of the image or video are extracted. By doing so, it is possible to accurately represent information for uniquely identifying the content.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  Note that, in the drawings shown below, the same reference numerals are given to components having the same functions in the embodiments, and repeated descriptions thereof are omitted.

[First Embodiment]
FIG. 3 is an apparatus configuration diagram according to the first embodiment of the present invention.

  The content information specifying device shown in FIG. 1 includes an image input unit 10, an area dividing unit 20, a color information mapping unit 30, a color information clustering unit 40, a cluster representative information extracting unit 50, an information sequence forming unit 60, and an information sequence output unit 70. , A database 80, and a storage memory 90.

  The image input unit 10, the region dividing unit 20, the color information mapping unit 30, the color information clustering unit 40, the cluster representative information extracting unit 50, the information sequence forming unit 60, and the information sequence output unit 70 are connected to the storage memory 90. .

  The image input unit 10 stores the input image in the storage memory 90.

  The area dividing unit 20 acquires an image from the storage memory 90, divides the image into a plurality of areas, and stores the areas in the storage memory 90.

  The color information mapping unit 30 creates a pixel value (for example, RGB component) corresponding to the coordinates assigned to each area acquired from the storage memory 90 and stores it in the storage memory 90.

  The color information clustering unit 40 acquires a mapping result for each area from the storage memory 90, and generates a cluster by combining pixel values from the same small area that are located in the vicinity in the color space. The cluster number is assigned to each pixel (labeling), and the result is stored in the storage memory 90.

  The cluster representative information extraction unit 50 acquires a clustering result from the storage memory 90, obtains representative information from a set of pixel values having the same label for each region, and stores the result in the storage memory 90.

  The information sequence forming unit 60 acquires representative information from the storage memory 90, arranges the information sequences based on a certain order, and passes them to the information sequence output unit 70.

  The information sequence output unit 70 outputs the data acquired from the information sequence formation unit 60 to the database 80.

  The operation in the above configuration will be described below.

  FIG. 4 shows a processing flow in one embodiment of the present invention, and FIG. 5 shows a processing image in one embodiment of the present invention.

  Hereinafter, a case where an image is a target will be described, but the present invention can also be applied to a case of a video by replacing processing in each frame of the video with an image.

  Step 110) First, when an image is input by the image input unit 10, the input image is stored in the storage memory 90.

  Step 120) Next, the area dividing unit 20 divides the image on the storage memory 90 into a plurality of small areas, and stores each small area in the storage memory 90. Each small region has the same size for the entire image as shown in FIG. In the division into small areas, the number of pixels in each small area may be fixed, or the number of divisions may be fixed (the number of pixels in the small area depends on the image size).

  Step 130) Thereafter, the color information mapping unit 30 maps the pixel values in the small region on the color space for the small region obtained by the region dividing unit 20, and stores the result in the storage memory 90 for each region. Store. As shown in FIG. 6B, when a plurality of types of colors are included in one small region such as region 1 or region 2, the pixel values are in the form of a color block in the RGB space. Distributed. In addition, although the example which used RGB space as a color space which maps a pixel value was shown here, it is not limited to RGB space actually, L * a * b *, L * u * v *, etc. Or a three-dimensional space centered on XYZ tristimulus values, or may be mapped to a two-dimensional space such as an xy chromaticity diagram.

  Step 140) The color information clustering unit 40 acquires the result of the color information mapping process from the storage memory 90, and puts together pixel values from the same small area that are located in the vicinity in the color space. The cluster is generated and the clustering result is stored in the storage memory 90. The determination of the neighborhood in the color space is performed by, for example, calculating the Euclidean distance in the three-dimensional space, and assuming that the pixel is the origin if the value is equal to or smaller than a certain threshold, and the pixel in the color space is the origin. If the angle between the vectors is equal to or less than a certain threshold value, it is regarded as a neighborhood.

  Step 150) The cluster representative information extraction unit 50 acquires the clustering result in the color space of the pixel values included in each small region from the storage memory 90, extracts representative information for the cluster, and stores the storage memory 90. To store. As the representative information, for example, an average pixel value or variance of pixels constituting a cluster, a mode value of pixel values, or the like is used.

  Step 160) The information sequence forming unit 60 acquires the representative information of the cluster stored in the storage memory 90, generates an information sequence based on the representative information, and stores it in the storage memory 90. When there are a plurality of types of clusters included in each small region, the representative information of the clusters is sequentially connected to each small region to form an information string. FIG. 7 shows a state in which cluster information is arranged in ascending numerical order for each small region to form an information string. In region 1, since the two types of clusters are generated in the color information clustering unit 40, the representative information of each cluster is arranged in order. On the other hand, since three types of clusters are generated from the region 2, representative information is arranged corresponding to the number of clusters. Note that the order of the small areas when generating the information string may be arranged in a raster format starting from the upper left of the image, and the luminance is the highest among the small areas located outside at the stage of division by the area dividing unit 20. The data of the small area may be arranged in a spiral shape starting from a higher one, or may be arranged in accordance with other predetermined rules.

  Step 170) The information sequence generated by the information sequence forming unit 60 and stored in the storage memory 90 is stored in the database 80.

[Second Embodiment]
In the present embodiment, another example between step 120 and step 130 in the first embodiment will be described.

  FIG. 8 is an apparatus configuration diagram according to the second embodiment of the present invention. Between the area dividing unit 20 and the color information mapping unit 30 in FIG. 3, a frequency characteristic calculating unit 110 and a small area characteristic determining unit 120 are illustrated. The small area integration / division unit 130 is added.

  FIG. 9 is a flowchart of the operation after the area division processing according to the second embodiment of the present invention. The processing described below is processing performed between the above-described step 120 and step 130, and the other processing is the same as that of the above-described first embodiment, and thus description thereof is omitted.

  Step 120) As in the first embodiment, the area dividing unit 20 divides the entire image into small areas having the same size.

  Step 121) Next, the frequency characteristic calculation unit 110 calculates the frequency characteristic for each small area. Here, the frequency characteristic is a power spectrum calculated by the equation (2) with respect to a numerical value obtained from the Fourier transform shown in the following equation (1).

なお、式（１）において、ｆ（ｘ）は座標系ｘにおける画素値であり、Ｆ（ｆ）はフーリエ変換結果である。

In equation (1), f (x) is a pixel value in the coordinate system x, and F (f) is a Fourier transform result.

P (f) = │F (f ) │ 2 formula (2)
Step 122) Based on the frequency characteristics obtained above, the small area characteristic determination unit 120 determines whether the small areas initially divided in Step 120 are to be divided, integrated, or left as they are. When there is only one type of color region in one small region, the power spectrum has a peak only in the lowest frequency component as shown in FIG. On the other hand, when a plurality of types of color areas exist in one small area, as shown in FIG. 10B, there are peaks for a plurality of frequency components. Based on the frequency characteristics described above, when there are a plurality of peaks of the power spectrum, it is determined that the small region is divided. Further, when there is only one type of peak, it is determined that these areas are integrated as one area if the adjacent areas on the top, bottom, left, and right on the image have similar characteristics.

  Step 123) Finally, the small area integration / division unit 130 performs integration and division on the division result of the first small area in Step 120 based on the result in the small area characteristic determination step 122. When the small area is further divided, there are a method of fixing the number of pixels of a smaller small area and a method of fixing the number of divisions, either of which may be used.

  Here, the determination of dividing the small area is performed only once. However, the frequency characteristics may be calculated again for the divided small area, and the determination of whether or not the division is performed may be repeated.

  In the above description, the method of using the frequency characteristics for the determination of the division / integration of the small regions has been described. However, there is a method of using the characteristics by principal component analysis instead of using the frequency characteristics. The operation in this case is shown below.

  FIG. 11 is a flowchart of an operation when an image is divided into small areas according to the second embodiment of the present invention.

  Step 120) The image is divided into small areas having the same size.

  Step 124) The principal component is obtained for each small region. The principal component is obtained by creating a covariance matrix from pixel values included in a small region and obtaining its eigenvalue and eigenvector. In order from the largest eigenvalue, the first principal component vector, the second principal component vector,... Here, since a color space composed of three components R, G, and B is targeted, only the third principal component can be obtained.

  Step 125) Next, based on the relative relationship of the contribution ratios of the principal components (magnitude relationship of eigenvalues), determination of integration and division of small regions is performed. When the contribution ratio of the first principal component is overwhelmingly large compared to the contribution ratios after the other second principal components, there is a high possibility that the target small area has one kind of color. In addition, if regions adjacent to each other vertically and horizontally on the image have similar characteristics, it is determined that they are integrated as one region. On the other hand, when the contribution ratios of the first principal component and the second principal component are approximate, there is a high possibility that a plurality of colors are included in one small region, and therefore, it is determined to divide into small regions.

  Step 123) Finally, based on the result in the principal component characteristic determination step 125, the division and division into the first small region division result in Step 120 is performed.

  FIG. 12 shows the result of integrating and dividing the small areas, taking the image shown in FIG. 6 as an example. FIG. 12 shows an example in which the integrated area 3 and the integrated area 4 are areas integrated by the small area integration / division step (step 123), and the divided areas 5 and 6 are divided. When a plurality of colors are included in one small area, the small area is further divided. When the small area is occupied by almost the same color, the same color is displayed on the top, bottom, left, and right. It is shown to be integrated with an area with the characteristics of

  In the above description, the processing for performing both integration and division of small regions based on the result of the small region characteristic determination step (step 122) has been described. However, only the processing of either division or integration is executed. It doesn't matter.

  The next color information mapping step (step 130) is the same as in the first embodiment described above, and a description thereof will be omitted.

  After mapping pixel values from each small area to the color space (color information mapping step (step 130)), pixels located in the vicinity in the color space are clustered together as one (step 140). As described in the first embodiment, pixels in different small areas are generated as separate clusters even when they are located in the vicinity in the color space. Furthermore, after the cluster is generated, it is determined whether or not it can be generated as a cluster, and only those satisfying the conditions are left as clusters. Here, whether or not the cluster can be generated is determined by, for example, whether the number of pixels included in the cluster is a predetermined number or more, or the ratio of the number of pixels included in the cluster to the number of pixels in the small region is constant. A criterion such as whether or not the above is used is used. Note that the determination of the neighborhood in the color space is the same as that described in the first embodiment, and thus the description thereof is omitted.

  Further, the next cluster representative information extraction step (step 150) is also the same as that in the first embodiment, so this description is also omitted.

  In the information sequence forming step (step 160), an information sequence expressing the characteristics of the image is formed based on the result obtained in the cluster representative information extracting step (step 150). In this process, when there are a plurality of types of clusters included in each small area, one representative cluster is selected and an information string is generated. Hereinafter, a representative cluster selection method in the cluster representative information extraction unit 50 will be described.

  FIG. 13 is a flowchart of representative cluster selection processing according to the second embodiment of the present invention.

  Step 161) The cluster representative information extraction unit 50 calculates cluster feature information for each small region. Here, the cluster feature information is, for example, information such as the number of pixels included in one cluster, the dispersion of pixel values in the cluster, and the radius of the cluster. Hereinafter, a case where the number of pixels in one cluster is used as feature information will be described. However, dispersion or a radius may be used.

  Step 162) From the calculation result of the cluster feature information for each small region, the one having the largest number of pixels contained in one cluster is selected as the representative cluster. The fact that the number of pixels included in the cluster is large indicates that the proportion of the color area in the small area is the largest. Further, when using the variance of the number of pixels in the cluster as the feature information of the cluster, the cluster having the largest variance is selected, and when using the cluster radius as the feature information, the cluster having the largest radius is selected.

  Step 160) With the above processing, one representative cluster is obtained in one small area, so an information string is generated from the representative information of these clusters.

  In the generation of the information sequence, as described in the first embodiment, the raster format with the small area as the upper left starting point, the spiral starting from the starting point determined in advance based on the magnitude of the brightness, etc. It is obvious that you can arrange them in order.

In the first embodiment, the case where all the clusters in one small area are used as the cluster used for the information sequence, and in the second embodiment, the case where one is selected from a plurality of clusters has been described. Before these processes, a determination step of selecting one of the methods based on the statistical properties of each cluster for each small area may be provided. For example, for multiple clusters obtained from a small area, calculate the inter-cluster variance and intra-cluster variance,
Intercluster distribution / Intracluster distribution> th formula (3)
Is determined to use representative information of all clusters,
Inter-cluster dispersion / intra-cluster dispersion ≦ th formula (4)
If it is, it is determined that one is selected from a plurality of clusters. Here, th is a threshold value.

  In the above determination, when the pixel values of the small area are mapped to the RGB color space, a plurality of clusters are distributed apart as in area 7 in FIG. There are cases where the distribution is distributed, and the fact that each of the values of the inter-cluster variance with respect to the intra-cluster variance has different characteristics is used. In other words, the former is a state in which there are a plurality of distinctly different color regions in a small region, and the ratio of the inter-cluster variance to the intra-cluster variance is large (Equation (3)), while the latter is compared with the former. And small (formula (4)). In addition, in Formula (3) and Formula (4), although the equal sign was attached | subjected to Formula (4), it is obvious that you may attach | subject to Formula (3).

  Step 170) Finally, the information string generated above is stored in the database 80.

  In addition, the present invention constructs the functions shown in FIG. 3 and FIG. 8 shown in the first embodiment and the second embodiment as programs and installs them in a computer for execution, or creates a network. It is possible to circulate through.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk device or a flexible disk / CD-ROM, and installed in a computer or distributed.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention can be applied to prevention of unauthorized use of content distributed on the Internet or outside, content search, and a reference service using content as a key.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is an apparatus block diagram in the 1st Embodiment of this invention. It is a flowchart in the 1st Embodiment of this invention. It is a figure which shows the process image in the 1st Embodiment of this invention. It is an example of the area | region division in the 1st Embodiment of this invention. It is an example of information sequence formation in the 1st embodiment of the present invention. It is an apparatus block diagram in the 2nd Embodiment of this invention. It is a flowchart of operation | movement after the area | region division process in the 2nd Embodiment of this invention. It is an example of a typical power spectrum in the case of dividing | segmenting / not dividing in the small region division | segmentation judgment in the 2nd Embodiment of this invention. It is a flowchart of the operation | movement at the time of dividing | segmenting the image in the 2nd Embodiment of this invention into a small area. It is an example of the result of having performed integration and division determination after the area division processing in the second exemplary embodiment of the present invention. It is a flowchart of the selection process of the representative cluster in the 2nd Embodiment of this invention. It is a figure which shows the mode of the cluster distribution obtained from the small area | region in the 2nd Embodiment of this invention.

Explanation of symbols

1 area 2 area 3 integrated area 4 integrated area 5 divided area 6 divided area 7 area 8 area 10 image input unit 20 area dividing unit, area dividing unit 30 color information mapping unit, color information mapping unit 40 color information clustering unit, color information Clustering unit 50 Cluster representative information extracting means, cluster representative information extracting unit 60 Information sequence forming means, information sequence forming unit 70 Information sequence output unit 80 Database 90 Storage memory 110 Frequency characteristic calculating unit 120 Small region characteristic determining unit 130 Small region integration / Dividing part

Claims (10)

A content information identification method using a local color distribution that extracts information from the image itself for the purpose of identifying the image,
An area dividing step in which the image dividing means divides the input image into a plurality of small areas;
A color information mapping step for mapping color information included in each small region to a color space with respect to the small region divided in the region dividing step;
A color information clustering step in which color information clustering means performs clustering in a color space based on the result mapped in the color information mapping step;
A cluster representative information extracting unit is a cluster representative information extracting step for obtaining representative information for each cluster in the color space obtained in the color information clustering step;
An information sequence forming step, wherein the information sequence forming means obtains a feature amount of the small area from the representative information of each cluster, and forms an information sequence of the entire image;
A content information specifying method characterized by: A content information specifying method using a local color distribution that extracts information from the video itself for the purpose of specifying the video,
An area dividing step in which the image dividing means divides each frame of the input video into a plurality of small areas;
A color information mapping step for mapping color information included in each small region to a color space with respect to the small region divided in the region dividing step;
A color information clustering step in which color information clustering means performs clustering in a color space based on the result mapped in the color information mapping step;
A cluster representative information extracting unit is a cluster representative information extracting step for obtaining representative information for each cluster in the color space obtained in the color information clustering step;
An information sequence forming step, wherein the information sequence forming means obtains a feature amount of a small area from the representative information of each cluster, forms an information sequence for each frame, and forms an information sequence arranged in a time axis direction;
A content information specifying method characterized by: In the region dividing step,
The content information specifying method according to claim 1 or 2, wherein the step of dividing the entire frame of the image or video into regions of a plurality of types according to the characteristics of the image is performed. In the color information clustering step,
The content information specifying method according to claim 1 or 2, wherein the step of clustering is performed based on an adjacent relationship in each frame of an image or a video. A content information identification device using a local color distribution that extracts information from the image itself for the purpose of identifying the image,
Area dividing means for dividing the input image into a plurality of small areas;
Color information mapping means for mapping color information contained in each small area to a color space for the small areas divided by the area dividing means;
Color information clustering means for clustering in a color space based on the result mapped by the color information mapping means;
Cluster representative information extraction means for obtaining representative information for each cluster in the color space obtained by the color information clustering means;
An information sequence forming means for obtaining a feature amount of a small area from representative information of each cluster and forming an information sequence of the entire image;
A content information specifying device comprising: A content information specifying device that uses a local color distribution to extract information from the video itself for the purpose of specifying the video,
Area dividing means for dividing each frame of the input video into a plurality of small areas;
Color information mapping means for mapping color information contained in each small area to a color space for the small areas divided by the area dividing means;
Color information clustering means for clustering in a color space based on the result mapped by the color information mapping means;
Cluster representative information extraction means for obtaining representative information for each cluster in the color space obtained by the color information clustering means;
An information sequence forming means for obtaining a feature amount of a small area from representative information of each cluster, forming an information sequence for each frame, and arranging the information sequence in the time axis direction;
A content information specifying device comprising: The region dividing means includes
7. The content information specifying device according to claim 5, further comprising means for dividing an entire frame of an image or video into areas of a plurality of types according to image characteristics. The color information clustering means includes
7. The content information specifying device according to claim 5, further comprising means for clustering based on an adjacent relationship in each frame of an image or a video. Computer
9. A content information specifying program that functions as the content information specifying device according to claim 5. Computer
9. A computer-readable recording medium storing a program that functions as the content information specifying device according to claim 5.

Images