JP2003323602A - Image retrieving device and image retrieving method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image retrieving device capable of drastically reducing a processing time for retrieving a still; mage and a moving image. <P>SOLUTION: An image whose possibility to be a similar video candidate is judged to be zero by a correlation calculating process is removed from candidates. Therefore, it is possible to narrow down candidates, or retrieval results, with less matchings than when matching with respect to all designated moving image blocks in round robin system. Therefore, it is possible to drastically reduce processing quantity. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image search device and an image search method.

[0002]

2. Description of the Related Art Conventionally, an image feature amount generating device, an image retrieving device, and a generating method and a retrieving method thereof are generally applied to retrieve a part of a stored moving image.

An example of a conventional compressed moving image search method using a reference moving image is described in Japanese Patent Laid-Open No. 10-13832. The moving image search method disclosed in this publication extracts at least one of the DCT coefficients of each block as a feature vector of each screen, and extracts the feature vector of each frame included in a specific moving image pattern serving as a search key. Probabilistic state transitions are modeled by learning, and the time domain with a high frequency for the state transition model obtained by learning the feature vector extracted from the frames included in the series of moving images to be searched is output as the search result. .

Japanese Patent Laid-Open No. 6-231254 discloses that
This is a method of searching an image including an area similar to the specific area of the reference image from the compressed moving image data. When searching for a reference image, pattern matching is performed using the average value of pixel values included in the encoded information as a first step to select candidate images, and a second step is to perform reduction of the reduced image on the reference screen and the input screen. The search is performed by calculating the correlation value with the frequency component.

[0005]

However, the above-mentioned conventional method has the following problems. First, since the size of the feature quantity is large, the scale of the storage device for storing the feature quantity is large. Secondly, the search speed decreases due to the large size of the feature amount. Thirdly, for each time point of the moving image data to be searched in the database,
Since matching is performed with the entire input moving image feature amount, matching time becomes enormous. This problem becomes more noticeable as the scale of the database increases.

According to the present invention, similar still images and moving images are searched at high speed from still image and moving image data,
An object is to provide an image feature amount generation device, an image search device, and a generation method and a search method for generating a still image and a moving image feature amount with a smaller storage capacity.

[0007]

In order to achieve the above object, the invention according to claim 1 calculates a frequency component by performing frequency analysis on a reduced image generated from image data, and calculates the frequency component. A DC component and a part of the AC component are extracted from the frequency component, and a feature amount of the image data generated by using the extracted DC component and a part of the AC component is used to extract the feature amount of the image data. It is characterized in that similar image data is searched by calculating a correlation value between them.

According to the second aspect of the present invention, the reduced image generated from the image data is subjected to frequency analysis to calculate the frequency component, and the calculated DC component and a part of the AC component are calculated. An image storage unit with a feature amount that extracts and stores the feature amount of the image data generated using the extracted DC component and a part of the AC component together with the image data, and a reduced image generated from the input image data. The frequency component is calculated by performing a frequency analysis on, the DC component and some AC components are extracted from the calculated frequency component, and the extracted DC component and some AC components are used. An image correlation measurement unit that inputs a feature amount of the generated input image data and calculates a correlation value between the feature amount of the input image data and the feature amount of the image data stored in the image storage unit with the feature amount. Prepare And wherein the door.

According to a third aspect of the present invention, in the second aspect of the present invention, frequency analysis means for performing frequency analysis on the reduced image generated from the image data to calculate a frequency component, and the calculated frequency. And a direct current component and a partial alternating current component extracting means for extracting a direct current component and a part of the alternating current component from the component.

The invention according to claim 4 is the invention according to claim 2 or 3.
In the invention described in (3) above, a reduced image generating means for generating a reduced image from the image data is provided.

According to a fifth aspect of the present invention, frequency analysis means for performing frequency analysis on the reduced image generated from the input image data to calculate a frequency component, and a direct current component and a part of the calculated frequency component. Extract the AC component and
DC component and partial AC component extraction means for generating the feature amount of the input image data by using the extracted DC component and a part of the AC component, and frequency analysis for the reduced image generated from the image data. By calculating the calculated frequency component, extracting the DC component and a part of the AC component from the calculated frequency component, of the image data generated using the extracted DC component and a part of the AC component. An image correlation measuring unit that calculates a correlation value between the feature amount of the image data stored in the image storage unit with the feature amount that stores the feature amount together with the image data and the feature amount of the input image data. .

According to a sixth aspect of the present invention, frequency analysis is performed on the second image data generated using the set of color information representing the first image data to calculate frequency components, The direct current component and a part of the alternating current component are extracted from the calculated frequency component, and the characteristic amount of the first image data generated using the extracted direct current component and the partial alternating current component is used as the first image. A frequency component is calculated by performing frequency analysis on the third image data generated using the image storage unit with the feature amount stored together with the data and the set of color information representing the input image data, and the calculation is performed. The DC component and a part of the AC component are extracted from the frequency component, and the feature amount of the input image data generated by using the extracted DC component and a part of the AC component is input, and the input image data is input. Features and features And image correlation measuring unit for calculating a correlation value between the feature quantity of the first image data stored in the image storage unit,
It is characterized by including.

According to a seventh aspect, in the invention according to the sixth aspect, frequency analysis is performed on the third image generated using the set of color information representing the input image data to calculate the frequency component. Frequency analysis means to extract the direct current component and a part of the alternating current component from the calculated frequency component,
A direct current component and a partial alternating current component extracting means for generating a feature amount of the input image data by using the extracted direct current component and a part of the alternating current component are provided.

The invention according to claim 8 is claim 6 or 7
In the invention described in, the input image data is divided into blocks, color information representative of each block is extracted from each of the divided blocks, and third image data is generated using the extracted representative color information. An image generating means is provided.

According to a ninth aspect of the present invention, frequency analysis means for performing frequency analysis on the first image data generated using a set of color information representing the input image data to calculate frequency components, A direct current component and a part of the alternating current component are extracted from the calculated frequency component, and a direct current component and a part of which generate the feature amount of the input image data by using the extracted direct current component and the partial alternating current component. A frequency component is calculated by performing frequency analysis on the third image data generated by using the AC component extracting means and the set of color information representing the second image data, and the DC component is calculated from the calculated frequency component. A component and a part of the AC component, and stores the feature amount of the second image data generated by using the extracted DC component and a part of the AC component together with the second image data. Image storage And image correlation measuring unit for calculating a correlation value between the feature quantity of the feature quantity and the input image data of the second image data stored, characterized in that it comprises a.

The invention according to claim 10 is the invention according to claim 3, 5,
In the invention described in 7 or 9, the DC component and partial AC component extracting means extracts either one or both of a part of the coefficient of the luminance component and a part of the coefficient of the color difference component,
The feature amount of the image data is generated by using one or both of the extracted coefficient of the luminance component and the extracted coefficient of the color difference component.

The eleventh aspect of the present invention is characterized in that, in the sixth aspect to the eighth aspect, the second image data and the third image data are 8 × 8 fixed size image data. To do.

The twelfth aspect of the invention is characterized in that, in the ninth aspect of the invention, the first image data and the second image data are image data of a fixed size of 8 × 8.

According to a thirteenth aspect of the present invention, image data is converted into first 8 × 8 fixed size image data, and frequency conversion is performed on the converted first 8 × 8 fixed size image data. To calculate the frequency component, and extract the direct current component of the luminance component, the direct current component of the color difference component, and a part of the alternating current component from the calculated frequency component to obtain the feature amount of the image data as the image data. The image storage unit with a feature amount to be stored together with the input image data is stored in the second 8 × 8
Second fixed size image data
Frequency analysis is performed on the input image data having a fixed size of 8 × 8, and the direct current component of the luminance component, the direct current component of the color difference component, and a part of the alternating current component obtained by performing the frequency analysis are extracted. By inputting the feature amount of the input image data generated by, the image correlation measuring unit that calculates the correlation value between the feature amount of the input image data and the feature amount of the image data stored in the image storage unit with the feature amount, It is characterized by including.

According to a fourteenth aspect of the present invention, there is provided image data determination means for determining whether the input image data is a still image or a moving image, and an image determined by the image data determination means as a still image. Reduced image generation means for generating a reduced image from data, frequency analysis means for performing frequency analysis of the generated reduced image, and DC component for extracting a DC component and a partial AC component obtained as a result of the analysis as a still image feature amount, and Still image feature amount calculating means having a part AC component extracting means, and data corresponding to all or part of frames are extracted from the image data determined to be a moving image by the image data determining means and selected as a processing target. Feature amount extraction target frame selection means, and reduced image generation means for generating reduced images from the respective data of the extracted frames. Frequency analysis means for performing frequency analysis of the reduced image, direct current component and partial alternating current component extraction means for extracting direct current component and partial alternating current component obtained as a result of analysis as image feature amount, and direct current component and partial alternating current component extraction means A moving image feature amount calculating means having moving image feature amount generating means for collecting frame feature amounts output from the moving image feature amount generating means, and still image feature amount calculating means for still image data. Image feature amount storage means for storing the still image feature amount and the moving image feature amount previously calculated by the moving image feature amount calculation means for the moving image data,
An image database including an image data storage unit that stores still image data and moving image data, a still image feature amount calculated by the still image calculation unit based on the input still image data, and an image feature included in the image database. Static image feature amount correlation measuring means for calculating a correlation value with the amount, moving image feature amounts calculated by the moving image feature amount calculating means based on the input moving image data, and image feature amounts included in the image database. And a moving image feature amount correlation measuring unit for calculating a correlation value of the still image feature amount correlation measuring unit, the still image feature amount correlation measuring unit detects a still image in an image database having a high correlation value as a candidate of a similar still image, and a moving image The feature amount correlation measuring means is characterized by detecting a moving image section in an image database having a high correlation value as a candidate for a similar moving image section.

According to a fifteenth aspect of the present invention, in the invention according to the fourteenth aspect, the moving image characteristic amount correlation measuring means calculates the moving image characteristic amount calculated by the moving image calculating means with respect to the input moving image data. The partial correlation value between the moving image feature amount of the divided moving image included in the moving image database and each of the divided moving image feature amounts is sequentially calculated, and the partial correlation value is smaller than a predetermined threshold value. When it is low, the feature is that the moving image section included in the moving image database having the feature amount is removed from candidates of similar moving images.

According to a sixteenth aspect of the present invention, in the invention according to the fourteenth aspect or the fifteenth aspect, the moving image feature amount correlation measuring means is the moving image feature calculated by the moving image calculating means with respect to the input moving image data. The amount is divided, and a partial correlation value between a part of the moving image feature amount of the moving image included in the moving image database and the divided partial moving image feature amount is sequentially calculated, and the partial correlation value is predetermined. It is characterized in that it is removed from similar moving image candidates when the number of partial moving image feature amounts lower than the threshold value becomes equal to or more than a predetermined threshold value.

According to the seventeenth aspect of the present invention, the reduced image generated from the image data is subjected to frequency analysis to calculate the frequency component, and the direct current component and a part of the alternating current component are calculated from the calculated frequency component. Similar image data is extracted by calculating the correlation value between the feature amounts of the image data by using the feature amount of the image data generated by using the extracted DC component and a part of the AC component. It is characterized by searching.

In the eighteenth aspect of the present invention, frequency analysis is performed on the reduced image generated from the image data to calculate the frequency component, and the direct current component and a part of the alternating current component are calculated from the calculated frequency component. A step of extracting and storing the feature amount of the image data generated by using the extracted DC component and a part of the AC component in the image storage unit with the feature amount together with the image data; A frequency analysis is performed on the reduced image to calculate a frequency component, a DC component and a part of the AC component are extracted from the calculated frequency component, and the extracted DC component and a part of the AC component are extracted. An image correlation measurement step for inputting the feature amount of the input image data generated by using the input image data and calculating a correlation value between the feature amount of the input image data and the feature amount of the image data stored in the image storage unit with the feature amount. Tsu and up, characterized in that it comprises a.

According to a nineteenth aspect of the present invention, in the invention according to the eighteenth aspect, a frequency analysis step of performing frequency analysis on a reduced image generated from image data to calculate a frequency component, and the calculated frequency. A DC component and a partial AC component extracting step of extracting a DC component and a part of the AC component from the component.

The invention described in Item 20 is the invention described in Item 18 or 19 characterized by further comprising a reduced image generating step of generating a reduced image from the image data.

According to a twenty-first aspect of the present invention, a frequency analysis step of performing frequency analysis on a reduced image generated from input image data to calculate a frequency component, and a DC component and a part of the calculated frequency component. An AC component is extracted, and a DC component and a partial AC component extraction step of generating a feature amount of the input image data by using the extracted DC component and a part of the AC component; A frequency analysis is performed on the reduced image to calculate a calculated frequency component, a DC component and a part of the AC component are extracted from the calculated frequency component, and the extracted DC component and a part of the AC component are extracted. Calculate the correlation value between the feature amount of the image data stored in the image storage unit with the feature amount that stores the feature amount of the image data generated using the image data and the feature amount of the input image data And image correlation measurement step that, characterized in that it comprises a.

According to a twenty-second aspect of the present invention, frequency analysis is performed on the second image data generated using the set of color information representing the first image data to calculate frequency components, The direct current component and a part of the alternating current component are extracted from the calculated frequency component, and the characteristic amount of the first image data generated using the extracted direct current component and the partial alternating current component is used as the first image. A step of storing the data in the image storage unit with a feature amount together with the data, and a frequency component is calculated by performing frequency analysis on the third image data generated using the set of color information representing the input image data, A DC component and a part of the AC component are extracted from the calculated frequency component, and the feature amount of the input image data generated by using the extracted DC component and a part of the AC component is input, and the input is performed. Image data features Characterized in that and an image correlation measuring step of calculating a correlation value between the feature quantity of the first image data stored in the feature amount with the image storage unit.

According to a twenty-third aspect of the present invention, in the twenty-third aspect of the present invention, frequency analysis is performed on the third image generated using the set of color information representing the input image data to obtain frequency components. Frequency analysis step to calculate,
A direct current component and a partial alternating current component that extract a direct current component and a partial alternating current component from the calculated frequency component, and generate a feature amount of input image data using the extracted direct current component and a partial alternating current component. And an extraction step.

According to a twenty-fourth aspect of the invention, in the invention according to the twenty-second or the twenty-third aspect, the input image data is divided into blocks, color information representing each block is extracted from each of the divided blocks, and the extracted color information is extracted. It is characterized by comprising a reduced image generating step of generating the third image data using representative color information.

According to a twenty-fifth aspect of the present invention, a frequency analysis step of performing frequency analysis on the first image data generated using the set of color information representing the input image data to calculate frequency components, A direct current component and a part of the alternating current component are extracted from the calculated frequency component, and a direct current component and a part of which generate the feature amount of the input image data by using the extracted direct current component and the partial alternating current component. An AC component extraction step and frequency analysis is performed on the third image data generated using the set of color information representing the second image data to calculate the frequency component, and the DC component is calculated from the calculated frequency component. A component and a part of the AC component, and stores the feature amount of the second image data generated by using the extracted DC component and a part of the AC component together with the second image data. Characterized in that and an image correlation measuring step of calculating a correlation value between the feature quantity of the feature quantity and the input image data of the second image data stored in the image storage unit.

The invention described in claim 26 is,
In the invention described in 1, 23, or 25, in the DC component and partial AC component extraction step, one or both of a part of the coefficient of the luminance component and a part of the coefficient of the color difference component are extracted and extracted. It is characterized in that the feature amount of the image data is generated by using one or both of a part of the coefficient of the luminance component and a part of the coefficient of the color difference component.

The invention as defined in claim 27 is based on claims 22 to 2.
In the invention described in 4, the second image data and the third image data are image data of 8 × 8 fixed size.

According to a twenty-eighth aspect of the invention, in the twenty-fifth aspect of the invention, the first image data and the second image data are image data of 8 × 8 fixed size.

According to a twenty-ninth aspect of the invention, the image data is converted into first 8 × 8 fixed size image data, and the frequency analysis is performed on the converted first 8 × 8 fixed size image data. To calculate the frequency component, and extract the direct current component of the luminance component, the direct current component of the color difference component, and a part of the alternating current component from the calculated frequency component to obtain the feature amount of the image data as the image data. Together with the step of storing in the image storage unit with the feature amount, converting the input image data into second 8 × 8 fixed size image data, and converting to the converted second 8 × 8 fixed size input image data. Then, the frequency analysis is performed, and the feature amount of the input image data generated by extracting the DC component of the luminance component, the DC component of the color difference component, and a part of the AC component obtained by performing the frequency analysis is input. And characterized in that it comprises an image correlation measuring step of calculating a correlation value between the feature amount and the feature amount with the image feature amount of image data stored in the storage unit of the input image data.

[0036]

BEST MODE FOR CARRYING OUT THE INVENTION Next, embodiments of an image feature amount generating apparatus, an image retrieving apparatus, a generating method and a retrieving method according to the present invention will be described in detail with reference to the accompanying drawings.
1 to 9, there are shown an embodiment of an image feature amount generation device, an image search device, and a generation method and a search method thereof according to the present invention.

First, a first embodiment of an image feature amount generating apparatus and method according to the present invention will be described in detail with reference to FIG.

The first embodiment shown in FIG. 1 has a still image characteristic amount generating section 22 for inputting still image data and generating a characteristic amount of the still image data. The still image feature amount generation unit 22 includes a reduced image generation unit 221, a frequency analysis unit 222, a DC component and a partial AC component extraction unit 2.
23 and.

The reduced image generation means 221 generates a reduced image 12 for the input still image data 11, and the reduction ratio at this time is arbitrary. As an example of a method of generating a reduced image, there is a method of dividing an image into blocks and extracting only color information representative of each block. As representative color information, the average color, the most frequent color, etc. in each block can be used. Further, when the image data is compressed in block units, only the DC component of each block can be extracted and generated.

The frequency analysis means 222 analyzes the frequency of the reduced image 12 and calculates the frequency component 13. It is also possible to perform the frequency analysis after dividing the reduced image 12 into a plurality of regions. As a frequency analysis method for reduced images,
DCT transform, wavelet transform, Fourier transform, Hadamard transform, etc. can be used.

DC component and partial AC component extracting means 22
3 extracts a DC component and a partial AC component 14 from the frequency component 13 calculated by the frequency analysis means 222.
As a specific example of the frequency analysis unit 222 and the DC component and partial AC component extraction unit 223, the frequency analysis unit 222 is executed by a two-dimensional DCT for the entire reduced image size to extract the DC component and n AC components. The method and the like can be used. Also, a coefficient of only the luminance component, a coefficient of only the color difference component, a method of extracting only a part of both coefficients, or the like can be used.

In this image feature amount generating apparatus and method, since the frequency analysis is performed on the reduced image and only a part of the obtained coefficients is used, it is possible to generate the image feature amount requiring a small storage capacity.

Next, a second embodiment of the image feature amount generating apparatus and method of the present invention will be described in detail with reference to FIG.

The second embodiment shown in FIG. 2 has a moving image characteristic amount generating section 2 which inputs moving image data and generates a characteristic amount of the moving image data. The moving image feature amount generation unit 2 includes a feature amount extraction target frame selection unit 21, a still image feature amount generation unit 22, and a feature amount generation unit 23. Further, the still image feature amount generation unit 22 has the above-described reduced image generation unit 221, frequency analysis unit 222, and DC component and partial AC component extraction unit 223. The moving image is composed of a plurality of still images, and the still image feature amount generation unit 22 used in the first embodiment can be used even when the above-mentioned object to be handled is a moving image.

The feature amount extraction target frame selecting means 21 which is a component of the feature amount generating unit 2 selects the frame data 11 to be the feature amount extraction target from the input moving image data 1.

The feature amount extraction target frame selecting means 21 is
It is possible to take out only the compressed data corresponding to the intra-frame encoded frame in which the compression processing is completed within one frame in the moving image data. It is possible to select all the intra-coded frames or only some of the intra-coded frames. In addition, regarding an inter-coded frame in which processing is not completed within the frame, for example, see the document; Boon-Lock Yeo and Bede Li.
u: ”On the extraction of DC sequence from MPEG Co
mpressed Video ”, IEEE International Conference o
By using the method described in n Image Processing (1995), an intra-coded frame can be generated in a pseudo manner. Therefore, it is also possible to artificially generate an intra-coded frame and select it.

The reduced image generation means 221 constituting the still image feature amount extraction section 22 generates a reduced image from the still image data 11 selected by the feature amount extraction target frame selection means 21. The frequency analysis unit 222 decomposes the reduced image 12 into frequency components 13. The DC component and partial AC component extraction means 223 uses the frequency component 13 generated by the frequency analysis means 222 to determine the DC component and partial AC component 14 from the frequency component 13.
To extract.

The feature amount generating means 23 is a direct current component and a partial alternating current component 14 provided from the still image feature amount extracting section 22.
Is extracted for all selected frames to generate the feature amount 15.

Also in this embodiment, since the frequency analysis is performed on the reduced image and only a part of the obtained coefficients is used, it is possible to generate a feature amount that requires a small storage capacity.

Next, with reference to FIG. 3, a third embodiment of the image feature amount generating apparatus and method of the present invention will be described.

In the third embodiment shown in FIG. 3, a still image feature amount generator 22 for inputting still image data and generating a feature amount of the still image data, and the input still image data is stored. The still image data storage unit 61 and the still image feature amount storage unit 62 that stores the still image feature amount generated by the still image feature amount generation unit 22 are included in the still image database storage unit 6 with a feature amount. The still image feature amount generation unit 22 includes a reduced image generation unit 221, a frequency analysis unit 222, a DC component and a partial AC component extraction unit 22.
3 and 3.

The still image feature amount generation unit 22 generates a reduced image from the input still image image data by the reduced image generation unit 221, and the frequency analysis unit 222 performs a frequency analysis of the generated reduced image to perform analysis. The resulting DC component and partial AC component are extracted by the DC component and partial AC component extraction means 223 to generate a still image feature amount.

The generated still image feature amount is stored in the still image feature amount storage unit 62, and the still image data used for generating the still image feature amount is stored in the still image data storage unit 61. .

Also in this embodiment, since the frequency analysis is performed on the reduced image and only a part of the obtained coefficients is used, it is possible to generate a feature amount which requires a small storage capacity.

As shown in FIG. 4, the fourth embodiment relating to the image feature amount generating apparatus and method has a moving image feature amount generating section 2 and a moving image database storage section 3 with characteristic amount. To be done. The description of the moving image feature amount generation unit 2 is omitted.

In this embodiment, the feature amount 15 is generated from the moving image data 1 and both are stored in the moving image database storage unit 3 with the feature amount.

Moving image database storage unit with feature amount 3
Is a moving image data storage unit 31 and a moving image feature amount storage unit 3
2 and stores the moving image data 1 and the characteristic amount 15 generated from the moving image data 1 by the characteristic amount generation unit 2 for various moving image data 1.

The moving image data storage unit 31, which is one of the components of the moving image database storage unit 3 with the feature amount, holds the moving image data 1. In addition, the moving image data storage unit 31
Holds the feature amount 15 generated by the feature amount generating unit 2 from the moving image data 1.

The image feature amount generating apparatus constituted by the above-mentioned respective units uses the feature amount extraction target frame selecting means 2 to extract data corresponding to all or part of a frame from moving image data.
In step 1, the reduced image generation unit 221 generates a reduced image from the data corresponding to the extracted frames, and the frequency analysis unit 222 performs frequency analysis of the generated reduced image. The DC component and AC component extraction unit 223 extracts a part of the AC component, and the feature amount generation unit 23 executes it for all the selected frames to generate the feature amount.

In this moving image feature amount generation apparatus, frequency analysis is performed on the reduced image of some frames, and only a part of the obtained coefficients is used. Therefore, the moving image feature amount that requires a small storage capacity is used. Can be generated.

Next, with reference to FIG. 5, a fifth embodiment of the image feature amount generating apparatus and method of the present invention will be described.

The fifth embodiment according to the image feature amount generation apparatus shown in FIG. 5 is an image data determination unit 100, a still image feature amount generation unit 22, a moving image feature amount generation unit 2, and a feature amount. And the attached image database storage unit 5. Note that the image data targeted by the present embodiment includes moving image data and still image data, and the moving image is composed of a plurality of still images.

The image data 7 input to this embodiment is
The image data determination unit 100 determines whether the image is a still image or a moving image. If the image is a still image, it is input to the still image feature amount generation unit 22, and if it is a moving image, it is input to the moving image feature amount generation unit 2.

The still image feature amount generating unit 22 generates a reduced image from the input still image image data by the reduced image generating unit 221, and the frequency analyzing unit 222 performs the frequency analysis of the generated reduced image for analysis. The resulting DC component and partial AC component are extracted by the DC component and partial AC component extraction unit 223, and the feature amount generation unit 23 executes them for all selected frames to generate the feature amount.

The moving image feature amount generating section 2 extracts data corresponding to all or a part of the frame from the moving image data by the feature amount extraction target frame selecting means 21, and reduces the reduced image from the data corresponding to each extracted frame. Generation means 2
21 generates a reduced image, the frequency analysis unit 222 performs frequency analysis of the generated reduced image, and the DC component and partial AC component extraction unit 223 extracts the DC component and the partial AC component obtained as a result of the analysis. The feature quantity generation means 2
3 performs on all selected frames to generate a feature amount.

Each feature amount 1 generated by the above processing
4 and 15 are stored in the image feature amount storage unit 52 of the image database storage unit 5 with the feature amount. Further, image data including a moving image and a still image is stored in the image data storage unit 51.

When a still image is input as described above, frequency analysis is performed on the reduced image, a feature amount is generated using only a part of the obtained coefficients, and a moving image is input. In this case, the frequency analysis is performed on the reduced image of some frames, and the feature amount is generated using only a part of the obtained coefficients. Can generate quantity.

Next, a first embodiment of the image retrieval apparatus and method of the present invention will be described with reference to FIG.

As shown in FIG. 6, according to the present embodiment, a still image feature amount generating section 22 for inputting still image data and generating a feature amount of the still image data, and a still image feature data for the still image data. A still image database storage unit 6 with a feature amount including a still image feature amount storage unit 62 that stores still image feature amounts generated in advance by the feature amount generation unit 22 and a still image data storage unit 61 that stores still image data. , The still image feature amount generation unit 2 based on the input still image data
The static image correlation measurement unit 41 calculates a correlation value between the static image feature amount generated in 2 and the static image feature amount stored in the static image database with feature amount stored in the storage unit 6.

The still image feature amount generation unit 22 generates a reduced image from the still image image data input by the reduced image generation unit 221, and the frequency analysis unit 222 performs a frequency analysis of the generated reduced image to perform analysis. The input still image data 1 is obtained by extracting the resulting direct current component and partial alternating current component by the direct current component and partial alternating current component extraction means 223.
The still image feature amount 14 for 1 is generated.

In the still image database with feature amount storage unit 6, still images are stored in the still image data storage unit 61 according to the procedure described in the third embodiment of the image feature amount generating apparatus. The still image feature amount calculated by the still image feature amount generation unit 22 using the still image stored in the data storage unit 61 is stored in the still image feature amount storage unit 62.

The still image correlation measuring unit 41 calculates the correlation value between the feature amount 14 generated by the input still image data and the feature amount generated by the image included in the image database, and the correlation value is high. Detect still images in the database as candidates for similar still images.

Next, a second embodiment of the image retrieval apparatus and method of the present invention will be described with reference to FIGS. 7 and 8. It should be noted that FIG.
8 is a block diagram showing a configuration of the embodiment of FIG. 8, and FIG. 8 is a flowchart showing a correlation measurement procedure of the image search method.

In the second embodiment relating to the image retrieval apparatus and method shown in FIG. 7, the feature amount generating section 2, the moving image database storage unit with characteristic amount 3, and the feature amount 17 in the moving image database are input. A correlation measuring unit 4 for calculating a correlation value between the moving image data 1 and the feature amount 15 generated by the feature amount generating unit 2 is included.

First, the moving image data 1 used for retrieval is input, and the characteristic amount generating section 2 generates the characteristic amount 15. When performing a search, the feature amount generation unit 2 is generated from the moving image to be referred to and input to the correlation measurement unit 4.

The correlation measuring unit 4 calculates the correlation value between the feature amount 17 in the moving image database and the feature amount 15 generated by the feature amount generating unit 2 with respect to the input moving image data 1 and is searched. The partial moving image storage position information 18 is output as a search result.

The operation of the moving picture correlation measuring section 4 will be described in detail with reference to FIG. The image search method according to the present invention is input as a processing step for obtaining a correlation between the feature amount generated from the input moving image data and a part of the feature amount generated from the moving image included in the moving image database. The feature amount generated from the moving image is divided, the partial correlation is sequentially calculated for each divided portion, and if the partial correlation is lower than a predetermined threshold, the moving image in the database having the feature amount is calculated. There is a step of removing from the candidates. By adopting such a method, it is possible to narrow down the search result candidates with a smaller number of matchings compared to the case of performing a brute force match for the entire specified moving image section, so that the moving image is searched. Processing time can be shortened. This processing procedure will be described in detail below.

First, as an initial setting, an identifier number for calling a moving image feature amount to be searched is initialized (step S1). Here, the identifier is uniquely assigned to each moving image data when the moving image database is generated. In addition, the feature amount 15 is also acquired in advance.

Next, the moving image feature amount to be searched corresponding to the identifier is acquired, and the partial feature amount to be searched (hereinafter,
A pointer indicating the acquisition position of the target portion feature amount) is set at the beginning (step S2).

A feature amount having a predetermined time width is acquired from the reference reference feature amount. This data is called a reference partial feature amount. Further, the target portion feature amount is acquired from the location indicated by the pointer indicating the acquisition position (step S3).

The correlation between the reference partial feature amount and the target partial feature amount obtained in step S3 is obtained (step S4).

If the correlation value is lower than the predetermined threshold value (hereinafter referred to as mismatch determination), it is checked in step S5 whether or not the target portion feature quantity to be acquired next is present. The feature position acquisition position pointer is shifted by a predetermined value (step S6), and the process returns to step S3. If the next target partial feature amount does not exist, the identifier is increased by one and the same process is performed for the next moving image feature amount (step S13). However, when matching is performed with respect to all the moving image features in the database, the process ends (step S12).

If the correlation value is higher than the predetermined threshold value in step S4, the next target partial feature quantity and reference partial feature quantity that continue in time from the acquisition position are acquired (step S4).
7) Obtain the correlation (step S8). Thereafter, the matching is repeated in the same manner until a mismatch is determined, and when all the reference partial feature amounts are matched (step S9),
The search result is output (step S10).

It is possible to output the moving image data identifier number including the search result and the start position or the end position in the moving image data. In addition to this, the correlation value and the like can be output.

When it is judged that there is a mismatch, it is judged whether or not the mismatch judgment has exceeded a preset number of times or a predetermined time (step S11). If not exceeded, the process returns to step S7. When it exceeds, it returns to step S5. In step S11, the difference in the selection position of the frame to be intra-coded, the difference in camera work,
Temporary mismatches due to changes in the feature amount due to screen noise can be covered by appropriately determining the number of times or time. Further, setting the number of times or time to "0" is equivalent to omitting step S11, and the circuit design can be lightened.

On the other hand, it is also possible to perform a search by comparing the correlation with the search partial feature quantity at once with the time width when the reference partial feature quantity is obtained as the entire reference feature quantity.

Next, a third embodiment of the image retrieval apparatus and method of the present invention will be described in detail with reference to FIG.

In the embodiment shown in FIG. 9, the image feature amount generating section 10, the image data storage section 51 for storing still images and moving images, and the image data stored in the image data storage section are used. Image data storage unit 5 with a feature quantity, which includes an image feature quantity storage unit 52 in which still image feature quantities and moving image feature quantities generated in advance by the image feature quantity generation unit 10 are stored, and a feature quantity in the image database. A moving image correlation value measuring unit 4 for calculating a correlation value between the moving image data 1 input with 17 and the characteristic amount 15 generated by the characteristic amount generating unit 2;
Still image correlation value measuring unit 41 for calculating a correlation value between the feature amount 17 in the image database and the feature amount generated by the image feature amount generating unit 10 for the input still image data 11.
Including and

First, the image data 1 used for the search is input, and the characteristic amount generating section 2 generates the characteristic amount 15. When the image data to be referred to is a moving image, the characteristic amount generation unit 10 generates a characteristic amount from the moving image and inputs the characteristic amount to the moving image characteristic amount correlation measurement unit 4. If the image is a still image, the feature amount generation unit 10 generates a feature amount from the still image and inputs the feature amount to the still image feature amount correlation measurement unit 19.

The moving image correlation measuring unit 4 correlates the calculation of the correlation value between the characteristic amount 17 in the moving image database and the characteristic amount 15 generated by the characteristic amount generating unit 2 with respect to the input moving image data 1. The measurement position 4 is performed, and the storage position information 18 of the searched partial moving image is output as a search result. Note that the description of this processing operation is omitted because it has already been described in the description of the second embodiment related to the moving image feature amount generation device described above.

The still image correlation measuring unit 41 calculates the correlation value between the feature amount 14 generated by the input still image data and the feature amount generated by the image included in the image database, and the correlation value is high. Detect still images in the database as candidates for similar still images.

(Examples) Next, examples of the present invention will be described. Using about 20 hours of MPEG-1, 2 encoded video data, a video section corresponding to the opening of a certain specific article, CM, or program was specified, and an experiment was conducted to detect video sections that seemed to be the same. .

A time code and a DC component are extracted for each intra-coded frame in the whole moving image to be searched. Here, the reduced image generated by extracting the DC component is further reduced to a fixed size of 8 × 8 and then DCT is performed to obtain three DC components of a luminance component and a color difference component (referred to as DC_Y, DC_Cb, and DC_Cr). , And two low-frequency components (A
C1 and AC2) are normalized to the range of [0, 255] and then stored together with the time code. For each intra-frame coded frame, the feature amount is 5 bytes and the time code is 4 bytes, and the total is 9 bytes.
For this reason, when the intra-frame coded frames are present at 0.5 second intervals, the data size of the feature amount corresponding to the video for one day also falls within about 1.5 megabytes.

The feature amount of the leading frame of the reference moving image is matched with the feature amount of the search target moving image in intra-frame coding frame units. DC_Y, DC_Cb, DC_C
r difference value (hereinafter referred to as DC difference value), AC1, AC
The target moving image candidates are those in which the difference values of 2 satisfy the specified conditions (threshold values).

When it is regarded as the target moving image candidate, the subsequent frame is examined in time series. If it is not regarded as a target video candidate, the same procedure is repeated for the next frame. After that, it is assumed that the DC difference values satisfying the specified condition are matched, and they are sequentially examined. For the purpose of reducing the number of times of matching, AC1 and AC2 may not be used thereafter.

Considering the error due to the sample position, if there is a mismatch for three consecutive frames, it is excluded from the candidates. If the difference in the feature quantity satisfies the specified condition (threshold value) for the length corresponding to the specified moving image section, it is regarded as the target moving image and the time code of the start point and the end point is output as the search result. . Further, when the target moving image section is detected, the start point of the next search may be started from the frame next to the end point of the detected section.

With this procedure, it is possible to significantly reduce the processing amount as compared with the case where the entire specified moving image section is subjected to brute force matching.

As a result of conducting an experiment on a set of eleven moving image sections which seem to be the same, it was found that the other can be obtained by designating one of all eleven. Further, when the length of the target moving image section was set to 10 seconds or more, no excessive detection was found.

The above-described embodiment is an example of a preferred embodiment of the present invention. However, it is not limited to this,
Various modifications can be made without departing from the scope of the present invention. Further, in any of the embodiments, the similar moving image search can be easily expanded by defining and using the distance between the feature amounts instead of removing the one that does not meet the condition from the candidates.

[0100]

As is apparent from the above description, the image feature amount generating apparatus and method of the present invention generate a reduced image from image data, perform a frequency analysis of the generated reduced image, and obtain the analysis result. A feature amount is generated by extracting a direct current component and a partial alternating current component. Further, the image search apparatus and method of the present invention extracts data corresponding to all or part of a frame from image data, and generates a reduced image from the extracted data corresponding to each frame. The frequency analysis of the generated reduced image is performed, the direct current component and the partial alternating current component obtained as a result of the analysis are extracted, and the extraction is executed for all selected frames.

The image search apparatus and method of the present invention can remove a candidate from a candidate when it is determined that it cannot be a similar image candidate in the correlation calculation process. Therefore, it is possible to narrow down the search result candidates with a smaller number of times of matching, as compared with the case of performing brute force matching for the entire specified moving image section, and it is possible to significantly reduce the processing amount. Therefore, it is possible to reduce the processing time for searching the moving image.

Further, only the coefficients obtained by performing the frequency analysis on the reduced image of some frames are used. Therefore, it is possible to generate an image feature amount that requires a small storage capacity.

INDUSTRIAL APPLICABILITY The present invention can be widely applied to surveys of the number of broadcasts of a specific video, such as commercial video, specific news articles, and program opening, statistics of broadcast information, trend analysis, and the like.

INDUSTRIAL APPLICABILITY The present invention can be applied to, for example, a similar video search for searching a part of the stored image having similar color and frequency distribution to the moving image to be referred to.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of an embodiment to which an image feature amount generation device and method of the present invention are applied.

FIG. 2 is a block diagram showing a configuration example of an embodiment to which the image feature amount generation device and method of the present invention are applied.

FIG. 3 is a block diagram showing a configuration example of an embodiment to which an image feature amount generation device and method of the present invention are applied.

FIG. 4 is a block diagram showing a configuration example of an embodiment to which the image feature amount generation device and method of the present invention are applied.

FIG. 5 is a block diagram showing a configuration example of an embodiment to which the image feature amount generation device and method of the present invention are applied.

FIG. 6 is a block diagram showing a configuration example of an embodiment to which the image search device and method of the present invention are applied.

FIG. 7 is a block diagram showing a configuration example of an embodiment to which an image search device and method of the present invention are applied.

FIG. 8 is a flowchart illustrating an example of a processing procedure of an image search method.

FIG. 9 is a block diagram showing a configuration example of an embodiment to which an image search device and method of the present invention are applied.

[Explanation of symbols]

1 Moving image data 2 Feature quantity generator 3 Moving image database storage with features 4 Correlation measurement unit 5 Moving image database storage with features 6 Still image database storage with features 7 image data 10 Image feature amount generation unit 11 still image data 12 reduced images 13 frequency components 14 DC component and some AC components 15 features 16 Video data identifier 17 Search target moving image features 18 Storage location information of searched partial moving images 19 Storage location information of retrieved still images 21 Feature amount extraction target frame selection means 22 Still image feature extraction unit 23 Feature Generation Unit 31 Moving image data storage unit 32 Moving image feature storage 41 Still image correlation measurement unit 51 Image data storage 52 Moving image feature amount storage unit 61 Still image data storage unit 62 Still image feature amount storage unit 100 image data determination unit 221 Reduced image generation means 222 Frequency analysis means 223 DC component and partial AC component extraction means

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ identification code FI theme code (reference) G06T 7/00 300 G06T 7/00 300H

Claims (29)

[Claims] 1. A frequency analysis is performed on a reduced image generated from image data to calculate a frequency component, a DC component and a part of an AC component are extracted from the calculated frequency component, and the extraction is performed. It is possible to search for similar image data by calculating a correlation value between the feature amounts of the image data by using the feature amount of the image data generated by using the DC component and a part of the AC component. Characteristic image retrieval device. 2. A reduced image generated from image data is subjected to frequency analysis to calculate a frequency component, a DC component and a part of the AC component are extracted from the calculated frequency component, and the extraction is performed. An image storage unit with a feature amount that stores the feature amount of the image data generated using the DC component and a part of the AC component together with the image data, and a frequency for the reduced image generated from the input image data. Perform an analysis to calculate the frequency component, extract the DC component and a part of the AC component from the calculated frequency component,
The feature amount of the input image data generated using the extracted DC component and a part of the AC component is input, and the feature amount of the input image data and the image stored in the image storage unit with the feature amount. An image retrieving apparatus comprising: an image correlation measuring unit that calculates a correlation value with a feature amount of data. 3. A frequency analyzing means for performing frequency analysis on a reduced image generated from image data to calculate a frequency component, and a direct current for extracting a direct current component and a part of the alternating current component from the calculated frequency component. The image retrieval apparatus according to claim 2, further comprising: a component and a partial AC component extracting unit. 4. The image retrieving apparatus according to claim 2, further comprising reduced image generation means for generating a reduced image from image data. 5. A frequency analyzing means for performing frequency analysis on a reduced image generated from input image data to calculate a frequency component, and a DC component and a part of an AC component are extracted from the calculated frequency component. A DC component and a partial AC component extracting means for generating a feature amount of the input image data by using the extracted DC component and a part of the AC component, and a reduced image generated from the image data. A frequency component is calculated by performing frequency analysis, a DC component and some AC components are extracted from the calculated frequency component, and the generated DC component and some AC components are generated. An image phase for calculating a correlation value between the feature amount of the image data stored in the image storage unit with the feature amount that stores the feature amount of the image data together with the image data and the feature amount of the input image data. Image retrieval apparatus characterized by comprising: a measuring unit. 6. A frequency component is calculated by performing frequency analysis on the second image data generated by using a set of color information representing the first image data, and the frequency component is calculated from the calculated frequency component. A DC component and a part of the AC component are extracted, and the characteristic amount of the first image data generated by using the extracted DC component and a part of the AC component is stored together with the first image data. And a frequency component is calculated for the third image data generated by using the image storage unit with the feature amount and the set of color information representing the input image data, and the calculated frequency is calculated. A DC component and a part of the AC component are extracted from the component, and the feature amount of the input image data generated by using the extracted DC component and a part of the AC component is input, and the input image data Feature quantity and the feature quantity Image retrieval apparatus comprising: an image correlation measuring unit for calculating a correlation value between the feature quantity of the first image data stored in the image storage unit can, a. 7. A frequency analysis means for performing frequency analysis on the third image generated using a set of color information representing the input image data to calculate a frequency component, and the calculated frequency component. From the DC component and a part of the AC component from the DC component and a partial AC component extraction means for generating the feature amount of the input image data using the extracted DC component and a part of the AC component, The image search device according to claim 6, further comprising: 8. The input image data is divided into blocks, color information representing each block is extracted from each of the divided blocks, and the third image data is generated using the extracted representative color information. The image search device according to claim 6 or 7, further comprising reduced image generation means. 9. A frequency analysis means for performing frequency analysis on first image data generated using a set of color information representing input image data to calculate a frequency component, and from the calculated frequency component A direct current component and a partial alternating current component, and a direct current component and a partial alternating current component extracting means for generating a feature amount of the input image data by using the extracted direct current component and a partial alternating current component. , A frequency component is calculated by performing frequency analysis on the third image data generated using the set of color information that represents the second image data, and a DC component and a part of the calculated frequency component are calculated. An image storage unit with a feature amount that extracts an AC component and stores the feature amount of the second image data generated using the extracted DC component and a part of the AC component together with the second image data. Remembered by Image retrieval apparatus characterized by and an image correlation measuring unit for calculating a correlation value between the feature quantity of the feature quantity and the input image data of the second image data. 10. The direct current component and partial alternating current component extracting means extracts either one or both of a part of a coefficient of a luminance component and a part of a coefficient of a color difference component, and extracts the coefficient of the extracted luminance component. The feature amount of the image data is generated by using one or both of a part or a part of a coefficient of a color difference component.
The image search device described in 5, 7, or 9. 11. The image search device according to claim 6, wherein the second image data and the third image data are image data of a fixed size of 8 × 8. 12. The image search device according to claim 9, wherein the first image data and the second image data are image data of a fixed size of 8 × 8. 13. The image data is converted into first 8 × 8 fixed size image data, and the frequency component is analyzed by performing frequency analysis on the converted first 8 × 8 fixed size image data. A feature that stores the feature amount of the image data generated by extracting the DC component of the luminance component, the DC component of the color difference component, and a part of the AC component from the calculated frequency component together with the image data. An image storage unit with a quantity and input image data are converted into second 8 × 8 fixed size image data, and frequency analysis is performed on the converted second 8 × 8 fixed size input image data. , Inputting the characteristic amount of the input image data generated by extracting the DC component of the luminance component and the DC component of the color difference component and a part of the AC component obtained by performing the frequency analysis, and inputting the input image De An image retrieving apparatus, comprising: an image correlation measuring unit that calculates a correlation value between the characteristic amount of the data and the characteristic amount of the image data stored in the image storage unit with the characteristic amount. 14. Image data determining means for determining whether the input image data is a still image or a moving image,
Reduced image generation means for generating a reduced image from image data determined to be a still image by the image data determination means, frequency analysis means for performing frequency analysis of the generated reduced image, and direct current obtained as a result of the analysis. Minute image data and a partial alternating current component as still image characteristic amounts, and a still image feature amount calculating means having a direct current component and a partial alternating current component extracting means, and image data determined by the image data determining means to be a moving image. Feature amount extraction target frame selection means for selecting data corresponding to all or some of the frames as extraction processing targets, reduced image generation means for generating reduced images from the respective data of the extracted frames, and the generated reduction Frequency analysis means for frequency analysis of images, and DC and partial AC components obtained as a result of analysis are used as image features. A moving image having direct-current component and partial alternating-current component extracting means for extracting, and moving-image characteristic amount generating means for collecting moving-image characteristic amounts by collecting frame characteristic amounts output from the direct-current component and partial alternating-current component extracting means. Feature amount calculation means, still image feature amount previously calculated by the still image feature amount calculation means for still image data, and moving image previously calculated by the moving image feature amount calculation means for moving image data An image database including an image feature amount storage unit that stores a feature amount and an image data storage unit that stores the still image data and the moving image data, and the still image calculation unit based on the input still image data. Still image feature amount correlation measuring means for calculating a correlation value between the calculated still image feature amount and the image feature amount included in the image database; And a moving image feature amount correlation measuring unit for calculating a correlation value between the moving image feature amount calculated by the moving image feature amount calculating unit based on the generated moving image data and the image feature amount included in the image database. However, the still image feature amount correlation measuring means detects a still image in an image database having a high correlation value as a candidate of a similar still image, and the moving image feature amount correlation measuring means is in an image database having a high correlation value. An image search device, wherein the moving image section is detected as a candidate of a similar moving image section. 15. The moving image feature amount correlation measuring unit divides the moving image feature amount calculated by the moving image calculating unit with respect to the input moving image data, and a moving image included in the moving image database. A part of the moving image features of
A partial correlation value with each of the divided moving image feature amounts is sequentially calculated, and when the partial correlation value is lower than a predetermined threshold value, a moving image section included in the moving image database having the feature amount is calculated. 15. The image search device according to claim 14, wherein the image search device removes from similar moving image candidates. 16. The moving image feature amount correlation measuring unit divides the moving image feature amount calculated by the moving image calculating unit with respect to the input moving image data, and a moving image included in the moving image database. A part of the moving image features of
A partial correlation value with the divided partial moving image feature amount is sequentially calculated, and the number of the partial moving image feature amount in which the partial correlation value is lower than a predetermined threshold value is equal to or more than a predetermined threshold value. The image search device according to claim 14 or 15, wherein the image search device is removed from candidates of similar moving images at a time point. 17. A reduced image generated from image data is subjected to frequency analysis to calculate a frequency component, a DC component and some AC components are extracted from the calculated frequency component, and the extraction is performed. Using the feature amount of the image data generated using the DC component and a part of the AC component,
An image search method characterized in that similar image data is searched by calculating a correlation value between feature amounts of the image data. 18. A reduced image generated from image data is subjected to frequency analysis to calculate a frequency component, a DC component and a part of an AC component are extracted from the calculated frequency component, and the extraction is performed. Storing the feature amount of the image data generated using the DC component and a part of the AC component in the image storage unit with the feature amount together with the image data, for the reduced image generated from the input image data. Frequency analysis is performed to calculate the frequency component, and the DC component and a part of the AC component are extracted from the calculated frequency component,
The feature amount of the input image data generated using the extracted DC component and a part of the AC component is input, and the feature amount of the input image data and the image stored in the image storage unit with the feature amount. An image retrieval method comprising: an image correlation measurement step of calculating a correlation value with a feature amount of data. 19. A frequency analysis step of performing a frequency analysis on a reduced image generated from image data to calculate a frequency component, and a direct current for extracting a direct current component and a part of the alternating current component from the calculated frequency component. 19. The image retrieval method according to claim 18, further comprising: a component and a partial AC component extracting step. 20. A reduced image generating step of generating a reduced image from image data is provided.
The image search method described in 8 or 19. 21. A frequency analysis step of performing frequency analysis on a reduced image generated from input image data to calculate a frequency component, and extracting a DC component and a part of an AC component from the calculated frequency component. A DC component and a partial AC component extracting step of generating the feature amount of the input image data using the extracted DC component and a part of the AC component, and for the reduced image generated from the image data. A frequency component is calculated by performing frequency analysis, a DC component and some AC components are extracted from the calculated frequency component, and the generated DC component and some AC components are generated. Calculate a correlation value between the feature amount of the image data stored in the image storage unit with the feature amount that stores the feature amount of the image data together with the image data and the feature amount of the input image data. Image retrieval method characterized by comprising the image correlation measuring step, the that. 22. A frequency component is calculated by performing a frequency analysis on the second image data generated by using a set of color information representing the first image data, and the frequency component is calculated from the calculated frequency component. The direct current component and a part of the alternating current component are extracted, and the characteristic amount of the first image data generated by using the extracted direct current component and the partial alternating current component is set to the first value.
And a step of storing the image data with the image data in the image storage unit with the feature amount, and a frequency component is calculated by performing a frequency analysis on the third image data generated by using the set of color information representing the input image data. Then, the DC component and a part of the AC component are extracted from the calculated frequency component, and the feature amount of the input image data generated by using the extracted DC component and a part of the AC component is input. And an image correlation measuring step of calculating a correlation value between the feature amount of the input image data and the feature amount of the first image data stored in the image storage unit with the feature amount. Image search method. 23. A frequency analysis step of calculating a frequency component by performing a frequency analysis on the third image generated using a set of color information representing the input image data, and the calculated frequency component. From the direct current component and a part of the alternating current component is extracted, and the direct current component and the partial alternating current component extracting step of generating the feature amount of the input image data by using the extracted direct current component and the partial alternating current component, The image retrieval method according to claim 22, further comprising: 24. The input image data is divided into blocks, color information representing each block is extracted from each of the divided blocks, and the third image data is generated using the extracted representative color information. 24. The image search method according to claim 22, further comprising a reduced image generation step. 25. A frequency analysis step of performing frequency analysis on first image data generated using a set of color information representing input image data to calculate a frequency component, and from the calculated frequency component A DC component and a partial AC component extraction step of extracting a DC component and a part of the AC component, and using the extracted DC component and a part of the AC component to generate a characteristic amount of the input image data; , A frequency component is calculated by performing frequency analysis on the third image data generated using the set of color information that represents the second image data, and a DC component and a part of the calculated frequency component are calculated. Image storage with feature quantity for extracting an AC component and storing the feature quantity of the second image data generated using the extracted DC component and a part of the AC component together with the second image data Image retrieval method characterized by and an image correlation measuring step of calculating a correlation value between the feature quantity of the input image data and feature amount of the stored second image data to the. 26. In the step of extracting the direct current component and the partial alternating current component, either one or both of a coefficient of a luminance component and a coefficient of a color difference component are extracted, and the extracted coefficient of the luminance component is extracted. 20. The feature amount of the image data is generated by using one or both of a part or a part of a coefficient of a color difference component.
The image search method described in 21, 23 or 25. 27. The image search method according to claim 22, wherein the second image data and the third image data are image data having a fixed size of 8 × 8. 28. The image retrieval method according to claim 25, wherein the first image data and the second image data are image data having a fixed size of 8 × 8. 29. Converting image data into first 8 × 8 fixed size image data, and performing frequency analysis on the converted first 8 × 8 fixed size image data to determine frequency components. Calculate the characteristic amount of the image data generated by extracting the direct current component of the luminance component, the direct current component of the color difference component, and a part of the alternating current component from the calculated frequency component together with the characteristic amount of the image data. A step of storing the image data in the image storage unit, converting the input image data into second 8 × 8 fixed size image data, and performing a frequency analysis on the converted second 8 × 8 fixed size input image data. And input the feature amount of the input image data generated by extracting the direct current component of the luminance component and the direct current component of the color difference component and a part of the alternating current component obtained by performing the frequency analysis, An image retrieval method comprising: an image correlation measuring step of calculating a correlation value between a feature amount of input image data and a feature amount of the image data stored in the image storage unit with the feature amount.