JP2007034613A - Image processing apparatus and method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform high-speed and accurate retrieval for an image of optional shape. <P>SOLUTION: Optional shape images having optional shapes are preliminarily registered in a database with image characteristic quantities for each block thereof, and a retrieval image is input with image characteristic quantities for each block (S401). An optional shape image as a retrieval object is narrowed in the database based on the image characteristic quantities for each block of the optional shape images and the retrieval image (S402), and the similarity of image characteristic quantity between the retrieval image and the narrowed optional shape image is calculated for each block(S403). Based on the calculated similarities, retrieval results are output (S404 and S405). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method, and more particularly, to an image processing apparatus and method for searching for similar images having an arbitrary shape.

  In recent years, technology has been developed to analyze images, extract features such as colors, edges, and textures, and determine the similarity between two images by comparing features. It is used for detecting cut points of moving images.

  A technique for determining such similarity has been developed centering on a technique effective for rectangular images, but a technique applicable to an image having an arbitrary shape has also been proposed.

  For example, according to Patent Document 1, a rectangular image including an arbitrary shape is generated, the background region portion is complemented with a representative value obtained from the arbitrary shape image portion, and conventional image feature extraction is applied to the rectangular image. Thus, the similarity is determined.

Also, for example, according to Patent Document 2, a special feature amount is set such that a distance from any other feature amount is set to 0 for the background region portion outside the attention region of arbitrary shape, and the distance calculation is performed. The influence of is eliminated.
JP 2002-245456 JP Japanese Patent Laid-Open No. 11-312248

  However, the technique disclosed in Patent Document 1 has the following problems.

  In other words, since a rectangular image is generated regardless of the shape, and the image is registered, first, regardless of the shape of the query image that is the search source image, all the images registered in advance as the search destination image Comparison is required. Therefore, the search speed decreases as the number of registered images increases.

  Further, since the background region portion is complemented with the representative value obtained from the image portion having an arbitrary shape, the difference between the images of the arbitrary shape is accumulated also on the background region portion, and the distance is calculated, so that high-precision search cannot be performed.

  On the other hand, the technique disclosed in Patent Document 2 also has the following problems.

  In other words, the search target can be narrowed down using the most frequent feature amount for each image as a key, but the feature amount is extracted and registered as usual for the block of only the background region, so the search target narrowing down It is not valid when searching for an area. Therefore, as the number of registered images increases, the search speed for an arbitrarily shaped image (region of interest) decreases.

  In addition, for blocks that contain images of arbitrary shape, feature values are extracted including the background region part. If there are many background region parts, effective feature values are not extracted, and high-precision search is performed. Can not.

  The present invention has been made to solve the above-described problems individually or collectively, and an object thereof is to provide an image processing apparatus and method capable of performing high-speed search for an image having an arbitrary shape.

  It is another object of the present invention to provide an image processing apparatus and method capable of performing high-precision search for an image having an arbitrary shape.

  As a means for achieving the above object, an image processing apparatus of the present invention comprises the following arrangement.

  That is, an image input unit that inputs an image of an arbitrary shape, a circumscribed rectangle creating unit that creates a circumscribed rectangle for the image, a block dividing unit that divides the circumscribed rectangle into blocks, and an image feature amount of the block Image feature quantity extracting means for extracting, intra-block shape information extracting means for extracting shape information in the block, registration means for registering the image feature quantity and the in-block shape information in association with the image, and the image Image search means for searching for an image using the feature quantity and the in-block shape information is provided.

  The image processing method of the present invention has the following configuration.

  That is, an image input step for inputting an image of an arbitrary shape, a circumscribed rectangle creating step for creating a circumscribed rectangle for the image, a block dividing step for dividing the circumscribed rectangle into blocks, and an image feature amount of the block An image feature amount extraction step to extract, an in-block shape information extraction step to extract shape information in the block, a registration step to register the image feature amount and the in-block shape information in association with the image, and the image And an image search step of searching for an image using the feature amount and the in-block shape information.

  According to the present invention having the above-described configuration, it is possible to quickly search for similar images of arbitrary shapes by performing narrowing down before similarity calculation based on image feature amounts extracted in units of image blocks. It becomes.

  Further, by calculating the similarity based on the image feature amount extracted for each block of the image, it is possible to search for a similar image having an arbitrary shape with high accuracy.

  Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings.

<First Embodiment>
System Configuration FIG. 1 is a block diagram showing the configuration of the image processing apparatus according to this embodiment. In the figure, reference numeral 101 denotes a CPU which executes various controls in the image processing apparatus of the present embodiment. Reference numeral 102 denotes a ROM which stores a boot program executed when the apparatus is started up and various data.

A RAM 103 stores a control program for processing by the CPU 101 and provides a work area when the CPU 101 executes various controls. A keyboard 104 and a mouse 105 provide various input operation environments for the user.
Reference numeral 106 denotes an external storage device, which includes a hard disk, a flexible disk, an optical disk, a magnetic disk, a magneto-optical disk, a magnetic tape, a nonvolatile memory card, and the like. Reference numeral 107 denotes a display, which includes a display or the like, and displays the result and the like to the user. A network interface 108 enables communication with each device on the network. Reference numeral 109 denotes an interface such as IEEE1394 or USB, which communicates with devices such as the scanner 110 and the digital camera 111. Reference numeral 112 denotes a bus connecting the above-described components.

  In the above configuration, the external storage device 106, the scanner 110, and the digital camera 111 can be replaced with those arranged on the network.

  FIG. 2 is a block diagram showing a functional configuration of the image processing apparatus having the configuration shown in FIG. In the figure, reference numeral 201 denotes an image input unit. An image acquired by a device such as the scanner 110 or the digital camera 111 is displayed via an interface 109 or an image existing on a network such as the Internet or a LAN. Enter via 108. An area extracting unit 202 extracts an image area to be registered from the image input by the image input unit 201. A normalization unit 203 normalizes the image area extracted by the area extraction unit 202. A feature extraction unit 204 extracts an image feature amount from the image area normalized by the normalization unit 203.

  An image DB unit 205 is configured by the external storage device 106 and stores an image region and an image feature amount in association with each other. An operation unit 206 includes a keyboard 104, a mouse 105, and the like, and enables a search instruction from the user. A distance calculation unit 207 calculates the distance between the feature amount of the search source image and the feature amount of the search destination image in the image DB unit 205. Reference numeral 208 denotes a display unit, which includes the display unit 107, displays search candidates based on the results of the distance calculation unit 207, and presents them to the user.

Image Registration Processing The image registration processing in this embodiment will be described below using the flowchart in FIG.

  First, an image to be registered is input by the image input unit 201 (S301). The input image may be a single image such as a clip art to be registered, or a plurality of images arranged like a document image. In the present embodiment, the following description will be given assuming that the latter document image is input.

  Next, the region extraction unit 202 performs region extraction on the input image, and extracts an image region such as a clip art to be registered (S302). Details of this region extraction processing will be described later.

  Next, the normalizing unit 204 normalizes the image area extracted as a registration target in step S302 (S303). For example, when clip art is pasted on a document image, processing such as rotation may be performed. Details of the normalization process will be described later.

  Next, the feature extraction unit 204 extracts block information from the normalized image region (S304). The block information includes an image feature amount, a block occupation rate, and block shape information. The image feature amount is not particularly limited as long as it allows vector expression, but here it is assumed to be a color label representing the arrangement of colors on the image. The block occupation ratio and block shape information will be described later. The extracted block information is associated with the image and registered in the image DB unit 205 (S305). Details of the color label extraction process and details of the index in the image DB unit 205 will be described later.

  Next, it is confirmed whether or not the processing in steps S302 to S305 has been performed for all regions to be registered in the input image (S306). If the processing has been performed for all regions, this registration processing is terminated. If an unprocessed area remains, the process returns to step S302.

Area extraction process (S302)
Hereinafter, the region extraction processing in step S302 will be described in detail with reference to FIG. Here, the region extraction processing is, for example, by recognizing the raster image 901 shown in FIG. 4 as a block for each meaningful block as shown in the region classification image 902 and classifying each block for each attribute. This is a process of extracting an area. Here, as shown in the region classification image 902, the block attributes include text (TEXT), drawing (PICTURE), photograph (PHOTO), line (LINE), table (TABLE), and the like.

  Specifically, first, the input image is binarized to black and white, and contour tracking is performed to extract a block of pixels surrounded by a black pixel contour. For a black pixel block with a large area, the white pixel block is extracted by tracing the outline of the white pixel inside, and the black pixel block is recursively extracted from the white pixel block with a certain area or more. Extract a block of pixels.

  The black pixel blocks thus obtained are classified by size and shape, and are classified into regions having different attributes. For example, first, an area having an aspect ratio close to 1 and a constant size is assumed to be a pixel block corresponding to a character. In addition, the area where adjacent characters can be grouped in a well-aligned area is the character area, the flat pixel block is a line region, and the area occupied by the black pixel block that is a certain size or larger and includes a rectangular white pixel block aligned is shown. This is an area. Also, an area where irregular shaped pixel clusters are scattered is classified as a photographic area, and a pixel cluster of any other shape is classified as a graphic area. In the present embodiment, for example, a pictorial area having an arbitrary shape such as clip art is extracted as a registration target.

Normalization processing (S303)
Hereinafter, the normalization process in step S303 will be described in detail with reference to the flowchart of FIG.

  First, in step S501, a circumscribed rectangle is created for an arbitrary shape image to be registered as shown in FIG. Nine circumscribed rectangles are created by arranging rectangles tilted by 10 degrees from 0 degrees to 80 degrees clockwise with respect to the horizontal so as to circumscribe the arbitrary shape image. FIG. 6 shows examples of rectangles of 0 degrees and 60, 70, and 80 degrees among these circumscribed rectangles. In this embodiment, nine circumscribed rectangles tilted by 10 degrees are created, but the present invention is not particularly limited to this example.

  Of the nine circumscribed rectangles obtained in this way, the smallest circumscribed rectangle having the smallest area is selected in step S502. In the example of FIG. 6, the circumscribed rectangle of 80 degrees is the minimum circumscribed rectangle.

  In step S503, if the inclination angle of the minimum circumscribed rectangle is 0 degree, the process is terminated without modifying the arbitrary shape image. When the inclination angle of the minimum circumscribed rectangle is 10 to 40 degrees, in step S504, the arbitrary shape image is rotated counterclockwise by the inclination angle, and the process ends. When the inclination angle of the minimum circumscribed rectangle is 50 to 80 degrees, in step S505, the arbitrary shape image is rotated clockwise (90-inclination angle), and the process ends.

  The circumscribed rectangle thus normalized is hereinafter referred to as a normalized circumscribed rectangle. Also, in the normalized circumscribed rectangle of an arbitrary shape image, the portion of the arbitrary shape image is called an effective region, and the other portion is called an invalid region.

● Index in Image DB Unit Here, the index in the image DB unit 205 will be described in detail. FIG. 7 is a diagram for explaining the data storage state of the index in the image DB unit 205. In the figure, image IDs are numbered in the order of registration. Also, the file name of the full path of the image, the block occupancy matrix, the block shape matrix, and the color label matrix are stored as image data in the data format shown in FIG. 7 together with the image ID. Details of the block occupancy matrix, the block shape matrix, and the color label matrix will be described later.

Block information extraction process (S304)
Hereinafter, the block information extraction processing in step S304 will be described in detail with reference to the flowchart of FIG.

  In this process, the normalized circumscribed rectangle of the image of the arbitrary shape to be processed is divided into a plurality of blocks, and the color label having the most frequent color in the color histogram of each block is associated with the position information of each block. The attached information is extracted as a color feature amount (color label). In addition, information in which the proportion of the effective area in each block is associated with the position information of each block is extracted as a block occupancy rate. Further, the approximate shape of the effective area in each block is labeled, and extracted as block shape information in association with the position information of each block.

  First, in step S1020, the image is divided into a plurality of blocks. In this embodiment, as shown in FIG. 9, the vertical and horizontal directions of an image are each divided into 9 blocks. In this embodiment, an example of dividing into 9 × 9 = 81 blocks is shown for convenience of description, but the present invention is not particularly limited to this example.

  In step S1030, the target block to be processed is set as the upper left block. Note that the block of interest is set with reference to an order determination table in which the processing order is determined in advance as shown in FIG. 10, for example.

  In step S1040, the presence / absence of an unprocessed block of interest is determined. If there is no unprocessed target block, the process ends. If there is an unprocessed target block, the process proceeds to step S1050.

  In step S1050, the density values of all pixels in the effective area in the block of interest are projected onto a color bin that is a partial space created by dividing the color space as shown in FIG. 11, and a color histogram for the color bin is generated. . In this embodiment, as shown in FIG. 11, the density value of all the pixels of the block of interest is projected onto a color bin obtained by dividing the RGB color space into 3 × 3 × 3 = 27. It is not limited to examples.

  In step S1060, the color bin label of the most frequent color bin in the color histogram is determined as the representative color of the target block, and the representative color is temporarily stored in the RAM 103 in association with the target block and its position.

  Next, in step S1070, the 1st place of the effective area ratio (%) in the target block is rounded to 10% increments from 0 to 100%, and converted into 11 types of labels from 0 to 0. The block occupancy is temporarily stored in the RAM 103 in association with the position. In this embodiment, an example in which the ratio is divided into 11 stages is shown, but the present invention is not limited to this example. For example, as an extreme example, it may be divided into two types of 0% and the other or 100% and the other, or may be divided into three types of 0%, 100% and the other. Further, the step may be finer or coarser, and the step width may not be uniform.

  In step S1080, the shape of the effective area in the block of interest is labeled, and temporarily stored in the RAM 103 as block shape information in association with the block of interest and its position. The labeling of the shape is performed by determining which one of the block approximate figures (a) to (j) shown in FIG. 12 is most similar and assigning labels 0 to 9 respectively. In the present embodiment, an example in which ten types of approximate figures are prepared has been shown, but the present invention is not limited to this example.

  To determine which shape most closely resembles the area of the area where the target block is an effective area and the approximate figure is an invalid area, and the area of the area where the target block is an invalid area and the approximate figure is an effective area A method is used to select a figure that minimizes the sum of. Instead of using the area, it can be determined in consideration of the occupancy ratio of the effective area of the target block on each side.

  Next, in step S1090, the target block to be processed next is set with reference to the order determination table shown in FIG. At this time, color label information, block occupancy information, and block shape information temporarily stored in the RAM 103 are stored in the image DB unit 205 as a block information matrix as shown in FIG.

  Thereafter, the process returns to step S1040, and the processes of steps S1040 to S1080 are recursively repeated until there is no unprocessed block of interest.

Similar Image Search Processing Through the image registration processing described above, a block information matrix of a plurality of images is registered in the image DB unit 205 in the format shown in FIG.

  Hereinafter, similar image search processing in the image processing apparatus of this embodiment will be described with reference to the flowchart of FIG.

  First, a search source image is designated (S401). As a method for designating the search source image, for example, an image in the image DB unit 205 is randomly selected and displayed on the display unit 208, and the user specifies an image close to a desired image from the operation unit 206, There is a method of reading the corresponding block information matrix. In addition, a method in which an image itself similar to a desired image is input from the image input unit 201 by the user is also effective. In this case, the image feature amount for the input search source image is completely the same as that in the image registration process described above. Extracted in similar steps.

  A similar image can be searched for by calculating the distance between the feature amount of the specified search source image and the feature amount of the search destination image stored in the image DB unit 205. However, in the present embodiment, prior to this distance calculation, presearch is performed by narrowing down the search destination image to only those having a feature amount close to the feature amount of the search source image (S402). Details of the pre-search process in the distance calculation unit 207 will be described later.

  Next, the distance calculation unit 207 calculates the distance between the feature amount of the search destination image extracted as the search target from the image DB unit 205 by the pre-search and the feature amount of the search source image, and the calculation result is set as the image ID. A distance list is temporarily stored in the RAM 103 (S403). Details of the distance calculation will be described later.

  Next, the distance calculation unit 207 sorts the list in ascending order of distance, and then sequentially reads the image from the image DB unit 205 by referring to the index (full path file name) from the image ID, and generates a thumbnail image by reducing the image. (S404).

  Next, the thumbnail images are displayed as a search result side by side on the matrix together with the calculated distance on the display unit 208 (S405). For example, the first candidates are displayed in order of increasing distance from the left to the right of the column and from the upper column to the lower column. With this display, the user can confirm whether or not a desired image has been searched.

● Presearch processing (S402)
Hereinafter, the pre-search process in step S402 will be described in detail.

  As described above, it is possible to determine whether each block is an effective area, an invalid area, or both areas based on the block occupancy rate information stored in the image DB unit 205. That is, it reflects the approximate shape (in blocks) of the arbitrarily shaped image. In this embodiment, this is utilized to narrow down the search destination image group in the image DB unit 205 to only those whose approximate shape is the same as that of the search source image.

  Therefore, first, only the block occupancy matrices of the search source image and all the search destination images are compared. As a comparison method, for example, in the block occupancy matrix of the search destination image, after converting the block with 0% of the effective area in the block to 0 and the other blocks to 1, the block occupancy of the search source image There is a method of taking a logical product with a matrix. As a result, if the block occupancy matrix of the search source image has not changed, the image ID is temporarily stored in the RAM 103 as the next distance calculation target, and if it has changed, it may be excluded from the distance calculation target. it can. Note that this is a method of narrowing down to images having the same invalid area, but conversely, a method of narrowing down to images having the same valid area is also possible. In any case, since the comparison can be performed by a method with a low calculation cost such as logical product, even if the comparison with all the search destination images is performed, the calculation can be performed at a very high speed as compared with the distance calculation.

Distance calculation (S403)
Hereinafter, the distance calculation in step S403 will be described in detail.

…（式1） In this embodiment, the block ID occupancy matrix, block shape matrix, and label matrix (here, color label matrix) are acquired with reference to the image IDs of the images narrowed down by the pre-search in step S402, and the search source image Is calculated using Equation 1.

... (Formula 1)

  Expression 1 is an expression for obtaining a distance (final solution) between the label matrices. Here, Distk is the distance between the label of the kth block of the search source image and the label of the kth block of the search destination image. Further, WeightPPk is a weight for the ratio of the effective area in the kth block of the search source image and the ratio of the effective area in the kth block of the search destination image. WeightSSk is a weight for a block shape difference between the block shape of the kth block of the search source image and the block shape of the kth block of the search destination image.

  FIG. 14 is a diagram illustrating an example of a penalty matrix between labels used when the distances are obtained by comparing the label matrices. The smaller the value (penalty) in the matrix, the higher the similarity of the labels, that is, the penalty value indicates the distance between the labels. For example, the penalty for label 2 and label 6 is “7”. Also, the penalty between the same labels is naturally "0". The purpose of using this matrix is to perform distance determination according to the similarity of labels. That is, in the present embodiment, since the RGB color space is used as the feature amount space, the distance can be determined according to the color similarity.

  For example, in the label matrix of the search source image and the search destination image, the distance between the labels is obtained from the label value of the corresponding position (block) with reference to the penalty matrix of FIG.

  The weight for the block ratio is obtained by calculating the average of the ratio of the effective area in the k-th block of the search source image and the ratio of the effective area in the k-th block of the search destination image. It is obtained by giving. Giving a linear weight to the average value means that the average value (0 to 100%) is converted to 0 to 1 by a linear function. In the present embodiment, an example is shown in which the average of effective area ratios in a block is used. However, the present invention is not limited to this example, and a value with a larger ratio is used or a ratio is smaller. Alternatively, the square value may be used. Moreover, when converting such values into weights, an example in which linear weights are given to values from 0% to 100% has been shown, but the present invention is not limited to this example, and 0% to 100%. You may convert into a weight by using a quadratic function with respect to the value of%.

  The weighting for the block shape difference is performed by creating a table as shown in FIG. In FIG. 15, the letters (a) to (j) written on the vertical and horizontal axes of the table correspond to the block approximate shapes (a) to (j) shown in FIG. For example, if the block shape of the search source is (c) and the block shape of the search destination is (g), the weight is multiplied by “1.2”. The weight is determined according to the overlapping area of the block approximate shape information. The weights shown in the table of FIG. 15 are merely examples, and the present invention is not limited to this example.

  FIG. 16 shows a specific example of label comparison. In FIG. 16, the label matrix of the search source image is “11231344X”, and the label matrix of the search destination image is “11322445X”. For the last block, since the value of the block occupancy matrix is 0 (all areas in the block are invalid areas), the label value is undefined. For such a label matrix, Dist, WeightP, and WeightS are determined by performing matching using the penalty matrix shown in FIG. 14 and Equation (1), and the distance (final solution) is obtained.

  As described above, the distance calculation of this embodiment is characterized in that a weighting table as shown in FIG. 15 is introduced so that the weight can be changed according to the difference in the shape of the approximate figure of the block. Further, the ratio of the effective area in the block is used as a weight for distance calculation. Also, when pattern matching between labels, a penalty matrix between labels as shown in Fig. 14 is introduced to reduce the penalty (distance) between adjacent cells and to give a large penalty to distant cells. It is characterized by that.

  As described above, according to the present embodiment, when searching for an image of an arbitrary shape, it is possible to perform a high-speed search process for an arbitrary shape image by performing pre-search according to the proportion of the effective area in the block. Become.

  In addition, when searching for an image having an arbitrary shape, a high-precision search for an arbitrary shape image can be performed by setting a weight for distance calculation in accordance with the ratio of effective areas in blocks or / and approximate shape information.

<Modification>
In the pre-research processing of this embodiment, an example is shown in which the block occupancy matrix of the search source image is compared with the block occupancy matrices of all the search destination images. Here, by configuring the index of the image DB unit 205 as shown in FIG. 17, it is possible to realize a similar search process at a higher speed and with a smaller memory capacity.

  That is, at the time of image registration, the block occupancy matrix of an already registered image is compared with the block occupancy matrix of the registration target image by the above-described method. Then, when it can be determined that the shapes are almost the same, the color labels are held together to create an index as shown in FIG.

  In FIG. 17, the block occupancy matrix key is a matrix that is used when comparing the block occupancy matrices and is obtained by converting a block whose effective area in the block is 0% to 0 and other blocks to 1. A set of an image ID, a block occupancy matrix, and a color label matrix (hereinafter referred to as a label matrix group) that is the same block occupancy matrix key is collectively used as an index.

  Therefore, at the time of search, the logical product of the block occupancy matrix of the search source image and the block occupancy matrix key is taken, and only when the block occupancy matrix of the search source image does not change, the corresponding label matrix group is read into the RAM 203, Narrow down the search target image. Then, the distance calculation is performed only on the images in the label matrix group narrowed down on the RAM 203.

  In this way, since the number of comparisons of the block occupancy matrix is reduced, processing faster than the method described in the present embodiment described above can be performed. The set of image ID and full path file name is created as a separate index, and when reading the image file itself, such as when displaying search results, the full path file name information is based on the image ID. Use it. Of course, the full path file name information may be included in the label matrix group, but in this case, the memory consumption increases.

  If the purpose is only to enable pre-search, only the block occupancy matrix key described above may be used, and the block occupancy matrix and block shape matrix need not be recorded. Sometimes it is possible to implement the present invention with the smallest index.

  Also, if the search time can be long, fairness without pre-search is also possible. In this case, of course, the index is as shown in FIG. In the distance calculation of the image feature amount, both the block occupancy rate and the weight obtained from each of the block shape information are used. If the search accuracy is within an allowable range, only the weight obtained from the block occupancy rate, or Only the weight obtained from the block shape information may be used. Of course, if only the weights obtained from the block occupancy are used, it is not necessary to extract and record the block shape information. If only the weights obtained from the block shape information are used, the block occupancy must be extracted and recorded. Absent.

<Other embodiments>
Although the embodiments have been described in detail above, the present invention can take embodiments as, for example, a system, an apparatus, a method, a program, or a storage medium (recording medium). The present invention may be applied to a system composed of a single device or an apparatus composed of a single device.

  In the present invention, a software program (in the embodiment, a program corresponding to the flowchart shown in the figure) that realizes the functions of the above-described embodiment is directly or remotely supplied to the system or apparatus, and the computer of the system or apparatus Is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  As a recording medium for supplying the program, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card , ROM, DVD (DVD-ROM, DVD-R).

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instructions of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

It is a block diagram which shows the structure of the image processing apparatus in one Embodiment which concerns on this invention. It is a block diagram which shows the function structure of the image processing apparatus in this embodiment. It is a flowchart which shows an image registration process. It is a figure which shows an example of an area | region extraction process. It is a flowchart which shows the normalization process of an image. It is a figure which shows the example of creation of the circumscribed rectangle with respect to arbitrary shapes. It is a figure which shows the data storage format of the index in an image DB part. It is a flowchart which shows a block information extraction process. It is a figure which shows the example of a block division of an image. It is a figure which shows an example of an order determination table. It is a figure which shows the structural example of the color bin on color space. It is a figure which shows an example of a block approximate figure. It is a flowchart which shows an image search process It is a figure which shows an example of the penalty matrix between labels. It is a figure which shows an example of the weighting table with respect to the difference between block approximate figures. It is a figure which shows the specific example of a label comparison process. It is a figure which shows the data storage format of the index in an image DB part.

Claims (13)

An image input means for inputting an image of an arbitrary shape;
Circumscribed rectangle creating means for creating a circumscribed rectangle for the image;
Block dividing means for dividing the circumscribed rectangle into blocks;
Image feature amount extraction means for extracting the image feature amount of the block;
In-block shape information extracting means for extracting shape information in the block;
Registration means for registering the image feature quantity and the in-block shape information in association with the image;
Image search means for searching for an image using the image feature quantity and the shape information in the block;
An image search processing apparatus comprising: The image search means includes
Weight determining means for determining a weight for each block from the in-block shape information;
Similarity calculation means for calculating the similarity between images by weighting the distance of the image feature amount for each block by the weight;
The image search processing apparatus according to claim 1, further comprising: The image search means includes
The image search processing apparatus according to claim 1, further comprising a narrowing search unit that performs a narrowing search using the in-block shape information.   The image search processing device according to claim 1, wherein the intra-block shape information includes occupancy information of the image in the block.   The image search processing apparatus according to claim 4, wherein the occupancy information includes an occupancy rate of the image in a block.   6. The image search processing device according to claim 1, wherein the in-block shape information includes approximate shape information of the image in the block. The occupancy information includes indicating by 0 or 1 whether a block includes the image;
The narrow-down search unit performs the narrow-down by calculating the occupancy information of the search source image and the occupancy information of the search destination image by a logical product. Image search processing device. The occupancy information includes indicating by 0 or 1 whether a block includes the image;
The registration means includes
The feature information and the in-block shape information having the same occupation information represented by using 0 or 1 are collected, and the occupation information is registered as a key.
7. The narrowing-down search unit performs narrowing-down by extracting an image feature amount and shape information in a block that match the occupancy information of the search source image and the key. Image processing search device. The image search processing apparatus according to claim 1, wherein the circumscribed rectangle creating unit gives and creates a normalized circumscribed rectangle. The image search processing device according to claim 1, wherein the image feature amount extraction unit extracts an image feature amount from only a range including the image in each block. An image input step for inputting an image of an arbitrary shape;
A circumscribed rectangle creating step for creating a circumscribed rectangle for the image;
A block dividing step of dividing the circumscribed rectangle into blocks;
An image feature amount extracting step for extracting an image feature amount of the block;
In-block shape information extraction step for extracting shape information in the block;
A registration step of registering the image feature quantity and the in-block shape information in association with the image;
An image search step of searching for an image using the image feature amount and the shape information in the block;
An image processing method comprising:   A program that, when executed by an information processing apparatus, controls the information processing apparatus to operate as the information processing apparatus according to any one of claims 1 to 10.   A recording medium on which the program according to claim 12 is recorded.

Images