JP4820433B2 - Image information retrieval apparatus and program - Google Patents

Description

The present invention relates to an image information search apparatus and program, and in particular, based on an image information extraction method and program for extracting information necessary to uniquely identify an image, and an obtained image expression, The present invention relates to an image information search device and a program for searching and determining whether or not an image is identical to the image.

  Conventionally, for the purpose of preventing unauthorized use of content, there are three types of methods for determining whether the content on the Internet or the like is the same as the content stored in the local database, such as the Internet. It has been.

  The first is a digital watermark technique, the second is a hash function, and the third is a feature amount matching technique. Hereinafter, an overview of each technology will be described.

  First, in the method using the first digital watermark technology, information that uniquely represents the content, for example, information such as an ID is embedded in the content in advance as a digital watermark, and watermark information is detected from the content that is available outside. The identity is determined based on whether or not the identification information such as the ID matches (for example, refer to Non-Patent Document 1).

  In the method using the second hash function, the same is determined by whether or not the hash value of the original content matches the hash value of the content that is available outside (for example, see Non-Patent Document 2).

  In the method using the third feature amount matching technique, the content is divided into blocks of an appropriate size, and a vector (feature amount) having a feature amount (for example, an average value) of each block as an element is generated. Whether or not they are the same is determined based on the similarity to the vector representing the feature amount of the content that is circulating outside (see, for example, Non-Patent Document 3).

http://www.jpo.go.jp/shiryou/s_sonota/hyoujun_gijutsu/denshi_sukashi/index.htm http://e-words.jp/w/E3838FE38383E382B7E383A5E996A2E695B0.html Jia Li, James Z Wang, Gio Wiederhold, "IRM: Integrated Region Matching for Image Retrieval", Proceedings of the eighth ACM international conference on Multimedia, pp. 147-156 (2000)

  However, the above-described conventional methods have the following problems.

  Since the digital watermark technique superimposes information for identification on the content itself, there is a problem that the quality of the content is deteriorated, and at the same time, the work of embedding the watermark in advance occurs.

  On the other hand, since the method using the hash function has a characteristic that it is converted into a different hash value even for a minute change of the bit string, for example, by format conversion (including compression) performed when publishing to the Internet. There is a problem that the hash value is completely different from that of the original content and the content is determined to be different while being the same content.

  In addition, for the problems of the above two techniques, that is, the problem of degradation of image quality and complexity of information embedding work in the digital watermark technique, and the problem that the identification performance with respect to minute fluctuations in the hash function is reduced, In essence, the problem can be solved by using information of the content itself and using representative information (feature amount) of a small area (block) of the content, that is, a feature amount matching technique. However, in such a technique, the generation method of vectors (features) depends on the arrangement order of blocks, and the rotation of content affects the order of blocks (the order of vector elements). There is a problem that it is determined as different contents.

The present invention has been made in view of the above, and aims to while using the information of the content itself, and to provide a robust image information extraction and retrievable image information retrieval apparatus and program a rotation transformation To do.

The present invention (claim 1), the same image as the input image, an image information retrieval unit that searches a database of information sequence of a plurality of images are stored,
Area dividing means for dividing the input target image into equal small areas (blocks);
Partial representative information extracting means for extracting representative information based on a value obtained by processing pixel value information in the block for a part of the divided block;
Using the representative information of some blocks obtained by the partial representative information extraction means , the possibility of coincidence for determining the coincidence with the information sequence of image information corresponding to each of a plurality of images stored in the database A determination means ;
Representative information extraction means for obtaining representative information for all the blocks of the target image when the match possibility determination means determines that there is a match possibility;
Starting point determination means for determining the position of the starting block from the information sequence corresponding to the image stored in the database;
An information sequence forming means for forming an information sequence based on the representative information for each block obtained by the representative information extracting means , with the starting block determined by the starting point determining means as the head ;
Information sequence matching that determines the degree of coincidence between the information sequence obtained by the information sequence forming unit and the information sequence corresponding to a plurality of images stored in the database that is determined to be a match by the match possibility determination unit A determination means ;
And an image output means for outputting a corresponding image when the information string match determination means determines that they match .

The present invention (Claim 2 ) is a program for causing a computer to function as each means of the image information retrieval apparatus according to Claim 1 .

  As described above, according to the present invention, an image can be expressed as rotation-invariant image information by dividing the image into uniform blocks and arranging the representative information of each block in order from the starting position, and from an image with an unknown rotation angle. The same image can be searched.

It is a block diagram of the image information extraction device in the 1st Embodiment of this invention. It is a flowchart of outline | summary operation | movement of the image information extraction apparatus in the 1st Embodiment of this invention. It is a flowchart of operation | movement of the origin determination part in the 1st Embodiment of this invention. It is a figure which shows a mode that the starting point block in the 1st Embodiment of this invention is determined. It is a figure which shows the mode of the information sequence production | generation in the 1st Embodiment of this invention. It is a figure which shows the mode of the information sequence production | generation in the 1st Embodiment of this invention. It is a flowchart of operation | movement of the origin determination part in the 2nd Embodiment of this invention. It is a figure which shows a mode that the starting point block in the 2nd Embodiment of this invention is determined. It is a block diagram of the image information search device in the 3rd Embodiment of this invention. It is a flowchart of outline | summary operation | movement of the image information search device in the 3rd Embodiment of this invention. It is a block diagram of the image information search device in the 4th Embodiment of this invention. It is a flowchart of outline | summary operation | movement of the image information search device in the 4th Embodiment of this invention. It is a figure which shows operation which calculates | requires the block starting point in the 4th Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  Note that in the drawings used in this embodiment, components having the same function are denoted by the same reference numerals, and repeated description thereof is omitted.

[First Embodiment]
In the present embodiment, a process for creating an information sequence of an image and storing it in a database will be described.

  FIG. 1 is a configuration diagram of an image information extraction apparatus according to the first embodiment of the present invention.

  The image information extraction apparatus shown in FIG. 1 includes an image input unit 101, a memory 102 that temporarily stores an input image and processing results of each component, and an area dividing unit 103 that divides the input image into blocks. , Representative information extraction unit 104 that extracts representative information based on a value obtained by processing pixel value information in each block, and a starting point determination unit that obtains a starting block for arranging the representative information in order 105, an information sequence forming unit 106 that forms an information sequence including representative information of all or some of the blocks starting from the starting block, and a database (DB) 107 that stores the information sequence.

  Next, the operation in the above configuration will be described.

  FIG. 2 is a flowchart of an outline operation in the first embodiment of the present invention.

  Step 201) First, when an image is input to the image input unit 101, it is stored in the memory 102.

  Step 202) Next, the area dividing unit 103 divides the input image stored in the memory 102 into equal small areas (blocks). When dividing an image into small areas, a method of dividing the image into blocks of a predetermined size (number of pixels) and a method of dividing the image with a fixed number of blocks in the vertical and horizontal directions (in this case, Each block has a different size (number of pixels) for each image), and either method may be selected.

  Step 203) Thereafter, the representative information extracting unit 104 calculates representative information for each block obtained by the region dividing unit 103. Here, the representative information is a process of pixel value information in the block, such as statistical values such as the average value, variance, standard deviation, etc. of the pixel values in the block, and edge strength information obtained by a Gaussian filter, etc. Any value can be used as long as it is obtained. Further, the same type of representative information may be used for all blocks, or different representative information may be used for each block. Furthermore, the color space that is the source of pixel values when obtaining representative information may be a RGB three-component (RGB space), or a RGB space that is converted to a space consisting of only luminance components. It is obvious that a uniform color space close to human perception may be used.

  Step 204) Next, the starting point determination unit 105 obtains a starting point for arranging the representative information for each block obtained by the representative information extracting unit 104 in order. A method of obtaining the starting point will be described with reference to the flowchart of FIG. 3 showing the detailed processing flow of step 204 and FIG.

  FIG. 3 is a flowchart of the process of the starting point determination unit in the first embodiment of the present invention, and FIG. 4 is a diagram showing how the starting point block is determined in the first embodiment of the present invention.

  First, a block near the square of the image is selected (step 301). In FIG. 4A, the image is divided into seven blocks in the vertical and horizontal directions, and blocks located in the upper left (UL), upper right (UR), lower left (LL), and lower right (LR) are selected as blocks near the square. Is shown.

  Next, representative information of the same type is obtained for the blocks near the square, and the average value is calculated (step 302). Note that, in the process of the representative information extraction step 203 in the representative information extraction unit 104, when the same type of representative information is obtained for all the blocks, in step 302, only an operation for obtaining an average may be performed. . Among the average values of the four types of representative information obtained in step 302, a block near the square having the maximum value is obtained, and the block located at the corner is set as the starting point (step 303).

  FIG. 4B shows a state in which the LR is the largest among the blocks near the square, and the block at the corner (the dark block in the figure) is determined as the starting point.

  In the above description, as the blocks around the square, two vertical and horizontal blocks including a corner are targeted. However, in actuality, only the corner (one vertical and one horizontal block) may be used, or three vertical and horizontal blocks. It doesn't matter. Furthermore, in the case of an image that is long in the horizontal direction as shown in FIG. 4C, the number of vertical and horizontal blocks may be changed.

  Step 205) Finally, the information sequence forming unit 106 forms an information sequence including the representative information of each block as a component while selecting the blocks clockwise from the starting point block obtained by the starting point determining unit 105. In FIG. 5 (a), starting from the starting block (the block at the lower right corner) shown in FIG. 4 (b), the blocks are selected in a spiral fashion from the outside to the inside in the clockwise direction. It shows a state. FIG. 5B shows a state of information string generation, and shows a state in which an information string is created by fixing the number of digits of information in order from the representative information of block 1. Also, as shown in FIG. 6, the starting block (lower right corner) in FIG. 4B may be regarded as the upper left of the image, and the block may be selected in a raster format.

[Second Embodiment]
The apparatus configuration diagram and the outline operation flowchart in the present embodiment are the same as those in FIGS. 1 and 2 of the first embodiment, respectively.

  Hereinafter, the operation of the starting point determination step 204 in the starting point determination unit 105 will be described.

  FIG. 7 is a flowchart of the starting point determination unit in the second embodiment of the present invention, and FIG. 8 is a diagram showing how the starting point block is determined in the second embodiment of the present invention.

  First, the outer peripheral blocks of the image are divided by direction, and blocks belonging to each direction are selected as a group (step 304). In FIG. 8A, the image is divided into 7 blocks each in the vertical and horizontal directions, and the upper and lower sides (UP), the lower side (DW), the left side (LT), and the right side (RT) with respect to the outer peripheral blocks of the image. It shows how it was divided. Next, as in the first embodiment, the same type of representative information is obtained for the block selected in step 304, and the average value is calculated (step 302). Note that, when the representative information extraction unit 104 calculates the representative information of the same type for all blocks during the processing of the representative information extraction step 203, in step 302, only the operation for obtaining the average may be performed. Among the average values of the representative information for each direction of the peripheral block obtained in step 302, the average value having the maximum value is obtained, and the block located at the center is set as the starting point (step 303).

  FIG. 8B shows a state in which the middle block of the DW area is determined as the position of the starting block, with DW being the area that is the largest average representative information among the four areas. As described above, the method of determining the starting block is not limited to the method in which the block is located in the middle of the area obtained for each direction, but the block having the largest representative information may be used as the starting point. The next corner may be the starting point. In addition, as for the area of the outer peripheral block by direction, it is not necessary to stick to the upper, lower, left, and right four areas as described above, but eight areas such as the upper left, upper right, lower left, lower right, etc. It is self-evident that it may be targeted.

  In the operation at the information sequence forming step 205 in the next information sequence forming unit 106, as described in the first embodiment, each block is selected while selecting the block starting from the starting block in the clockwise direction. An information string having a representative component is formed. In the information sequence forming step 205 described above, if the starting block is a corner, as in the description in the first embodiment, it is regarded as the upper left position of the image, and the block is selected as the raster format. However, an information sequence having the representative information as a component may be formed. If the starting block is not a corner, it is obvious that the same operation can be performed if the corner that comes first in the clockwise or counterclockwise direction is regarded as the block in the upper left position of the image.

[Third Embodiment]
FIG. 9 is a block diagram of an image information retrieval apparatus according to the third embodiment of the present invention.

  The image information search apparatus shown in FIG. 1 includes an information sequence match determination unit 108 that determines the degree of match with the information sequence of image information corresponding to a plurality of images already stored in the database 107 in the configuration of FIG. In this configuration, an image output unit 109 that outputs images that match in the column match determination unit 108 is added.

  In the present embodiment, it is assumed that an image information sequence has already been obtained and stored in the database 107 according to the first embodiment or the second embodiment, and the same image as the input image is stored in the database 107. A method for searching whether or not there is will be described.

  FIG. 10 is a flowchart of an outline operation of the image information retrieval apparatus in the third embodiment of the present invention.

  In the flowchart shown in FIG. 10, steps 201 to 205 are the same as the flow from the image input unit 101 to the information sequence forming unit 106 in steps 201 to 205 in FIG. 2 in the first embodiment described above. Therefore, the description is omitted.

  Hereinafter, operations of the information sequence match determination unit 108 and the image output unit 109 will be described.

  Step 206) The information sequence match determination unit 108 compares the information sequence of the input image obtained by the information sequence formation unit 106 with the information sequence stored in the database 107. The information string is expressed in a format as shown in FIG. 5B, and the degree of coincidence is calculated based on the method shown below to determine whether or not they are coincident.

  Assume that x is an information string obtained from the input image, and y is an information string of the searched image stored in the database 107.

x = {a ₁ , a ₂ ,..., a _n } (1)
y = {b ₁ , b ₂ ,..., b _n } (2)
However, a _i and b _i (i = 1, 2,..., N) are representative information of each block. n is the total number of blocks.

  At this time, the degree of coincidence D is expressed by the following equation.

D = [{Σ (a _i −b _i ) ² } / n] ^1/2 (3)
The degree of coincidence D expressed by the above equation (3) corresponds to the distance between the point x and the point y in the n-dimensional Euclidean space. This value is determined and, if the threshold th ₁ is smaller than a match. If it is not determined to match, the matching degree D is calculated for the information sequence of other images in the database 107, and the same determination is performed.

  In the above description, the degree of coincidence D between two information strings is determined by regarding each information string as represented by Equation (3) as a point in an n-dimensional space, and the distance between the two points. However, the method for determining the degree of coincidence need not be limited to this, and the following method may be used.

For example, it is possible to make a determination based on an angle formed between vectors by regarding an information string as a vector in an n-dimensional space. In this case, when the following equation θ is smaller than th ₂ , it is determined that they are coincident.

θ = ArcCos [(Σ [a _i · b _i ]) / (| x | · | y |)] (4)
Furthermore, for example, for each representative information corresponding to each block of the information sequence, a method of determining that there is a match when the number M of mismatches is smaller than the threshold th ₃ is possible.

M = Σm _i where m _i = 0 (when a _i = b _i ) (5)
m _i = 1 (when a _i ≠ b _i )
Step 207) Finally, in the image output unit 109, the information sequence match determination unit 108 outputs the image determined to match the information sequence of the input image to an external output device such as a monitor.

[Fourth Embodiment]
Also in this embodiment, an image information search apparatus and its operation will be described.

  FIG. 11 is a configuration diagram of an image information search apparatus according to the fourth embodiment of the present invention.

  The image information search apparatus shown in FIG. 9 has the configuration shown in FIG. 9 and a match possibility determination unit 110 that determines the match possibility with the information sequence of image information corresponding to a plurality of images already stored in the database 107. It is the structure which added.

  In the present embodiment, as in the third embodiment, it is assumed that the information sequence of the image is already stored in the database 107 according to the first embodiment or the second embodiment, and the input image and A method for searching whether or not the same image exists in the database 107 will be described.

  FIG. 12 is a flowchart of an outline operation of the image information retrieval apparatus in the fourth embodiment of the present invention.

  In the flowchart shown in FIG. 12, steps 201 and 202 are the same as the processing from the image input unit 101 to the region dividing unit 103 in steps 201 and 202 in FIG. 2 in the first embodiment. Omitted. Hereinafter, operations after the representative information extraction unit 104 (step 208) will be described.

  Step 208) As described in the first embodiment, the representative information extraction unit 104 is a part for obtaining the representative information of each block obtained by the area dividing unit 103 for the input image. In this mode, first, the possibility of matching is determined, the number of comparison target images is reduced, and at the same time, for the purpose of obtaining the position of the starting point (block) of the information sequence, some blocks are represented. Ask for information. Here, blocks belonging to the outer periphery are selected as some blocks. FIG. 13A shows a state in which blocks are extracted in order of the outer one row in order clockwise from the upper left block as a starting point. Representative information is obtained from the first block to the 24th block.

  Step 209) Based on the representative information of some blocks obtained in step 208, the match possibility determination unit 110 compares the information information sequence of the image in the database 107 and determines the match possibility.

  If it is determined that there is a possibility of matching, the process proceeds to the next step 203. If it is determined that there is no possibility, the operation from step 208 is repeated on the other images of the database 107.

  A method for comparing / determining the information sequence corresponding to the input image and the image in the database 107 in the information sequence match determination step 209 will be described below.

In FIG. 13B, a ₁ , a ₂ ,..., A ₂₄ are obtained by sequentially arranging partial information sequences corresponding to blocks outside the input image, starting from the block located at the upper left of the image, b ₁ , b ₂ ,..., B ₂₄ are partial information strings corresponding to blocks outside the comparison target image in the database 107. The partial information sequence in the database 107 is obtained by arranging representative information of each block in order from the starting block determined based on the method described in the first embodiment or the second embodiment. Yes, the position of the starting block varies depending on the image. For comparison of partial information strings, there is a method of checking in sequence up to b ₂₄ , such as whether it is the same as b ₁ or b ₂ based on a ₁ , but here, in a short time, Describes how to identify match / mismatch.

By using the property that the number of blocks in the image is the same and the direction of the order is the same (clockwise here) and only the starting block is different when extracting the partial information sequence, the number of times is small. Realize the match / mismatch judgment in. Block representative information is extracted from the partial information sequence of the input image at regular intervals. FIG. 13B shows a state where every sixth block is selected. For _{example, a 1, a 7, a} 13, a 19, to pick up. At the same interval of every 6th, even for a substring of b, _first , a combination of b ₁ , b ₇ , b ₁₃ , b ₁₉ is compared, and if they do not match, another combination, that is, b ₂ , B ₈ , b ₁₄ , b ₂₀ , and then the candidates are sequentially shifted one by one, and if there is no match even if the combination of b ₆ , b ₁₂ , b ₁₈ , b ₂₄ is confirmed, there is no match If there is no possibility of matching, if it matches even once, it is determined that there is a possibility of matching.

  Further, in the above description, the comparison is made with every pair of every six blocks, but it is obvious that every fourth or every eight blocks may be used.

  If it is determined that there is a “possibility of matching” by the above operation, the position of the starting block of the image on the database 107 side is also revealed at the same time.

  Step 203) When it is determined that there is a match possibility in the match possibility determination step 209, the representative information extraction unit 104 extracts representative information for all blocks of the input image again.

  In the subsequent procedures, that is, formation of an information sequence in the information sequence formation unit 106 (step 205), determination of the degree of coincidence of the information sequence in the information sequence match determination unit 108 (step 206), and information sequence match determination step 206 Since the procedure until the image determined to match is output to the image output unit 109 (step 207) is the same as that in the third embodiment, the description thereof is omitted.

  In the present invention, the operation of the image information extraction method in the first and second embodiments and the operation of the image information search method in the third and fourth embodiments are constructed as a program. It can be installed and executed on a computer used as an information extraction device or an image information retrieval device, or can be distributed via a network.

  Further, the constructed program is stored in a portable storage medium such as a hard disk device or a flexible disk / CD-ROM, and installed in a computer used as an image information extraction device or an image information retrieval device to be executed. Or can be distributed.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  Further, the constructed program can be stored in a portable storage medium such as a hard disk, a flexible disk, or a CD-ROM, and can be installed or distributed in a computer.

  The present invention is not limited to the above-described embodiments and examples, and various modifications and applications can be made within the scope of the claims.

  The present invention can be applied to a reference service using an image as a key to prevent unauthorized use of content on the Internet or outside, search for content, and images.

DESCRIPTION OF SYMBOLS 101 Image input part 102 Memory 103 Area division part 104 Representative information extraction part 105 Starting point determination part 106 Information sequence formation part 107 DB (database)
108 information string match determination unit 109 image output unit 110 match possibility determination unit

Claims (2)

The same image as the input image, an image information retrieval unit that searches a database of information sequence of a plurality of images are stored,
Area dividing means for dividing the input target image into equal small areas (blocks);
Partial representative information extracting means for extracting representative information based on a value obtained by processing pixel value information in the block for a part of the block divided into regions;
Using the representative information of some blocks obtained by the partial representative information extracting means , a match is performed to determine the possibility of matching with an information sequence of image information corresponding to each of a plurality of images stored in the database A possibility determination means ;
Representative information extraction means for obtaining representative information for all the blocks of the target image when the match possibility determination means determines that there is a match possibility;
Starting point determination means for obtaining the position of a block as a starting point from an information sequence corresponding to an image stored in the database;
Information sequence forming means for forming an information sequence based on representative information for each block obtained by the representative information extracting means , with the starting point block determined by the starting point determining means as the head ;
Determining the degree of coincidence between the information sequence obtained by the information sequence forming unit and the information sequence corresponding to the plurality of images stored in the database that has been determined to have a match possibility by the match possibility determining unit Information string match determination means to perform,
An image output means for outputting a corresponding image when the information string match determination means determines a match;
An image information retrieval apparatus comprising: Computer
The program for functioning as each means of the image information search device of Claim 1.

Images