KR100810002B1 - Normalization method for shape descriptors and image retrieval method using the same - Google Patents

Abstract

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates to a normalization method for calculating shape descriptors, an image retrieval method using the same, and a computer-readable recording medium recording a program for realizing the above methods.

2. The technical problem to be solved by the invention

In order to compensate for this, the general shape descriptor used to search for a specific image required by the user in a wide image database is vulnerable to affine transformed objects such as noise in the image, distortion in one direction, and application of perspective. A normalization method capable of obtaining a shape descriptor invariant to affine transformation, an image retrieval method using the same, and a computer-readable recording medium recording a program for implementing the methods.

3. Summary of Solution to Invention

In the image normalization method, an object image is converted into a standard image in a deformation due to movement, rotation, size change, noise, or distortion of an object in the image, and as a result, various shape descriptors are added to the image. A first image normalization step of applying a shape descriptor that is invariant to the affine transformation, and normalizing the image by applying a compact algorithm to noise, shift, size change, or distortion during deformation; And a second image normalization step of normalizing an image by calculating a rotation invariant shape descriptor using Jonikey-based moments or AR (AR) transforms after going through the compact algorithm to extract an object of invariably similar shape to the rotation angle. Included.

4. Important uses of the invention

The present invention is used for character image search, trademark search, and the like.

Shape Descriptor, Image Retrieval, AR Transform, Affine Transform, Normalization, Compact Algorithm

Description

Normalization method for shape descriptors and image retrieval using the same             

1 is an exemplary configuration of a content-based image retrieval system to which the present invention is applied.

2 is a flowchart of an embodiment of an image retrieval method according to the present invention;

3 is a diagram illustrating performance improvement test results for experimental data for MPEG-7 international standard according to an exemplary embodiment of the present invention.

* Explanation of symbols for the main parts of the drawings

10: video input device 20: storage device

30: Q & A device

The present invention relates to a normalization method for calculating shape descriptors in a content-based image retrieval system, an image retrieval method using the same, and a computer-readable recording medium recording a program for realizing the above methods. The present invention relates to a preprocessing process for calculating a revolving blower shape descriptor used for retrieval.

In general, the shape information of an object in an image is very useful for recognizing an object. General descriptors of this shape information are very robust to object rotation, movement, size change and noise. However, these general shape descriptors are very vulnerable to objects that are distorted or obliquely to one side, objects to which perspective is applied, that is, Affine transformed objects, and therefore, the same objects are likely to be recognized as different objects.

However, in the conventional calculation of the rotation invariant descriptor, the normalization process of changing the overall shape of the object, such as an obliquely inclined object or an object to which the perspective is applied, is absent even if the object is not normalized. There was no.

Therefore, in order to obtain a shape descriptor that can improve retrieval performance among various descriptors used in retrieval of still images, a method of obtaining an invariant descriptor for affine conversion is required as a preprocessing process of shape descriptor calculation. .

The present invention has been proposed to meet the above-mentioned demands, and a general shape descriptor used to search for a specific image requested by a user in an extensive image database may have noise in the image, distortion in only one direction, and application of perspective. Because it is vulnerable to phosphorous-converted objects, to compensate for this, a normalization method that can obtain a shape descriptor that is invariant to affine transformation, an image retrieval method using the same, and a computer-readable recording medium recording a program for realizing the methods. The purpose is to provide.

In order to achieve the above object, the present invention provides a method for normalizing an image, wherein the object image is converted into a standard image when the object is transformed due to movement, rotation, size change, noise, or distortion in the image. A first image normalization step of applying various shape descriptors to a shape descriptor to obtain an invariant shape descriptor for affine transformation, and normalizing an image by applying a compact algorithm to noise, shift, size change, or distortion during deformation; And a second image normalization step of normalizing an image by calculating a rotation invariant shape descriptor using Jonikey-based moments or AR (AR) transforms after going through the compact algorithm to extract an object of invariably similar shape to the rotation angle. Characterized in that the made up.

On the other hand, the present invention, in the image retrieval method in the content-based image retrieval system, the movement of the object by normalizing the image by applying a compact algorithm during the movement, rotation, size change, noise, or distortion of the object in the image An image normalization step of obtaining a normalized image that is invariant to size change, or distortion; Calculating a shape descriptor from the normalized image; And searching for an image by calculating a similarity level in the image database using the calculated shape descriptor.

The present invention for achieving the above object, in the image normalization system equipped with a processor, by transforming the object image to a standard image at the time of deformation due to the movement, rotation, size change, noise, or distortion of the object in the image, As a result of applying various shape descriptors to the image to obtain an invariant shape descriptor for affine transformation, and normalizing the image by applying a compact algorithm to noise, shift, size change, or distortion during deformation; And a method for normalizing an image by calculating a rotation invariant shape descriptor using Jonikey-based moment or AR transform after going through the compact algorithm to extract an object invariably similar to the rotation angle. Provide a computer readable recording medium having recorded thereon.

Meanwhile, the present invention provides a content-based image retrieval system equipped with a processor to normalize an image by applying a compact algorithm when moving, rotating, changing size, noise, or distortion in an object in the image. The ability to obtain a normalized image that is invariant to changes, or distortions; Calculating a shape descriptor from the normalized image; And a computer-readable recording medium having recorded thereon a program for realizing a function of searching for an image by calculating similarity in the image database using the calculated shape descriptor.

The present invention can be used as a preprocessing process of the calculation of the shape descriptor among various descriptors used in the retrieval of still images and moving images, and it is possible to obtain a shape descriptor that can further improve the search performance.

In the normalization method, even when a search based on the shape of an object (such as a character image or a trademark image search) is performed in the image search, distortion of the object image or inclination due to perspective may occur. It can be recognized as the same object.

After this normalization method, the shape descriptor can be calculated to obtain a descriptor invariant to the affine transformation. That is, if the normalization method is applied to the input image, it is converted into a standard image, and if various types of rotation invariant descriptors are applied to the standard image, the shape descriptor invariant to the affine transformation is obtained. The shape descriptor thus obtained can be used to search for various images.

Therefore, by using the normalization method as a preprocessing process before calculating the shape descriptor, it is possible to compensate for the inclination by the enlargement / reduction in one direction or the perspective in the image.

The shape descriptor preprocessed by the present invention can be applied in various fields, and can be usefully used for all content-based searches that focus on objects in the image. In particular, image descriptors having high value-added intellectual property such as character image search and trademark search can be used. Can also be used for searching.                     

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary configuration diagram of a content-based image retrieval system to which the present invention is applied, and will be described by taking a content-based image retrieval system through image similarity weight adjustment as an example.

As shown in FIG. 1, the content-based image retrieval system using the image similarity weight control according to the present invention receives an image from an operator, stores the image, extracts and stores a feature (shape descriptor) of the stored image, and calculates the similarity. And a video input device 10 for generating and storing a small image having a predetermined size, a storage device 20 for storing an image, a feature vector, a small image, and a similarity table, and receiving a query from a user. It includes a query response device 30 for outputting a small image according to the similarity of the query, receiving the user's selection information on the output small image to change the similarity weight and output the image according to the result.

The image input apparatus 10 divides the image currently input under the control of the database input unit 11 into a small image and a database input unit 11 for analyzing and databaseting the image as a database input request from the operator is input. The feature vector directory database 24 generates a feature vector for a certain number of extraction specifications for the image currently input under the control of the database input unit 11 and the small image generation unit 12 stored in the small image directory database 21. The feature extractor 13 for storing the information in the feature extractor 13 and a predetermined number of extracted feature vectors, and calculates the similarity with respect to each feature of the feature vector of the other image in the image directory database 22, Normalize the similarity of the features to have the same importance, and normalize the similarity in each feature Is combined with a single similarity to form an index, and then the similarity calculator 14 stores the constructed index in the index database 23.

The storage device 20 includes a small image directory database 21 storing a small image directory, an image directory database 22 storing an image directory, an index database 23 storing a similarity table, A feature vector directory database 24 for storing feature vector directories is provided.

The query response device 30 analyzes the query image to output a similarity table from the index database 23, and a small image corresponding to the similarity table output by the query unit 32. The result display unit 33 reads from the directory database 21 and shows it to the search user, and receives a feedback signal input from the user, adjusts the weight, obtains the similarity according to the adjusted weight, and obtains the similarity from the corresponding index database 23. A weight adjusting unit 31 for outputting the table to the result display unit 23 is provided.

The operation of the content-based image retrieval system through weight adjustment to which the present invention having the above configuration is applied will be described in detail as follows.

The database input unit 11 stores the image to be processed in the image directory database 22 and then drives the small image generating unit 12 to display a new large image (Thumbnail, that is, a large image to facilitate screen display). A small image of about 100 pixels) is stored in the small image directory database 21.

In addition, since there is no feature vector for the new image, the feature extractor 13 is driven to extract the invariant feature quantities for the shape of the new object image to form the feature vector, and the feature vector is extracted for each feature. Stored in the directory database 24.

The similarity calculator 14 calculates and calculates similarity for each feature with the feature vector of another image in the image directory database 22 using a predetermined number of feature vectors extracted by the feature extractor 13. The similarity for each feature is normalized to have the same importance, the normalized similarity in each feature is combined into a single similarity, and then made into a table to generate an index and stored in the index database 23.

The similarity calculator 14 is driven by the control of the database input unit 11 when the feature extraction operation by the feature extractor 13 is completed for all new images input by the operator.

As described above, the normalized similarity in each feature of all images is calculated as a single combined similarity in the similarity calculator 14 to form a table, and when the similarity calculator 14 generates an index, Databaseization ends.                     

The query unit 32 represents a query tool used when a general user wants to search for an image in the image directory database 22 that has been input. After analyzing the query image according to a user's search request, the feature extractor 13 ) And the similarity calculator 14 transmits the image retrieved from the index database 23 to the result display unit 33.

In this case, when the user selects a desired image and an undesired image and inquires again about the result shown on the result display unit 33, the weight adjusting unit 31 analyzes the user's opinion and adjusts the weight for each feature, Again, the result of the changed weight is derived from the index database 23 and sent to the result display unit 33.

The query unit 32 may be a sample image that is arbitrarily selected and displayed by the result display unit 33, or another image data or a user input from an external image input apparatus such as an image drawn by the user (ie, a picture) or a scanner. It can be a specified color and its ratio.

In addition, the result display unit 33 receives the input image searched from the index database 23 and retrieves the small image from the small image directory database 21 and shows it to the search user.

Now, as a preprocessing of the shape descriptor using Zernike Moment, a rotation invariant shape descriptor used for content-based image retrieval, or the AR transform shape descriptor calculation, the normalization method to obtain the shape descriptor invariant to the affine transformation It will be described in more detail.

Normalization of an image refers to a process of transforming a distorted or deformed image into a standardized image of a certain form. Such deformations include noise in an object in the image, movement of the object, rotation, size change in one or two directions during x-y, and perspective change. This variant is normalized using a compact algorithm. From the normalized object image, we can calculate the Zernike Moment or AR transform shape descriptors, which are rotation invariant shape descriptors.

The affine transformation on the 2D image may be expressed as Equation 1 below.

In Equation 1, [uv 1] ^T is an affine transformed point of point [xy 1] ^T and A is an affine matrix. Therefore, in the above Equation 1, size change, rotation, inclination due to perspective, and transformation appear.

Equation 1 shows a relationship between two points associated by an affine matrix. However, it is often difficult to obtain affine coefficients here, because it is not only obvious to find the corresponding two points on two images, but also sensitive to changes caused by noise. Therefore, Equation 2 below provides a new form of affine relationship (using the equation of moment).

In Equation (2), m _pq is the moment of the pq order in the normalized image f (x, y), m ' _pq is the moment of the pq order of the affine transformed image f (u, v). The moment definition of the pq order is shown in Equation 3 below.

The purpose of the compact algorithm is to adjust the image through two linear transformations so that the covariance matrix of the algorithmic image is an identity matrix.

In one image, the covariance matrix may be calculated as shown in Equation 4 below.

Assume that matrix E consists of rows of eigenvectors of matrix C (the eigenvectors corresponding to the smallest eigenvalues are placed as the last row). As a result, the rotation transformation matrix E can be made as shown in Equation 5 below.

Here, e _ix and e _iy are eigenvectors corresponding to respective eigenvalues. The covariance matrix C 'of the transformed image is expressed by Equation 6 below.

As a result, the converted image becomes an image with no correlation in the new coordinate system.

The next step is to transform the coordinate system by moving the origin to the center of the image and multiplying the coordinates by the matrix E. In this way, the new coordinate system matches the eigenvectors of C. If [u, v] ^T is the new coordinate, the following equation (7) is established.

In Equation (7), the coordinate system is rotated to remove the correlation in the new coordinate system. Therefore, the size of each element can be dealt with independently. The final goal is to obtain an image with a covariance matrix, such as a transformed identity matrix. Thus, the size of the two new coordinates corresponding to the eigenvalues is converted as shown in Equation (8).

Here, the magnitude transformation matrix is defined as follows.

In Equation (9), c is a size conversion constant.                     

Now, from the affine-converted image f (u, v), an algorithm for obtaining an image f (x ', y') that has undergone a compact algorithm such as Equation (10) can be obtained.

As a result, a normalized image invariant to the movement, the change in size, and the distortion of the object using Equation 10 may be obtained.

Improved retrieval efficiency can be obtained by using AR transform from the normalized image.

According to the present invention has the following effects.

First, the search efficiency can be improved when searching the image database. 3 shows that the performance is improved when using the normalization method. This shows the results of performance improvement experiments on experimental data for the MPEG-7 (MPEG-7) international standard (scale of evaluation: BEP).

In general, image search is performed by using three kinds of descriptors such as color, shape, and texture. Among them, the efficiency of searching by shape is improved. Image search, which is important for shape technicians, includes character image search and trademark image search.

Second, since the present invention is a normalization method which is a preprocessing process before applying the shape descriptor, after the step, any shape descriptor can be applied to improve the search efficiency.

As an example, an image search method according to the present invention will be described with reference to FIG. 2.

As shown in FIG. 2, the image retrieval method according to the present invention first receives an original image (201), and performs affinity of an original image as a preprocessing process of calculating an AR transform shape descriptor used for content-based image retrieval. A normalization method is performed (202). Normalization of an image refers to a process of making a distorted or deformed image into a normalized image. Such deformations include noise, movement, rotation, size change, and distortion of objects in the image. Noise, shifts, magnitude variations, and distortions during these strains are normalized using a compact algorithm. Normalization for rotation can be done by going through a compact algorithm and computing either ART-based shape descriptors or Jonikey moment-based shape descriptors.

Then, the shape descriptor is calculated (feature extraction) using the AR transform on the normalized image (203), the similarity is calculated in the image database using the calculated shape descriptor (204), and the image is searched and retrieved. Output 205.

The method of the present invention as described above may be implemented as a program and stored in a computer-readable recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.).

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited to the drawings shown.

The present invention as described above is the same even if the distortion of the object, the inclination by perspective, etc., which are often prone to occur when conducting a search based on the shape of the object (character image, trademark image search, etc.) in the image search. By allowing the object to be recognized as an object, there is an effect that can be usefully used for a content-based search that focuses on the objects in the image except the natural image search.

Claims (10)

In the image normalization method, In case of deformation due to movement, rotation, size change, noise, or distortion of the object in the image, the object image is converted to the standard image, and as a result, various shape descriptors are applied to the image, resulting in an invariant shape descriptor for affine transformation. But get A first image normalization step of normalizing an image by applying a compact algorithm to noise, movement, magnitude change, or distortion during deformation; And A second image normalization step of normalizing an image by calculating a rotation invariant shape descriptor using Jonikey-based moments or AR (AR) transforms after going through the compact algorithm to extract an object invariably similar to the rotation angle. Normalization method for calculating the shape descriptor comprising a. The method of claim 1, The compact algorithm, Adjust the image through two linear transformations so that the covariance matrix of the image that has undergone the algorithm is an identity matrix, and move, change, or distort the object from the affine transformed image f (u, v) by the following equation. And a normalized image f (x ', y') which is invariant to.

(here,

(Where c is the size conversion constant),

(Where e _ix and e _iy are eigenvectors corresponding to their respective eigenvalues), m _pq is the moment of order _pq in the normalized image f (x, y), and m ' _pq is the affine transformed image f (u, moment of order pq of v)

)being) The method according to claim 1 or 2, The first image normalization step, Using the compact algorithm, noise, object movement, rotation, simple size change on the xy coordinate axis, or size change only on one side, or obliquely tilted or perspective are applied to the object in the image. In the case of distortion, a normalization method for calculating a shape descriptor, wherein the object image is first normalized to a standard image by adjusting the covariance matrix calculated from the coordinate values of the object on the xy coordinate axis to be the same. The method of claim 3, wherein The second image normalization step, A normalization method for calculating a shape descriptor, characterized by normalizing a feature amount of an object image by calculating a feature amount for a rotation invariant shape descriptor such as a Jonikey moment or an AR transformation. An image retrieval method in a content-based image retrieval system, An image normalization step of normalizing an image by applying a compact algorithm when moving, rotating, changing size, noise, or distortion in an object in the image to obtain a normalized image invariant to movement, size change, or distortion of the object; Calculating a shape descriptor from the normalized image; And Searching for an image by calculating a similarity level in an image database using the calculated shape descriptor Image search method comprising a. The method of claim 5, The shape descriptor, Image retrieval method, characterized in that the rotation invariant descriptor. The method according to claim 5 or 6, The image normalization step, An image retrieval method characterized by normalizing by applying a compact algorithm for shifting, size change, noise, or distortion during deformation to obtain a shape descriptor that is invariant to affine transformation for the image after preprocessing. The method of claim 7, wherein The compact algorithm, Adjust the image through two linear transformations so that the covariance matrix of the image that has undergone the algorithm is an identity matrix, but the object's movement, size change, and distortion can be changed from the affine transformed image f (u, v) by the following equation. And a constant normalized image f (x ', y').

(here,

(Where c is the size conversion constant),

(Where e _ix and e _iy are eigenvectors corresponding to their respective eigenvalues), m _pq is the moment of order _pq in the normalized image f (x, y), and m ' _pq is the affine transformed image f (u, moment of order pq of v)

)being) In an image normalization system with a processor, In case of deformation due to movement, rotation, size change, noise, or distortion of the object in the image, the object image is converted to the standard image, and as a result, various shape descriptors are applied to the image, resulting in an invariant shape descriptor for affine transformation. But get A function of normalizing an image by applying a compact algorithm to noise, shift, size change, or distortion during deformation; And Function to normalize the image by calculating rotation invariant shape descriptor using Jonikey-based moment or AR (AR) transformation after going through the compact algorithm to extract an object of invariably similar shape to rotation angle A computer-readable recording medium having recorded thereon a program for realizing this. In the content-based image retrieval system having a processor, A function of obtaining a normalized image invariant to movement, size change, or distortion of an object by applying a compact algorithm to normalize the image when the object moves, rotates, changes in size, noise, or distortion in the image; Calculating a shape descriptor from the normalized image; And A function of searching for an image by calculating a similarity in an image database using the calculated shape descriptor A computer-readable recording medium having recorded thereon a program for realizing this.

Images