WO2008060022A1

WO2008060022A1 - System and method for evaluating and certifying image identifier

Info

Publication number: WO2008060022A1
Application number: PCT/KR2007/002469
Authority: WO
Inventors: Weongeun Oh
Original assignee: Electronics And Telecommunications Research Institute
Priority date: 2006-11-13
Filing date: 2007-05-22
Publication date: 2008-05-22

Abstract

A system and method for evaluating and certificating image identifying software are provided. The system includes: a receiving module for receiving image identifying software as an object to evaluate and certificate and a certificate application related to the image identifying software; an identifying test module for identifying modified image data using the received image identifying software; an evaluating module for evaluating the received image identifying software using the identifying result from the identifying test module; and a certificate generating module for generating a certificate using statistic data and evaluating result from the evaluating module.

Description

SYSTEM AND METHOD FOR EVALUATING AND CERTIFYING IMAGE IDENTIFIER

Technical Field

[1] The present invention relates to a system and method for evaluating and certificating software for identifying an image, and more particularly, to an evaluating and certifying system and method for defining an evaluating system and a format of a certificate to evaluate a software product for identifying an image modified by duplication or malicious attack, or transformed according to a type of Internet or a reproducing device and to certify the software product according to the evaluating result in order to allow a user to select a proper software product based on the contents of the certificate of an image identifying technology. Background Art

[2] As the demand of digital contents increases, an enormous amount of multimedia contents including video, audio, and image have been produced, distributed and serviced constantly. Among the multimedia contents, numerous digital still images (hereinafter images) have been explosively produced and distributed due to the popularization of high-performance portable digital cameras and the price drop of large capacity storage devices or portable storage mediums. There were many researches in progress to develop a technology for effectively searching a target image from numerous images or accurately identifying the target image from others. Such a technology refers to an image searching technology.

[3] Conventionally, many image searching methods using meta data (key word) or contents of an image have been introduced. Since such conventional image searching methods use previously inputted information or original information included in an image, for example, color and texture, the conventional image searching methods were only useful to search an original image which is not transformed or modified. The conventional image searching methods, however, cannot be used if the image is unlawfully modified through duplication or malicious attack, or if the original characteristics of the image, for example, the size, the format, and the quality, change according to a type of Internet or a reproducing device. In order to search such modified image, a technology for identifying a modified image is needed. Such a technology refers an image identifying technology.

[4] As the amount of using images increases, numerous image identifying methods have been introduced, and it is expected that more image identifying methods will be introduced because many related researches have been in progress. However, none of the conventional image identifying methods can perfectly identify a modified image in every possible situation. That is, the conventional image identifying methods may or may not identify a modified image according to a certain environment. Therefore, the performance thereof may differ according to each of image identifying software products. However, users cannot be provided with objective evaluation information for conventional image identifying software products, and developers also do not have a chance to confirm or to advertise the superiority of their image identifying software products.

Disclosure of Invention Technical Problem

[5] Accordingly, the present invention is directed to a system and method for evaluating and certifying a software product for identifying an image, which substantially obviates one or more problems due to limitations and disadvantages of the related art.

[6] It is an object of the present invention to provide a system and method for evaluating the performances of image identifying technologies durable for various modifications and automatically issuing certificates having the evaluating results.

[7] It is another object of the present invention to provide a system and method for evaluating and certifying image identifying software, which automatically processes an evaluating system and a format of a certificate to evaluate and certify image identifying technologies and objectively and systemically compares and analyzes the performances of image identifying software in order to allow a user to select a proper technology based on the contents of the certificate of the image identifying technology. Technical Solution

[8] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[9] To achieve these objects and other advantages and in accordance with the purpose of the invention, as embodied and broadly described herein, there is provided a system for evaluating and certificating image identifying software including: a receiving module for receiving image identifying software as an object to evaluate and certificate and a certificate application related to the image identifying software; an identifying test module for identifying modified image data using the received image identifying software; an evaluating module for evaluating the received image identifying software using the identifying result from the identifying test module; and a certificate generating module for generating a certificate using statistic data and evaluating result from the evaluating module.

[10] The system may further include an evaluating environment setup module for setting up evaluating environment suitable to the image identifying software when the certificate application is received.

[1 1] In the evaluating environment, evaluation criteria, evaluation parameters, and evaluation scenario may be automatically setup according to an application field suitable to the image identifying software,

[12] The modified image data may be obtained by modifying an image stored in a test image database based on a modification algorithm setup according to the evaluating environment. The modification algorithm may include non-geometric modification and geometric modification.

[13] The non-geometric modification may include brightness change, color/monochrome conversion, JPEG compression, color reduction, Gaussian noise transform, histogram equalization, and sharpness change. The geometric modification may include rotation, scaling, translation, flip, crop, and skew.

[14] The system may further include a transmission module for transmitting the certificate created by the certificate generating module to an external authority.

[15] The certificate may include performance evaluating information having information about identifying accuracy, computation complexity and calculator size. The identifying accuracy may be shown as SR=K/(1+N) where SR denotes identifying accuracy, K denotes the number of modified images of identified query image, I denotes a query image, and N denotes the number of modified images.

[16] In another aspect of the present invention, there is provided a method for evaluating and certificating image identifying software including the steps of: receiving image identifying software as an object to evaluate and certificate and a certificate application related to the image identifying software; identifying modified image data using the received image identifying software; evaluating the received image identifying software using the identifying result from the identifying test module; and generating a certificate using statistic data and evaluating result from the evaluating module.

[17] The step of receiving the image identifying software may further include the step of setting up an evaluating environment suitable to the image identifying software when the certificate application is received.

[18] The step of identifying modified image data may include generating modified image data by modifying an image stored in a test image database based on a modification algorithm setup according to the evaluating environment.

[ 19] The method may further include the step of transmitting the composed certificate to an external authority. [20] It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

Advantageous Effects [21] A system and method for evaluating an image identifying software product according to an embodiment of the present invention can objectively and systemically compare and analyze the performances of image identifying software products which are already developed or will be developed. [22] The system and method for evaluating an image identifying software product according to an embodiment of the present invention shows a direction to a developer to develop superior image identifying technology and allows a user to select a suitable image identifying technology. [23] The system and method for evaluating an image identifying software product according to an embodiment of the present invention can provide objectively certified mutual trust through systemically evaluating and certificating an image identifying software product.

Brief Description of the Drawings [24] The accompanying drawings, which are included to provide a further understanding of the invention, are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the principle of the invention. In the drawings: [25] FIG. 1 is a block diagram illustrating a system for evaluating and certifying image identifying software according to an embodiment of the present invention; [26] FIG. 2 is a block diagram illustrating a system for evaluating and certifying image identifying software according to an embodiment of the present invention; and [27] FIG. 3 is a flowchart illustrating a method for evaluating and certifying image identifying software according to an embodiment of the present invention.

Best Mode for Carrying Out the Invention [28] Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. [29] Hereinafter, a system and method for evaluating and certifying image identifying software according to an embodiment of the present invention will be described with reference to accompanying drawings. [30] FIG. 1 is a block diagram illustrating a system for evaluating and certifying image identifying software according to an embodiment of the present invention. [31 ] The system for evaluating and certifying image identifying software according to an embodiment of the present invention (hereinafter, an evaluating/certifying system) includes a receiving module 110, an evaluating environment setup module 130, an identifying test module 150, an evaluating module 170, a certificate generating module 180, a test image DB 120, and a modified image DB 140.

[32] The receiving module 110 receives image identifying software and a certificate application for certifying the image identifying software. The image identifying software denotes software developed to identify modified images. The certificate application includes information about a certificate authority, a certificate applicant, and an image identifying software product. Table 1 shows detailed information included in the certificate application.

[33] Table 1 [Table 1] [Table ] Certificate application

[34] The certificate authority information includes a name and address of a certificate authority and authority identification (ID) number. The certificate applicant information includes a name and address of a certificate applicant and an applicant identification (ID) number. The image identifying product information includes a product identification (ID) number, a patent number, and the description of an algorithm.

[35] The evaluating environment setup module 130 automatically sets up an evaluating environment according to an application field suitable to the received image identifying software when the receiving module 110 receives the certificate application. The evaluating environment includes evaluation criteria, evaluation parameters, and evaluation scenario. The certificate authority may provide the information about the application field.

[36] The application field of the image identifying software is a wide scope, for example, copyright protection, illegal duplication and copying prevention, digital library, data authentication, copy control, and finger printing. Also, the evaluation criteria and parameters may be variously set up according to the application field.

[37] The types of images and modified images stored in the image DB 120 and the modified image DB 140, and the evaluation parameters may diversely change according to the application field. Furthermore, the types and the evaluation parameters may change while evaluating.

[38] According to needs, the evaluating environment setup module 130 can additionally setup supplementary tools for a support system to evaluate an image identifying software and the evaluating environment thereof.

[39] The supplementary tools for the support system and the evaluating environment thereof includes information about classifying of application fields for image identifying software, setting up of evaluation reference suitable to application field, evaluating environment (signal processing and DB tool), and developing of statistical processing method and environment (spread sheet and statistical process).

[40] The support system for evaluating image identifying software supports the applicants or the certificate authority to easily setup evaluating environment although they are not experts,

[41] The image database for evaluating image identifying software includes a test image

DB 120 and a modified image DB 140.

[42] The test image DB 120 stores color images, and the color images are used to calculate a first threshold value in order to calculate a success rate of an image identifying software product to identify a predetermined image. A threshold value of an algorithm for identifying an original input image using an image identifying software product may be defined by dividing the sum of threshold values of identified images by the total number of calculation. For example, when the test image DB 120 include 50,000 color images, the threshold value of the algorithm is 50,000 x (50,000 - 1)12.

[43] When the test image DB 120 includes 50,000 color images, the modified image DB

140 stores 2,000 images selected from 50,000 color images by analyzing energy values of a spatial domain and a frequency domain in the color images. The selected 2,000 images are diversely modified using modification algorithms, and the modified images are used to evaluate the performance of an image identifying software product.

[44] The image modification algorithms can be classified into non-geometrical modification and geometrical modification as follows.

[45] A. non-geometrical modification.

[46] 1.1. brightness change

[47] It changes the brightness of an input image at following rates.

[48] ±20%, ±10%, ±5% [49] 1.2. color/monochrome conversion [50] It converts an input image by following equation. [51] I = 0.299R + 0.587G + 0.114B [52] In equation, R, G, and B denote Red, Green, and Blue component, respectively. [53] 1.3. JPEG compression [54] It compresses an input image based on JPEG compression at quality factor (QF) 65, 80, and 95. The Joint Photographic Experts Group (JPEG) compression is the standard of lossy compression method for compressing images, which is defined by international organization for standardization (ISO) and ITU telecommunication standardization sector (ITU-T). QF denotes a method for measuring the quality of an image.

[55] 1.4. color reduction [56] It reduces the number of bits that represent a pixel color of an input image from 24-bits (R=8bits, G=8bits, B=8bits) to 16-bits or to 8-bits.

[57] 1.5. Gaussian noise [58] It changes the standard deviation of an input image varies by 8.0, 4.5, and 2.5. [59] 1.6. histogram equalization [60]

[61] 1.7. blurring [62] It degrades the sharpness of an input image using following masks. [63] 5*5, 3*3 [64] B. geometrical modification [65] 2.1. rotation [66] It rotates an input image at following angles. [67] 45°, 25°, 10° [68] 2.2. scaling [69] It scales down a width and a height of an input image at following rates. [70] 50%, 70%, 90% [71] 2.3. translation [72] It shifts the location of an input image at following rates. [73] 40%, 20%, 10% [74] 2.4. flip [75] It changes the right and left of an input image [76] 2.5. aspect ratio change [77] It changes a horizontal/vertical ratio of an input image at following rates, and performs a crop modification.

[78] 4:3 -> 16:9 + crop [79] 2.6. crop [80] It crops the width and height ratio of an input image and a predetermined size of an input image at following rate.

[81] 50%, 70%, 90% [82] 2.7. skew [83] It transforms an input image using following equation. That is,

06 changes as much as +/-10°, +/-6°and +1-2° ,

[84]

[85] In the equation, OC denotes a skew angle, (u, v) denotes a coordinate of an input image, and

denotes a coordinate of a skewed image.

[86] 2.8. perspective [87] It modifies an input image using following equation. In the equation, changes as much as +/-10 °, +1-6 °, +/-2 °.

[88]

T T

[ x_p, y_p, P] =M_Vϊoj ' [W- V₅ O, !]

[89] where (u_o,v_o,w) denotes a coordinate of a input image. [90]

^■MprςT

[91] Herein, d always has a positive value. A horizontal axis and a vertical axis are defined as x-axis and y-axis, respectively. _x denotes a rotation angle, and w and h denote a width and a height of an input image.

[92] The identifying test module 150 performs an identifying operation for identifying an image modified according to the evaluating environment set by the evaluating environment setup module 130 using the received image identifying software product. The identifying operation is performed in the evaluating environment set by the evaluating environment setup module 130 to test the received image identifying software product according to a predetermined test image, an image modification function, evaluation criteria, evaluation parameters, and evaluation scenario.

[93] The evaluating module 170 statistically processes the result of identifying a modified image, which is performed at the identifying test module 150, according to evaluation criteria, a statistical processing method, and a statistic environment, which are set by the evaluation environment processing module 130.

[94] The evaluation environment set by the evaluation environment setup module 130 may change according to the application field of image identifying software. That is, the application field of the image identifying software is a wide scope, for example, copyright protection, illegal duplication and copying prevention, digital library, data authentication, copy control, and finger printing. Also, the evaluation criteria and parameters may be variously set up according to the application field. [95] The types of images and modified images and the evaluation parameters may diversely change according to the application field. The types and the evaluation parameters may also change while evaluating. [96J Particularly, following supplementary tools and environments are needed to evaluate an image identifying software product.

[97] 1) classifying application fields of an image identifying software product

[98] 2) setting up an evaluating criteria suitable to an application field

[99] 3) evaluation environment (for example, signal processing, D/B tool)

[100] 4) developing a statistical processing method and an environment (for example, spread sheet, statistical processing package).

[101] The evaluating module 170 uses the tools to calculate statistical data related to reliability and performance for identifying a modified image based on the identifying operation result. The evaluating module 170 calculates information about identifying accuracy, computing complexity, an calculator size of the image identifying software product with the information about reliability.

[102] The certificate generating module 180 generates a certificate including the statistical processing result from the evaluating module 170. Therefore, the certificate generating module 180 composes the certificate according to a certificate format shown in Table 2 with the result of testing the predetermined image identifying software product according to a test image, an image modification function, evaluation criteria, evaluation parameters, and evaluation scenario, which are set in the evaluation environment. [103] Table 2

[Table 2]

[Table ]

Format of certificate

[104] [105] The performance evaluating information included in the certificate includes identifying accuracy, computing complexity, and a calculator size.

[106] The identifying accuracy is represented by SR=K/(I+N). SR denotes Success Ratio, I denotes a query image, N denotes the number of modified images, and K denotes the number of modified images of identified query image I. [107] The computation complexity represents the complexity of computation to identify a query image using a predetermined image identifying software product using commands and an amount of memory used.

[108] The calculator size shows the size of data to express a calculator by the number of bits.

[109] FIG. 2 is a block diagram illustrating a system for evaluating and certifying image identifying software according to an embodiment of the present invention.

[110] A certificate applicant transmits a certificate application 211 and an image identifying software product to the certificate authority through the Internet according to an interface and a communication protocol assigned by the certificate authority. Then, the evaluating/certificating system according to the present embodiment receives the transmitted certificate application 211 and the image identifying software product.

[I l l] The certificate authority performs an identifying test according to the certificate application and an environment setup by the image identifying software. Then, the certificate authority issues a certificate 270 having the evaluating result to an owner of an image identifying software product or the certificate applicant. The certificate authority also builds a certificate database and manages the issued certificates.

[112] The evaluating/certificating system 200 according to the present embodiment sets up an evaluating environment including evaluation criteria and parameters 262 and evaluation scenario 263 according to application fields 261.

[113] After setting up the evaluating environment, the image identifying module 251 performs an identifying operation for images and modified images stored in the test image DB 221 and the modification tool DB 222 using the received image identifying software product.

[114] The modified image is image data generated by modifying an image stored in the test image DB 221 according to a modification algorithm 256 setup by the evaluating environment.

[115] The evaluating module (Image Identifier Evaluation Tool) 252 calculates statistical data of reliability and performance based on the identifying result. The evaluating module 252 calculates identifying accuracy, computation complexity, and a calculator size of the image identifying software product with information about reliability.

[1 16] The evaluating/certificating system 200 composes a certificate according to a predetermined certification application form 254 based on the calculating result of the evaluating module (Image Identifier Evaluation Tool) 252, and transmits the composed certificate to the certificate applicant through the Internet.

[117] FIG. 3 is a flowchart illustrating a method for evaluating and certifying image identifying software according to an embodiment of the present invention.

[118] The method for evaluating and certifying image identifying software according to the present embodiment is performed as follows. At step S301, an image identifying software product and a certificate application related thereto are received as an object of evaluating and certificating from an external certificate application authority or a certificate applicant.

[119] At step S302, an evaluating environment including an evaluation DB, attack function, evaluation criteria, evaluation parameters, and evaluation scenario is automatically setup according to the application files of the received image identifying software product when the certificate application is received.

[120] After setting up the evaluating environment, modified image data is generated by modifying test image data according to various modification algorithms step S3O3.

[121] After setting up the evaluating environment and generating the modified image data, the modified image is identified using the received image identifying software at step S304.

[122] Then, statistical data related to reliability of the image identifying software product is calculated using the identifying result at step S305.

[123] Finally, a certificate including the calculated reliability statistical information is composed and transmitted to the external certificate applying authority or the certificate applicant at step S306.

[124] The method for evaluating and certificating an image identifying software product according to the present embedment can be embodied into a computer readable recording medium including CD-ROM, RAM, floppy disk, hard disk, and optical magnetic disk.

[125] It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention. Thus, it is intended that the present invention covers the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.

[126]

Claims

[1] A system for evaluating and certificating image identifying software comprising: a receiving module for receiving image identifying software as an object to evaluate and certificate and a certificate application related to the image identifying software; an identifying test module for identifying modified image data using the received image identifying software; an evaluating module for evaluating the received image identifying software using the identifying result from the identifying test module; and a certificate generating module for generating a certificate using statistic data and evaluating result from the evaluating module.

[2] The system of claim 1, further comprising an evaluating environment setup module for setting up evaluating environment suitable to the image identifying software when the certificate application is received.

[3] The system of claim 2, wherein in the evaluating environment, evaluation criteria, evaluation parameters, and evaluation scenario are automatically setup according to an application field suitable to the image identifying software.

[4] The system of claim 2, wherein the modified image data is obtained by modifying an image stored in a test image database based on a modification algorithm setup according to the evaluating environment.

L5] The system of claim 4, wherein the modification algorithm includes non- geometric modification and geometric modification.

[6] The system of claim 5, wherein the non-geometric modification includes brightness change, color/monochrome conversion, JPEG compression, color reduction, Gaussian noise transform, histogram equalization, and sharpness change.

[7] The system of claim 5, wherein the geometric modification includes rotation, scaling, translation, flip, crop, and skew.

[8] The system of claim 1, further comprising a transmission module for transmitting the certificate created by the certificate generating module to an external authority.

[9] The system of claim 1, wherein the certificate includes performance evaluating information having information about identifying accuracy, computation complexity and calculator size.

[10] The system of claim 9, wherein the identifying accuracy is shown as

SR=K/(1+N) where SR denotes identifying accuracy, K denotes the number of modified images of identified query image, I denotes a query image, and N denotes the number of modified images. [11] The system of claim 9, wherein the computation complexity is calculated based on commands and an amount of memory used. [12] A method for evaluating and certificating image identifying software comprising the steps of: receiving image identifying software as an object to evaluate and certificate and a certificate application related to the image identifying software; identifying modified image data using the received image identifying software; evaluating the received image identifying software using the identifying result from the identifying test module; and generating a certificate using statistic data and evaluating result from the evaluating module. [13] The method of claim 12, wherein the step of receiving the iamge identifying software further includes the step of setting up an evaluating environment suitable to the image identifying software when the certificate application is received. [14] The method of claim 13, wherein in the evaluating environment, evaluation criteria, evaluation parameters, and evaluation scenario are automatically setup according to an application field suitable to the image identifying software. [15] The method of claim 13, wherein the step of identifying modified image data includes generating modified image data by modifying an image stored in a test image database based on a modification algorithm setup according to the evaluating environment. [16] The method of claim 12, further comprising the step of transmitting the composed certificate to an external authority. [17] A computer readable recording medium storing a program for executing a method of claim 12 in a computer.