JP2007018174A - Information processing unit and control method therefor, computer program, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for discriminating an image data type, and a high-speed image retrieval technique using the above discrimination technique. <P>SOLUTION: An information processing unit includes an acquisition means for obtaining image data constituted of one or more tiles, each at least including a header having coding information in regard to the coding of the tile concerned, and a discrimination means for discriminating the image type of each tile based on the above coding information. The above tile is divided on the basis of a plurality of frequency components. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for quickly identifying the type of image data and a technique for high-speed image retrieval using the technique.

  Recent high-performance computers, storage devices such as hard disk drives (HDDs) and large-capacity memories, digital cameras, digital videos, scanners, printers, multifunction peripherals (MFPs), digital copiers, etc. With the popularization and high performance of digital cameras, opportunities to handle digital images are increasing. Also, with the widespread use of high-speed Internet lines, it is not uncommon to send and receive these images. An MFP, a digital copying machine, a computer, or the like equipped with an HDD may be used as an image server.

  With such a background, the digital images that are handled are becoming increasingly high-definition and large-scale. As data definition increases, each data size tends to increase, and an image compression / decompression algorithm such as JPEG or JPEG2000 that has a high compression ratio while maintaining high image quality is essential and standardized. In addition, an image search algorithm has been proposed in order to solve the problems associated with the increase in volume.

  Japanese Patent Laid-Open No. 2004-133867 has a configuration in which image data is extracted in advance by analyzing image data, and similar images are searched at high speed by comparing the feature amounts based on the arrangement of the image feature amounts. It is disclosed.

Also, a technique for determining the type of image when performing an image search is known. Patent Document 2 discloses a method of creating a histogram of image data, discriminating a binary image when the frequency distribution is biased, and discriminating a natural image when there is no bias. Further, in Patent Document 3, when the correlation between the target pixel and the neighboring pixels in the vicinity thereof is strong, it is determined as a computer graphics image, and when the correlation between the target pixel and the peripheral pixels is weak, it is determined as a natural image. Is disclosed. In Patent Document 4, when the number of colors of image data is 1, it is determined as a single color image, when the number of colors is small, it is determined as a palette image, and when the number of colors is large, it is determined as a natural image. Is disclosed.
Japanese Patent Laid-Open No. 11-288418 JP 2000-222564 A JP 2002-15327 A JP-A-11-88700

  However, the configuration disclosed in Patent Document 1 has a problem in that it takes time for preprocessing and initial search because image feature amounts must be extracted from image data to be searched in advance. In addition, when an operation such as movement, deletion, addition, or editing is performed on an image file outside the management of the image database, the image feature amount is extracted again for the image data, and the pre-registered image feature amount is obtained. Had to be updated.

  Further, in the image discrimination methods disclosed in Patent Documents 2 to 4, all of the image pixel values are read and analyzed to determine the type of the image. Therefore, a memory for reading the image data is required, and a time for analyzing the image data is required.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for discriminating the type of image data and a technique for high-speed image search using the technique.

According to the present invention,
Acquisition means for acquiring image data composed of one or more tiles, wherein the tile includes at least a header including encoding information about encoding of the tile;
Discrimination means for discriminating the image type for each tile based on the encoding information;
With
An information processing apparatus is provided in which the tile is divided into a plurality of frequency components.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which discriminate | determines the kind of image data, and the technique of the high-speed image search using the said technique can be provided.

  Embodiments according to the present invention will be described below in detail with reference to the accompanying drawings. However, the constituent elements described in this embodiment are merely examples, and are not intended to limit the scope of the present invention only to them.

[About JPEG2000 format]
In the present embodiment, the JPEG2000 algorithm is used as one of the image file formats compressed and encoded according to the hierarchization algorithm. Therefore, first, the JPEG2000 format will be described. Since the JPEG2000 algorithm is well known from various documents, only the outline of the portion related to the present embodiment will be described here.

  When saving an image in the JPEG2000 format, the image can be divided into rectangles of the same size called tiles. FIG. 3 is a diagram schematically showing tile division of a JPEG2000 image.

  For each tile, as shown in FIG. 4, compression coding is performed in the order of color transformation, discrete wavelet transformation, quantization, entropy coding, and bitstream formation. FIG. 4 is a diagram showing an encoding process procedure according to the JPEG2000 algorithm.

  In FIG. 4, the color conversion 401 is a process of converting a component from a color system such as RGB composed of R (red), G (green), and B (blue) to a color system such as YCbCr.

  The discrete wavelet transform 402 is a process for decomposing the image subjected to color transformation in 401 into a frequency space using the discrete wavelet transform. FIG. 5 is a schematic diagram showing discrete wavelet transform according to the JPEG2000 system. The tile image (0LL) in (a) is subjected to discrete wavelet transform and decomposed into subbands 1LL, 1HL, 1LH, and 1HH (b). Subsequently, a discrete wavelet transform is applied to the low frequency component 1LL to decompose it into subbands 2LL, 2HL, 2LH, and 2HH (c). In this example, two-level transformation is performed, but the number of discrete wavelet transformations is not particularly limited.

  The quantization of 403 is a process of linearly quantizing the transform coefficient of the discrete wavelet transform.

  As shown in FIG. 6, the entropy encoding 404 is a process of performing encoding in the order of precinct division, code block division, bit plane division, division into coding paths, and binary arithmetic coding. FIG. 6 is a diagram showing an entropy encoding procedure according to the JPEG2000 system.

  In FIG. 6, the precinct division 601 is a portion that divides the subband coefficients such as 2HL and 2LH into areas called precincts. FIG. 7 is a diagram schematically showing precinct division according to the JPEG2000 system. In FIG. 7, a portion 701 is obtained by frequency-converting the portion of the same region in the original image, and these portions belong to the same precinct.

  The code block division 602 is a process of dividing the precinct into smaller areas called code blocks. FIG. 8 is a schematic diagram showing code block division according to the JPEG2000 system. This code block unit is a basic unit when entropy coding is performed.

  The bit plane division of 603 is a process of developing the linearly quantized discrete wavelet transform coefficients for each code block for each bit. FIG. 9 is a schematic diagram showing bit plane division according to the JPEG2000 system.

  In FIG. 9, reference numeral 901 exemplifies transform coefficients of the linear quantized discrete wavelet transform in a certain code block. Reference numeral 902 denotes a bit string representing the sign of the transform coefficient 901. A value 0 means a positive value and a value 1 means a negative value. Reference numeral 903 denotes a bit plane in which the absolute value of the transform coefficient 901 is binary-developed from MSB (Most Significant Bit) to LSB (Least Significant Bit).

  For example, since the transform coefficient (904) having a value of +12 is a positive value, the corresponding sign bit is 0 (905). Since the binary representation of the absolute value 12 of +12 is (1100), the values of the corresponding portions 906a to 906d of the bit plane are “1”, “1”, “0”, and “0”, respectively. Similarly, since the conversion coefficient (907) having a value of −6 is a negative value, the corresponding sign bit is 1 (908). Also, since the binary representation of the absolute value 6 of −6 is (0110), the values of the corresponding portions 909a to 909d of the bit plane are “0”, “1”, “1”, and “0”, respectively.

  A bit plane that is all zeros on the MSB side is called a zero bit plane, and no data is stored. On the other hand, the number of zero bit planes described later is counted for each code block.

  The division into the coding path 604 is a process of further dividing the bit plane into a signature propagation path, a magnesium refinement path, and a cleanup path. FIG. 10 is a schematic diagram showing division into coding passes according to the JPEG2000 system.

  As shown in FIG. 10, each bit plane 1001a to 1001d (hereinafter collectively referred to as 1001) is divided into coding propagation paths 1002b to 1002d (hereinafter collectively referred to as 1002) by dividing into coding paths. , Divided refinement paths 1003b to 1003d (hereinafter collectively referred to as 1003) and cleanup paths (1004b to 1004d). However, the most significant bit (MSB side) bit plane 1001a is made to correspond only to the cleanup path (1004a). Hereinafter, the cleanup paths 1004a to 1004d are collectively referred to as 1004.

  Each bit plane 1001 and each coding pass 1002 to 1004 have the same size according to the coordinate length in the vertical and horizontal directions. Each coding pass 1002 to 1004 has a position where a bit value is defined and a position where it is not defined. In FIG. 10, for example, positions where bit values are defined, such as 1006 and 1007, are shaded (shaded). The bit values defined in the shaded portions of the coding paths 1002 to 1004 (for example, 1002b to 1004b) are equal to the bit values at corresponding positions on the bit plane 1001 (for example, 1001b) before the division.

  The process of dividing the bit plane into coding passes is executed based on the following (Process 1) and (Process 2).

  (Processing 1) For the most significant bit (MSB side) bit plane, a corresponding cleanup path is generated. The cleanup path corresponding to the most significant bit plane has the same bit value as the most significant bit plane defined at all positions. For example, for the bit plane 1001a, a cleanup path 1004a in which the same bit value as that of the bit plane 1001a is defined at all positions is generated.

  (Process 2) The following (Process a) to (Process c) are performed on all the second and subsequent bit planes in order from the second most significant bit plane.

(Processing a) Generation of a signature propagation path A signature in which a bit value is defined at a position corresponding to the surrounding of a position where a bit value 1 is defined in any of the bit planes higher than the bit plane to be processed Generate a propagation path. For example, when (processing a) is performed on the bit plane 1001b, since the bit value 1 is defined at the position 1005 in the bit plane 1001a that is higher than the bit plane 1001b, the position 1006 corresponding to the periphery of the 1005 A significance propagation path 1002b in which a bit value is defined is generated. The bit value defined at position 1006 is equal to the bit value at the same position in bit plane 1001b.

(Processing b) Generation of a magnesium refinement path A magnesium refinement path in which a bit value is defined at a position corresponding to a position where a bit value 1 is defined in any bit plane higher than the bit plane to be processed. Generate. For example, when (processing b) is performed for the bit plane 1001b, since the bit value 1 is defined at the position 1005 in the bit plane 1001a that is higher than the bit plane 1001b, the bit value is set at the position 1007 corresponding to 1005. Generate a definition refinement path 1003b. The bit value defined at the position 1007 is equal to the bit value at the same position in the bit plane 1001b.

(Processing c) Generation of a cleanup path A cleanup path in which bit values are defined at all positions other than the positions where the bit values are defined is generated in the signature propagation path and the magnesium refinement path. For example, when performing (processing c) on the bit plane 1001b, bit values are defined at all positions other than the position 1006 where the bit value is defined at 1002b and the position 1007 where the bit value is defined at 1003b. A cleanup path 1004b is generated. The bit value defined in the cleanup path 1004b is equal to the bit value at the same position in the bit plane 1001b.

  In the example of FIG. 10, the bit planes 1001c and 1001d are also divided into coding passes by repeating (Process 2).

  A coding pass is distributed to one or more layers. FIG. 11 is a schematic diagram showing layer division according to the JPEG2000 system. As shown in FIG. 11, the layers are divided and distributed at the boundary of each coding pass of each code block. In general, the efficiency is determined from the increase in the amount of code and the degree of image quality improvement when each coding path is transmitted, and layer division is performed.

  The binary arithmetic encoding 605 is a process of arithmetically encoding the data after the coding pass division.

  After entropy coding 404 is performed as described above, a bit stream for writing data to a file is formed at 405. Data having the same layer hierarchy, resolution level (decomposition level of discrete wavelet transform), component, and position (precinct) are collected to form one packet. Further, the packet includes a zero length packet indicating whether the packet length is zero (empty packet), a code block inclusion indicating whether the current code block is already included in a packet in the previous layer, zero bit A packet header including the number of planes, the number of coding passes, and the length of the compressed image data of the code block is included.

  An image of one tile is expressed by collecting the number of layer layers × the number of resolution levels × the number of components × the number of precincts. All of these packets may be combined into one tile part as shown in FIG. 12A or may be divided into a plurality of tile parts as shown in FIG. As shown in FIG. 12, the tile part header is attached to the head of the tile part.

  Furthermore, as shown in FIG. 13, one image is expressed by arranging tile parts. FIG. 13 is a schematic diagram showing an arrangement of tile parts according to the JPEG2000 system. The tile parts may be arranged in the order of the tiles of the image as shown in FIG. 13A, or may be arranged in order from the tile to be prioritized as shown in FIG. 13B. When a tile is divided into a plurality of tile parts, the tile parts are arranged as shown in FIG.

[Hardware configuration of information processing device]
The information processing apparatus according to the present embodiment is realized by an information processing apparatus such as a personal computer (PC), a workstation (WS), or a personal digital assistant (PDA). Next, the hardware configuration of the information processing apparatus will be described with reference to FIG. FIG. 1 is a block diagram showing a hardware configuration of the information processing apparatus according to the present embodiment.

  In FIG. 1, reference numeral 101 denotes a CPU that executes various controls of the information processing apparatus according to the present embodiment. A ROM 102 stores a boot program executed when the information processing apparatus is started up and various data. Reference numeral 103 denotes a RAM which stores a control program for processing by the CPU 101 and provides a work area when the CPU 101 executes various controls.

  A keyboard 104 and a mouse 105 provide various input operation environments for the user. An external storage device 106 includes a hard disk, a floppy (registered trademark) disk, a CD-ROM, a DVD-ROM, and the like.

  Reference numeral 107 denotes a display that displays processing contents and processing results and transmits them to the user. A network interface 108 enables communication with each device (not shown) on the network. Reference numeral 109 denotes an interface (I / F) such as IEEE 1394 or USB, which communicates with devices such as the scanner 110 and the digital camera 111. Reference numeral 112 denotes a bus connecting the above-described components.

  In the above configuration, the scanner 110, the digital camera 111, and the external storage device 106 may be replaced with those arranged on the network. Moreover, it can also be comprised as an alternative of a hardware apparatus with the software which implement | achieves a function equivalent to the above each apparatus.

  In the present embodiment, an example in which the program and related data according to the present embodiment are directly loaded into the RAM 103 from the external storage device 106 and executed is shown, but the present invention is not limited to this. For example, it is possible to record the program according to the present embodiment in the ROM 102, configure the program so as to form a part of the memory map, and execute the program directly by the CPU 101.

  In the present embodiment, for convenience of explanation, a configuration in which the information processing apparatus according to the present embodiment is realized by a single device will be described. However, a configuration in which resources are distributed to a plurality of devices may be realized. For example, storage and calculation resources may be distributed in a plurality of devices. Alternatively, resources may be distributed for each component virtually realized on the information processing apparatus, and parallel processing may be performed.

[Functional configuration of information processing device]
FIG. 2 is a block diagram showing a functional configuration of the information processing apparatus according to the present embodiment. Each functional block shown in FIG. 2 is realized by the CPU 101 of the information processing apparatus described above with reference to FIG. 1 executing a program loaded in the RAM 103 and cooperating with each hardware shown in FIG. The Of course, some or all of the functional blocks may be realized by dedicated hardware.

  In FIG. 2, reference numeral 201 denotes a search source image, which is designated by the user. An image storage unit 202 exists on the external storage device 106 and stores one or more image data. The information processing apparatus according to the present embodiment searches for image data similar to the search source image 201 among the image data stored in the image storage unit 202 in a later-described similar image search process.

  An image header reading unit 203 reads the image header for each of the search source image 201 and the image data (hierarchically encoded) stored in the image storage unit 202. As will be described later, the image header describes coding information such as the number of coding passes, the length of the compressed image data of the code block, and the number of zero bit planes.

  An image type determination unit 204 analyzes the image header portion and determines the image type based on the encoded information described in the image header. An image type label matrixing unit 205 arranges the image types determined by the image type determining unit 204 as labels, and creates an image type label matrix. Details of processing (image type label matrix creation processing) executed by the image type determination unit 204 and the image type label matrixing unit 205 will be described later.

  Reference numeral 206 denotes an image type label matrix comparison unit. Details of the processing (image type label matrix comparison processing) executed by the image type label matrix comparison unit 206 will be described later.

  An image data reading unit 207 reads the hierarchically encoded image data from the search source image 201 and the image storage unit 202.

  Reference numeral 208 denotes an image feature amount extraction unit. Reference numeral 209 denotes a feature quantity label matrixing unit that arranges the image feature quantity labels obtained by the image feature quantity extraction unit 208 to create a feature quantity label matrix. Details of processing (feature amount label matrix creation processing) executed by the image feature amount extraction unit 208 and the feature amount label matrixing unit 209 will be described later.

  Reference numeral 210 denotes a feature amount label matrix comparison unit. Details of processing (feature amount label matrix comparison processing) executed by the feature amount label matrix comparison unit will be described later.

  An operation example of the information processing apparatus according to this embodiment having the above-described configuration will be described below.

[Similar image search processing]
The similar image search process will be described with reference to the flowchart of FIG. FIG. 14 is a flowchart showing a procedure of similar image search processing executed by the information processing apparatus according to the present embodiment.

  First, in step S1401, designation of a similar search source image (search source image) 201 is received from the user. The information processing apparatus according to the present embodiment displays, for example, the identification information of the similar search source image 201 on the display unit 107 so as to be selectable, and accepts input from the keyboard 104, the mouse 105, and the like.

  Next, in step S1402, an image type label matrix is created for the designated similar search source image 201 by an image type label matrix creation process described later.

  In step S1403, a feature amount label matrix is created for the similarity search source image 201 by a feature amount label matrix creation process described later.

  Next, in step S1404, one of the image data stored in the image storage unit 202 is selected as a similar comparison destination image, and the selected similar comparison destination image is subjected to image type label matrix creation processing to be described later. Create a label matrix.

  Next, in step S1405, the image type label matrix of the similar search source image 201 is compared with the image type label matrix of the similar comparison target image selected in step S1404, and whether or not the image type label matrix does not match. Determine whether. If comparison is not possible due to a difference in tile shape or the like, or if the comparison results indicate a match (NO in step S1405), the process advances to step S1406. If the comparison results in a mismatch (YES in step S1405), the process advances to step S1408. The comparison of the image type label matrix is executed based on an image type label matrix comparison process described later.

  In step S1406, a feature amount label matrix is created for the similar comparison target image selected in step S1404 by a feature amount label matrix creation process described later.

  In step S1407, the feature amount label matrix of the similar search source image 201 is compared with the feature amount label matrix of the similar comparison target image selected in step S1404 to calculate the similarity. The comparison of the feature amount label matrix is executed based on a feature amount label matrix comparison process described later.

  In the similar image search processing, the processing from step S1404 to step S1407 is repeatedly executed so that all registered data (image data) stored in the image storage unit 202 becomes a similar comparison destination image.

  In step S1408, it is determined whether or not the processing from step S1404 to step S1407 has been executed for all the image data stored in the image storage unit 202. If it is being executed (YES in step S1408), the process advances to step S1409. If not executed (NO in step S1408), the process returns to step S1404, image data that has not yet been processed in steps S1404 to S1407 is selected from the image storage unit 202, and the processes in steps S1404 to S1407 are performed. Execute.

  In step S1409, the identification information of the similar comparison destination images is sorted in descending order of similarity obtained in step S1407, and the process proceeds to step S1410.

  In step S1410, the sorted list of identification information of the similar comparison target images is displayed on the display unit 107 as a search result and presented to the user. Then, the process ends.

[Image type label matrix creation processing]
Next, the image type label matrix creation process executed in steps S1402 and S1404 will be described. The image type label matrix creating process includes an image type determining process executed by the image type determining unit 204 and an image type label matrixing process executed by the image type label matrixing unit 205.

(Image type discrimination processing)
The image type determination process is performed by determining the type of image (in the example of this embodiment, whether it is a photograph (natural) image, a character image, or a graphic image) in units of tiles divided when the image is stored. This is a process for determining. The image type is determined based on at least one of coding information such as the number of coding passes, the length of the compressed image data of the code block, and the number of zero bit planes written in the packet header. .

  When discriminating image types based on the number of coding passes, the total number of coding passes in the low frequency component (2LL in FIG. 5C) and the total number of coding passes in the high frequency components (1HL, 1LH, 1HH in FIG. 5C). Compare When the ratio of the total number of coding passes in the high frequency component to the total number of coding passes in the low frequency component exceeds a predetermined value, it is a character image / graphic image, and in the high frequency component with respect to the total number of coding passes in the low frequency component When the ratio of the total number of coding passes is below a predetermined value, it is determined that the image is a natural image.

  When the number of bit plane layers is plural, the number of coding passes may be calculated for each layer and determined. In that case, by using the number of coding passes of the highest layer, discrimination with higher accuracy becomes possible.

  When the image type is determined based on the length of the compressed image data of the code block (compressed image data length), the length of the compressed image data of the code block and the high frequency component (FIG. 5) in the low frequency component (2LL in FIG. 5C). The lengths of the compressed image data of the code blocks in (c) 1HL, 1LH, 1HH) are compared. When the ratio of the length of the compressed image data of the code block in the high frequency component to the length of the compressed image data of the code block in the low frequency component exceeds a predetermined value, it is a natural image, and the code block in the low frequency component If the ratio of the length of the compressed image data of the code block in the high-frequency component to the length of the compressed image data is less than a predetermined value, it is determined that the image is a character image / graphic image.

  When the image type is determined based on the number of zero bit planes, if the number of zero bit planes in the high frequency components (1HL, 1LH, 1HH in FIG. 5C) exceeds a predetermined value, the image is a natural image. When the number of zero bit planes in the component (1HL, 1LH, 1HH in FIG. 5C) is below a predetermined value, it is determined that the image is a character image / graphic image.

  The method based on at least one of the coding information such as the number of coding passes, the length of the compressed image data of the code block, the number of zero bit planes, etc., determines the image type with a certain accuracy. It is empirically known that it is possible.

  As described above, in the configuration according to this embodiment, the amount of memory used is reduced by reading only the information written in the header portion and analyzing only the header information without reading the entire image data of the image file. It is possible to perform image type discrimination with little and high speed.

  The image type discriminating unit 204 assigns a label of 0 for tiles that are discriminated as photograph (natural) images as a result of executing the image type discriminating process, and 1 for tiles discriminated as character images / graphic images. That is, for each tile, information for identifying the tile and label information indicating the type of image are associated and stored in a storage device such as the RAM 103 or the external storage device 106.

(Image type label matrix processing)
The image type label matrixing process is a process of creating the image type label matrix by arranging the label information obtained as a result of the discrimination by the image type discrimination process based on the arrangement of the corresponding tiles in the entire image data.

  FIG. 15 is a diagram for explaining a tile order sequence when an image type label matrix is created. In FIG. 15, 1501 schematically illustrates image data, and 1502 schematically illustrates tiles constituting the image data 1501. The numbers described in the tiles 1502 are serial numbers indicating the tile arrangement order. The image type label matrixing unit 205 arranges the image type label information based on the serial numbers of the divided image tiles illustrated in FIG. 5 to create a label matrix. In the present embodiment, the label matrix is a matrix in which label information is two-dimensionally arranged corresponding to the arrangement of tiles, but a matrix arranged one-dimensionally in a predetermined order is also called a label matrix.

[Image type label matrix comparison processing]
Next, the image type label matrix comparison process executed by the image type label matrix comparison unit 206 in step S1405 will be described.

  When the tile division mode is the same between the search source image 201 and the comparison destination image, a simple comparison is made between the image type label matrix of the search source image 201 and the image type label matrix of the comparison destination image to match or match Judge whether to do. That is, it is determined whether or not the values of the label information at the corresponding tile positions (component values of the image type label matrix) are all equal. If they are all equal, it is determined that they match. Note that the tile division mode is equal when the image data is divided into tiles, the number of tiles in the vertical direction is equal to the number of tiles in the horizontal direction, that is, the number of rows and the number of columns in the image type label matrix. Means that both are equal.

  When the tile division mode is different between the search source image 201 and the comparison destination image, the following processing is performed. That is, for each of the search source image 201 and the comparison target image, the components of the corresponding image type label matrix are decomposed into a plurality of groups (grouping) so that both the number of rows and the number of columns of the image type label matrix are equal. And processing for combining the components included in the group into one for each group. The combined image type label matrix is determined to match when the component values at the corresponding positions of the search source image 201 and the comparison target image are all equal, and is determined to be mismatched when they are not equal.

  Here, the process of combining the plurality of components of the corresponding image type label matrix into one corresponds to the process of combining a plurality of tiles into one tile for the image data. In addition, when the relative position of the tile boundary in the image data after combining a plurality of tiles is close between the search source image 201 and the comparison target image, the similarity can be determined with high accuracy. .

For this reason, in this embodiment, the grouping of the components of the image type label matrix is executed so as to satisfy the following condition.
(1) The number of rows of the image type label matrix after synthesis is the number of rows of the image type label matrix corresponding to the search source image 201 and the number of rows of the image type label matrix corresponding to the comparison destination image before synthesis. The greatest common divisor.
(2) The number of columns of the image type label matrix after combining is the number of columns of the image type label matrix corresponding to the search source image 201 before combining and the number of columns of the image type label matrix corresponding to the comparison destination image. The greatest common divisor.

This is equivalent to grouping tiles so as to satisfy the following conditions.
(1) The number of tiles in the vertical direction after tile combination is the greatest common divisor of the number of tiles in the vertical direction of the search source image 201 before combination and the number of tiles in the vertical direction of the comparison target image.
(2) The number of tiles in the horizontal direction after combining the tiles is the greatest common divisor of the number of tiles in the horizontal direction of the search source image 201 before combining and the number of tiles in the horizontal direction of the comparison target image.

  By grouping the matrix components (tiles) in this way, the matrix components (tiles) can be grouped so that the number and arrangement of the matrix components (tiles) included in the group related to the synthesis are all equal. it can. Since grouping and synthesizing matrix components in this way corresponds to combining the tiles so that the relative position in the image data of the tile boundary is close to the search source image 201 and the comparison target image, The determination can be performed with high accuracy.

Note that the synthesis of matrix components (combination of tiles) is executed so as to satisfy the following conditions.
(Condition 1) If the total number of matrix components after combination (combined tiles) is less than a predetermined value, the image type label matrix cannot be compared and the process is terminated.
(Condition 2) When the labels of tiles to be combined, that is, when the component values included in the group related to composition are all 1 (character image / graphic image), the label (component value) after combination (composition) is set to 1. . If the ratio of 0 (photo image) of the labels of the tiles to be combined, that is, the ratio of the component values with the value 0 included in the group related to combination exceeds a predetermined value, the combined label (after combination) Component value) is set to 0. In other cases, the image type label matrix cannot be compared and the process is terminated.

  In addition, said conditions can be changed suitably according to a use and the objective. For example, instead of (Condition 1), “If the number of combined tiles exceeds a predetermined value, the image type label matrix comparison is impossible and the process is terminated”.

  As described above, by determining the similarity of images based on the result determined in the image type determination process, the similarity of images can be determined at high speed.

[Feature label creation process]
Next, the feature amount label matrix creation process executed in steps S1403 and S1406 will be described. The feature amount label matrix creation process includes a feature amount extraction process executed by the image feature amount extraction unit 208 and a feature amount label matrix formation process executed by the feature amount label matrixing unit 209. In the present embodiment, a color label described later is used as the image feature amount.

(Feature extraction process)
First, the image is divided into a plurality of blocks. In this embodiment, an image is divided into vertical and horizontal blocks. FIG. 16 is a diagram showing an example of block division in this embodiment. As shown in FIG. 16, in this embodiment, it is assumed that the image is divided into a total of 9 × 3 × 3. Note that the 3 × 3 division used in the present embodiment is for illustrative purposes only, and any division form can be adopted. Actually, for example, for natural images, the number of divisions is preferably 10 × 10 or more. In addition, if the product is shown on a white plain background, the number of divisions is preferably 13 × 13 or more. The mode of block division can be freely selected regardless of the image division unit in the JPEG2000 standard such as tile.

  FIG. 17 is a diagram for explaining a multidimensional feature amount space in the present embodiment. As shown in FIG. 17, the multidimensional feature space (RGB color space) is divided into a plurality of blocks (color blocks), that is, cells (color cells), and each cell (color cell) is unique by a serial number. Label is given. In the example of FIG. 17, the RGB color space is divided into 3 × 3 × 3 = 27 blocks (cells). Here, the multi-dimensional feature amount space (RGB color space) is divided into a plurality of blocks each having a fixed space in order to absorb strict differences in image feature amounts (color, RGB value). This is because the degree of similarity of feature amounts is compared with high accuracy by giving a certain width to the range.

  Next, image feature amount calculation processing determined for each divided block is performed to determine which cell in the multidimensional feature amount space belongs to and a corresponding label. This image feature amount calculation processing is performed as follows, for example. That is, first, RGB values are acquired for all pixels included in the processing target block, and color cells belonging to the respective pixels are obtained. Then, the label of the color cell having the largest number of belonging pixels is determined as the parameter label (color label) of the divided image block. Such processing is performed for all blocks.

  In the above example, the configuration in which the RGB value is used as it is as the image feature amount is described, but the configuration according to the present embodiment is not limited to this. For example, analysis using a principal component analysis or the like is performed on a plurality of sample image data in advance, and an image feature amount normalization condition that accurately reflects image similarity is obtained, and this normalization is performed. You may comprise so that an image feature-value calculation process may be performed on condition.

(Feature label matrix processing)
After assigning parameter labels to each block as described above, a parameter label matrix (hereinafter referred to as a label matrix) is generated by arranging the parameter labels assigned to each block in a predetermined block order.

  Also in the case of generating a label matrix, a serial number is assigned to each divided image block as shown in FIG. 15, and the parameter labels are arranged based on this serial number to create a label matrix. In the present embodiment, the label matrix is a matrix in which the label information is two-dimensionally arranged corresponding to the arrangement of the blocks, but the one arranged in one dimension in a predetermined order is also called a label matrix.

[Feature Label Label Comparison Processing]
Next, the processing (feature amount label matrix comparison processing) performed by the feature amount label matrix comparison unit 210 in step S1407 to perform similarity comparison (similarity calculation) between feature amount label (color label) matrices is shown in FIG. Will be described with reference to FIG. FIG. 18 is a diagram illustrating an example of a penalty matrix between labels used when comparing label matrices and obtaining similarity.

  In FIG. 18, reference numerals 1801 and 1802 denote component values (labels) of feature quantity label matrices to be compared. The values shown in the table are penalties indicating the similarity between two labels to be compared, and the smaller the penalty value, the more similar the labels are. For example, as in 1803, the penalty of label 2 and label 6 is “7”. Further, for example, as in 1804, the penalty between the same labels is naturally “0”.

  The purpose of this matrix is to perform distance determination according to the similarity of labels. That is, in this embodiment, since the RGB color space is used as the feature amount space, the distance determination according to the similarity of colors can be performed. A penalty matrix is defined so that a penalty (distance) is reduced between adjacent cells when pattern matching between labels is performed, and a large penalty is given to a distant cell.

  In the present embodiment, distances are obtained with reference to the penalty matrix in FIG. 18 for the label values at corresponding positions in the feature amount label matrix of the search source image 201 and the feature amount label matrix of the search target image. Then, the similarity is obtained based on the penalty values obtained for all the components (labels) of the feature amount label matrix. The similarity is calculated so as to be an increasing function for each penalty value. For example, the similarity can be the sum of distances (penalties) for all labels in the label row.

FIG. 19 is a diagram for explaining the calculation of the distance between label strings by matching. For example, in the example of FIG. 19, since the label column 1901 of the search source image 201 is “112313441” and the label column 1902 of the search target image is “1132244452”, matching is performed using the penalty matrix of FIG. The penalty column is “002217011”. Therefore, if similarity is the sum of penalties,
The distance (final solution), that is, the similarity is
0 + 0 + 2 + 2 ++ 1 + 7 + 0 + 1 + 1 = 14
It becomes.

  In this embodiment, the similarity search is performed by expressing the feature quantity as a label. However, the similarity search may be performed without labeling the feature quantity.

  As described above, since the feature amount label matrix creation processing and the feature amount label matrix comparison processing are executed only for the image data determined to be similar in the image type determination processing in step S1405, the image search can be executed at high speed. .

[Other Embodiments]
According to the above configuration, high-speed image retrieval can be provided without extracting image feature amounts in advance, but it is also possible to extract image feature amounts in advance and use the image type discrimination matrix as a pre-search. It is. As described above, since the amount of calculation related to the image feature amount extraction processing is small, it can be performed at high speed even when pre-processing is performed. When pre-processing is performed, a higher-speed image search can be realized.

  Further, although color information is selected as the image feature amount in each of the above embodiments, the embodiment according to the present invention is not limited to this, and other image parameters (for example, luminance values and edge histograms) are set for each image division block. It is also possible to implement it by asking for.

  In each of the above embodiments, an example of recognition based on one feature amount has been described. However, a high-speed search based on a plurality of feature amounts is performed by performing a logical operation on a search result with other feature amounts. It is also possible.

  When performing a search using a plurality of image feature amounts for one image, the similarity obtained in the above configuration is regarded as one new image feature amount, and a multivariate analysis using a plurality of parameters is performed. It is also possible to perform a search using a statistical distance measure. In the above-described embodiment, similar images having a degree of similarity exceeding a predetermined value are obtained as search results. However, a predetermined number of images may be output as search results in order from images with high similarity. Needless to say.

  Note that the present invention can be applied to an apparatus (for example, a copier, a facsimile machine, etc.) composed of a single device even if it is applied to a system composed of a plurality of devices such as a host computer, an interface device, a reader, and a printer. You may apply.

  Another object of the present invention is to supply a storage medium storing software program codes for implementing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can also be achieved by reading and executing the program code stored in the.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is the block diagram which showed the hardware constitutions of the information processing apparatus which concerns on this embodiment. It is the block diagram which showed the function structure of the information processing apparatus which concerns on this embodiment. It is the figure which showed typically the tile division | segmentation of a JPEG2000 image. It is the figure which showed the encoding process sequence based on a JPEG2000 system. It is the schematic diagram which showed the discrete wavelet transform based on a JPEG2000 system. It is the figure which showed the entropy encoding procedure which concerns on a JPEG2000 system. It is the figure which represented the precinct division | segmentation based on a JPEG2000 system typically. It is the model showing the code block division | segmentation based on a JPEG2000 system. It is the model showing the bit plane division | segmentation which concerns on a JPEG2000 system. It is a schematic diagram showing the division | segmentation into the coding pass based on a JPEG2000 system. It is a schematic diagram showing layer division according to the JPEG2000 system. It is the schematic diagram which showed the structure of the tile part which concerns on a JPEG2000 system. It is the schematic diagram which showed the arrangement | sequence of the tile part which concerns on a JPEG2000 system. It is the flowchart which showed the procedure of the similar image search process which concerns on this embodiment. It is a figure explaining the tile order sequence at the time of creating an image kind label matrix. It is a figure which shows the example of a block division | segmentation in this embodiment. It is a figure explaining the multidimensional feature-value space in this embodiment. It is a figure which shows an example of the penalty matrix between labels. It is a figure explaining calculation of the distance between label strings by matching.

Claims (12)

Acquisition means for acquiring image data composed of one or more tiles, wherein the tile includes at least a header including encoding information about encoding of the tile;
Discrimination means for discriminating the image type for each tile based on the encoding information;
With
The information processing apparatus according to claim 1, wherein the tile is divided into a plurality of frequency components. The encoded information includes a first coding pass number in a low frequency component and a second coding pass number in a high frequency component among the plurality of frequency components constituting the tile,
The information processing apparatus according to claim 1, wherein the determination unit performs the determination based on a ratio between the first coding pass number and the second coding pass number. The encoded information includes a first compressed image data length in a low-frequency component and a second compressed image data length in a high-frequency component among the plurality of frequency components constituting the tile,
The information processing apparatus according to claim 1, wherein the determination unit performs the determination based on a ratio between the first compressed image data length and the second compressed image data length. The encoding information is the number of zero bit planes in a high frequency component among the plurality of frequency components constituting the tile,
The information processing apparatus according to claim 1, wherein the determination unit performs the determination based on a size of the number of zero bit planes.   The information processing apparatus according to claim 1, wherein the image type is at least one of a character image and a graphic image and a natural image.   The information processing apparatus according to claim 1, wherein the image data is encoded data based on a JPEG2000 system. The acquisition means further acquires first image data composed of a first tile and second image data composed of a second tile,
The determining means determines the image type of the first and second tiles for each tile,
Further, the similarity between the first and second image data is determined based on the comparison between the image type of the first tile and the image type of the second tile determined by the determining unit. The information processing apparatus according to claim 1, further comprising a first determination unit. The tile further includes one or more pixel information,
When the first determination unit determines that the first and second image data are similar, the pixel information included in the first tile and the pixel information included in the second tile The information processing apparatus according to claim 7, further comprising: a second determination unit that determines the similarity between the first and second image data based on the second image data. The pixel information includes at least RGB color information,
Storage means for storing a plurality of areas defined in the RGB color space;
The second determination means includes
The determination is performed based on an area to which the pixel information included in the first tile belongs and an area to which the pixel information included in the second tile belongs. The information processing apparatus according to claim 8. An acquisition step of acquiring image data composed of one or more tiles, wherein the tile includes at least a header including encoding information about encoding of the tile;
A determination step of determining an image type for each tile based on the encoding information;
With
The method for controlling an information processing apparatus, wherein the tile is divided into a plurality of frequency components.   A computer program for causing a computer to function as the information processing apparatus according to claim 1.   A computer-readable storage medium storing the computer program according to claim 11.

Images