JP4362537B2 - Image processing apparatus, image forming apparatus, image transmitting apparatus, image reading apparatus, image processing system, image processing method, image processing program, and recording medium thereof - Google Patents

Description

  The present invention relates to an image processing apparatus, an image forming apparatus, an image transmission apparatus, an image reading apparatus, an image processing system, and an image processing method for extracting a feature quantity of image data. The present invention relates to an image processing program and its recording medium.

  Conventionally, various techniques for comparing the input image data obtained by reading a document image with a scanner and a registered image registered in advance and determining the similarity between the two have been proposed.

  As a method for determining the similarity, for example, a character image is extracted and matching is performed using a keyword extracted from the image using an OCR (Optical Character Reader) or the like, or a feature of a ruled line included in the image is extracted and matched. A method for performing the above has been proposed.

  Patent Document 1 discloses a technique for recognizing a format of a form image or the like by recognizing a character, a character string frame, a frame or the like from an input image and performing matching for each frame based on the frame information. .

Patent Document 2 also extracts, as feature points, the centroid of words in English documents, the centroid of connected components of black pixels, the closed space of kanji, and specific parts that repeatedly appear in the image. In contrast, a set of local feature points is determined, a subset of feature points is selected from each determined set, and a plurality of combinations of feature points in the subset are determined as quantities that characterize each selected subset. A technique is disclosed in which invariants with respect to a scientific transformation are obtained, each obtained invariant is used as a feature quantity, and document matching is performed based on the thus obtained feature quantity.
JP-A-8-255236 (Publication date: October 1, 1996) International Publication No. WO2006 / 092957A1 (publication date: September 8, 2006) Japanese Laid-Open Patent Publication No. 4-282968 (Released on October 8, 1992)

  However, in the techniques of Patent Documents 1 and 2, when the input image data is data read in a state where the input image data is inclined with respect to a predetermined arrangement angle at the reading position of the image reading apparatus, In the case of data that has been subjected to such processing, there is a problem that feature points cannot be extracted with high accuracy.

  For example, in the technique of Patent Document 1, the recognition result of characters, character string frames, frames, and the like fluctuates due to the influence of the inclination, enlargement, reduction, etc., so that format recognition cannot be performed with high accuracy.

  Further, in the technique of Patent Document 2, the result of extraction of the centroid of a word in an English document, the centroid of a connected component of black pixels, a closed space of kanji, and a specific portion that repeatedly appears in an image due to the influence of the tilt, enlargement, reduction, etc. As a result, the accuracy of the document collation result decreases.

  When extracting feature points from an image including handwritten characters (for example, an image written by handwriting on a document printed with a predetermined font), the handwritten characters are registered in the image processing apparatus. Since the degree of difference with respect to the shape of the font is high and inherently prone to misjudgment, the techniques described in Patent Documents 1 and 2 reduce the accuracy of judgment due to the tilt, enlargement, reduction, etc. In particular, erroneous determination is likely to occur.

  Further, in the technique of Patent Document 2, when extracting feature points, binarization and labeling of image data is performed, and then a word centroid in an English document, a centroid of connected components of black pixels, and a closed space for kanji. Alternatively, since it is necessary to extract a specific portion that repeatedly appears in the image, the processing is complicated, and there is a problem that the circuit scale for performing these processing increases.

  Also, when extracting the centroid of a word or the centroid of a connected component of black pixels as a feature point as in the technique of Patent Document 2, for example, in the case of input image data of a document that occupies most of the table and has few characters For example, since the number of feature points to be extracted is reduced, there is a problem that the accuracy of collating image data is lowered.

  The present invention has been made in view of the above-mentioned problems, and the object thereof is image data read in a state of being inclined with respect to a predetermined arrangement angle at the reading position of the image reading apparatus, The feature point is to extract feature points that can appropriately specify the image data regardless of the inclination, enlargement, reduction, or the like, from the image data that has been subjected to processing such as enlargement or reduction.

  In order to solve the above problems, an image processing apparatus according to the present invention is based on a feature point detection unit that detects a feature point included in input image data and a relative position between the feature points detected by the feature point detection unit. A feature amount calculation unit that calculates a feature amount of the input image data, wherein the feature point detection unit extracts a partial image including a plurality of pixels including a target pixel from the input image data. Included in the partial image when the partial image extracting unit to extract, the rotated image generating unit that generates a self-rotating image obtained by rotating the partial image by a predetermined angle, and the partial image and the self-rotating image are superimposed The coincidence degree determination unit that determines whether or not the image pattern and the image pattern included in the self-rotation image match, and the target pixel in the partial image that is determined to match, The block consisting of a plurality of pixels including a target pixel is characterized by comprising a detection unit for detecting a point above features.

  According to the above configuration, the partial image extraction unit extracts a partial image including a plurality of pixels including the target pixel from the input image data, and the rotation image generation unit generates a self-rotation image obtained by rotating the partial image by a predetermined angle. The coincidence degree determination unit determines whether or not the image pattern included in the partial image matches the image pattern included in the self-rotated image when the partial image and the self-rotated image are superimposed. That is, it is determined whether or not the image patterns included in both images match when the partial images are superimposed on the partial images while being rotated by a predetermined angle. Then, the detection unit detects, as a feature point, a pixel of interest in the partial image determined to be coincident by the coincidence degree determination unit or a block including a plurality of pixels including the pixel of interest.

  As a result, when the input image data is image data read in a state of being inclined with respect to a predetermined arrangement angle at the reading position of the image reading apparatus, or processing such as enlargement or reduction is performed. A pixel of interest corresponding to a partial image including an image pattern that is not affected by the inclination, enlargement, reduction, or the like or an image pattern that is less affected by the inclination, enlargement, reduction, etc. A block including pixels can be detected as a feature point. Therefore, by calculating the feature amount of the input image data based on the relative positions of the feature points, it is possible to calculate a feature amount that can accurately identify the input image data regardless of the inclination, enlargement, reduction, or the like. .

  In addition to the above configuration, at least one of a storage unit that stores the feature amount of the registered image and a registered image acquisition unit that acquires a feature amount of the registered image from a communicably connected external device, and the feature It is good also as a structure provided with the similarity calculation part which compares the feature-value of the input image data which the quantity calculation part calculated with the feature-value of a registration image, and calculates the similarity of both images.

  According to the above configuration, when the image data is read in a state of being inclined with respect to a predetermined arrangement angle at the reading position of the image reading apparatus, or an image subjected to processing such as enlargement or reduction Even in the case of data, the similarity between the input image and the registered image can be calculated with high accuracy.

  Further, a storage unit for storing a feature amount of image data and identification information for identifying the image data, a feature amount calculated from the input image data by the feature amount calculation unit, and the input image data are identified. It is good also as a structure provided with the registration process part which matches with this identification information and memorize | stores it in the said memory | storage part.

  According to the above configuration, when the image data is read in a state of being inclined with respect to a predetermined arrangement angle at the reading position of the image reading apparatus, or an image subjected to processing such as enlargement or reduction Even in the case of data, it is possible to calculate a feature amount that can specify the input image data with high accuracy, and store the feature amount and the input image data in the storage unit.

  A pattern detection processing unit configured to determine whether or not the partial image includes an image pattern; and the rotated image generation unit is a portion determined to include the image pattern in the pattern detection processing unit. The self-rotating image may be generated for the image.

  According to the above configuration, the pattern detection processing unit determines whether or not each partial image includes an image pattern, and the rotated image generation unit self-rotates the partial image that is determined to include the image pattern. Generate an image. As a result, the self-rotated image generation process and the coincidence determination process can be omitted for partial images that do not include an image pattern, so that the process can be simplified.

  Further, the rotated image generation unit generates a plurality of self-rotated images having different rotation angles for each partial image, and the coincidence degree determination unit is configured to rotate each of the partial images and the partial images. And when the partial image matches at least one self-rotating image, the detection unit determines from the target pixel in the partial image or a plurality of pixels including the target pixel. A block may be detected as the feature point.

  According to said structure, compared with the case where only one self-rotation image is produced | generated about each partial image, more feature points can be extracted. In addition, at least the partial image extraction process and the matching degree determination process can be processed for each self-rotated image by a common algorithm or processing circuit, so that the algorithm is not complicated and the circuit scale of the processing circuit is not increased. More feature points can be extracted.

  The partial image extraction unit may be configured to extract a plurality of types of partial images by changing the size of the extraction target area of the partial image.

  If the size of the extraction target area of the partial image is different, the extraction result of the partial image is also different. Therefore, according to said structure, the number of the feature points extracted can be increased by extracting the multiple types of partial image from which the size of the extraction object area | region of a partial image differs. Thereby, since the similarity between the input image and the registered image is determined based on more feature points, the accuracy of the similarity determination can be further increased.

  The partial image extraction unit sets the size of the extraction target area of the partial image to the first size and extracts the partial image, and sets the size of the extraction target area of the partial image to the first size. A second process for extracting a partial image with a second size larger than the second size, and when performing the second process, the target pixel is centered from the second size extraction target area centered on the target pixel. It is good also as a structure which excludes the extraction object area | region of the 1st size to perform partial image extraction.

  According to said structure, the number of the feature points extracted can be increased further, and the precision of similarity determination can be raised further.

  Even if it is the same input image data, if attention is paid to different color components, the extraction results of the partial images are different. Therefore, according to said structure, the number of the feature points extracted can be increased by extracting a partial image for every color component about a several color component. Thereby, since the similarity between the input image and the registered image is determined based on more feature points, the accuracy of the similarity determination can be further increased.

  The image processing apparatus may further include a smoothing processing unit that performs a smoothing process on the input image data, and the feature point detection unit may detect the feature point based on the input image data that has been subjected to the smoothing process. Good.

  According to the above configuration, by detecting feature points based on input image data that has been subjected to smoothing processing, it is possible to prevent inappropriate feature amounts from being extracted due to the influence of halftone dots and noise components.

  An image forming apparatus of the present invention includes any one of the above-described image processing apparatuses and an image output unit that forms an image corresponding to input image data on a recording material. An image transmission apparatus according to the present invention includes any one of the above-described image processing apparatuses and a transmission apparatus that transmits input image data to another apparatus that is communicably connected. The image reading apparatus of the present invention includes a scanner device that reads an original image and acquires input image data, and any one of the image processing devices described above.

  According to the image forming apparatus, the image transmitting apparatus, and the image reading apparatus, the input image data is image data that is read in a state where the input image data is inclined with respect to a predetermined arrangement angle at the reading position of the image reading apparatus. In some cases or even when the image data has been subjected to processing such as enlargement or reduction, it is possible to calculate a feature quantity that can accurately identify input image data regardless of the above-described inclination, enlargement, reduction, etc. .

  In order to solve the above-described problems, an image processing system of the present invention includes an image processing device and a server device connected to the image processing device so as to communicate with each other. A feature point detection unit to detect and a feature amount calculation unit to calculate a feature amount of the input image data based on the relative positions of the feature points detected by the feature point detection unit are included in the image processing apparatus or the server apparatus. An image processing system provided in a distributed manner in the image processing apparatus and the server apparatus, wherein the feature point detection unit includes a plurality of pixels including a target pixel from the input image data. A partial image extracting unit that extracts a partial image, a rotated image generating unit that generates a self-rotating image obtained by rotating the partial image by a predetermined angle, and the partial image and the self-rotating image. A portion that is determined to match between the matching degree determination unit that determines whether or not the image pattern included in the partial image matches the image pattern included in the self-rotating image, and the matching degree determination unit And a detection unit that detects a target pixel in the image or a block including a plurality of pixels including the target pixel as the feature point.

  According to the above configuration, the partial image extraction unit extracts a partial image including a plurality of pixels including the target pixel from the input image data, and the rotation image generation unit generates a self-rotation image obtained by rotating the partial image, and matches The degree determination unit determines whether or not the image pattern included in the partial image matches the image pattern included in the self-rotating image. Then, the detection unit detects, as a feature point, a pixel of interest in the partial image determined to be coincident by the coincidence degree determination unit or a block including a plurality of pixels including the pixel of interest.

  As a result, when the input image data is image data read in a state of being inclined with respect to a predetermined arrangement angle at the reading position of the image reading apparatus, or processing such as enlargement or reduction is performed. A pixel of interest corresponding to a partial image including an image pattern that is not affected by the inclination, enlargement, reduction, or the like or an image pattern that is less affected by the inclination, enlargement, reduction, etc. A block including pixels can be detected as a feature point. Therefore, by calculating the feature amount of the input image data based on the relative positions of the feature points, it is possible to calculate a feature amount that can accurately identify the input image data regardless of the inclination, enlargement, reduction, or the like. .

  In order to solve the above problems, an image processing method of the present invention is based on a feature point detection step for detecting feature points included in input image data and a relative position between the feature points detected in the feature point detection step. A feature amount calculation step of calculating a feature amount of the input image data, wherein the feature point detection step extracts a partial image including a plurality of pixels including a target pixel from the input image data. An image included in the partial image when the partial image is extracted, a rotated image generating step of generating a self-rotated image obtained by rotating the partial image by a predetermined angle, and the partial image and the self-rotated image A matching degree determining step for determining whether or not a pattern and an image pattern included in the self-rotating image match, and a partial image determined to match in the matching degree determining step. That is the pixel of interest, or a block consisting of a plurality of pixels including the pixel of interest characterized in that it comprises a detection step of detecting a point above features.

  According to the above method, a partial image consisting of a plurality of pixels including the target pixel is extracted from the input image data in the partial image extraction step, and a self-rotation image obtained by rotating the partial image in the rotation image generation step is generated and matched. In the degree determination step, it is determined whether or not the image pattern included in the partial image matches the image pattern included in the self-rotating image. Then, in the detection step, the pixel of interest in the partial image determined to match in the coincidence degree determination step or a block composed of a plurality of pixels including the pixel of interest is detected as a feature point.

  As a result, when the input image data is image data read in a state of being inclined with respect to a predetermined arrangement angle at the reading position of the image reading apparatus, or processing such as enlargement or reduction is performed. A pixel of interest corresponding to a partial image including an image pattern that is not affected by the inclination, enlargement, reduction, or the like or an image pattern that is less affected by the inclination, enlargement, reduction, etc. A block including pixels can be detected as a feature point. Therefore, by calculating the feature amount of the input image data based on the relative positions of the feature points, it is possible to calculate a feature amount that can accurately identify the input image data regardless of the inclination, enlargement, reduction, or the like. .

  The image processing apparatus may be realized by a computer. In this case, an image processing program for causing the image processing apparatus to be realized by the computer by causing the computer to operate as the respective units, and the program are recorded. Computer-readable recording media are also included in the scope of the present invention.

  An image processing apparatus and an image processing system according to the present invention generate a partial image extraction unit that extracts a partial image including a plurality of pixels including a target pixel from input image data, and a self-rotated image obtained by rotating the partial image by a predetermined angle. The degree of coincidence determination that determines whether or not the image pattern included in the partial image and the image pattern included in the self-rotating image match when the rotated image generation unit and the partial image and the self-rotating image overlap each other And a detection unit that detects, as a feature point, a pixel of interest in a partial image that is determined to be coincident by the coincidence degree determination unit or a block that includes a plurality of pixels including the pixel of interest.

  Further, the image processing method of the present invention includes a partial image extraction step for extracting a partial image including a plurality of pixels including a target pixel from input image data, and a rotated image for generating a self-rotated image obtained by rotating the partial image by a predetermined angle. A matching step for determining whether or not the image pattern included in the partial image and the image pattern included in the self-rotating image match when the generating step and the partial image and the self-rotating image are overlapped; And a detection step of detecting, as a feature point, a pixel of interest in a partial image determined to be coincident in the coincidence degree determination step, or a block composed of a plurality of pixels including the pixel of interest.

  Therefore, according to the image processing device, the image processing system, and the image processing method of the present invention, the input image data is read in a state in which the input image data is arranged at an inclination with respect to a predetermined arrangement angle at the reading position of the image reading device. The input image data can be accurately identified regardless of the above-described inclination, enlargement, reduction, etc., even when the image data is processed or when the image data has been subjected to processing such as enlargement, reduction, etc. A feature amount can be calculated.

Embodiment 1
An embodiment of the present invention will be described. In the present embodiment, an example in the case where the present invention is applied to a digital color multifunction peripheral (MFP) will be described.

(1-1. Configuration of Digital Color Multifunction Machine 1)
FIG. 2 is a block diagram showing a schematic configuration of a digital color multifunction peripheral (image processing apparatus, image forming apparatus, image reading apparatus) 1 according to the present embodiment. The digital color multifunction peripheral 1 has a copy function, a printer function, a facsimile transmission function, a scanner function, a scan to e-mail function, and the like.

  As shown in FIG. 2, the digital color MFP 1 includes a color image input device 2, a color image processing device 3, a color image output device 4, a communication device 5, and an operation panel 6.

  The color image input device (image reading device) 2 is composed of a scanner unit (not shown) having a device that converts optical information into an electrical signal, such as a CCD (Charge Coupled Device), for example, and a reflected light image from a document. Are output to the color image processing apparatus 3 as analog signals of RGB (R: red, G: green, B: blue).

  The color image processing apparatus 3 includes an A / D conversion unit 11, a shading correction unit 12, a document matching processing unit 13, an input tone correction unit 14, a region separation processing unit 15, a color correction unit 16, and a black generation and under color removal unit 17. A spatial filter processing unit 18, an output tone correction unit 19, and a tone reproduction processing unit 20. An analog signal output from the color image input device 2 to the color image processing device 3 is converted into an A / D conversion unit 11, a shading correction unit 12, a document matching processing unit 13, and an input tone correction unit in the color image processing device 3. 14, the region separation processing unit 15, the color correction unit 16, the black generation and under color removal unit 17, the spatial filter processing unit 18, the output gradation correction unit 19, and the gradation reproduction processing unit 20 are sent in this order, and CMYK digital color The signal is output to the color image output device 4 as a signal.

  The A / D (analog / digital) converter 11 converts RGB analog signals into digital signals.

  The shading correction unit 12 performs a process for removing various distortions generated in the illumination system, the imaging system, and the imaging system of the color image input apparatus 2 on the digital RGB signal sent from the A / D conversion unit 11. Is. Further, the shading correction unit 12 performs a process of converting color balance adjustment and a signal such as a density signal that can be easily handled by the image processing system employed in the color image processing apparatus 3.

  The document matching processing unit 13 extracts feature points from the input image data, and calculates feature amounts based on the extracted feature points. Further, the document matching processing unit 13 stores (registers) the feature amount calculated as described above in association with image data in a hash table described later. In addition, the document matching processing unit 13 determines the similarity between the input image and the registered image by comparing the feature amount calculated as described above from the input image data with the feature amount of the registered image stored in the hash table. To do. Further, the document matching processing unit 13 outputs the input RGB signal as it is to the input tone correction unit 14 at the subsequent stage. Details of the document collation processing unit 13 will be described later.

  The input tone correction unit 14 performs image quality adjustment processing such as removal of background color (background color density component: background density) and contrast on the RGB signal from which various distortions have been removed by the shading correction unit.

  The region separation processing unit 15 separates each pixel in the input image into one of a character region, a halftone dot region, and a photo region from the RGB signal. Based on the separation result, the region separation processing unit 15 generates a region identification signal indicating which region the pixel belongs to, the color correction unit 16, the black generation and under color removal unit 17, the spatial filter processing unit 18, and the gradation reproduction. The output signal is output to the processing unit 20, and the input signal output from the input tone correction unit 14 is output to the subsequent color correction unit 16 as it is.

  The color correction unit 16 performs a process of removing color turbidity based on spectral characteristics of CMY (C: cyan, M: magenta, Y: yellow) color materials including unnecessary absorption components in order to realize faithful color reproduction. Is.

  The black generation and under color removal unit 17 generates black (K) signals from the CMY three-color signals after color correction, and subtracts the K signals obtained by black generation from the original CMY signals to generate new CMY signals. The process to generate is performed. As a result, the CMY three-color signal is converted into a CMYK four-color signal.

  The spatial filter processing unit 18 performs spatial filter processing using a digital filter on the image data of the CMYK signal input from the black generation and under color removal unit 17 to correct the spatial frequency characteristics. As a result, blurring of the output image and deterioration of graininess can be reduced. Similar to the spatial filter processing unit 18, the gradation reproduction processing unit 20 also performs predetermined processing on the image data of the CMYK signal based on the region identification signal.

  For example, the region separated into characters by the region separation processing unit 15 has a high frequency enhancement amount in the sharp enhancement processing in the spatial filter processing by the spatial filter processing unit 18 in order to improve the reproducibility of black characters or color characters. Increased. At the same time, the tone reproduction processing unit 20 selects binarization or multi-value processing on a high-resolution screen suitable for high-frequency reproduction.

  Further, with respect to the region separated into halftone dot regions by the region separation processing unit 15, the spatial filter processing unit 18 performs low-pass filter processing for removing the input halftone component. The output tone correction unit 19 performs an output tone correction process for converting a signal such as a density signal into a halftone dot area ratio that is a characteristic value of the color image output device 4. Then, gradation reproduction processing (halftone generation) is performed so that the image is finally separated into pixels and each gradation is reproduced. For the region separated into photographs by the region separation processing unit 15, binarization or multi-value processing is performed on the screen with an emphasis on gradation reproducibility.

  The image data subjected to the above-described processes is temporarily stored in a storage device (not shown), read out at a predetermined timing, and input to the color image output device 4.

  The color image output device 4 outputs the image data input from the color image processing device 3 onto a recording material (for example, paper). The configuration of the color image output device 4 is not particularly limited, and for example, a color image output device using an electrophotographic method or an inkjet method can be used.

  The communication device 5 is composed of a modem or a network card, for example. The communication device 5 performs data communication with other devices (for example, personal computers, server devices, other digital multifunction peripherals, facsimile devices, etc.) connected to the network via a network card, a LAN cable, or the like.

  When the image data is transmitted, the communication device 5 performs the transmission procedure with the other party and secures a state in which the image data can be transmitted. Then, the communication device 5 compresses the image data (image data read by the scanner) in a predetermined format. ) Is read from the memory, and necessary processing such as changing the compression format is performed, and then sequentially transmitted to the other party via the communication line.

  Further, when receiving the image data, the communication device 5 performs a communication procedure and receives the image data transmitted from the other party and inputs it to the color image processing device 3. The received image data is subjected to predetermined processing such as expansion processing, rotation processing, resolution conversion processing, output gradation correction, gradation reproduction processing, and the like by the color image processing apparatus 3 and is output by the color image output apparatus 4. The received image data may be stored in a storage device (not shown), and the color image processing device 3 may read it out as necessary to perform the predetermined processing.

  The operation panel 6 includes, for example, a display unit such as a liquid crystal display and setting buttons (none of which are shown), and information corresponding to an instruction from a main control unit (not shown) of the digital color multifunction peripheral 1 is described above. While displaying on a display part, the information input from a user via the said setting button is transmitted to the said main control part. The user inputs a processing request (for example, processing mode (copying, printing, transmission, editing, etc.), number of copies (number of copies, number of copies), destination of input image data, etc.) for the input image data via the operation panel 6. can do. The main control unit includes, for example, a CPU (Central Processing Unit) and the like, and is based on programs and various data stored in a ROM (not shown) and the like, information input from the operation panel 6, and the like. To control the operation.

(1-2. Configuration of Document Collation Processing Unit 13)
Next, details of the document matching processing unit 13 will be described. The document matching processing unit 13 according to the present embodiment extracts a plurality of feature points from input image data, determines a set of local feature points for each extracted feature point, and features points from the determined sets. As a quantity that characterizes each selected subset, an invariant for geometric transformation is obtained based on a plurality of combinations of feature points in the subset, and the obtained invariants are combined. A hash value (feature value) is calculated. In addition, by voting for a registered image corresponding to the calculated hash value, a registered image similar to the input image data is searched, and similarity determination processing (similarity / non-similarity determination) for the registered image is performed. It is also possible to perform processing for associating the calculated hash value with the image from which the hash value has been extracted and storing (registering) it in the hash table.

  FIG. 1 is a block diagram illustrating a schematic configuration of the document matching processing unit 13. As shown in this figure, the document collation processing unit 13 includes a preprocessing unit 30, a feature point calculation unit (feature point detection unit) 31, a feature amount calculation unit 32, a voting processing unit 33, a similarity determination processing unit 34, and a registration. A processing unit 37, a control unit 7, and a memory 8 are provided.

  The control unit 7 controls the operation of each unit of the document collation processing unit 13. The control unit 7 may be provided in a main control unit for controlling the operation of each unit of the digital color multifunction peripheral 1, and is provided separately from the main control unit. It may control the operation of the processing unit 13.

  The memory 8 includes a hash table 103 that stores an index for specifying a registered image and a feature amount extracted from the registered image in association with each other. In addition to the hash table 103, the memory 8 includes a storage unit (not shown) that stores various data used for processing of each unit of the document collation processing unit 13, processing results, and the like. Details of the hash table 103 will be described later.

  FIG. 3 is a block diagram illustrating a schematic configuration of the preprocessing unit 30. As shown in this figure, the pre-processing unit 30 includes a signal conversion processing unit (achromatic processing unit) 41, a resolution conversion unit 42, and an MTF processing unit 43.

  When the image data (RGB signal) input from the shading correction unit 12 is a color image, the signal conversion processing unit 41 achromatizes the image data and converts it into a lightness signal or a luminance signal.

  For example, the signal conversion processing unit 41 converts the RGB signal into the luminance signal Y by the following equation.

Yi = 0.30Ri + 0.59Gi + 0.11Bi
Here, Y is a luminance signal of each pixel, R, G, and B are each color component in the RGB signal of each pixel, and the subscript i is a value assigned to each pixel (i is an integer of 1 or more). It is.

Alternatively, the RGB signal may be converted into a CIE 1976 L ^* a ^* b ^* signal (CIE: Commission International de l'Eclairage, L ^* : brightness, a ^* , b ^* : chromaticity).

  The resolution conversion unit 42 performs a scaling process on the input image data. For example, when the input image data is optically scaled by the color image input device 2, the resolution conversion unit 42 scales the input image data again so that the predetermined resolution is obtained. Further, the resolution conversion unit 42 may perform resolution conversion for lowering the resolution than the resolution read at the same magnification in the color image input device 2 in order to reduce the processing amount in each processing unit in the subsequent stage. (For example, image data read at 600 dpi (dot per inch) is converted to 300 dpi).

  An MTF (modulation transfer function) processing unit 43 is used to absorb (adjust) that the spatial frequency characteristics of the color image input device 2 are different for each model. In the image signal output by the CCD, MTF degradation occurs due to optical parts such as lenses and mirrors, aperture aperture of the light receiving surface of the CCD, transfer efficiency and afterimage, integration effects due to physical scanning, and scanning unevenness. ing. Due to the deterioration of the MTF, the read image is blurred. The MTF processing unit 43 performs a process of repairing the blur caused by the deterioration of the MTF by performing an appropriate filter process (enhancement process). It is also used to suppress high-frequency components that are unnecessary for the feature point extraction process described later. That is, enhancement and smoothing processing is performed using a mixing filter (not shown). FIG. 4 shows an example of the filter coefficient in this mixed filter.

  Note that the configuration of the pre-processing unit 30 is not limited to the above-described configuration. For example, in addition to the above-described processes, or in place of some or all of the above-described processes, a differential process for extracting an edge portion or an Noise reduction processing using a sharp mask, binarization processing for reducing the amount of data to be handled, and the like may be performed.

  FIG. 5 is a block diagram illustrating a schematic configuration of the feature point calculation unit (feature point detection unit) 31. As shown in this figure, the feature point calculation unit 31 includes a pattern detection processing unit 45, a rotated image generation unit 46, a coincidence degree calculation unit 47, and a coincidence degree determination unit 48.

  The pattern detection processing unit 45 extracts a partial image of M × M pixels (M is an integer of 3 or more) centered on the target pixel from the input image, and determines whether or not a valid pattern exists in the partial image. Perform pattern detection processing. The pattern detection processing unit 45 performs the pattern detection processing on each pixel by sequentially raster-scanning the target pixel for each pixel. When the number of pixels included in the partial image is less than M × M at the edge of the image, for example, processing such as folding back the image edge so as to compensate for the insufficient pixels may be performed. Here, the size of the partial image is M × M. However, the present invention is not limited to this. For example, a partial image of M × N pixels (N is an integer of 3 or more and M ≠ N) is extracted. Also good. However, by using M × M pixels, the number of pixels used for the determination and the number of pixels of the partial image can be matched even when the self-rotation angle described later is other than 180 °. More preferably.

  Here, details of the pattern detection processing will be described. In the present embodiment, the pattern detection processing unit 45 first calculates a variance value busy indicating the complexity of the image data calculated by the following equation. In the following formula (1), N represents the number of pixels of the partial image, I represents the signal value of each pixel, and i represents a value for identifying each pixel (i is an integer from 1 to N).

  Then, the pattern detection processing unit 45 determines the presence / absence of a pattern by comparing the magnitude relationship between the variance value busy calculated as described above and a preset threshold value TH1. For example, it is determined that there is an effective pattern when busy ≧ TH1, and it is determined that there is no effective pattern when busy <TH1. The threshold value TH1 may be set as appropriate so that the pattern is appropriately extracted. Although the case where the variance value busy is used has been described here, the present invention is not limited to this. For example, the determination may be made using an index indicating the complexity of the image other than the variance value.

  The rotation image generation unit 46 rotates the partial image determined to have a pattern by the pattern detection processing unit 45 by a predetermined angle (predetermined self-rotation angle) R ° around the center coordinate (target pixel) in the partial image. Generate a self-rotating image.

  FIG. 6 is an explanatory diagram illustrating an example of an input image and a self-rotation image (a 90-degree rotation image, a 180-degree rotation image, and a 120 ° C. rotation image) generated by the rotation image generation unit 46.

Note that a method for rotating the image is not particularly limited, and for example, affine transformation using a rotation matrix can be used. Also, in the case of rotating 90 degrees, if the C column R row of the input partial image (I) is (I) _CR , the self-rotated image (90-degree self-rotated image) is (T) _CR = (I) _RM- You may produce | generate by calculating _{C + 1} .

  When the number of vertical and horizontal pixels does not match before and after self-rotation (for example, when rotating 120 degrees), the size of the partial image is set larger than the image size used for the matching degree determination process described later. Alternatively, the partial image may be extracted again so that the image size used in the matching degree determination process is obtained after the self-rotation.

  For example, when the size of the partial image extracted from the input image is set to the default image size (M × M) as shown by the broken line in FIG. 7A, if this partial image is self-rotated 120 degrees, FIG. ) As indicated by a broken line. As a result, as is clear when comparing the solid line and the broken line shown in FIG. 7C (or the thick line and the broken line shown in FIG. 7B), the partial image that is finally cut out (used to determine the degree of coincidence). In the partial image), a portion not including an image is generated.

  On the other hand, as shown by a solid line in FIG. 7A, the size of the partial image extracted from the input image is larger than the default image size (M × M) (for example, N × N (N is M When the partial image is self-rotated by 120 degrees, the partial image becomes a solid line portion shown in FIG. 7B, and an area corresponding to the size of the partial image used for determining the degree of coincidence is cut out therefrom. As a result, it is possible to obtain an appropriate partial image for use in the coincidence determination.

  The coincidence calculation unit 47 calculates a correlation value (normalized correlation value; coincidence) S between the input image (input partial image) and the self-rotated image.

  Here, the correlation value calculation method and the degree-of-match determination method will be described more specifically. In general, the correlation value S between two images Input (I) and Target (T) composed of N pixels is expressed by the following equation (2). In addition, A, B, and C in the formula (2) are values represented by the following formulas (3) to (5).

  In the present embodiment, Input (I) and Target (T) are self-rotating images, so it is clear that B = C. Therefore, the correlation value S may be calculated as S = (A / B) × 1000, and the calculation can be simplified. Further, since B is the same as the above-described variance value busy, the value of the already calculated variance value busy may be used as the B, and it is not necessary to recalculate B.

  The coincidence determination unit (detection unit) 48 determines whether or not the input image matches the self-rotation image by comparing the correlation value S calculated by the coincidence calculation unit 47 with a preset threshold TH_est. To do. That is, when the partial image included in the input image and the self-rotated image obtained by rotating the partial image by a predetermined angle are superimposed, the image pattern included in the partial image matches the image pattern included in the self-rotated image. Whether or not (overlapping or not) is determined. Specifically, the coincidence degree calculation unit 47 determines that both images match when S> TH_est, and uses the center pixel (target pixel) of this partial image as a feature point. On the other hand, if S ≦ TH_est, it is determined that the two images do not match. In addition, the coincidence degree determination unit 48 outputs information indicating the pixel as the feature point to the feature amount calculation unit 32. Alternatively, the degree-of-match determination unit 48 may store information indicating the pixel as the feature point in the memory 8, and the feature amount calculation unit 32 may read out this information from the memory 8. Note that the threshold value TH_est may be set as appropriate so that feature points are appropriately extracted.

  The feature amount calculation unit 32 includes a feature point extraction unit 32a, an invariant calculation unit 32b, and a hash value calculation unit 32c. The feature point calculated by the feature point calculation unit 31 is used to rotate and parallelize a document image. A feature amount (hash value and / or invariant) that is an invariable amount with respect to geometric deformation such as movement, enlargement, reduction, and parallel movement is calculated.

  As shown in FIG. 8, the feature point extraction unit 32a sets one feature point as a target feature point, and sets a predetermined number of feature points in the vicinity of the target feature point in order from the closest distance from the target feature point (here 4 points) are extracted as peripheral feature points. In the example of FIG. 8, when the feature point a is the target feature point, the four feature points b, c, d, and e are extracted as the peripheral feature points, and when the feature point b is the target feature point, Four points of feature points a, c, e, and f are extracted as peripheral feature points.

  Further, the feature point extraction unit 32a extracts a combination of three points that can be selected from the four peripheral feature points extracted as described above. For example, as shown in FIGS. 9A to 9C, when the feature point a shown in FIG. 8 is the feature point of interest, three of the peripheral feature points b, c, d, and e are selected. Combinations, that is, combinations of peripheral feature points b, c, d, peripheral feature points b, c, e, and peripheral feature points b, d, e are extracted.

  Next, the invariant calculation unit 32b calculates an invariant (one of feature quantities) Hij with respect to geometric deformation for each extracted combination. Here, i is a number indicating the feature point of interest (i is an integer equal to or greater than 1), and j is a number indicating a combination of three peripheral feature points (j is an integer equal to or greater than 1). In the present embodiment, the ratio of two of the lengths of the line segments connecting the peripheral feature points is set as the invariant Hij. The length of the line segment may be calculated based on the coordinate value of each peripheral feature point. For example, in the example of FIG. 9A, if the length of the line segment connecting the feature point c and the feature point d is A11 and the length of the line segment connecting the feature point c and the feature point b is B11, The variable H11 is H11 = A11 / B11. In the example of FIG. 9B, if the length of the line segment connecting the feature point c and the feature point b is A12 and the length of the line segment connecting the feature point b and the feature point e is B12, The variable H12 is H12 = A12 / B12. In the example of FIG. 9C, if the length of the line segment connecting the feature point d and the feature point b is A13 and the length of the line segment connecting the feature point b and the feature point e is B13, The variable H13 is H13 = A13 / B13. In this way, invariants H11, H12, and H13 are calculated in the examples of FIGS. 9A to 9C. In the above example, the line segment connecting the peripheral feature point located on the left side in the horizontal direction and the peripheral feature point located in the center in the horizontal direction is Aij, and the peripheral feature point located in the center in the horizontal direction is located on the right side in the horizontal direction. The line segment connecting the peripheral feature points is Bij. However, the present invention is not limited to this, and the line segment used for calculating the invariant Hij may be selected by an arbitrary method.

Next, the hash value calculation unit 32c calculates a remainder value of (Hi1 × 10 ² + Hi2 × 10 ¹ + Hi3 × 10 ⁰ ) / D as a hash value (one of feature quantities) Hi and stores it in the memory 8. . Note that D is a constant set in advance according to how much the range of values that the remainder can take is set.

  Note that the method for calculating the invariant Hij is not particularly limited. For example, the cross ratio of five points in the vicinity of the feature point of interest, or the double ratio of five points extracted from the neighboring n points (n is an integer of n ≧ 5). The value calculated based on the arrangement of m points extracted from n points in the vicinity (m is an integer of m <n and m ≧ 5) and the cross ratio of the five points extracted from m points, etc. The invariant Hij may be used. The cross ratio is a value obtained from four points on a straight line or five points on a plane, and is known as an invariant with respect to projective deformation which is a kind of geometric transformation.

Also, the formula for calculating the hash value Hi is not limited to the above formula, that is, the configuration in which the remainder of (Hi1 × 10 ² + Hi2 × 10 ¹ + Hi3 × 10 ⁰ ) / D is calculated as the hash value Hi. Other hash functions (for example, any one of hash functions described in Patent Document 2) may be used.

  Further, each part of the feature quantity calculation unit 32, after completing the extraction of the peripheral feature points for one target feature point and the calculation of the hash value Hi, changes the target feature point to another feature point, A hash value is calculated, and hash values for all feature points are calculated.

  In the example of FIG. 8, when the extraction of the peripheral feature points and the hash value when the feature point a is the target feature point is finished, the peripheral feature points and the hash value are extracted when the feature point b is the target feature point. I do. In the example of FIG. 8, when the feature point b is the target feature point, four feature points a, c, e, and f are extracted as the peripheral feature points. Then, as shown in FIGS. 10A to 10C, a combination of three points selected from these peripheral feature points a, c, e, and f (peripheral feature points a, e, f, and peripheral points) Feature points c, e, f, peripheral feature points a, c, f) are extracted, and a hash value Hi is calculated for each combination and stored in the memory 8. Then, this process is repeated for each feature point, and a hash value when each feature point is a feature point of interest is obtained and stored in the memory 8.

  Note that the invariant calculation method when the feature point a is the feature point of interest is not limited to the above method. For example, as shown in FIGS. 22A to 22D, when the feature point a shown in FIG. 8 is the feature point of interest, three of the peripheral feature points b, c, d, and e are selected. Combinations, that is, peripheral feature points b, c, d, peripheral feature points b, c, e, peripheral feature points b, d, e, peripheral feature points c, d, e are extracted, and each extracted combination , An invariant (one of feature quantities) Hij with respect to geometric deformation may be calculated.

Further, when the feature point b shown in FIG. 8 is set as the feature point of interest, as shown in FIGS. 23 (a) to 23 (d), four peripheral feature points of feature points a, c, e, and f are used. A combination of three points (peripheral feature points a, e, f, peripheral feature points a, c, e, peripheral feature points a, f, c, peripheral feature points e, f, c) from You may make it calculate the invariant Hij with respect to geometric deformation about each combination. In this case, the remainder of (Hi1 × 10 ³ + Hi2 × 10 ² + Hi ³ × 10 ¹ + Hi4 × 10 ⁰ ) / D may be calculated as a hash value and stored in the memory 8.

  In the above example, the line segment connecting the peripheral feature point closest to the target feature point and the second closest peripheral feature point is Aij, and the peripheral feature point closest to the target feature point and the third closest peripheral feature point are The line segment connecting the two is connected to Bij. However, the present invention is not limited to this, and the line segment used for calculating the invariant Hij may be selected by an arbitrary method such as selecting based on the length of the line segment connecting the neighboring feature points. That's fine.

  Note that, when performing registration processing for registering input image data as a registered image, the feature amount calculation unit 32 performs registration processing for hash values (feature amounts) for each feature point of the input image data calculated as described above. Send to part 37. In addition, when performing a determination process (similarity determination process) on whether or not the input image data is image data of a registered image that has already been registered, the feature amount calculation unit 32 calculates the input image calculated as described above. A hash value for each feature point of the data is sent to the voting processing unit 33.

  The registration processing unit 37 sequentially registers a hash value for each feature point calculated by the feature amount calculation unit 32 and an index (document ID) representing a document (input image data) in the hash table 103 provided in the memory 8. (See FIG. 11A). If the hash value is already registered, the document ID is registered in association with the hash value. Document IDs are sequentially assigned numbers without duplication. When the number of documents registered in the hash table 103 exceeds a predetermined value (for example, 80% of the number of documents that can be registered), the old document ID may be searched and sequentially deleted. . The deleted document ID may be used again as the document ID of new input image data. Further, when the calculated hash values are the same value (H1 = H5 in the example of FIG. 11B), these may be combined and registered in the hash table 103.

  The voting processing unit 33 compares the hash value of each feature point calculated from the input image data with the hash value registered in the hash table 103, and votes for registered images having the same hash value. In other words, for each registered image, the number of times the same hash value as the hash value of the registered image is calculated from the input image data is counted, and the count value is stored in the memory 8. FIG. 12 is a graph showing an example of the number of votes for registered images ID1, ID2, and ID3.

  The similarity determination processing unit 34 reads the voting result (the index of each registered image and the number of votes for each registered image; similarity) of the voting processing unit 33 from the memory 8 and obtains the maximum number of votes and the maximum number of votes obtained. Extract image index. Then, the extracted maximum number of votes is compared with a predetermined threshold THa to determine similarity (whether or not the input image data is image data of a registered image), and a determination signal indicating the determination result is sent to the control unit Send to 7. That is, when the maximum number of votes is greater than or equal to a predetermined threshold value THa, it is determined that there is similarity (the input image data is image data of a registered image). No determination is made (the input image data is not image data of a registered image) ”.

  Alternatively, the similarity determination processing unit 34 calculates the similarity by dividing and normalizing the number of votes obtained for each registered image by the total number of votes (the total number of feature points extracted from the input image data). And the threshold value THa (for example, 80% of the total number of votes) may be compared to determine the similarity.

  In addition, the similarity determination processing unit 34 normalizes the number of votes obtained for each registered image by dividing by the number of registered hash values (maximum registered number) for the registered image having the largest number of registered hash values. The degree of similarity may be determined by calculating the degree and comparing the degree of similarity with a predetermined threshold THa (for example, 80% of the total number of votes). That is, if the calculated similarity is equal to or higher than the threshold THa, it is determined as “similar”, and if it is lower than the threshold THa, it is determined as “no similarity”. In this case, the total number of hash values extracted from the input image data may be larger than the maximum registered number (particularly when at least a part of the original and / or registered image has a handwritten part). The calculated value of similarity may exceed 100%.

  The threshold value THa for determining similarity may be constant for each registered image, or may be set for each registered image according to the importance of each registered image. The importance of the registered image depends on the registered image, for example, the importance is maximized for banknotes, securities, confidential documents, confidential documents, etc., and the importance is lower for banknotes, etc. May be set step by step. In this case, a weighting coefficient corresponding to the importance of the registered image is stored in the memory 8 in association with the index of the registered image, and the similarity determination processing unit 34 corresponds to the registered image that has obtained the maximum number of votes. The similarity may be determined using the threshold THa.

  In determining similarity, the threshold THa is kept constant, while the similarity is determined by multiplying the number of votes for each registered image (the number of votes obtained for each registered image) by the weight coefficient of each registered image. Good. In this case, a weighting coefficient corresponding to the importance of each registered image is stored in the memory 8 in association with the index of each registered image, and the similarity determination processing unit 34 determines the number of registered images for each registered image. The number of corrected votes multiplied by the weighting coefficient is calculated, and the similarity may be determined based on the number of corrected votes. For example, the maximum correction vote number may be compared with the threshold value THa, the maximum correction vote number normalized with the total number of votes may be compared with the threshold value THa, and the maximum correction vote number is normalized with the maximum registration number. This may be compared with the threshold value THa. In this case, for example, the weighting factor may be set to a value larger than 1, and to a larger value as the importance of the registered image becomes higher.

  In this embodiment, one hash value is calculated for one feature point (attention feature point). However, the present invention is not limited to this, and a plurality of one feature point (attention feature point) may be calculated. A hash value may be calculated. For example, 6 points are extracted as the peripheral feature points of the feature point of interest, and for each of the 6 combinations obtained by extracting 5 points from these 6 points, 3 points are extracted from 5 points to obtain an invariant and a hash value is calculated. You may use the method to do. In this case, six hash values are calculated for one feature point.

(1-3. Processing in Digital Color Multifunction Machine 1)
Next, processing in the digital color multifunction peripheral 1 will be described with reference to a flowchart shown in FIG.

  First, the control unit 7 acquires input image data and a processing request (instruction input) input from the user via the operation panel 6 or the communication device 5 (S1, S2). The input image data may be acquired by reading a document image with the color image input device 2, or input image data transmitted from an external device may be acquired with the communication device 5. The input image data may be read and acquired from various recording media via a card reader (not shown) provided in FIG.

  Next, the control unit 7 causes the preprocessing unit 30 to perform preprocessing (for example, achromatic processing, resolution conversion processing, and MTF processing) on the input image data (S3), and the feature point calculation unit 31 calculates feature points. The process is executed (S4), and the feature amount calculation unit 32 calculates the feature amount (S5). Details of the feature point calculation process will be described later.

  Next, the control unit 7 determines whether or not the process requested by the process request is a registration process (S6). If it is determined that the registration process is being performed, the control unit 7 registers the feature amount calculated by the feature amount calculation unit 32 and the document ID (registered image ID) in the hash table 103 in association with each other (S7).

  On the other hand, when determining that it is not a registration process (when determining that it is a similarity determination process), the control unit 7 causes the voting processing unit 33 to execute a voting process (S8), and causes the similarity determination processing unit 34 to perform the voting process. A similarity determination process is executed (S9).

  If it is determined that there is similarity, execution of image processing (for example, processing such as copying, printing, electronic distribution, facsimile transmission, filing, image data correction, editing, etc.) on the input image data is prohibited (S10). The process is terminated. On the other hand, if it is determined that there is no similarity, the execution of the image processing on the input image data is permitted (S11), and the processing ends. In the present embodiment, an example is described in which execution of image processing is permitted when there is similarity, and execution of image processing is prohibited when there is no similarity, but this is not a limitation. For example, the similarity determination result may be notified to a predetermined notification destination. Further, it is necessary to record the input image data according to the similarity determination result, whether to superimpose a predetermined symbol or the like on the output image corresponding to the input image data, whether to perform user authentication, similarity It may be determined whether or not to display the sex determination result.

  FIG. 14 is a flowchart showing the flow of the feature point calculation process (the process of S4) in the feature point calculation unit 31.

  As shown in this figure, when input image data is input from the preprocessing unit 30 to the feature point calculation unit 31, the control unit 7 causes the pattern detection processing unit 45 to extract partial images (for example, M × M partials). The image is cut out) (S21), and the pattern detection process is executed (S22). Then, the control unit 7 determines whether there is a valid pattern in the partial image based on the result of the pattern detection process (S23).

  If it is determined that a valid pattern exists, the control unit 7 causes the rotated image generation unit 46 to generate a self-rotated image of the partial image extracted in S21 (S24). Next, the control unit 7 causes the coincidence calculating unit 47 to calculate the coincidence between the partial image extracted in S21 and the self-rotating image generated in S24 (S25). Further, the control unit 7 causes the coincidence degree determination unit 48 to determine whether or not the partial image and the self-rotation image match (S26).

  If it is determined in S26 that the partial image matches the self-rotating image, the control unit 7 registers the central pixel (target pixel) in this partial image as a feature point in the memory 8 (S27). Alternatively, information indicating that the center pixel (target pixel) in the partial image is a feature point is output to the feature point calculation unit 31.

  In addition, when the control unit 7 determines that there is no effective pattern after the processing of S27, or in S23, or if it is determined in S26 that the partial image and the self-rotation image do not match, It is determined whether pattern detection processing has been performed for all pixels (S28). That is, it is determined whether or not the pattern detection process has been performed for each partial image in which all pixels in the input image data are pixels of interest.

  If there remains a pixel for which pattern detection processing has not been performed, the pixel of interest is raster-scanned, and the processing of S22 and subsequent steps is performed on the next pixel of interest (S29). On the other hand, when determining that the pattern detection process has been performed for all the pixels, the control unit 7 ends the feature point calculation process.

  As described above, the document collation processing unit 13 of the digital color multifunction peripheral 1 according to the present embodiment extracts a partial image from input image data, determines whether or not the extracted partial image includes a pattern, and determines the pattern. Is included, it is determined whether or not the partial image matches the self-rotated image obtained by rotating the partial image. If they match, the pixel of interest in this partial image is extracted as a feature point.

  As a result, when the input image data is data read in a state where the document is inclined with respect to a predetermined arrangement angle at the reading position of the image reading apparatus, or processing such as enlargement or reduction is performed. Even in the case of data, a center point (a pixel of interest) of a partial image that is not affected (or has a small effect) by the tilt, enlargement, reduction, or the like can be extracted as a feature point. That is, regardless of the inclination, enlargement, reduction, etc., feature points of the same pattern can be extracted with high accuracy, and the calculated ratio of the distances between feature points remains unchanged. Therefore, by calculating the similarity of the image based on the feature amount calculated based on the extracted feature point, the similarity between the input image and the registered image can be obtained regardless of the inclination, enlargement, reduction, or the like. It can be judged with high accuracy.

  FIG. 15 shows a partial image in input image data read at an appropriate angle with respect to a predetermined arrangement angle at the reading position of the image reading apparatus, a 90-degree rotated image obtained by self-rotating the partial image by 90 degrees, and an image reading apparatus It is explanatory drawing which shows an example of the partial image in the input image data read in the state inclined 30 degrees with respect to the predetermined arrangement | positioning angle in the reading position of this, and the 90 degree rotation image which rotated this partial image 90 degree | times. . As shown in this figure, even in the case of input image data read in a state in which the document is inclined with respect to a predetermined arrangement angle at the reading position of the image reading apparatus, the partial image and the partial image By using the target pixel of the partial image that matches the self-rotated image as the feature point, the feature point of the same pattern can be accurately extracted without being affected by the inclination.

  FIG. 16 shows a partial image in the input image data reduced to 70%, a 90-degree rotated image obtained by self-rotating the partial image by 90 degrees, a partial image in the input image data reduced to 150%, and the partial image in 90 degrees. It is explanatory drawing which shows an example of the partial image in the 90 degree rotation image which carried out self rotation, and the input image which has not been changed in magnification. As shown in this figure, even when scaling processing (reduction or enlargement) is performed on the input image data, attention is paid to the partial image in which the partial image and the self-rotation image of the partial image match. By using pixels as feature points, feature points of the same pattern can be accurately extracted without being affected by the scaling process.

  In the present embodiment, a set of local feature points is determined for each feature point extracted as described above, a subset of feature points is selected from each determined set, and each selected subset Is obtained based on a plurality of combinations of feature points in the subset, and invariants for geometric transformation are obtained, and hash values (feature values) are calculated by combining the obtained invariants. Thereby, the similarity between the input image and the registered image can be determined with higher accuracy.

  Further, according to the above configuration, the partial image is extracted, and it is only determined whether or not the extracted partial image matches the self-rotated image obtained by rotating the partial image (whether there is an autocorrelation). Thus, the feature points can be easily extracted, so that the circuit configuration of the feature point calculation unit 31 can be simplified and hardware can be easily realized.

  In general, handwritten characters do not have a high degree of matching (autocorrelation) with self-rotating images. For this reason, for example, even when handwritten writing is present in the input image, it is possible to prevent inappropriate feature points from being extracted by handwritten characters. Therefore, according to the above configuration, even if there is handwritten writing in the input image, it is possible to appropriately extract the feature points of the input image.

  In the present embodiment, when the partial image matches the self-rotated image of the partial image, the target pixel in the partial image is extracted as the feature point. However, the present invention is not limited to this. For example, a block composed of a plurality of pixels including the target pixel in the partial image may be extracted as a feature point.

  In the present embodiment, the case of 90 °, 120 °, and 180 ° has been described as an example of the self-rotation angle R. However, the method for setting the self-rotation angle R is not particularly limited, and may be set arbitrarily. That's fine.

  In this embodiment, feature points are extracted by determining whether or not a partial image matches a self-rotation image obtained by rotating the partial image at a predetermined self-rotation angle R. It is not limited to. For example, the degree of coincidence between the partial image and each self-rotated image obtained by rotating the partial image at a plurality of self-rotation angles is calculated, and the center of the partial image is calculated using the calculation result of the degree of coincidence with respect to each self-rotated image. You may make it determine whether a pixel is made into a feature point.

  Table 1 shows the determination results of the degree of coincidence with respect to the patterns A to E shown in FIG. 6 with respect to the 90 ° rotated image, the 180 ° rotated image, and the 120 ° rotated image.

In the table, “◯” indicates a pattern determined to match, and “x” indicates a pattern determined not to match.

  As shown in this table, the determination result of the degree of coincidence varies depending on the self-rotation angle R. Therefore, by calculating the degree of coincidence for a plurality of self-rotation images, a different pattern for each self-rotation angle is obtained between the partial image and the self-rotation image. Can be extracted as a matching pattern.

  Note that, for example, a target pixel of a pattern determined to match at each of a plurality of self-rotation angles may be used as a feature point, and is determined to match at any one of the plurality of self-rotation angles. The target pixel of the pattern may be used as the feature point, and the target pixel of the pattern determined to match at a predetermined number or more of the plurality of self-rotation angles may be used as the feature point.

  As described above, the feature points are extracted by using the determination result of the degree of coincidence with respect to the self-rotated images at a plurality of angles, so that the number of feature points can be easily increased. Thereby, since the similarity between the input image and the registered image can be determined based on more feature points, the accuracy of the similarity determination can be further increased.

  Further, in the present embodiment, the control unit 7 illustrated a case of cutting out one partial image having a size of M × M when the pattern detection processing unit 45 extracts a partial image. Is not limited to one. For example, in addition to a partial image of M × M size, attention is also paid to a partial image of size L × L (L> M), and the same calculation may be performed to obtain a feature point.

  For example, assume that M = 7 and L = 11, and a partial image is extracted with the central point of the patterns A to C shown in FIGS. Note that the frame region surrounded by a broken line in FIGS. 24 to 26 is a partial image having a size of 7 × 7, and the frame region surrounded by a solid line is a partial image having a size of 11 × 11.

  In consideration of the degree of coincidence between the extracted partial image and the self-rotated image obtained by rotating the partial image by 180 degrees with respect to the partial images of the above sizes, in the case of the pattern A, 7 pixels × 7 pixels coincide, but 11 pixels × 11 pixels do not match. In the case of pattern B, neither 7 pixels × 7 pixels nor 11 pixels × 11 pixels match. In the case of the pattern C, both 7 pixels × 7 pixels and 11 pixels × 11 pixels match. In this way, even with partial images extracted from the same pattern, the match / mismatch with the self-rotating image changes according to the size of the partial image cut out (pixel size of the partial image).

  In the case of pattern B, although 7 pixels × 7 pixels and 11 pixels × 11 pixels do not match, the partial image area of 7 pixels × 7 pixels is excluded (masked) from the calculation target in the partial image of 11 pixels × 11 pixels. If the coincidence with the self-rotating image is determined, the coincidence results.

In addition, as a calculation method of exclusion, paying attention to the number of target pixels, a mask portion may be drawn for each sigma symbol in the above formulas (3) to (5). If it writes about the above-mentioned formula (3),
7 × 7 pixels: Number of pixels Nm (= 49)

11 × 11 pixels: Number of pixels Nl (121)

Masked image: Number of pixels Nk (121-49 = 72)

It becomes. Each sigma calculation value used at the time of calculating Ak is obtained when calculating Am and Al, and thus does not need to be newly calculated. Further, B in the above formula (4) and C in the above formula (5) can be calculated in the same manner.

  Table 2 shows coincidence / non-coincidence between the partial image obtained as described above and a self-rotation image obtained by rotating the partial image by 180 degrees. In the table, “◯” indicates a pattern determined to match, and “x” indicates a pattern determined not to match.

  As shown in Table 2, the determination result of the degree of coincidence varies depending on the cut-out size of the partial image and the presence or absence of a mask. Therefore, by calculating the degree of coincidence for a plurality of clipping methods, a pattern that differs for each clipping method can be extracted as a pattern that matches the partial image and the self-rotating image.

  Note that, for example, a target pixel of a pattern determined to match in each of a plurality of clipping methods may be used as a feature point, and a pattern determined to match in any one of the plurality of clipping methods. The pixel of interest may be used as a feature point, and the pixel of interest of a pattern determined to match in a predetermined number or more of a plurality of clipping methods may be used as the feature point.

  As described above, by extracting feature points using the determination result of the degree of coincidence for a plurality of clipping methods, the number of extracted feature points can be easily increased. Thereby, since the similarity between the input image and the registered image can be determined based on more feature points, the accuracy of the similarity determination can be further increased.

  In the present embodiment, an example in which the feature point calculation unit 31 extracts feature points based on the image data of the multi-value image input from the preprocessing unit 30 has been described. This is not a limitation.

  For example, the preprocessing unit 30 may perform binarization processing and extract feature points based on the binarized image.

  In this case, the preprocessing unit 30 binarizes the image data by comparing the achromatic image data (luminance value (luminance signal) or lightness value (lightness signal)) with a predetermined threshold value. .

  Then, the pattern detection processing unit 45 of the feature point calculation unit 31 extracts a partial image from the image data binarized by the preprocessing unit 30, counts the number of ON pixels (number of black pixels) CountOn, and the counted value The presence / absence of a pattern is determined depending on whether or not is within a predetermined range. For example, thresholds TH2 and TH3 are set in advance, and if TH2 ≦ CountOn ≦ TH3, it is determined that there is a pattern, and if TH2> CountOn or CountOn> TH3, it is determined that there is no pattern. Note that the thresholds TH2 and TH3 may be set as appropriate so that the pattern is appropriately extracted.

  The rotated image generation unit 46 generates a self-rotated image obtained by rotating the partial image determined to have a pattern by the pattern detection processing unit 45 by a predetermined self-rotation angle R. At this time, the rotation process may be performed after performing a process such as rounding off the decimal point.

  Thereafter, the degree-of-match calculation unit 47 counts the number of matches Sn between the pixel value of the partial image (image before rotation) and the pixel value of the self-rotation image.

  FIG. 17A is an explanatory diagram illustrating an example of a partial image, and FIG. 17B is an explanatory diagram illustrating a self-rotated image obtained by self-rotating the partial image illustrated in FIG. 17A by 90 °. In the examples of FIGS. 17A and 17B, the pixel values of the partial image and the self-rotating image are the same as those of the pixels painted in black in FIG. 17B (5 rows and 5 columns). Eyes and 8th row, 8th column). Therefore, in this example, the number of pixels with the same pixel value between the partial image and the self-rotating image is “Sn = 2”.

  Thereafter, the coincidence degree determination unit 48 compares the value of the number of coincidences Sn counted by the coincidence degree calculation unit 47 with a preset threshold value TH_est2 to determine whether or not the partial image and the self-rotation image coincide with each other. Determine. For example, when Sn> TH_est2, it is determined that both images match, and when Sn ≦ TH_est2, it is determined that both images do not match. Note that the threshold value TH_est2 may be set as appropriate so that feature points are appropriately extracted.

  Further, a residual value Sz that is the sum of absolute values of pixel value differences for each pixel corresponding to each other in the partial image and the self-rotating image is calculated, and a feature point is calculated based on the residual value Sz. Also good.

  In this case, the processes of the pattern detection processing unit 45 and the rotated image generation unit 46 are the same as those described above. The coincidence calculation unit 47 calculates a sum of absolute values (residual values Sz) of pixel value differences for each pixel corresponding to each other in the partial image and the self-rotated image, as shown in the following equation. The image data input to the pattern detection processing unit 45 may be binary or multivalued.

  Thereafter, the coincidence degree determination unit 48 compares the residual value Sz calculated by the coincidence degree calculation unit 47 with a preset threshold TH_est3 to determine whether or not the partial image and the self-rotation image match. Determine. Note that, when the residual value Sz is used, the degree of coincidence between both images increases as the residual value Sz decreases. Therefore, for example, when Sz <TH_est3, it is determined that both images match, and when Sz ≧ TH_est3, it is determined that both images do not match. Note that the threshold value TH_est3 may be set as appropriate so that feature points are appropriately extracted.

  In this embodiment, the input image data is a multi-channel signal of one channel, and an example in which similarity determination is performed using a binary signal obtained by binarizing the input image data has been described. The configuration of is not limited to this. For example, the input image data is a color signal composed of a plurality of channels (for example, three channels of R, G, and B or four channels of C, M, Y, and K), or a signal from a visible light source outside the color signal. It may be combined data. In this case, the processing of the signal conversion processing unit 41 in the preprocessing unit 30 is through (no processing is performed). Whether or not the processing in the signal conversion processing unit 41 is to be determined is, for example, whether a color image selection instruction input by the user via the operation panel 6 of the image forming apparatus 3 (color image and monochrome Automatic determination of whether or not the document image (input image data) is a color image in the preceding stage of the document collation processing unit 13. A color selection unit (not shown) may be provided and determined according to the determination result.

  As a method for automatic color selection, for example, the method described in Patent Document 3 can be used. In this method, whether each pixel is a color pixel or a monochrome pixel is determined, and when the presence of a predetermined number or more of continuous color pixels is detected, this continuous color pixel is recognized as a color block. If there are more than a predetermined number of color blocks in the line, the line is counted as a color line. If a predetermined number of color lines are present in the document, it is determined that the image is a color image. Otherwise, it is determined that the image is a monochrome image.

  For example, when the color image input signal is a general three-channel signal of R, G, and B, as described above, each channel is handled as an independent multi-value signal in the pre-processing unit 30 or is binarized. As described above, feature points can be extracted for each channel as in the above-described embodiment.

  FIG. 27A is an explanatory diagram showing an example of input image data made up of color images, and FIGS. 27B to 27D show the R channel and G channel corresponding to the image data of FIG. FIG. 6 is an explanatory diagram showing multi-value image data of B channel. As shown in FIGS. 27 (b) to 27 (d), the positions of the extracted feature points are different for each channel. That is, the images of the respective channels for the three color marks (A: black, B: green, C: red) as shown in FIG. 27A (FIGS. 27B to 27D). The extraction result of the feature points based on is as shown in Table 3. In the table, ◯ indicates a pattern determined as a feature point, and x indicates a pattern not determined as a feature point.

  As shown in this table, since the partial images from which feature points are extracted differ for each channel, the feature points extracted based on these partial images and their self-rotated images also differ for each channel. Therefore, by extracting feature points using a plurality of channels, more feature points can be extracted than H.

  Note that, for example, the feature points extracted in each channel may be set as individual feature points, and it is determined that the partial image and the self-rotation image match in any one of the plurality of channels. The pixel of interest of the pattern may be used as the feature point, and the pixel of interest of the pattern determined to match the partial image and the self-rotated image in a predetermined number of channels among the plurality of channels is set as an individual feature point. You may make it do.

  In this way, by extracting feature points using the determination result of the degree of coincidence between partial images and self-rotating images in a plurality of channels, it is possible to easily increase the number of feature points or information associated therewith. Thereby, since the similarity between the input image and the registered image can be determined based on more feature points, the accuracy of the similarity determination can be further increased.

  In the present embodiment, the case where the present invention is applied to the digital color multifunction peripheral 1 has been described. However, the application target of the present invention is not limited to this. For example, the present invention may be applied to a monochrome multifunction device. Further, the present invention is not limited to a multifunction machine, and may be applied to an image processing apparatus such as a single facsimile communication apparatus, a copying machine, and an image reading apparatus.

  FIG. 18 is a block diagram showing a configuration example when the present invention is applied to a flatbed scanner (image reading apparatus, image processing apparatus) 1 ′.

  As shown in this figure, the flatbed scanner 1 'includes a color image input device 2 and a color image processing device 3'. The color image processing apparatus 3 ′ includes an A / D conversion unit 11, a shading correction unit 12, a document collation processing unit 13, a control unit 7 (not shown in FIG. 18), and a memory 8 (not shown in FIG. 18). The color image input device 2 is connected to the image reading device 1 ′ as a whole. The functions of the A / D conversion unit 11, the shading correction unit 12, the document collation processing unit 13, the control unit 7, and the memory 8 in the color image input device (image reading unit) 2 are substantially the same as those of the digital color multifunction peripheral 1 described above. Since it is the same, description is abbreviate | omitted here.

  Further, the function of the document collation processing unit 13 may be realized by an image processing system including an image processing device and a server device that is communicably connected to the image processing device. FIG. 19 shows image processing apparatuses (MFPs A, B,..., Printers A, B,..., Facsimiles A, B,..., Computers A, B,. (A, B,..., Scanners A, B,...) And a server apparatus 50 are explanatory diagrams showing the configuration of an image processing system 100 that is communicably connected. The configuration of the image processing system 100 is not limited to this. For example, the server device 50, a multifunction device, a printer (image forming device), a facsimile, a computer, a digital camera (image reading device), a scanner (image reading device). ) May be included.

  The scanner includes a document table, an optical scanning unit, a CCD (charge coupled device), etc., and scans the document image placed on the document table by the optical scanning unit to read the document image and generate image data. To do. The digital camera includes an imaging lens, a CCD, and the like (image input device), and shoots a document image, a person, a landscape, and the like to generate image data. The scanner and the digital camera may have a function of performing predetermined image processing (for example, various correction processes) in order to appropriately reproduce the image. The printer prints an image based on image data generated by a computer, a scanner, and a digital camera on a sheet (recording paper). Further, the facsimile performs processing such as binarization processing, resolution conversion processing, and rotation on the image data read from the image input device, and transmits the image data compressed to a predetermined format to the other party. Then, the image data transmitted from the other party is decompressed, subjected to rotation processing, resolution conversion processing, and halftone processing according to the performance of the image output apparatus, and output as a page unit image. The multifunction machine has at least two of a scanner function, a facsimile transmission function, and a printing function (copying function, printer function). The computer edits image data read by a scanner or a digital camera, or creates a document using application software.

  In the image processing system 100, each unit of the document collation processing unit 13 described above is distributed and provided in a server device 50 and an image processing device connected to the server device 50 via a network. The image processing apparatus and the server apparatus 50 cooperate to realize the function of the document collation processing unit 13.

  FIG. 20 is a block diagram illustrating a configuration example in which the functions of the document collation processing unit 13 are provided in a distributed manner in the server device 50 and the digital color multifunction peripheral 1.

  As shown in FIG. 20, the color image processing apparatus 3 of the digital color multifunction peripheral 1 includes a document matching processing unit 13a including a preprocessing unit 30, a feature point calculating unit 31, and a feature amount calculating unit 32, and a document matching processing unit. A control unit 7a for controlling the operation of the document 13a, a memory 8a for storing information necessary for processing of the document collation processing unit 13a, and a communication device 5 for communicating with an external device are provided. The server device 50 includes a communication device 51 that performs communication with an external device, a voting processing unit 33, a similarity determination processing unit 34, and a document matching processing unit 13b that includes a registration processing unit 37, and a document matching processing unit. A control unit 7b for controlling 13b and a memory 8b for storing information necessary for processing of the document collation processing unit 13b. When data transmission / reception is required between each functional block provided in the digital color multifunction peripheral 1 and each functional block provided in the server device 50, the control unit 7a and the control unit 7b are connected to the communication devices 5 and 51, respectively. To transmit and receive data as appropriate. Other functions are the same as those described above.

  In the example of FIG. 20, all of the feature amount calculation unit 32 (the feature point extraction unit 32a, the invariant calculation unit 32b, and the hash value calculation unit 32c) are provided in the digital color multifunction peripheral 1, but this is not limitative. For example, as illustrated in FIG. 21, the feature point extraction unit 32 a and the invariant calculation unit 32 b may be provided in the digital color multifunction peripheral 1, while the hash value calculation unit 32 c may be provided in the server device 50.

  Each unit of the feature amount calculation unit 32 is provided in the server device 50, and data related to the feature points calculated by the feature point calculation unit 31 is transmitted from the digital color multifunction peripheral 1 to the server device 50, and is provided in the server device 50. The feature amount calculation unit 32 may calculate a hash value based on the hash table 103 stored in the memory 8b and the received feature point data. Further, the feature point calculation unit 31 and the feature amount calculation unit 32 are provided in the server device 50, and input image data is transmitted from the digital color multifunction peripheral 1 to the server device 50 to calculate the feature points provided in the server device 50. The unit 31 and the feature amount calculation unit 32 may calculate the hash value based on the input image data received from the server device 50 and the hash table 103 stored in the memory 8b.

  Further, in the above description, an example in which the similarity determination process is performed has been described. However, in the case of performing the registration process, the registration processing unit 37 provided in the server device 50 has received from the digital color multifunction peripheral 1. The document ID and the hash value (or the hash value calculated by the hash value calculation unit 32c provided in the server device 50) may be registered in the hash table 103 provided in the memory 8b. Whether the similarity determination process or the registration process is performed is designated by the user of the digital color multifunction peripheral 1 via the operation panel 6, and a signal indicating which process is performed is transmitted to the server device 50. Alternatively, the registration process may be performed for the input image that the server device 50 has determined as not similar as a result of the similarity determination process.

  When the server device 50 includes the hash value calculation unit 32c, the hash value is calculated and calculated using a method different from the hash value calculation method stored in the hash table 103 (using another hash function). The hash table 103 may be updated using a hash value. As a result, for example, an appropriate hash value referring to a feature value (invariant) according to the type of document image or the like can be registered (updated) in the hash table 103, and voting processing can be performed using it. The accuracy (similarity determination accuracy) can be improved.

  In each of the above embodiments, each unit (each block) constituting the document collation processing unit and the control unit provided in the digital color multifunction peripheral 1 and / or the server device 50 is realized by software using a processor such as a CPU. The That is, the digital color multifunction peripheral 1 and / or the server device 50 develops a central processing unit (CPU) that executes instructions of a control program that realizes each function, a read only memory (ROM) that stores the program, and the program. A random access memory (RAM), and a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is to read the program code (execution format program, intermediate code program, source program) of the control program of the digital color multifunction peripheral 1 and / or the server device 50, which is software that realizes the functions described above, with a computer. This is achieved by supplying the recording medium recorded as possible to the digital color multifunction peripheral 1 and / or the server device 50, and reading out and executing the program code recorded on the recording medium by the computer (or CPU or MPU). .

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the digital color multifunction peripheral 1 and / or the server device 50 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  In addition, each block of the digital color multifunction peripheral 1 and / or the server device 50 is not limited to being realized using software, and may be configured by hardware logic, and performs a part of the processing. A combination of hardware and arithmetic means for executing software for controlling the hardware and performing the remaining processing may be used.

  The computer system of the present invention includes an image input device such as a flatbed scanner, a film scanner, and a digital camera, a computer in which various processes such as the similarity calculation process and the similarity determination process are performed by loading a predetermined program, An image display device such as a CRT display or a liquid crystal display that displays the processing results of the computer, and an image forming device such as a printer that outputs the processing results of the computer to paper or the like may be used. Furthermore, a network card, a modem, or the like as communication means for connecting to a server or the like via a network may be provided.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  The present invention can be applied to an image processing apparatus that calculates feature points included in input image data and calculates the feature amount of the input image data based on the relative positions of the calculated feature points.

It is a block diagram which shows schematic structure of the document collation process part with which the image processing apparatus concerning one Embodiment of this invention is equipped. 1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to an embodiment of the present invention. It is a block diagram which shows schematic structure of the pre-processing part with which the document collation process part shown in FIG. 1 is equipped. It is explanatory drawing which shows an example of the filter coefficient of the mixing filter with which the MTF process part of the pre-processing part shown in FIG. 3 is equipped. It is a block diagram which shows schematic structure of the feature point calculation part with which the document collation process part shown in FIG. 1 is equipped. It is explanatory drawing which shows an example of the self-rotation image produced | generated by the rotation image production | generation part with which the feature point calculation part shown in FIG. 5 is equipped. It is explanatory drawing which shows the size of the partial image used in the feature point calculation part shown in FIG. It is explanatory drawing which shows an example of the attention feature point and periphery feature point which are extracted when the feature-value calculation part with which the document collation process part shown in FIG. 1 is provided calculates a feature-value. (A)-(c) is explanatory drawing which shows an example of the combination of the attention feature point and surrounding feature point which are extracted when the feature-value calculation part of the image processing apparatus shown in FIG. 2 calculates a feature-value. . (A)-(c) is explanatory drawing which shows an example of the combination of the attention feature point and surrounding feature point which are extracted when the feature-value calculation part of the image processing apparatus shown in FIG. 2 calculates a feature-value. . (A) And (b) is explanatory drawing which shows an example of the index showing the hash value registered into a hash table, and input image data in the document collation process part shown in FIG. 3 is a graph showing an example of the number of votes for each registered image in a voting processing unit provided in the document matching processing unit shown in FIG. 1. It is a flowchart which shows the flow of a process in the document collation process part shown in FIG. It is a flowchart which shows the flow of a process in the feature point calculation part with which the document collation process part shown in FIG. 1 is equipped. An example of the relationship between an inclination angle of a document with respect to a predetermined arrangement angle at a reading position of the image reading apparatus, a partial image read from the document and included in input image data, and a self-rotated image obtained by rotating the partial image is shown. It is explanatory drawing. It is explanatory drawing which shows an example of the relationship between the partial image contained in input image data, the partial image contained in the image data which scaled this input image data, and the self-rotation image which rotated this partial image. (A) is explanatory drawing which shows an example of the partial image extracted from binary image data, (b) is explanatory drawing which shows an example of the self-rotation image which rotated the partial image shown to (a). . It is a block diagram which shows the modification of the image processing apparatus concerning one Embodiment of this invention. It is explanatory drawing which shows the structure of the image processing system concerning one Embodiment of this invention. 1 is a block diagram illustrating a configuration example of an image processing system according to an embodiment of the present invention. It is a block diagram which shows the other structural example of the image processing system concerning one Embodiment of this invention. (A)-(d) is description which shows an example of the combination of the attention feature point and peripheral feature point which are extracted when the feature-value calculation part of the image processing apparatus concerning one Embodiment of this invention calculates a feature-value. FIG. (A)-(d) is description which shows an example of the combination of the attention feature point and peripheral feature point which are extracted when the feature-value calculation part of the image processing apparatus concerning one Embodiment of this invention calculates a feature-value. FIG. It is explanatory drawing which shows an example of a partial image. It is explanatory drawing which shows an example of a partial image. It is explanatory drawing which shows an example of a partial image. (A) is explanatory drawing which shows an example of the input image data which consists of a color image, (b)-(d) is multi-value image data of R channel, G channel, and B channel corresponding to the image data of (a). It is explanatory drawing which shows.

Explanation of symbols

1 Digital color MFP (image processing device, image forming device, image reading device, image processing device)
1 'Flatbed scanner (image processing device, image reading device)
2 Color image input device (input data acquisition unit, image input device)
3,3 'color image processing device (image processing device)
4 color image output device (image output unit)
5 Communication devices (input data acquisition unit, transmission device)
6 Operation panel (Processing input section)
7, 7a, 7b Control unit 8, 8a, 8b Memory (storage unit)
13, 13a, 13b Document collation processing unit (similarity calculation unit)
30 Pre-processing unit 31 Feature point calculation unit (feature point detection unit)
32 feature amount calculation unit 32a feature point extraction unit 32b invariant calculation unit 32c hash value calculation unit 33 voting processing unit (similarity calculation unit)
34 similarity determination processing unit 37 registration processing unit 45 pattern detection processing unit (partial image extraction unit)
46 Rotated Image Generation Unit 47 Matching Level Calculation Unit 48 Matching Level Determination Unit (Detection Unit)
50 Server apparatus 100 Image processing system 103 Hash table

Claims (16)

A feature point detector that detects feature points included in the input image data, and a feature amount calculator that calculates the feature amount of the input image data based on the relative positions of the feature points detected by the feature point detector. An image processing apparatus comprising:
The feature point detector is
A partial image extraction unit that extracts a partial image including a plurality of pixels centered on the target pixel from the input image data;
A rotated image generating unit that generates a self-rotating image obtained by rotating the partial image around the target pixel by 90 degrees, 180 degrees, or 120 degrees ;
A degree of coincidence calculation unit for calculating a correlation value S between the partial image and the self-rotating image;
A coincidence determination unit determines whether the above partial image and the self-rotation image matches by comparing the correlation value S and a preset threshold,
A detection unit that detects a pixel of interest in a partial image determined to match by the matching degree determination unit or a block including a plurality of pixels including the pixel of interest as the feature point ;
The coincidence calculation unit
If the partial image is I, the self-rotating image is T, the number of pixels of the partial image and the self-rotating image is N, and the pixel number of each pixel in the partial image and the self-rotating image is i,

The correlation value S is calculated based on the image processing apparatus. At least one of a storage unit that stores the feature amount of the registered image and a registered image acquisition unit that acquires the feature amount of the registered image from an external device that is communicably connected;
2. The image processing apparatus according to claim 1, further comprising: a similarity calculating unit that calculates a similarity between both images by comparing the feature amount of the input image data calculated by the feature amount calculating unit with the feature amount of the registered image. An image processing apparatus according to 1. A storage unit for storing the feature amount of the image data and identification information for identifying the image data;
The feature amount calculation unit includes a registration processing unit that associates a feature amount calculated from the input image data with identification information for identifying the input image data and stores the information in the storage unit. The image processing apparatus according to claim 1 or 2. A pattern detection processing unit for determining whether the partial image includes an image pattern;
The said rotation image production | generation part produces | generates the said self-rotation image about the partial image determined that the image pattern was contained in the said pattern detection process part, The any one of Claim 1 to 3 characterized by the above-mentioned. The image processing apparatus described. The rotation image generation unit generates a plurality of self-rotation images having different rotation angles for each partial image,
The degree-of-match determination unit determines whether or not the partial image and each self-rotated image obtained by rotating the partial image match,
When the partial image matches at least one self-rotating image, the detection unit detects a pixel of interest in the partial image or a block composed of a plurality of pixels including the pixel of interest as the feature point. The image processing apparatus according to any one of claims 1 to 4.   The image processing apparatus according to claim 1, wherein the partial image extraction unit extracts a plurality of types of partial images by changing the sizes of the extraction target areas of the partial images. . The partial image extraction unit
A first extraction process for extracting the partial image by setting the size of the extraction target area of the partial image to the first size;
Performing a second process of extracting a partial image by setting the size of the extraction target area of the partial image to a second size larger than the first size,
When performing the second process, a partial image is extracted by excluding the first size extraction target region centered on the target pixel from the second size extraction target region centered on the target pixel. The image processing apparatus according to claim 6. The partial image extraction unit
The image processing apparatus according to claim 1, wherein a partial image is extracted for each color component of a plurality of color components in the input image data. A smoothing processing unit that performs a smoothing process on the input image data;
The image processing apparatus according to claim 1, wherein the feature point detection unit detects the feature point based on input image data that has been subjected to the smoothing process.   An image forming apparatus comprising: the image processing apparatus according to claim 1; and an image output unit that forms an image corresponding to input image data on a recording material.   An image transmission apparatus comprising: the image processing apparatus according to claim 1; and a transmission apparatus that transmits input image data to another apparatus connected to be communicable. An image input device that reads a document image and obtains input image data;
An image reading apparatus comprising: the image processing apparatus according to claim 1. An image processing device and a server device communicably connected to the image processing device, a feature point detection unit for detecting a feature point included in input image data, and a feature detected by the feature point detection unit A feature amount calculation unit that calculates a feature amount of the input image data based on the relative positions of the points is provided in the image processing device or the server device, or distributed between the image processing device and the server device. An image processing system provided with
The feature point detector is
A partial image extraction unit that extracts a partial image including a plurality of pixels centered on the target pixel from the input image data;
A rotated image generating unit that generates a self-rotating image obtained by rotating the partial image around the target pixel by 90 degrees, 180 degrees, or 120 degrees ;
A degree of coincidence calculation unit for calculating a correlation value S between the partial image and the self-rotating image;
A coincidence determination unit determines whether the above partial image and the self-rotation image matches by comparing the correlation value S and a preset threshold,
A detection unit that detects, as the feature point, a pixel of interest in a partial image that is determined to be coincident by the matching degree determination unit, or a block that includes a plurality of pixels including the pixel of interest.
The coincidence calculation unit
If the partial image is I, the self-rotating image is T, the number of pixels of the partial image and the self-rotating image is N, and the pixel number of each pixel in the partial image and the self-rotating image is i,

The correlation value S is calculated based on the image processing system. A feature point detecting step for detecting a feature point included in the input image data, and a feature amount calculating step for calculating a feature amount of the input image data based on a relative position between the feature points detected in the feature point detecting step. An image processing method comprising:
The feature point detection step includes
A partial image extraction step of extracting a partial image composed of a plurality of pixels centered on the target pixel from the input image data;
A rotated image generation step of generating a self-rotated image obtained by rotating the partial image by 90 degrees, 180 degrees, or 120 degrees ;
A degree of coincidence calculation step of calculating a correlation value S between the partial image and the self-rotating image;
A coincidence determination step of determining whether the said partial image and the self-rotation image matches by comparing the correlation value S and a preset threshold,
Look including a detection step of detecting a block including a plurality of pixels including a target pixel or the pixel of interest, in the determination portion image to match in the match degree determining step as point above features,
In the coincidence calculation step,
If the partial image is I, the self-rotating image is T, the number of pixels of the partial image and the self-rotating image is N, and the pixel number of each pixel in the partial image and the self-rotating image is i,

The correlation value S is calculated based on the image processing method.   A program for operating the image processing apparatus according to any one of claims 1 to 9, wherein the program causes a computer to function as the feature point detection unit.   The computer-readable recording medium which recorded the program of Claim 15.

Images