JP4487000B2 - Image processing apparatus, image forming apparatus, image processing method, image processing system, image processing program, and recording medium thereof - Google Patents

Description

  The present invention relates to an image processing apparatus, an image forming apparatus, an image transmission apparatus, an image reading apparatus, an image processing system, and an image processing method for extracting a feature quantity of image data. The present invention relates to an image processing program and its recording medium.

  Conventionally, various techniques for comparing the input image data obtained by reading a document image with a scanner and a registered image registered in advance and determining the similarity between the two have been proposed.

  As a method for determining similarity, for example, a method of extracting a keyword from a character image using an OCR (Optical Character Reader) or the like and a method of matching by extracting a feature of a ruled line included in the image are proposed. Has been.

  Patent Document 1 discloses a technique for recognizing a format of a form image or the like by recognizing a character, a character string frame, a frame or the like from an input image and performing matching for each frame based on the frame information. .

Patent Document 2 also extracts, as feature points, the centroid of words in English documents, the centroid of connected components of black pixels, the closed space of kanji, and specific parts that repeatedly appear in the image. In contrast, a set of local feature points is determined, a subset of feature points is selected from each determined set, and a plurality of combinations of feature points in the subset are determined as quantities that characterize each selected subset. A technique is disclosed in which invariants with respect to a scientific transformation are obtained, each obtained invariant is used as a feature quantity, and document matching is performed based on the thus obtained feature quantity.
JP-A-8-255236 (Publication date: October 1, 1996) International Publication No. WO2006 / 092957A1 (publication date: September 8, 2006)

  However, in the technique of Patent Document 1 described above, since it is necessary to extract a plurality of types of elements such as characters, character string frames, frames, and straight lines indicating the frames from the input image and perform matching for each extracted element, There is a problem that it is complicated and processing time is long.

  In addition, when the character center of gravity is used as a feature point as in the technique of Patent Document 2 described above, the number of feature points to be extracted is reduced in the case of a document having a small number of characters, and the accuracy of document matching is reduced. There is a problem.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to easily and quickly extract a feature point that can appropriately specify the input image data from the input image data. An object is to provide an image processing apparatus and an image processing method.

  In order to solve the above problems, the image processing apparatus of the present invention is based on a feature point calculation unit that calculates a feature point of input image data, and a relative position between the feature points calculated by the feature point calculation unit. An image processing apparatus including a feature amount calculation unit that calculates a feature amount of input image data, wherein the feature point calculation unit includes a line image extending in a first direction and the first direction from the input image data. A line detection unit that detects a line image extending in a second direction different from the above, and calculating an intersection point between the line image extending in the first direction and the line image extending in the second direction as the feature point It is characterized by having a part.

  In order to solve the above-described problem, the image processing method of the present invention calculates the feature points of the input image data, and calculates the feature amount of the input image data based on the relative positions of the calculated feature points. A method for detecting a line image extending in a first direction and a line image extending in a second direction different from the first direction from the input image data, and extending in the first direction. And an intersection calculation step of calculating an intersection between the line image and the line image extending in the second direction as the feature point.

  In order to solve the above-described problems, an image processing system according to the present invention includes an image processing device and a server device that is communicably connected to the image processing device, and calculates feature points of input image data. The image processing device or the server device includes a feature point calculation unit and a feature amount calculation unit that calculates a feature amount of the input image data based on a relative position between the feature points calculated by the feature point calculation unit. Alternatively, the image processing system is provided in a distributed manner in the image processing device and the server device, wherein the feature point calculation unit includes a line image extending in a first direction from the input image data, and the A line detection unit that detects a line image extending in a second direction different from the first direction, a line image extending in the first direction, and a line image extending in the second direction The point is characterized in that it comprises a intersection calculation unit that calculates a point above features.

  According to the image processing apparatus, the image processing method, and the image processing system, the line image extending in the first direction and the line image extending in the second direction different from the first direction are detected from the input image data. An intersection point between the line image extending in one direction and the line image extending in the second direction is calculated as a feature point. Thus, by using the intersection of line images as a feature point, a large number of feature points can be extracted even in the case of input image data read from a document with few characters, for example. Can be identified appropriately. Further, since it is only necessary to extract the line image and calculate the intersection between the line images, for example, as in Patent Document 1, a plurality of types such as characters, character string frames, frames, and straight lines indicating the frames are input from the input image data. Compared with the case where the feature points are calculated based on these elements, the algorithm for the feature point calculation process can be simplified and the processing time can be shortened.

  The input image data may be acquired by reading a document image, or may be acquired by communication from another device communicably connected to the image processing device, It may be acquired by reading image data recorded on various recording media, or may be acquired by superimposing information input by a user on a predetermined format. .

  The first direction may be a horizontal direction in the input image data, and the second direction may be a vertical direction in the input image data.

  Since the ruled lines included in the input image data often include lines extending in the horizontal direction and the vertical direction, according to the above configuration, the intersection of the ruled lines included in the input image data can be suitably extracted as a feature point. it can.

  The line detection unit includes a first count unit that counts the number of pixels whose pixel values continuously existing in the horizontal direction are equal to or larger than a predetermined value, and a pixel value that exists continuously in the vertical direction is equal to or larger than a predetermined value. A second counting unit that counts the number of pixels that are, and when the number of pixels counted by the first counting unit is greater than or equal to a threshold value, an image composed of the pixels that are continuously present in the horizontal direction is displayed on the first A line image that is detected as a line image that extends in the direction, and that includes the pixels that are continuously present in the vertical direction when the number of pixels counted by the second count unit is greater than or equal to a threshold value, extends in the second direction It is good also as a structure detected as.

  According to the above configuration, a line image extending in the horizontal direction and a line image extending in the vertical direction can be easily detected, and an intersection of these two images can be used as a feature point.

  The first direction is a first oblique direction inclined downward to the right with respect to the horizontal direction in the input image data, and the second direction is a first oblique direction inclined downward to the left with respect to the horizontal direction in the input image data. Two diagonal directions may be used.

  According to said structure, by making the intersection of the line image extended in the 1st diagonal direction and the line image extended in the 2nd diagonal direction into a feature point, for example, the intersection of the line contained in a character, a symbol, etc. is characterized. It can be calculated as a point.

  Further, the input image data represents pixel values of a plurality of pixels arranged in a matrix, and the line detection unit is configured so that each of a plurality of lines including a plurality of pixels arranged in a horizontal direction The process of shifting the target pixel one pixel at a time from left to right is sequentially performed from the upper end line to the lower end line, and for each target pixel, the pixel value of the target pixel and the peripheral pixels of the target pixel A first labeling unit that performs a first labeling process that assigns one of a plurality of types of first labels based on a relationship with a value, and the number of pixels to which the first label is assigned by the first labeling unit For each of a plurality of lines made up of a plurality of pixels arranged in the horizontal direction from the right to the left of each line. The process of shifting the pixels one pixel at a time is sequentially performed from the upper end line to the lower end line, and for each target pixel, based on the relationship between the pixel value of the target pixel and the pixel values of the surrounding pixels of the target pixel. A second labeling unit that performs a second labeling process for applying any one of a plurality of types of second labels, and the number of pixels to which the second label is applied by the second labeling unit is determined for each type of second label. The first labeling unit (1) does not give the first label to the target pixel when the pixel value of the target pixel is less than a predetermined value. 2) When the pixel value of the target pixel is equal to or greater than a predetermined value and the first label is assigned to the pixel adjacent to the left side of the target pixel, the same type as the pixel adjacent to the left side is applied to the target pixel. Give the first label, 3) The pixel value of the target pixel is equal to or greater than a predetermined value, and the first label is not assigned to the pixel adjacent to the left of the target pixel, and the first label is applied to the pixel adjacent to the target pixel. Is given to the target pixel, the same type of first label as that of the upper adjacent pixel is assigned. (4) The pixel value of the target pixel is equal to or greater than a predetermined value, and When the first label is not assigned to the left adjacent pixel and the upper adjacent pixel, a first label of a type that is not applied to other pixels is assigned to the target pixel, and the second labeling unit is (A) When the pixel value of the target pixel is less than the predetermined value, the second label is not given to the target pixel, and (b) the pixel value of the target pixel is equal to or greater than the predetermined value, If the second label is assigned to the pixel to the right of A second label of the same type as the pixel on the right is given, and (c) the pixel value of the pixel of interest is equal to or greater than a predetermined value, and the second label is assigned to the pixel on the right of the pixel of interest If the second label is assigned to the pixel adjacent to the target pixel, the second label of the same type as the upper adjacent pixel is applied to the target pixel. If the pixel value of the pixel of interest is greater than or equal to a predetermined value and the second label is not assigned to the pixel adjacent to the right and top of the pixel of interest, the pixel of interest is assigned to another pixel. A line image in which a second label of a non-type is assigned, and an image composed of pixels to which a first label of the same type whose count result by the first label count unit is greater than or equal to a threshold is given is extended in the first direction. And the counting result by the second label counting unit May detect an image composed of pixels that have been granted a second label of the same type is the threshold value or more as a line image extending in the second direction.

  According to said structure, the intersection of the line image extended in the 1st diagonal direction and the line image extended in the 2nd diagonal direction is computable as a feature point.

  Further, the input image data represents pixel values of a plurality of pixels arranged in a matrix, and the line detection unit is configured so that each of a plurality of lines including a plurality of pixels arranged in a horizontal direction The process of shifting the target pixel one pixel at a time from left to right is sequentially performed from the upper end line to the lower end line, and for each target pixel, the pixel value of the target pixel and the peripheral pixels of the target pixel A first labeling unit that performs a first labeling process that assigns one of a plurality of types of first labels based on a relationship with a value, and a pixel that is given a first label by the first labeling unit, 1st label start end coordinate storage part which stores the start coordinate and end coordinate of the labeled area | region for every kind of 1st label, and several pixels arranged in a horizontal direction For each of the number of lines, the process of shifting the target pixel by one pixel from the right to the left of each line is sequentially performed from the upper end line to the lower end line, and for each target pixel, the pixel of the target pixel A second labeling unit that performs a second labeling process that assigns one of a plurality of types of second labels based on a relationship between a value and a pixel value of a peripheral pixel of the target pixel, and the second labeling unit A second label start / end coordinate storage unit that stores a start coordinate and an end coordinate of a labeled region for each type of second label with respect to a pixel to which a second label is assigned, and the first labeling unit includes: (1) If the pixel value of the pixel of interest is less than the predetermined value, the first label is not assigned to the pixel of interest, and (2) the pixel value of the pixel of interest is greater than or equal to the predetermined value. When the first label is assigned to the left adjacent pixel, a first label of the same type as the left adjacent pixel is assigned to the target pixel, and (3) the pixel value of the target pixel is a predetermined value. When the first label is not assigned to the pixel adjacent to the left of the pixel of interest and the first label is applied to the pixel adjacent to the pixel of interest, the attention is paid. A first label of the same type as the upper adjacent pixel is assigned to the pixel, and (4) the pixel value of the target pixel is equal to or greater than a predetermined value, In the case where the first label is not assigned, a first label of a type that is not given to other pixels is given to the target pixel, and the second labeling unit is configured such that (a) the pixel value of the target pixel is If it is less than the predetermined value, the second label is not assigned to this pixel of interest, and (b) When the pixel value of the pixel is equal to or greater than a predetermined value and the second label is given to the pixel on the right side of the pixel of interest, the second label of the same type as the pixel on the right side of the pixel of interest (C) the pixel value of the pixel of interest is greater than or equal to a predetermined value, the second label is not assigned to the pixel on the right side of the pixel of interest, and the pixel above the pixel of interest On the other hand, when the second label is given, a second label of the same type as the upper adjacent pixel is given to the target pixel, and (d) the pixel value of the target pixel is a predetermined value or more. When the second label is not assigned to the right adjacent pixel and the upper adjacent pixel of the target pixel, the second label of the type not assigned to other pixels is assigned to the target pixel, and the same An image composed of each pixel to which a first label of a type is assigned, wherein the first label An image in which the distance from the start coordinate to the end coordinate of the first label stored in the start / end coordinate storage unit is equal to or greater than a threshold is detected as a line image extending in the first direction, and the same type of second label is assigned. An image composed of each pixel that is stored in the second label start / end coordinate storage unit and whose distance from the start coordinate to the end coordinate of the second label is equal to or greater than a threshold value is extended in the second direction. The line image may be detected as a line image.

  According to said structure, the intersection of the line image extended in the 1st diagonal direction and the line image extended in the 2nd diagonal direction is computable as a feature point. Also, by detecting the line image according to the distance from the start coordinate to the end coordinate of the line image, the line image can be detected based on the length without depending on the thickness of the detection target image. it can.

  Further, the input image data represents pixel values of a plurality of pixels arranged in a matrix, and the line detection unit is configured so that each of a plurality of lines including a plurality of pixels arranged in a horizontal direction The process of shifting the target pixel one pixel at a time from left to right is sequentially performed from the upper end line to the lower end line, and for each target pixel, the pixel value of the target pixel and the peripheral pixels of the target pixel A first labeling unit that performs a first labeling process that assigns one of a plurality of types of first labels based on a relationship with a value, and the number of pixels to which the first label is assigned by the first labeling unit For each of a plurality of lines made up of a plurality of pixels arranged in the horizontal direction from the right to the left of each line. The process of shifting the pixels one pixel at a time is sequentially performed from the upper end line to the lower end line, and for each target pixel, based on the relationship between the pixel value of the target pixel and the pixel values of the surrounding pixels of the target pixel. A second labeling unit that performs a second labeling process for applying any one of a plurality of types of second labels, and the number of pixels to which the second label is applied by the second labeling unit is determined for each type of second label. The first labeling unit (1) does not give the first label to the target pixel when the pixel value of the target pixel is less than a predetermined value. 2) When the pixel value of the target pixel is equal to or greater than a predetermined value and the first label is assigned to the pixel adjacent to the left side of the target pixel, the same type as the pixel adjacent to the left side is applied to the target pixel. Give the first label, 3) The pixel value of the target pixel is equal to or greater than a predetermined value, and the first label is not assigned to the pixel adjacent to the left of the target pixel, and the first label is applied to the pixel adjacent to the target pixel. Is given to the target pixel, the same type of first label as that of the upper adjacent pixel is assigned. (4) The pixel value of the target pixel is equal to or greater than a predetermined value, and When the first label is not assigned to the left adjacent pixel and the upper adjacent pixel, and the first label is assigned to the upper left adjacent pixel, A first label of the same type as that of the pixel is assigned, and (5) the pixel value of the target pixel is equal to or greater than a predetermined value, and the first label is applied to the pixel adjacent to the left, upper, and upper left of the target pixel. If it is not assigned, this type of pixel is not assigned to other pixels. 1 label is assigned, and the second labeling unit (a) does not assign the second label to the pixel of interest when the pixel value of the pixel of interest is less than a predetermined value, and (b) the pixel of the pixel of interest If the value is equal to or greater than the predetermined value and the second label is assigned to the pixel on the right side of the pixel of interest, the second label of the same type as the pixel on the right is attached to the pixel of interest. (C) The pixel value of the target pixel is equal to or greater than a predetermined value, and the second label is not assigned to the pixel adjacent to the right of the target pixel. When two labels are given, the same type of second label as that of the upper adjacent pixel is given to the target pixel, and (d) the pixel value of the target pixel is equal to or greater than a predetermined value. The second label is not assigned to the pixel on the right side of the pixel and the pixel on the top side. When the second label is given to the element, a second label of the same type as the pixel on the upper right is given to the target pixel, and (e) the pixel value of the target pixel is equal to or greater than a predetermined value. In the case where the second label is not assigned to the right adjacent pixel, the upper adjacent pixel, and the upper right adjacent pixel of the target pixel, the second of the type that is not applied to the other pixels with respect to the target pixel. A label is applied, and an image composed of pixels to which a first label of the same type whose counting result by the first label counting unit is equal to or greater than a threshold is detected as a line image extending in a first direction, and the first It is good also as a structure which detects the image which consists of each pixel provided with the 2nd label of the same kind whose count result by a 2 label count part is more than a threshold value as a line image extended in a 2nd direction.

  According to the above configuration, when the first labeling unit labels the target pixel, the labeling result for the upper left adjacent pixel is considered in addition to the labeling result for the left adjacent and upper adjacent pixels of the target pixel. To label the pixel of interest. In addition, when the second labeling unit performs labeling, the target pixel is labeled in consideration of the labeling result for the pixel adjacent to the upper right side in addition to the labeling result for the right adjacent pixel and the upper adjacent pixel. . As a result, even if there is a deviation in the position of the diagonal line due to variations caused by errors at the time of image reading, different labels are attached to the originally identical lines as lines. It is possible to prevent the detection from being lost and improve the line detection accuracy.

  Further, the input image data represents pixel values of a plurality of pixels arranged in a matrix, and the line detection unit is configured so that each of a plurality of lines including a plurality of pixels arranged in a horizontal direction The process of shifting the target pixel one pixel at a time from left to right is sequentially performed from the upper end line to the lower end line, and for each target pixel, the pixel value of the target pixel and the peripheral pixels of the target pixel A first labeling unit that performs a first labeling process that assigns one of a plurality of types of first labels based on a relationship with a value, and a start of a pixel that is given the first label by the first labeling unit A first label start / end coordinate storage unit that stores coordinates and end coordinates for each type of first label, and each of a plurality of lines composed of a plurality of pixels arranged in the horizontal direction, The process of shifting the target pixel one pixel at a time from the right to the left of IN is sequentially performed from the upper end line to the lower end line, and for each target pixel, the pixel value of the target pixel and the peripheral pixels of the target pixel A second labeling unit for performing a second labeling process for applying any one of a plurality of types of second labels based on the relationship with the pixel value of the pixel, and a pixel to which the second label is applied by the second labeling unit And a second label start / end coordinate storage unit that stores the start coordinates and end coordinates of each of the second label types, wherein the first labeling unit is (1) when the pixel value of the target pixel is less than a predetermined value. Does not assign a first label to this pixel of interest, and (2) if the pixel value of the pixel of interest is greater than or equal to a predetermined value and the first label is assigned to the pixel adjacent to the left of this pixel of interest Is this featured picture Is assigned the same type of first label as the left adjacent pixel, and (3) the pixel value of the target pixel is equal to or greater than a predetermined value, and the first label is set to the left adjacent pixel of the target pixel. In the case where the first label is not given to the pixel adjacent above the target pixel, the first label of the same type as the above adjacent pixel is applied to the target pixel. (4) The pixel value of the pixel of interest is equal to or greater than a predetermined value, and the first label is not assigned to the pixel adjacent to the left and the pixel adjacent to the upper side of the pixel of interest. When one label is given, the same type of first label as the pixel on the upper left is given to the target pixel, and (5) the pixel value of the target pixel is equal to or greater than a predetermined value. When the first label is not assigned to the pixel on the left, top, and top left of the pixel Assigns a first label of a type not assigned to other pixels to the target pixel, and the second labeling unit (a) selects the target pixel when the pixel value of the target pixel is less than a predetermined value. If the pixel value of the target pixel is equal to or greater than a predetermined value and the second label is applied to the pixel on the right side of the target pixel, the target pixel And (c) the pixel value of the target pixel is equal to or greater than a predetermined value, and the second label is assigned to the right adjacent pixel of the target pixel. In the case where the second label is given to the pixel above the target pixel that is not given, the second label of the same type as the above neighboring pixel is given to the target pixel, (D) the pixel value of the pixel of interest is greater than or equal to a predetermined value, When the second label is not assigned to the adjacent pixel and the second label is assigned to the adjacent pixel on the upper right, (E) When the pixel value of the target pixel is equal to or greater than a predetermined value, and the second label is not applied to the pixel adjacent to the right side, the upper side, and the upper right side of the target pixel. Is a second label of a type that is not given to other pixels for this pixel of interest, and is an image composed of each pixel that is given the same type of first label, and the first label start / end coordinates An image in which the distance from the start coordinate to the end coordinate of the first label stored in the storage unit is greater than or equal to the threshold is detected as a line image extending in the first direction, and the second label of the same type is given. An image comprising each pixel, the second label May detect the image distance to the end coordinate is equal to or larger than the threshold from the second label of the start coordinates stored in the Le start end coordinate storage section as a line image extending in the second direction.

  According to the above configuration, when the first labeling unit labels the target pixel, the labeling result for the upper left adjacent pixel is considered in addition to the labeling result for the left adjacent and upper adjacent pixels of the target pixel. To label the pixel of interest. In addition, when the second labeling unit performs labeling, the target pixel is labeled in consideration of the labeling result for the pixel adjacent to the upper right side in addition to the labeling result for the right adjacent pixel and the upper adjacent pixel. . As a result, even if there is a deviation in the position of the diagonal line due to variations caused by errors at the time of image reading, different labels are attached to the originally identical lines as lines. It is possible to prevent the detection from being lost and improve the line detection accuracy. Also, by detecting the line image according to the distance from the start coordinate to the end coordinate of the line image, the line image can be detected based on the length without depending on the thickness of the detection target image. it can.

  The line detection unit detects a line image extending in the horizontal direction and a line image extending in the vertical direction, and the intersection calculation unit extends in the second direction and the line image extending in the first direction. In addition to the intersection with the line image, the intersection between the line image extending in the horizontal direction and the line image extending in the vertical direction may be calculated as the feature point.

  According to the above configuration, in addition to the intersection of the line image extending in the first oblique direction and the line image extending in the second oblique direction, the line image extending in the horizontal direction and the line image extending in the vertical direction Intersection points can be calculated as feature points. Therefore, since a larger number of feature points can be calculated, input image data can be specified more appropriately. Further, since the ruled lines included in the input image data often include lines extending in the horizontal direction and the vertical direction, the intersections of the ruled lines included in the input image data can be suitably extracted as feature points.

  In addition, the line detection unit performs detection processing of a line image extending in the horizontal direction and a line image extending in the vertical direction from detection processing of the line image extending in the first direction and the line image extending in the second direction. The intersection calculation unit is configured to perform the first direction when the number of intersections between the line image extending in the horizontal direction and the line image extending in the vertical direction is equal to or greater than a predetermined value. The line image extending in the second direction and the line image extending in the second direction may not be detected.

  According to the above configuration, when the number of intersections between the line image extending in the horizontal direction and the line image extending in the vertical direction is greater than or equal to a predetermined value, the line image extends in the first direction and the line image extends in the second direction. By not performing the line image detection process, it is possible to reduce the load and processing time required to detect the line image extending in the first direction and the line image extending in the second direction. In addition, when the predetermined value is the intersection of a line image extending in the horizontal direction and a line image extending in the vertical direction as a feature point, the number is such that input image data can be appropriately specified by the feature point. By setting to, it is possible to appropriately specify the input image data without calculating the intersection of the line image extending in the first direction and the line image extending in the second direction as a feature point.

  The intersection calculation unit includes an instruction input unit that receives an instruction input from a user, and extends in the line direction extending in the first direction and the second direction in accordance with the user instruction input to the instruction input unit. It may be configured to determine whether or not to perform line image detection processing.

  According to said structure, a user can select whether the detection process of the line image extended in a 1st direction and the line image extended in a 2nd direction is performed. Therefore, for example, the user can select a line image extending in the first direction and a second direction depending on whether the improvement in the accuracy of specifying the input image data or the reduction in the processing time required for the feature point calculation process is important. The necessity of the detection process of the line image to extend | stretch can be selected.

  In addition, when the intersection of the line image extending in the first direction and the line image extending in the second direction is composed of a plurality of adjacent pixels, the intersection calculation unit calculates the center coordinates and the barycentric coordinates of the plurality of pixels. Alternatively, the coordinates of a predetermined position in an area composed of a plurality of pixels may be calculated as the feature point. The predetermined position may be, for example, the coordinates of the pixel having the minimum or maximum coordinate value in the first direction and the minimum or maximum coordinate value in the second direction in the region including a plurality of pixels. For example, the coordinates corresponding to the average value of the coordinate values in one direction and the average value of the coordinate values in the second direction can be given.

  According to the above configuration, even when the intersection of the line image extending in the first direction and the line image extending in the second direction is composed of a plurality of adjacent pixels, the coordinates of the feature points are appropriately specified. be able to.

  The intersection calculation unit calculates a straight line expression corresponding to the line image extending in the first direction and a straight line expression corresponding to the line image extending in the second direction. An intersection of both straight lines may be calculated based on the equation, and coordinates corresponding to the calculated intersection may be calculated as the feature point.

  According to the above configuration, since the intersection point of both line images can be uniquely calculated by the calculation formula calculated based on the start coordinate and the end coordinate of each line image, the line extending in the first direction when calculating the intersection point Since it is not necessary to examine pixels included in both the image and the line image extending in the second direction on a pixel-by-pixel basis, the intersection calculation process can be simplified and the processing time required to calculate the intersection can be shortened.

  The storage unit stores identification information for identifying the registered image in association with the feature amount of the registered image, the feature amount calculated from the input image data by the feature amount calculation unit, and the storage unit. It is good also as a structure provided with the similarity determination part which determines the similarity of an input image and a registration image based on the feature-value of a registration image.

  According to the above configuration, the similarity between the input image and the registered image can be determined by comparing the feature amount acquired from the input image data with the feature amount of the registered image stored in the storage unit. . Further, by performing similarity determination using the feature points, it is possible to accurately determine the similarity between the input image and the registered image.

  A tilt detection unit configured to detect a tilt angle of the document image included in the input image data; and a tilt correction unit configured to correct the tilt of the document image based on the tilt angle detected by the tilt detection unit. The input image data after the inclination correction may be input to the intersection calculation unit.

  According to the above configuration, when the document image included in the input image data is tilted (for example, image data read while being tilted with respect to a predetermined placement angle at the reading position of the image reading device) Even if this is the case, this inclination can be corrected. Therefore, since the feature point can be calculated without being affected by the inclination, the feature point can be calculated with high accuracy.

  Further, the number of black pixels included in the pixel of interest and the peripheral pixels of the pixel of interest in the input image data is counted, and a pixel of interest having a count value equal to or greater than the first threshold is set as a black pixel, and the count value is less than the first threshold. A dilation processing unit that performs dilation processing with the target pixel as a white pixel, and the number of white pixels included in the pixel of interest and the surrounding pixels of the target pixel in the input image data after the dilation processing. A reduction processing unit that performs a reduction process that sets a target pixel that is equal to or greater than two thresholds as a white pixel and a target pixel that has a count value less than the second threshold, and inputs the input image data after the reduction process to the intersection calculation unit It is good also as composition to do.

  According to the above configuration, even if a gap (missing image data) occurs due to a reading error or a transmission / reception error in the line portion of the input image data by performing the expansion processing and the reduction processing. The line detection process can be performed after correcting the gap. Therefore, it is possible to prevent the line from being cut and detected by a gap (missing image data) and to accurately detect the feature points of the input image data.

  An image forming apparatus of the present invention includes any one of the above-described image processing apparatuses and an image output unit that forms an image corresponding to input image data on a recording material. An image transmission apparatus according to the present invention includes any one of the above-described image processing apparatuses and a transmission apparatus that transmits input image data to another apparatus that is communicably connected. The image reading apparatus of the present invention includes an image input apparatus that reads an original image and acquires input image data, and any one of the image processing apparatuses described above.

  According to the image forming apparatus, the image transmitting apparatus, and the image reading apparatus described above, a feature point is an intersection of line images. For example, even in the case of input image data read from a document with few characters, a large number of features Since a point can be extracted, input image data can be specified appropriately. Further, since it is only necessary to extract the line image and calculate the intersection between the line images, for example, as in Patent Document 1, a plurality of types such as characters, character string frames, frames, and straight lines indicating the frames are input from the input image data. Compared with the case where the feature points are calculated based on these elements, the algorithm for the feature point calculation process can be simplified and the processing time can be shortened.

  The image processing apparatus may be realized by a computer. In this case, an image processing program for causing the image processing apparatus to be realized by a computer by causing the computer to operate as the feature point calculation unit, and A computer-readable recording medium on which is recorded is also included in the category of the present invention.

  As described above, the image processing apparatus, the image processing method, and the image processing system of the present invention are configured to extract a line image extending in the first direction and a line image extending in the second direction different from the first direction from the input image data. Detecting and calculating the intersection of the line image extending in the first direction and the line image extending in the second direction as a feature point.

  Therefore, since a large number of feature points can be extracted, input image data can be specified appropriately. In addition, the algorithm for the feature point calculation process can be simplified and the processing time can be shortened.

[Embodiment 1]
An embodiment of the present invention will be described. In the present embodiment, an example in the case where the present invention is applied to a digital color multifunction peripheral (MFP) will be described.

(1-1. Configuration of Digital Color Multifunction Machine 1)
FIG. 2 is a block diagram showing a schematic configuration of a digital color multifunction peripheral (image processing apparatus, image forming apparatus, image reading apparatus) 1 according to the present embodiment. The digital color multifunction peripheral 1 has a copy function, a printer function, a facsimile transmission function, a scanner function, a scan to e-mail function, and the like.

  As shown in FIG. 2, the digital color MFP 1 includes a color image input device 2, a color image processing device 3, a color image output device 4, a communication device 5, and an operation panel 6.

  The color image input device (image reading device) 2 is composed of a scanner unit (not shown) having a device that converts optical information into an electrical signal, such as a CCD (Charge Coupled Device), for example, and a reflected light image from a document. Are output to the color image processing apparatus 3 as analog signals of RGB (R: red, G: green, B: blue).

  The color image processing apparatus 3 includes an A / D conversion unit 11, a shading correction unit 12, a document matching processing unit 13, an input tone correction unit 14, a region separation processing unit 15, a color correction unit 16, and a black generation and under color removal unit 17. A spatial filter processing unit 18, an output tone correction unit 19, and a tone reproduction processing unit 20. An analog signal output from the color image input device 2 to the color image processing device 3 is converted into an A / D conversion unit 11, a shading correction unit 12, a document matching processing unit 13, and an input tone correction unit in the color image processing device 3. 14, the region separation processing unit 15, the color correction unit 16, the black generation and under color removal unit 17, the spatial filter processing unit 18, the output gradation correction unit 19, and the gradation reproduction processing unit 20 are sent in this order, and CMYK digital color The signal is output to the color image output device 4 as a signal.

  The A / D (analog / digital) converter 11 converts RGB analog signals into digital signals.

  The shading correction unit 12 performs a process for removing various distortions generated in the illumination system, the imaging system, and the imaging system of the color image input apparatus 2 on the digital RGB signal sent from the A / D conversion unit 11. Is. Further, the shading correction unit 12 performs a process of converting color balance adjustment and a signal such as a density signal that can be easily handled by the image processing system employed in the color image processing apparatus 3.

  The document matching processing unit 13 extracts feature points from the input image data, and calculates feature amounts based on the extracted feature points. Further, the document matching processing unit 13 stores (registers) the feature amount calculated as described above in association with image data in a hash table described later. In addition, the document matching processing unit 13 determines the similarity between the input image and the registered image by comparing the feature amount calculated as described above from the input image data with the feature amount of the registered image stored in the hash table. To do. Further, the document matching processing unit 13 outputs the input RGB signal as it is to the input tone correction unit 14 at the subsequent stage. Details of the document collation processing unit 13 will be described later.

  The input tone correction unit 14 performs image quality adjustment processing such as removal of background color (background color density component: background density) and contrast on the RGB signal from which various distortions have been removed by the shading correction unit.

  The region separation processing unit 15 separates each pixel in the input image into one of a character region, a halftone dot region, and a photo region from the RGB signal. Based on the separation result, the region separation processing unit 15 generates a region identification signal indicating which region the pixel belongs to, the color correction unit 16, the black generation and under color removal unit 17, the spatial filter processing unit 18, and the gradation reproduction. The output signal is output to the processing unit 20, and the input signal output from the input tone correction unit 14 is output to the subsequent color correction unit 16 as it is.

  The color correction unit 16 performs a process of removing color turbidity based on spectral characteristics of CMY (C: cyan, M: magenta, Y: yellow) color materials including unnecessary absorption components in order to realize faithful color reproduction. Is.

  The black generation and under color removal unit 17 generates black (K) signals from the CMY three-color signals after color correction, and subtracts the K signals obtained by black generation from the original CMY signals to generate new CMY signals. The process to generate is performed. As a result, the CMY three-color signal is converted into a CMYK four-color signal.

  The spatial filter processing unit 18 performs spatial filter processing using a digital filter on the image data of the CMYK signal input from the black generation and under color removal unit 17 to correct the spatial frequency characteristics. As a result, blurring of the output image and deterioration of graininess can be reduced. Similar to the spatial filter processing unit 18, the gradation reproduction processing unit 20 also performs predetermined processing on the image data of the CMYK signal based on the region identification signal.

  For example, the region separated into characters by the region separation processing unit 15 has a high frequency enhancement amount in the sharp enhancement processing in the spatial filter processing by the spatial filter processing unit 18 in order to improve the reproducibility of black characters or color characters. Increased. At the same time, the gradation reproduction processing unit 20 selects binarization or multi-value processing on a high-resolution screen suitable for high frequency reproduction.

  Further, with respect to the region separated into halftone dot regions by the region separation processing unit 15, the spatial filter processing unit 18 performs low-pass filter processing for removing the input halftone component. The output tone correction unit 19 performs an output tone correction process for converting a signal such as a density signal into a halftone dot area ratio that is a characteristic value of the color image output device 4. Then, gradation reproduction processing (halftone generation) is performed so that the image is finally separated into pixels and each gradation is reproduced. With respect to the region separated into photographs by the region separation processing unit 15, binarization or multi-value processing is performed on the screen with an emphasis on gradation reproducibility.

  The image data subjected to the above-described processes is temporarily stored in a storage device (not shown), read out at a predetermined timing, and input to the color image output device 4.

  The color image output device 4 outputs the image data input from the color image processing device 3 onto a recording material (for example, paper). The configuration of the color image output device 4 is not particularly limited, and for example, a color image output device using an electrophotographic method or an inkjet method can be used.

  The communication device 5 is composed of a modem or a network card, for example. The communication device 5 performs data communication with other devices (for example, personal computers, server devices, other digital color multifunction peripherals, facsimile devices, etc.) connected to the network via a network card, a LAN cable, or the like.

  When the image data is transmitted, the communication device 5 performs the transmission procedure with the other party and secures a state in which the image data can be transmitted. Then, the communication device 5 compresses the image data (image data read by the scanner) in a predetermined format. ) Is read from the memory, and necessary processing such as changing the compression format is performed, and then sequentially transmitted to the other party via the communication line.

  Further, when receiving the image data, the communication device 5 performs a communication procedure and receives the image data transmitted from the other party and inputs it to the color image processing device 3. The received image data is subjected to predetermined processing such as expansion processing, rotation processing, resolution conversion processing, output gradation correction, gradation reproduction processing, and the like by the color image processing apparatus 3 and is output by the color image output apparatus 4. The received image data may be stored in a storage device (not shown), and the color image processing device 3 may read it out as necessary to perform the predetermined processing.

  The operation panel 6 includes, for example, a display unit such as a liquid crystal display and setting buttons (none of which are shown), and information corresponding to an instruction from a main control unit (not shown) of the digital color multifunction peripheral 1 is described above. While displaying on a display part, the information input from a user via the said setting button is transmitted to the said main control part. The user can make various processing requests for input image data via the operation panel 6 (for example, processing mode (copy, print, transmission, editing, feature point calculation method in the intersection calculation unit 45 (extract oblique line images)). And the like), the number of processed sheets (the number of copies, the number of printed sheets), the destination of the input image data, etc.). The main control unit includes, for example, a CPU (Central Processing Unit) and the like, and is based on programs and various data stored in a ROM (not shown) and the like, information input from the operation panel 6, and the like. To control the operation.

(1-2. Configuration of Document Collation Processing Unit 13)
Next, details of the document matching processing unit 13 will be described. The document matching processing unit 13 according to the present embodiment extracts a plurality of feature points from input image data, determines a set of local feature points for each extracted feature point, and features points from the determined sets. As a quantity that characterizes each selected subset, an invariant for geometric transformation is obtained based on a plurality of combinations of feature points in the subset, and the obtained invariants are combined. A hash value (feature value) is calculated. In addition, by voting for a registered image corresponding to the calculated hash value, a registered image similar to the input image data is searched, and similarity determination processing (similarity / non-similarity determination) for the registered image is performed. It is also possible to perform processing for associating the calculated hash value with the image from which the hash value has been extracted and storing (registering) it in the hash table.

  FIG. 1 is a block diagram illustrating a schematic configuration of the document matching processing unit 13. As shown in this figure, the document matching processing unit 13 includes a feature point calculation unit 31, a feature amount calculation unit 32, a voting processing unit 33, a similarity determination processing unit 34, a registration processing unit 37, a control unit 7, and a memory 8. I have.

  The control unit 7 controls the operation of each unit of the document collation processing unit 13. The control unit 7 may be provided in a main control unit for controlling the operation of each unit of the digital color multifunction peripheral 1, and is provided separately from the main control unit. The operation of the verification processing unit 13 may be controlled.

  The memory 8 includes a hash table 103 that stores an index for specifying a registered image and a feature amount extracted from the registered image in association with each other. In addition to the hash table 103, the memory 8 includes a storage unit (not shown) that stores various data used for processing of each unit of the document collation processing unit 13, processing results, and the like. Details of the hash table 103 will be described later.

  The feature point calculation unit 31 extracts line images from images such as characters and ruled lines included in the input image data, and calculates intersections between the line images as feature points.

  FIG. 3 is a block diagram illustrating a schematic configuration of the feature point calculation unit 31. As shown in this figure, the feature point calculation unit 31 includes a signal conversion processing unit (achromatic processing unit) 41, a resolution conversion unit 42, an MTF processing unit 43, a binarization processing unit 44, and an intersection calculation unit 45. ing.

  When the image data (RGB signal) input from the shading correction unit 12 is a color image, the signal conversion processing unit (achromatic processing unit) 41 achromatizes the image data to obtain a brightness signal or a luminance signal. It is to convert to.

  For example, the signal conversion processing unit (achromatic processing unit) 41 converts the RGB signal into the luminance signal Y by the following equation.

Yi = 0.30Ri + 0.59Gi + 0.11Bi
Here, Y is a luminance signal of each pixel, R, G, and B are each color component in the RGB signal of each pixel, and the subscript i is a value assigned to each pixel (i is an integer of 1 or more). It is.

Alternatively, the RGB signal may be converted into a CIE 1976 L ^* a ^* b ^* signal (CIE: Commission International de l'Eclairage, L ^* : brightness, a ^* , b ^* : chromaticity).

  The resolution conversion unit 42 performs a scaling process on the input image data. For example, when the input image data is optically scaled by the color image input device 2, the resolution conversion unit 42 scales the input image data again so that the predetermined resolution is obtained. Further, the resolution conversion unit 42 may perform resolution conversion for lowering the resolution than the resolution read at the same magnification in the color image input device 2 in order to reduce the processing amount in each processing unit in the subsequent stage. (For example, image data read at 600 dpi (dot per inch) is converted to 300 dpi).

  An MTF (modulation transfer function) processing unit 43 is used to absorb (adjust) that the spatial frequency characteristics of the color image input device 2 are different for each model. In the image signal output by the CCD, MTF degradation occurs due to optical parts such as lenses and mirrors, aperture aperture of the light receiving surface of the CCD, transfer efficiency and afterimage, integration effects due to physical scanning, and scanning unevenness. ing. Due to the deterioration of the MTF, the read image is blurred. The MTF processing unit 43 performs a process of repairing the blur caused by the deterioration of the MTF by performing an appropriate filter process (enhancement process). It is also used to suppress high-frequency components that are unnecessary for the feature point extraction process described later. That is, enhancement and smoothing processing is performed using a mixing filter (not shown). FIG. 4 shows an example of the filter coefficient in this mixed filter.

  The binarization processing unit 44 binarizes the image data by comparing the achromatic image data (luminance value (luminance signal) or brightness value (brightness signal)) with a preset threshold value. .

  The intersection calculation unit 45 extracts line images from the binarized image data and outputs the intersections of the extracted line images to the feature amount calculation unit 32 as feature points. Details of the processing in the intersection calculation unit 45 will be described later.

  The feature amount calculation unit 32 includes a feature point extraction unit 32a, an invariant calculation unit 32b, and a hash value calculation unit 32c. The feature point calculated by the feature point calculation unit 31 is used to rotate and parallelize a document image. A feature amount (hash value and / or invariant) that is an invariable amount with respect to geometric deformation such as movement, enlargement, reduction, and parallel movement is calculated.

  As shown in FIG. 5, the feature point extraction unit 32a sets one feature point as a target feature point, and sets a predetermined number of feature points in the vicinity of the target feature point in order from the closest distance from the target feature point (here 4 points) are extracted as peripheral feature points. In the example of FIG. 5, when the feature point a is the feature point of interest, four feature points b, c, d, e are extracted as the peripheral feature points, and when the feature point b is the feature point of interest, Four points of feature points a, c, e, and f are extracted as peripheral feature points.

  Further, the feature point extraction unit 32a extracts a combination of three points that can be selected from the four peripheral feature points extracted as described above. For example, as shown in FIGS. 6A to 6C, when the feature point a shown in FIG. 5 is the feature point of interest, three of the peripheral feature points b, c, d, and e are selected. Combinations, that is, combinations of peripheral feature points b, c, d, peripheral feature points b, c, e, and peripheral feature points c, d, e are extracted.

  Next, the invariant calculation unit 32b calculates an invariant (one of feature quantities) Hij with respect to geometric deformation for each extracted combination. Here, i is a number indicating the feature point of interest (i is an integer equal to or greater than 1), and j is a number indicating a combination of three peripheral feature points (j is an integer equal to or greater than 1). In the present embodiment, the ratio of two of the lengths of the line segments connecting the peripheral feature points is set as the invariant Hij. The length of the line segment may be calculated based on the coordinate value of each peripheral feature point. For example, in the example of FIG. 6A, if the length of the line segment connecting the feature point c and the feature point d is A11 and the length of the line segment connecting the feature point c and the feature point b is B11, The variable H11 is H11 = A11 / B11. In the example of FIG. 6B, if the length of the line segment connecting the feature point c and the feature point b is A12 and the length of the line segment connecting the feature point b and the feature point e is B12, The variable H12 is H12 = A12 / B12. In the example of FIG. 6C, if the length of the line segment connecting the feature point d and the feature point b is A13, and the length of the line segment connecting the feature point b and the feature point e is B13, The variable H13 is H13 = A13 / B13. In this way, invariants H11, H12, and H13 are calculated in the examples of FIGS. 6 (a) to 6 (c).

Next, the hash value calculation unit 32c calculates a remainder value of (Hi1 × 10 ² + Hi2 × 10 ¹ + Hi3 × 10 ⁰ ) / D as a hash value (one of feature amounts) Hi and stores it in the memory 8. . Note that D is a constant set in advance according to how much the range of values that the remainder can take is set.

  Note that the method for calculating the invariant Hij is not particularly limited. For example, the cross ratio of five points in the vicinity of the feature point of interest, or the double ratio of five points extracted from the neighboring n points (n is an integer of n ≧ 5). The value calculated based on the arrangement of m points extracted from n points in the vicinity (m is an integer of m <n and m ≧ 5) and the cross ratio of the five points extracted from m points, etc. The invariant Hij may be used. The cross ratio is a value obtained from four points on a straight line or five points on a plane, and is known as an invariant with respect to projective deformation which is a kind of geometric transformation.

  Also, the formula for calculating the hash value Hi is not limited to the above formula (2), and other hash functions (for example, any one of the hash functions described in Patent Document 4) may be used. Good.

  Further, each part of the feature quantity calculation unit 32, after completing the extraction of the peripheral feature points for one target feature point and the calculation of the hash value Hi, changes the target feature point to another feature point, A hash value is calculated, and hash values for all feature points are calculated.

  In the example of FIG. 5, when the extraction of the peripheral feature point and the hash value when the feature point a is the target feature point is finished, the peripheral feature point and the hash value are extracted when the feature point b is the target feature point. I do. In the example of FIG. 5, when the feature point b is set as the feature point of interest, four feature points a, c, e, and f are extracted as the peripheral feature points. Then, as shown in FIGS. 7A to 7C, a combination of three points selected from these peripheral feature points a, c, e, and f (peripheral feature points a, e, f, and peripheral points) Feature points c, e, f, peripheral feature points a, c, f) are extracted, and a hash value Hi is calculated for each combination and stored in the memory 8. Then, this process is repeated for each feature point, and a hash value when each feature point is a feature point of interest is obtained and stored in the memory 8.

  Note that the invariant calculation method when the feature point a is the feature point of interest is not limited to the above method. For example, as shown in FIGS. 30A to 30D, when the feature point a shown in FIG. 5 is the feature point of interest, the four neighboring feature points b, c, d closest to the feature point a are used. , E, and a combination of three of the selected four peripheral feature points, that is, peripheral feature points b, c, d, peripheral feature points b, c, e, peripheral feature points b, d, e Alternatively, each combination of the peripheral feature points c, d, and e may be extracted, and for each extracted combination, an invariant (one of feature quantities) Hij with respect to geometric deformation may be calculated. In this case, a method similar to the method described above can be used as the invariant calculation method.

  For example, a line connecting the peripheral feature point closest to the target feature point and the second closest peripheral feature point is Aij, and a line segment connecting the peripheral feature point closest to the target feature point and the third closest peripheral feature point Bij may be used, and the invariant Hij = Aij / Bij. In this case, in the example of FIG. 30A, if the length of the line segment connecting the feature point b and the feature point c is A11, and the length of the line segment connecting the feature point b and the feature point d is B11, The invariant H11 is H11 = A11 / B11. In the example of FIG. 30B, if the length of the line segment connecting the feature point b and the feature point c is A12 and the length of the line segment connecting the feature point b and the feature point e is B12, The variable H12 is H12 = A12 / B12. In the example of FIG. 30C, if the length of the line segment connecting the feature point b and the feature point d is A13 and the length of the line segment connecting the feature point b and the feature point e is B13, The variable H13 is H13 = A13 / B13. In the example of FIG. 30D, if the length of the line segment connecting the feature point c and the feature point d is A14 and the length of the line segment connecting the feature point c and the feature point e is B14, The variable H14 is H14 = A14 / B14.

  Further, when the feature point b shown in FIG. 5 is set as the feature point of interest, as shown in FIGS. 31 (a) to 31 (d), the four feature points a, c, e, closest to the feature point b are used. f is selected, and among these four peripheral feature points, a combination of three points (peripheral feature points a, e, f, peripheral feature points a, c, e, peripheral feature points a, f, c, peripheral Feature points e, f, c) may be extracted, and invariants Hij for geometric deformation may be calculated for each combination.

As a method for calculating the hash value based on the invariant, the same method as described above can be used. That is, the remainder value of (Hi1 × 10 ³ + Hi2 × 10 ² + Hi3 × 10 ¹ + Hi4 × 10 ⁰ ) / D may be calculated as a hash value and stored in the memory 8.

  In the above example, the line segment connecting the peripheral feature point closest to the target feature point and the second closest peripheral feature point is Aij, and the peripheral feature point closest to the target feature point and the third closest peripheral feature point are connected. The line segment is set as Bij. However, the present invention is not limited to this. For example, the line segment used for calculation of the invariant Hij may be selected by an arbitrary method such as selecting the line segment connecting the peripheral feature points as a reference. Good.

  Note that, when performing registration processing for registering input image data as a registered image, the feature amount calculation unit 32 performs registration processing for hash values (feature amounts) for each feature point of the input image data calculated as described above. Send to part 37. In addition, when performing a determination process (similarity determination process) on whether or not the input image data is image data of a registered image that has already been registered, the feature amount calculation unit 32 calculates the input image calculated as described above. A hash value for each feature point of the data is sent to the voting processing unit 33.

  The registration processing unit 37 has a hash table provided in the memory 8 in association with a hash value for each feature point calculated by the feature amount calculation unit 32 and an index (document ID) representing a document (input image data). The information is sequentially registered in 103 (see FIG. 8A). If the hash value is already registered, the document ID is registered in association with the hash value. Document IDs are sequentially assigned numbers without duplication. When the number of documents registered in the hash table 103 exceeds a predetermined value (for example, 80% of the number of documents that can be registered), the old document ID may be searched and sequentially deleted. . The deleted document ID may be used again as the document ID of new input image data. Further, when the calculated hash values are the same value (H1 = H5 in the example of FIG. 8B), these may be combined and registered in the hash table 103.

  The voting processing unit 33 compares the hash value of each feature point calculated from the input image data with the hash value registered in the hash table 103, and votes for registered images having the same hash value. In other words, for each registered image, the number of times the same hash value as the hash value of the registered image is calculated from the input image data is counted, and the count value is stored in the memory 8. FIG. 9 is a graph showing an example of the number of votes for registered images ID1, ID2, and ID3.

  The similarity determination processing unit 34 reads the voting result (the index of each registered image and the number of votes for each registered image; similarity) of the voting processing unit 33 from the memory 8 and obtains the maximum number of votes and the maximum number of votes obtained. Extract image index. Then, the extracted maximum number of votes is compared with a predetermined threshold THa to determine similarity (whether or not the input image data is image data of a registered image), and a determination signal indicating the determination result is sent to the control unit Send to 7. That is, when the maximum number of votes is greater than or equal to a predetermined threshold value THa, it is determined that there is similarity (the input image data is image data of a registered image). No determination is made (the input image data is not image data of a registered image) ”.

  Alternatively, the similarity determination processing unit 34 calculates the similarity by dividing and normalizing the number of votes obtained for each registered image by the total number of votes (the total number of feature points extracted from the input image data). And the threshold value THa (for example, 80% of the total number of votes) may be compared to determine the similarity.

  In addition, the similarity determination processing unit 34 normalizes the number of votes obtained for each registered image by dividing by the number of registered hash values (maximum registered number) for the registered image having the largest number of registered hash values. The degree of similarity may be determined by calculating the degree and comparing the degree of similarity with a predetermined threshold THa (for example, 80% of the total number of votes). That is, if the calculated similarity is equal to or higher than the threshold THa, it is determined as “similar”, and if it is lower than the threshold THa, it is determined as “not similar”. In this case, the total number of hash values extracted from the input image data may be larger than the maximum registered number (particularly when at least a part of the original and / or registered image has a handwritten part). The calculated value of similarity may exceed 100%.

  The threshold value THa for determining similarity may be constant for each registered image, or may be set for each registered image according to the importance of each registered image. The importance of the registered image depends on the registered image, for example, the importance is maximized for banknotes, securities, confidential documents, confidential documents, etc., and the importance is lower for banknotes, etc. May be set step by step. In this case, a weighting coefficient corresponding to the importance of the registered image is stored in the memory 8 in association with the index of the registered image, and the similarity determination processing unit 34 corresponds to the registered image that has obtained the maximum number of votes. The similarity may be determined using the threshold THa.

  In determining similarity, the threshold THa is kept constant, while the similarity is determined by multiplying the number of votes for each registered image (the number of votes obtained for each registered image) by the weight coefficient of each registered image. Good. In this case, a weighting coefficient corresponding to the importance of each registered image is stored in the memory 8 in association with the index of each registered image, and the similarity determination processing unit 34 determines the number of registered images for each registered image. The number of corrected votes multiplied by the weighting coefficient is calculated, and the similarity may be determined based on the number of corrected votes. For example, the maximum correction vote number may be compared with the threshold value THa, the maximum correction vote number normalized with the total number of votes may be compared with the threshold value THa, and the maximum correction vote number is normalized with the maximum registration number. This may be compared with the threshold value THa. In this case, for example, the weighting factor may be set to a value larger than 1, and to become a larger value as the importance of the registered image becomes higher.

  In this embodiment, one hash value is calculated for one feature point (attention feature point). However, the present invention is not limited to this, and a plurality of one feature point (attention feature point) may be calculated. A hash value may be calculated. For example, 6 points are extracted as the peripheral feature points of the feature point of interest, and for each of the 6 combinations obtained by extracting 5 points from these 6 points, 3 points are extracted from 5 points to obtain an invariant and a hash value is calculated. You may use the method to do. In this case, six hash values are calculated for one feature point.

(1-3. Processing in the sentence matching processing unit 13)
Next, processing in the document collation processing unit 13 will be described with reference to the flowchart shown in FIG.

  First, the control unit 7 acquires input image data and a processing request (instruction input) input from the user via the operation panel 6 or the communication device 5 (S1, S2). The input image data may be acquired by reading a document image with the color image input device 2, or input image data transmitted from an external device may be acquired with the communication device 5. The input image data may be read and acquired from various recording media via a card reader (not shown) provided in FIG.

  Next, the control unit 7 causes the feature point calculation unit 31 to execute a feature point calculation process (S3), and causes the feature amount calculation unit 32 to calculate a feature amount (S4).

  Next, the control unit 7 determines whether or not the process requested by the process request is a registration process (S5). If it is determined that the registration process is being performed, the control unit 7 associates the feature amount calculated by the feature amount calculation unit 32 with the document ID (ID of the registered image) and registers it in the hash table 103 (S6). The process ends.

  On the other hand, when determining that it is not a registration process (when determining that it is a similarity determination process), the control unit 7 causes the voting processing unit 33 to execute a voting process (S7), and causes the similarity determination processing unit 34 to perform the voting process. A similarity determination process is executed (S8).

  If it is determined that there is similarity, execution of image processing (for example, processing such as copying, printing, electronic distribution, facsimile transmission, filing, image data correction, editing, etc.) on the input image data is prohibited (S11). The process is terminated. On the other hand, if it is determined that there is no similarity, execution of image processing on the input image data is permitted (S12), and the processing ends. In the present embodiment, an example is described in which execution of image processing is permitted when there is similarity, and execution of image processing is prohibited when there is no similarity, but this is not a limitation. For example, the similarity determination result may be notified to a predetermined notification destination. Further, it is necessary to record the input image data according to the similarity determination result, whether to superimpose a predetermined symbol or the like on the output image corresponding to the input image data, whether to perform user authentication, similarity It may be determined whether or not to display the sex determination result.

(1-4. Configuration and processing of intersection calculation unit 45)
Next, the configuration of the intersection calculation unit 45 and details of processing in the intersection calculation unit 45 will be described. First, an outline of processing in the intersection calculation unit 45 will be described with reference to a flowchart shown in FIG. As shown in this figure, the control unit 7 causes the intersection point calculation unit 45 to use binarized image data (for example, represented by “1” (black pixels) and “0” (white pixels)). Then, the line image extending in the horizontal direction and the line image extending in the vertical direction are extracted (S21), and the intersection of the line images extending in both directions is extracted (S22). Note that the extraction process of line images extending in the horizontal direction and the line image extending in the vertical direction and the extraction process of intersections will be described later.

  Next, the control unit 7 determines whether or not to extract an intersection of line images extending in an oblique direction as a feature point (S23). In this determination, for example, when the number of feature points extracted in S22 is less than a predetermined value, an intersection of line images extending in an oblique direction may be extracted as a feature point, and the operation panel 6 or communication is possible. The determination may be made according to a user mode selection instruction input from an external device connected to the. For example, the user can select a mode in which diagonal line detection is not performed in order to reduce the processing time and processing load required for diagonal line detection, thereby further improving the accuracy of similarity determination processing. If desired, a mode for performing line detection in an oblique direction can be selected. In addition, for example, when the input image includes ruled lines, and there are many intersections of the ruled lines, and the input image has no skew, the intersection of the horizontal and vertical line images is used as a feature point. Since sufficient similarity determination accuracy can be ensured, detection of a line image in an oblique direction may not be performed.

  If it is determined that the intersection of the line images extending in the oblique direction is not extracted as the feature point, the control unit 7 outputs the intersection extracted in S22 to the feature amount calculation unit 32 as the feature point (S24), and calculates the intersection. The process ends.

  On the other hand, if it is determined in S23 that the intersection of the line images extending in the oblique direction is not extracted as the feature point, the control unit 7 causes the intersection calculation unit 45 to perform two oblique directions (from the upper left to the lower right (the lower right direction). ); A line image extending in the first diagonal direction and a direction from the upper right to the lower left (downward direction in the left); the second diagonal direction) is extracted (S25, S26), and an intersection of the line images extending in both directions is extracted. (S27). In addition, the process of S25 and the process of S26 may be performed in parallel, and either one may be performed first. The process of extracting line images extending in the oblique direction and the process of extracting intersections between line images extending in the oblique direction will be described later.

  And the control part 7 makes the intersection calculation part 45 output the intersection extracted by S22 and the intersection extracted by S27 to the feature-value calculation part 32 as a feature point (S28), and complete | finishes an intersection calculation process.

  In the above description, in S23, the control unit 7 determines whether or not to extract an intersection of line images extending in an oblique direction as a feature point. However, the present invention is not limited to this. The calculation of the intersection of line images extending in the direction and the calculation of the intersection of line images extending in two diagonal directions may be performed without fail, and only one of the preset intersections may be calculated. Also good. Further, it is necessary to calculate the intersection of the line images extending in the diagonal direction first, and according to the calculation result, the control unit 7 needs to calculate the intersection between the line image extending in the horizontal direction and the line image extending in the vertical direction. May be determined. However, since the processing algorithm of the diagonal line image detection process is more complex and requires more processing time than the horizontal and vertical line image detection processes, the horizontal and vertical line images It is preferable that the detection of the point and the calculation of the intersection are performed first, and the detection of the oblique line image and the calculation of the intersection are performed as necessary.

  Next, the line image extraction process and the intersection extraction process will be described in more detail. FIG. 12 is a block diagram illustrating a configuration of the intersection calculation unit 45. As shown in this figure, the intersection calculation unit 45 includes a horizontal counter 61, a vertical counter 62, a first labeling unit 63, a first diagonal counter 64, a second labeling unit 67, and a second diagonal counter 68. 60, a line determination unit 71, and an intersection extraction unit 72.

  First, horizontal line and vertical line image extraction processing and intersection extraction processing will be described.

  The horizontal counter 61 stores the count value of the horizontal run-length count for the binarized image data. Here, the run length count is a process of counting the number of pixels having a pixel value “1” continuous in a predetermined direction.

  More specifically, the control unit 7 scans the binarized image data from left to right (or right to left) in the horizontal direction to determine whether each pixel is a black pixel (“1”). If the pixel is a black pixel, the counter value of the horizontal counter 61 is incremented. If the pixel is not a black pixel, the counter value of the horizontal counter 61 is reset. When the scanning reaches the end in the horizontal direction, the counter value of the horizontal counter 61 is reset and the run length count process is performed for the next line adjacent in the vertical direction. Further, when resetting the counter value of the horizontal counter 61, the control unit 7 horizontally displays the counter value immediately before the reset and the coordinates of the pixel scanned when the reset is performed (or the image of the pixel scanned immediately before the reset). A line 61 is output from the counter 61.

  The vertical counter 62 stores the counter value of the run length count in the vertical direction for the binarized image data.

  In the present embodiment, the vertical counter 62 includes the same number of column counters 62_1 to 62_i as the number of pixels i in the horizontal direction. Then, the control unit 7 scans the binarized image data from left to right (or right to left) in the horizontal direction to determine whether each pixel is a black pixel (“1”), In the case of a black pixel, the counter value of the column counter is incremented according to the horizontal position (x direction) of the pixel. That is, the counter value of the run length count in the vertical direction for the binarized image data is stored for each column (for each x coordinate) by each column counter. If the pixel is not a black pixel, the counter value of the column counter corresponding to the horizontal position of the pixel is reset. When the scanning reaches the end in the horizontal direction, the run length count process is performed on the next line adjacent in the vertical direction. In addition, when resetting the counter value of the column counter, the control unit 7 uses the counter value immediately before the reset and the coordinates of the pixel scanned when the reset is performed (or the image of the pixel scanned immediately before the reset) as a vertical counter. The line determination unit 71 outputs the data from 62.

  When the counter value input from the horizontal counter 61 is a number m equal to or greater than a threshold value (for example, 5), the line determination unit 71 sets the counter values 1 to 4 based on the coordinates of the scanned pixels when resetting the counter value m. Each pixel corresponding to m is obtained, and an image composed of these pixels is determined as a line image. In addition, when the counter value input from the vertical counter 62 is a number n equal to or greater than a threshold value (for example, 5), the line determination unit 71 determines the counter based on the coordinates of the scanned pixel when resetting the counter value n. Each pixel corresponding to the values 1 to n is obtained, and an image composed of each pixel corresponding to each pixel is determined as a line image. If the line determination unit 71 determines that the line image is included, the coordinates of each pixel included in the line image (or the coordinates of the start pixel and the end pixel of the pixel value “1”) are output to the intersection extraction unit 72. To do.

  The intersection extraction unit 72 extracts an intersection between the horizontal line image and the vertical line image based on the input from the line determination unit 71 and outputs the extracted intersection to the feature quantity calculation unit 32 as a feature point. . Note that when a plurality of pixels adjacent to each other are extracted as intersections, the intersection extraction unit 72 coordinates the center coordinates, barycentric coordinates, or predetermined positions (for example, upper left) included in the area composed of the plurality of pixels. Is a feature point.

  FIG. 13 is an explanatory diagram illustrating an example of image data input to the intersection calculation unit 45. FIG. 14 is an explanatory diagram showing the transition of the counter values of the horizontal counter 61 and the vertical counter 62 when line determination processing is performed on the image data shown in FIG.

  First, at the start of the line determination process, column counters 62_1 to 62_i included in the horizontal counter 61 and the vertical counter 62 are initialized to “0”.

  Then, from the upper left pixel to the lower right pixel (coordinates (0,0) → (1,0) → (2,0) → ・ ・ ・ → (8,0) → (0,1) → (1 , 1) →...) Each pixel is scanned to determine whether it is a black pixel. The scanning direction is not limited to this. For example, scanning may be performed from the upper right to the lower left.

  In the example of FIG. 13, since there are no black pixels up to the coordinates (3, 1), both the column counters of the horizontal counter 61 and the vertical counter 62 remain at the initial value “0”. Since the black pixel exists at the coordinates (4, 1), the horizontal counter 61 and the vertical counter 62_4 (column counter corresponding to the x coordinate 4) are incremented. Then, since there is no black pixel at the next coordinate (5, 1), the horizontal counter 61 is reset to “0”, and the counter value “1” immediately before the reset is reset to the pixel scanned. The coordinate (4, 1) is output from the horizontal counter 61 to the line determination unit 71. In this case, since the horizontal counter immediately before the reset is “1” and is smaller than the line determination threshold (5), the line determination unit 71 determines that the line image is not a line image. At this time, since the x coordinate is 5, the vertical counter 62_4 is not reset. Further, since the vertical counter 62_5 is not a black pixel, it remains “0”.

  If the process proceeds in the same procedure, the horizontal counter 61 is set to the coordinate (6, 3) because the black pixels are continuously present in the horizontal direction at the coordinates (2, 3) to (6, 3). It is incremented to “5”. When the next coordinate (7, 3) is reached, there is no black pixel, so the horizontal counter 61 is reset to “0”. Since the horizontal counter immediately before the reset is “5” and is equal to or greater than the line determination threshold (5), the line determination unit 71 sets the pixels (coordinates (2) corresponding to the reset counter value count values “1” to “5”. , 3) to (6, 3)) is determined to be a line image extending in the horizontal direction.

  In addition, since black pixels exist at the coordinates (2, 3) to (6, 3), the column counters 62_2 to 62_6 are incremented. However, since there is no black pixel at the coordinates (2, 4), (3,4), (5, 4), (6, 4), the column counters 62_2, 62_3, 62_5, and 62_6 are reset to “0”. The Since the counter value immediately before the reset in each column counter is “1” and smaller than the line determination threshold value (5), the line determination unit 71 is not a line image for each pixel corresponding to the counter value “1”. Is determined.

  After that, when each pixel is scanned in the same procedure, the horizontal counter value before reset is “7” at the coordinates (8, 6), which is equal to or greater than the line determination threshold value (5). Each pixel (coordinates 1, 6) to (7, 6) corresponding to the values “1” to “7” is determined as a horizontal line image.

  Further, at the coordinates (4, 7), the counter value of the column counter 62_4 immediately before the reset is “6”, which is equal to or greater than the line determination threshold (5), and therefore corresponds to the reset counter value count values “1” to “6”. Each pixel (coordinates (4, 1) to (4, 6)) is determined to be a vertical line image.

  Thereafter, the intersection extraction unit 72 extracts pixels determined to be part of the line image in both the horizontal direction and the vertical direction as intersections, and outputs the extracted intersections to the feature quantity calculation unit 32 as feature points. Note that when a plurality of pixels adjacent to each other are extracted as intersections, the intersection extraction unit 72 selects the center coordinates, the barycentric coordinates, or a predetermined position (for example, upper left) included in the region composed of the plurality of pixels. The coordinates are calculated, and the calculation result is output to the feature amount calculation unit 32 as a feature point.

  FIG. 15 is an explanatory diagram showing the extraction results of horizontal and vertical line images and the extraction results of intersections for the image data shown in FIG. As shown in this figure, in the case of the image data of FIG. 13, the coordinates (4,3), (4,6) are determined as line images in both the horizontal direction and the vertical direction, and these two coordinates are used as intersections. Extracted.

  Next, two diagonal line image extraction processing and intersection extraction processing performed in the intersection calculation unit 45 shown in FIG. 12 will be described.

  When the first labeling unit 63 scans each pixel from the upper left to the lower right with respect to the black pixel in the binarized image data, whether or not each pixel is a black pixel and an adjacent pixel ( Each pixel is labeled according to the label assigned to the left adjacent pixel and the upper adjacent pixel), and the label assignment result is stored in the first label history table 66. When performing the above labeling, the first labeling unit 63 makes a determination based on information stored in the first label history table 66 for labels assigned to adjacent pixels. In addition, the first labeling unit 63 stores, for each pixel that has been labeled (each pixel whose label is other than “0”), the coordinates of each pixel and the label assigned to each pixel in association with each other. Remember me. Details of the labeling method will be described later.

  The first diagonal counter 64 includes a first label counter 65 and a first label history table 66.

  The first label history table 66 stores a label assigned to each pixel by the first labeling unit 63 for each column (for each x coordinate). In this embodiment, every time a label is assigned to each pixel, the label of the column corresponding to the pixel is updated. In other words, in the first label history table 66, each pixel on the right side from the upper adjacent pixel of the target pixel in the upper adjacent line of the target pixel, and from the leftmost pixel in the line including the target pixel to the left adjacent pixel of the target pixel. A total of one horizontal line label is stored. The label assigned to the column including the pixel of interest (the label assigned to the pixel adjacent to the pixel of interest) is updated to the label assigned to the pixel of interest. As a result, the storage capacity required for the first label history table 66 can be reduced without lowering the accuracy of line determination compared to storing labels assigned to all pixels. In addition, the labels of the pixels adjacent to the left and top of the target pixel can be extracted easily and quickly.

  The first label counter 65 counts the number of times each label is allocated for each label. Then, the control unit 7 causes the first label counter 65 to output the label and the count value of each label to the line determination unit 71 when scanning for all the pixels is completed.

  When the second labeling unit 67 scans each black pixel in the binarized image data from the upper right to the lower left, whether or not each pixel is a black pixel and an adjacent pixel ( Each pixel is labeled according to the label assigned to the right adjacent pixel and the upper adjacent pixel, and the label assignment result is stored in the second label history table 70. When performing the above labeling, the second labeling unit 67 makes a determination based on information stored in the second label history table 70 for labels assigned to adjacent pixels. The second labeling unit 67 is a storage unit (not shown) that associates the coordinates of each pixel and the label assigned to each pixel for each pixel that has been labeled (each pixel whose label is other than “0”). Remember me. Details of the labeling method will be described later.

  The second diagonal counter 68 includes a second label counter 69 and a second label history table 70.

  The second label history table 70 stores the label assigned to each pixel by the second labeling unit 67 for each column (for each x coordinate). In this embodiment, every time a label is assigned to each pixel, the label of the column corresponding to the pixel is updated. As a result, the storage capacity required for the second label history table 70 can be reduced without lowering the accuracy of line determination compared to storing labels assigned to all pixels. In addition, the labels of the pixels adjacent to the right and top of the target pixel can be extracted easily and quickly.

  The second label counter 69 counts the number of times each label is allocated for each label. Then, the control unit 7 causes the second label counter 69 to output the label and the count value of each label to the line determination unit 71 when scanning for all pixels is completed.

  When the counter value input from the first diagonal counter 64 is greater than or equal to a threshold value (for example, 5), the line determination unit 71 determines that an image including each pixel corresponding to the label corresponding to the counter value is a line image. . Similarly, when the counter value input from the second oblique counter 68 is greater than or equal to a threshold value (for example, 5), an image including pixels corresponding to the label corresponding to the counter value is determined as a line image. To do. If the line determination unit 71 determines that the line image is a line image, the line determination unit 71 reads out the coordinates of each pixel constituting the line image from the storage unit and outputs the coordinates to the intersection extraction unit 72.

  Based on the input from the line determination unit 71, the intersection extraction unit 72 extracts the intersections between the above-described two diagonal line images, and outputs the extracted intersections to the feature quantity calculation unit 32 as feature points. Note that when a plurality of pixels adjacent to each other are extracted as intersections, the intersection extraction unit 72 coordinates the center coordinates, barycentric coordinates, or predetermined positions (for example, upper left) included in the area composed of the plurality of pixels. Is a feature point.

Next, a labeling method in the first labeling unit 63 will be described. The first labeling unit 63 labels each pixel based on the following rules.
(1) When the pixel of interest is a black pixel and is labeled with respect to the pixel adjacent to the left of the pixel of interest (when the label of the pixel adjacent to the left is not “0”), Assign the same label. For example, as shown in FIG. 16A, when the label of the pixel adjacent to the left of the target pixel is “1” and the label of the pixel adjacent to the target pixel is “2”, the label of the target pixel is “1”. "
(2) The pixel of interest is a black pixel, and is not labeled with respect to the pixel adjacent to the left of the pixel of interest (the label of the pixel adjacent to the left is “0”). Is labeled, the same label as the adjacent pixel is assigned. For example, as illustrated in FIG. 16B, when the label of the pixel adjacent to the left of the target pixel is “0” and the label of the pixel adjacent to the target pixel is “2”, the label of the target pixel is “2”. "
(3) When the pixel of interest is a black pixel and neither the pixel adjacent to the left of the pixel of interest nor the pixel adjacent above is labeled, a new label is assigned to the pixel of interest. Therefore, as shown in FIG. 16C, when both the left adjacent pixel and the upper adjacent pixel label of the target pixel are “0”, the target pixel is scanned so far (upper left to lower right). A new label is assigned that is not assigned to any other pixel in the incoming scan.
(4) If the target pixel is not a black pixel, the label of the target pixel is set to “0”.

  The labeling method in the second labeling unit 67 is other than the point that each pixel is scanned from the upper right to the lower left, and the “left adjacent” in the above (1) to (4) is changed to “right adjacent”. Is the same as the labeling method in the first labeling unit 63.

  Therefore, at the time of scanning from the upper right to the lower left, for example, as shown in FIG. 17A, the label of the pixel adjacent to the right of the target pixel is “1”, and the label of the upper adjacent pixel of the target pixel is “2”. In this case, the label of the target pixel is “1”. Further, for example, as shown in FIG. 17B, when the label of the pixel adjacent to the right of the target pixel is “0” and the label of the pixel adjacent to the target pixel is “2”, the label of the target pixel is “2”. Also, as shown in FIG. 17C, when both the left adjacent pixel and the upper adjacent pixel label of the target pixel are “0”, the target pixel is scanned so far (from the upper right to the lower left). In scanning, a new label that is not assigned to another pixel is assigned.

  Next, the line image extraction process and the intersection extraction process in the oblique direction will be described in more detail using the process for the image data shown in FIG. 18 as an example.

  First, a labeling method when scanning from the upper left to the lower right will be described with reference to FIG. FIG. 19 is an explanatory diagram showing a labeling procedure when scanning from the upper left to the lower right.

  First, at the start of scanning, the first label counter 65 and the first label history table 66 are initialized to “0”. Then, from upper left to lower right (coordinates (0,0) → (1,0) → (2,0) → ・ ・ ・ → (8,0) → (0,1) → (1,1) → Scan in order.

  Until there is a coordinate (6, 1), there is no black pixel, so all the information stored in the first label counter 65 and the first label history table 66 remains the initial value “0”.

  Since the black pixel exists at the coordinates (6, 1), labeling is performed. Here, both the label of the pixel on the left (the value of x = 5 in the first label history table 66) and the label of the upper adjacent pixel (the value before the update of x = 6 in the first label history table 66) are both “ 0 ”, and no label is assigned to any of the pixels so far, the first labeling unit 63 assigns a new label (label“ 1 ”) to the pixel at coordinates (6, 1). Further, the coordinates (6, 1) and the label “1” assigned to the pixel at the coordinates (6, 1) are associated with each other and stored in a storage unit (not shown). Further, the label counter of the label “1” is incremented to “1”. Also, the label of x = 6 in the first label history table 66 is updated to “1”.

  Since there is a black pixel at the next coordinate (7, 1), labeling is performed. Here, since the label of the pixel on the left (the value of x = 6 in the first label history table 66) is not “0”, the first labeling unit 63 is adjacent to the pixel at the coordinate (7, 1). The same label (label “1”) as the label of the pixel is assigned. Further, the coordinate (7, 1) and the label “1” assigned to the pixel at the coordinate (7, 1) are associated with each other and stored in a storage unit (not shown). Further, the label counter of the label “1” is incremented to “2”. Further, the label of x = 7 in the first label history table 66 is updated to “1”.

  If scanning continues in the same procedure, a new label (label "2") is assigned at coordinates (2,2), and label "2" assigned to the pixels at coordinates (2,2) and coordinates (2,2) Are stored in a storage unit (not shown), the label counter of the label “2” is incremented to “1”, and the label of x = 2 in the first label history table 66 is updated to “1”. Further, at coordinate (3,2), label “2” is assigned, and coordinate (3,2) and label “2” assigned to the pixel at coordinate (3,2) are associated with each other and not shown. , The label counter of the label “2” is incremented to “2”, and the label of x = 3 in the first label history table 66 is updated to “2”.

  Also, a new label (label “3”) is assigned to the coordinate (5,2), and the label (3) assigned to the pixel of the coordinate (5,2) and the coordinate (5,2) is associated with the illustration. The label counter of the label “3” is incremented to “1”, and the label of x = 5 in the first label history table 66 is updated to “1”. Also, at coordinate (6,2), label “3” is assigned, and the storage unit (not shown) is associated with coordinate (6,2) and label “3” assigned to the pixel at coordinate (6,2). , The label counter of the label “3” is incremented to “2”, and the label of x = 6 in the first label history table 66 is updated to “3”.

  Further, at the coordinates (3, 3), it is a black pixel, the label of the pixel on the left side (x = 2 value in the first label history table 66) is “0”, and the label of the pixel on the upper side (the first pixel) Since the value of x = 3 in the one-label history table 66 before update) is “2”, the label “2” is assigned, and the label “2” assigned to the pixel at coordinates (3,3) and coordinates (3,3) ”Is stored in a storage unit (not shown), the label counter of the label“ 2 ”is incremented to“ 3 ”, and the label of x = 3 in the first label history table 66 is updated to“ 2 ”. .

  Thereafter, scanning and labeling are similarly performed up to the lower right pixel (coordinates (8, 6)). When the scanning and labeling are completed up to the lower right pixel (coordinate (8, 6)), the control unit 7 outputs each label and the count value of each label from the first diagonal counter 64 to the line determination unit 71. Let

  Next, a labeling method when scanning from the upper right to the lower left will be described with reference to FIG. FIG. 20 is an explanatory diagram showing a labeling procedure when scanning from the upper right to the lower left.

  First, at the start of scanning, the second label counter 69 and the second label history table 70 are initialized to “0”. Then, from upper right to lower left (coordinates (8,0) → (7,0) → (6,0) → ・ ・ ・ → (0,0) → (8,1) → (7,1) → ・(In order of)

  Until the coordinate (7, 1) is reached, there is no black pixel, so all the information stored in the first label counter 65 and the first label history table 66 remains the initial value “0”.

Since the black pixel exists at the coordinates (7, 1), labeling is performed. Here, the label of the pixel on the right (the value of x = 8 in the second label history table 70) and the label of the upper adjacent pixel (the value before x = 7 in the second label history table 70) are both “ 0 ”and no label is assigned to any of the pixels up to that point, the second labeling unit 76 uses coordinates (7
, 1) is assigned a new label (label “1”). Further, the control unit 7 stores the coordinates (7, 1) and the label “1” assigned to the pixel at the coordinates (7, 1) in association with each other in a storage unit (not shown). Further, the label counter of the label “1” is incremented to “1”. Further, the label of x = 7 in the second label history table 70 is updated to “1”.

  In the same manner, for the black pixel, a label is assigned to each pixel according to the label of the right adjacent pixel, the label of the upper adjacent pixel, and the label assigned so far, and in the second label history table 70, The column label including each pixel is updated. Each time a label is assigned, the label counter value of the label is incremented. For each pixel that has been labeled (each pixel whose label is other than “0”), the coordinates of each pixel and the label assigned to each pixel are associated with each other and stored in a storage unit (not shown).

  When scanning and labeling are completed up to the lower left pixel (coordinates (0, 6)), the control unit 7 causes the second oblique counter 68 to cause the line determination unit 71 to output each label and the count value of each label. .

  FIG. 21 is an explanatory diagram showing the result of extracting line images in two oblique directions and the result of extracting intersections for the image data shown in FIG. As shown in FIG. 18, in the case of the image data of FIG. 18, five types of labels (labels “1” to “5”) are assigned in the scanning from the upper left to the lower right. The count values of the labels are “2” for labels “1”, “3”, “4”, and “5”, and “7” for label “2”. Therefore, in this case, the line determination unit 71 determines that an image including pixels corresponding to the label “2” whose count value is equal to or greater than the threshold “5” is a line image. When determining that the line image is a line image, the line determination unit 71 reads out the coordinates of each pixel constituting the line image from the storage unit and outputs the coordinates to the intersection extraction unit 72.

  In scanning from the upper right to the lower left, as shown in FIG. 21, three types of labels (labels “1” to “3”) are assigned. The count values of the labels are “2” for labels “2” and “3” and “11” for label “1”. Therefore, in this case, the line determination unit 71 determines that an image including pixels corresponding to the label “1” whose count value is equal to or greater than the threshold “5” is a line image. When determining that the line image is a line image, the line determination unit 71 reads out the coordinates of each pixel constituting the line image from the storage unit and outputs the coordinates to the intersection extraction unit 72.

  The intersection extraction unit 72 then coordinates (3, 3), (4, 3), (5) of the pixels determined to be a line image in both the upper left to lower right scanning and the upper right to lower left scanning. , 3) is calculated as the intersection (4, 3), which is the center coordinate. In addition, the intersection extraction unit 72 outputs the calculated intersection to the feature amount calculation unit 32 as a feature point.

  As described above, in the document matching processing unit 13 of the digital color multifunction peripheral 1 according to the present embodiment, the intersection calculation unit 45 extracts a line image (for example, a line part in a character or ruled line) from the input image data, and the line image The intersection point between them is extracted and used as a feature point.

  Thus, by using the intersection of line images as a feature point, a large number of feature points can be extracted even in the case of input image data read from a document with few characters, for example. It is possible to specify appropriately, and it is possible to improve the accuracy of the similarity determination process (collation process).

  In addition, since feature points can be calculated simply by extracting line images and calculating intersections between line images, characters, character string frames, frames, straight lines indicating frames, etc. are extracted from the input image data and extracted. Compared to a configuration in which similarity determination processing is performed on types of elements, the algorithm for the feature point calculation processing can be simplified and the processing time can be shortened.

  In this embodiment, the intersection calculation unit 45 shown in FIG. 12 performs scanning from the upper left to the lower right and the upper right to the lower left to label each pixel, and the number of pixels having the same label. Although the configuration for determining a line image according to the above has been described, the method for detecting a line image is not limited to this.

  For example, when scanning from the upper left to the lower right and from the upper right to the lower left, each pixel is labeled by the method described above, and the start and end coordinates are stored for each type of label, and each label is stored. The line image may be detected using the start coordinate and the end coordinate. FIG. 22 is a block diagram showing the configuration of the intersection calculation unit 45 in this case, and is provided in the document matching processing unit 13 instead of the intersection calculation unit 45 shown in FIG.

  The intersection calculation unit 45 shown in FIG. 22 includes a first labeling information storage unit 64b instead of the first diagonal counter 64 in the intersection calculation unit 45 shown in FIG. A two-labeling information storage unit 68b is provided.

  The first labeling information storage unit 64b includes a first label start / end coordinate storage unit 65b and a first label history table 66. The first label start / end coordinate storage unit 65b stores the start coordinate and the end coordinate for each type of label based on the result of label assignment to each pixel by the first labeling unit 63. The first label history table 66 has substantially the same function as that described with reference to FIG.

  The second labeling information storage unit 68 b includes a second label start / end coordinate storage unit 69 b and a second label history table 70. The second label start / end coordinate storage unit 69b stores the start coordinate and the end coordinate for each type of label based on the result of label assignment to each pixel by the second labeling unit 67. The second label history table 70 has substantially the same function as that described with reference to FIG.

  Next, line image extraction processing and intersection extraction processing performed by the intersection calculation unit 45 illustrated in FIG. 22 will be described.

  The first labeling unit 63 performs the labeling process using the labeling method described above. At that time, each time a label is assigned to each pixel, the label allocation result is stored in the first label history table 66, and the start and end coordinates of the label stored in the first label start / end coordinate storage unit 65b are stored. Will be updated. Similarly, the second labeling unit 67 stores the label allocation result in the second label history table 70 and the second label start / end coordinate storage unit 69b each time a label is assigned to each pixel. Update the start and end coordinates of the label.

  At the time of labeling, the minimum and maximum coordinates of each label are obtained for each of the x coordinate and the y coordinate. Thereby, the coordinates of the four points of the circumscribed rectangle of the label can be obtained. When scanning from the upper left to the lower right, the upper left corner of the circumscribed rectangle of the image consisting of pixels with the same label is the start coordinate, the lower right corner of the circumscribed rectangle is the end coordinate, and the same label when scanning from the upper right to the lower left The upper right corner of the circumscribed rectangle of the image consisting of pixels marked with is the start coordinate, and the lower left corner of the circumscribed rectangle is the end coordinate.

The procedure for updating the start coordinates and the end coordinates is as follows.
(1) When assigning a new label, both the start coordinates (xst, yst) and the end coordinates (xend, yend) are set to the current coordinates (xcrt, ycrt). (2) When updating an existing label, xst, yst, xend, and yend are updated based on the following conditions (a) to (d) and (e) to (h), respectively.

When scanning from upper left to lower right,
(A) If xcrt <xst, xst is updated.
(B) If ycrt <yst, update yst.
(C) If xcrt> xend, xend is updated.
(D) If ycrt> yend, update yend.

When scanning from upper right to lower left,
(E) If xcrt> xst, xst is updated.
(F) If ycrt <yst, update yst.
(G) If xcrt <xend, xend is updated.
(H) If ycrt> yend, update yend.

  Next, the line image extraction process and the intersection extraction process in the oblique direction by the intersection calculation unit 45 illustrated in FIG. 22 will be described in more detail using the process for the image data illustrated in FIG. 23 as an example.

  FIG. 24 is an explanatory diagram showing a labeling procedure when scanning the image shown in FIG. 23 from the upper left to the lower right. With reference to this figure, the processing at the time of scanning from the upper left to the lower right will be described first.

  First, at the start of scanning, the first label history table 66 is initialized to “0”. Then, from upper left to lower right (coordinates (0,0) → (1,0) → (2,0) → ・ ・ ・ → (11,0) → (0,1) → (1,1) → Scan in order.

  Until the coordinate (7, 1) is reached, there is no black pixel, so all the information stored in the first label history table 66 remains the initial value “0”.

  Since the black pixel exists at the coordinates (7, 1), labeling is performed. Here, both the label of the pixel on the left (the value of x = 6 in the first label history table 66) and the label of the upper adjacent pixel (the value before the update of x = 7 in the first label history table 66) are both “ 0 ”and no label is assigned to any of the pixels up to that time, the first labeling unit 63 assigns a new label (label“ 1 ”) to the pixel at coordinates (7, 1). Further, the coordinates (7, 1) and the label “1” assigned to the pixel at the coordinates (7, 1) are associated with each other and stored in a storage unit (not shown). Further, the start coordinate of the label “1” is set to (7, 1), and the end coordinate is set to (7, 1). That is, xst = 7, yst = 1, xend = 7, and yend = 1 are set. Further, the label of x = 7 in the first label history table 66 is updated to “1”.

  Since the black pixel exists also at the next coordinate (8, 1), labeling is performed. Here, since the label of the pixel on the left side (the value of x = 7 in the first label history table 66) is not “0”, the first labeling unit 63 is adjacent to the pixel at the coordinate (8, 1) on the left side. The same label (label “1”) as the label of the pixel is assigned. Further, the coordinates (8, 1) and the label “1” assigned to the pixel at the coordinates (8, 1) are associated with each other and stored in a storage unit (not shown). Further, the end coordinate of the label “1” is updated to (8, 1) in order to satisfy the above formula (c). That is, the value of xend is updated to 8. Further, the label of x = 8 in the first label history table 66 is updated to “1”.

  By performing the same processing for the coordinates (9, 1) and (10, 1), the label “1” is assigned to each of these coordinates, and the coordinates (9, 1) (10, 1) are assigned to the pixels. The label “1” is associated and stored in a storage unit (not shown). In addition, when labeling the coordinates (10, 1), the end coordinates of the label “1” are updated to (10, 1) to satisfy the above formula (c) (also updated at the time of the coordinates (9, 1)). Is overwritten with information of coordinates (10, 1)). Also, the labels of x = 9, 10 in the first label history table 66 are updated to “1”.

  If scanning is continued in the same procedure, a new label (label “2”) is assigned at coordinates (2, 2), and label “2” assigned to the pixels at coordinates (2, 2) and coordinates (2, 2). Are stored in a storage unit (not shown), the start coordinate of the label “2” is (2,2), the end coordinate is (2,2), and x = 2 in the first label history table 66 The label is updated to “2”.

  Further, the label “2” is assigned to the coordinate (3, 2), and the label (2) assigned to the pixel of the coordinate (3, 2) and the coordinate (3, 2) is associated with a storage unit (not shown). The end coordinates of the label “2” are updated to (3, 2) to satisfy the above formula (c), and the label of x = 3 in the first label history table 66 is updated to “2”.

  Similarly, the label “2” is assigned to the coordinate (4, 2), and the label (2) assigned to the pixel at the coordinate (4, 2) is associated with the memory (not shown). Stored in the department. Further, the end coordinate of the label “2” is updated to (4, 2) to satisfy the above formula (c). Further, the label of x = 4 in the first label history table 66 is updated to “2”.

  In addition, a new label (label “3”) is assigned to the coordinate (6, 2), and the label (3) assigned to the pixel at the coordinate (6, 2) and the coordinate (6, 2) is associated with the illustration. Stored in the storage unit, the start coordinate of the label “3” is set to (6,2), the end coordinate is set to (6,2), and the label of x = 6 in the first label history table 66 is updated to “3”. Is done.

  The same processing is performed for the coordinates (7,2) to (10,2), and the label “3” is assigned to each of these coordinates, and the coordinates (7,2) to (10,2) are assigned to the pixels. The label “3” is stored in a storage unit (not shown) in association with it. Further, the end coordinate of the label “3” is updated to (10, 2) during the processing for the coordinate (10, 2) (although it is also updated at the time of the coordinates (7, 2) to (9, 2)) (Overwritten with information of (10, 2)). Further, the label of x = 7 to 10 in the first label history table 66 is updated to “3”.

  Further, at the coordinates (3, 3), the pixel is a black pixel, the label on the left adjacent pixel (the value of x = 2 in the first label history table 66) is “0”, and the label on the upper adjacent pixel (the first pixel) Since the value before update of x = 3 in the one label history table 66 is “2”, the label “2” is assigned, and the label “2” assigned to the pixel at the coordinates (3,3) and the coordinates (3,3) is assigned. ”Is stored in a storage unit (not shown), and the end coordinates of the label“ 2 ”are updated to (4, 3) to satisfy the above formula (d). Further, the label of x = 3 in the first label history table 66 is updated to “2”.

  The same processing is executed for the coordinates (4, 3) to (8, 3), and the label “2” is assigned to each of these coordinates, and the coordinates (4, 3) to (8, 3) and the coordinates (4 , 3) to (8, 3) are associated with the label “2” assigned to the pixels and stored in a storage unit (not shown). Further, the end coordinate of the label “2” is updated to (8, 3) in order to satisfy the above formula (c) (although updated at the time of the coordinates (5, 3) to (7, 3), the coordinate ( Overwritten with the information of 8,3)). Further, the labels of x = 4 to 8 in the first label history table 66 are updated to “2”.

  Thereafter, scanning and labeling are similarly performed up to the lower right pixel (coordinates (11, 8)). When scanning and labeling are completed up to the lower right pixel (coordinates (11, 8)), the control unit 7 obtains each label and the start coordinate and end coordinate of each label from the first labeling information storage unit 64b. The output is output to the line determination unit 71.

  Next, labeling processing at the time of scanning from upper right to lower left will be described with reference to FIG. FIG. 25 is an explanatory diagram showing a labeling procedure when scanning the image shown in FIG. 23 from the upper right to the lower left.

  First, at the start of scanning, the second label history table 70 is initialized to “0”. Then, from upper right to lower left (coordinates (11,0) → (10,0) → (9,0) → ・ ・ ・ → (0,0) → (11,1) → (10,1) → (In order of)

  Until the coordinate (10, 1) is reached, there is no black pixel, so all the information stored in the second label history table 70 remains the initial value “0”.

  Since the black pixel exists at the coordinates (10, 1), labeling is performed. Here, the label of the pixel on the right (the value of x = 11 in the second label history table 70) and the label of the upper adjacent pixel (the value before x = 10 in the second label history table 70) are both “ 0 ”and no label is assigned to any of the pixels so far, the second labeling unit 76 assigns a new label (label“ 1 ”) to the pixel at coordinates (10, 1). In addition, the control unit 7 stores the coordinates (10, 1) and the label “1” assigned to the pixel at the coordinates (10, 1) in association with each other in a storage unit (not shown). Further, the start coordinate of the label “1” is (10, 1) and the end coordinate is (10, 1). Also, the label of x = 10 in the second label history table 70 is updated to “1”.

  In the same manner, for the black pixel, a label is assigned to each pixel according to the label of the right adjacent pixel, the label of the upper adjacent pixel, and the label assigned so far, and in the second label history table 70, The column label including each pixel is updated. Further, each time a label is assigned, the start coordinate and end coordinate of the label are updated. For each pixel that has been labeled (each pixel whose label is other than “0”), the coordinates of each pixel and the label assigned to each pixel are associated with each other and stored in a storage unit (not shown).

  When scanning and labeling are completed up to the lower left pixel (coordinates (0, 8)), the control unit 7 determines the line and the start / end coordinates of each label from the second labeling information storage unit 68b. Output to the unit 71.

  Next, line determination processing in the line determination unit 71 and intersection extraction processing in the intersection extraction unit 72 will be described with reference to FIG. FIG. 46 is an explanatory diagram showing the image data shown in FIG. 23, the labeling result, the line determination result, and the intersection extraction result for this image data.

  As shown in FIG. 46, in the case of the image data in FIG. 23, five types of labels (labels “1” to “5”) are allocated in the scanning from the upper left to the lower right.

The line determination unit 71 sets the distance d between the start coordinates (x _st , y _st ) and the end coordinates (x _end , y _end ) of each label as d = √ ((xst−xend) ² + (yst−yend) ² ). Calculate based on In the example of FIG. 46, the distance d for each label is “3” for label “1”, “8.94” for label “2”, “4” for “3”, “2” for “4”, “5” becomes “3.16”.

  In addition, the line determination unit 71 compares the distance d for each label calculated as described above with a preset threshold value (“8” in the present embodiment), and copes with a label whose distance d is equal to or greater than the threshold value. An image made up of each pixel to be determined is a line image. In the example of FIG. 46, an image including pixels corresponding to the label “2” is determined as a line image.

  When determining that the line image is a line image, the line determination unit 71 reads out the coordinates of each pixel constituting the line image from the storage unit and outputs the coordinates to the intersection extraction unit 72.

  In the scanning from the upper right to the lower left, as shown in FIG. 46, four types of labels (labels “1” to “4”) are assigned. The distance d for each label is “10.82” for the label “1”, “2” for the label “2”, “2” for “3”, and “2” for “4”. Therefore, in this case, the line determination unit 71 determines that an image including pixels corresponding to the label “1” whose distance d is equal to or greater than the threshold “8” is a line image. When determining that the line image is a line image, the line determination unit 71 reads out the coordinates of each pixel constituting the line image from the storage unit and outputs the coordinates to the intersection extraction unit 72.

  The intersection extraction unit 72 calculates the center coordinates of the pixels determined to be a line image in both the upper left to lower right scanning and the upper right to lower left scanning as the intersection of the line images. The center coordinates may be calculated by, for example, calculating the average value of the x coordinate and the y coordinate, respectively, and rounding off after the decimal point. Thus, in the example of FIG. 46, the coordinates (3, 3) to (8, 3), (8) are determined as pixels determined to be a line image in both the upper left to lower right scanning and the upper right to lower left scanning. 4, 4) to (8, 4) are extracted, and coordinates (6, 3), which are the center coordinates of these pixels, are calculated as intersections of the line images.

  FIG. 47 relates to the detection processing of the line image in the oblique direction, when detecting the line image using the label count value and when detecting the line image using the distance from the start coordinate to the end coordinate of the line image. It is explanatory drawing which shows the difference in a detection result.

  The upper part of FIG. 47 shows the result of detecting the line image using the label count value for the same image as FIG. 23 and the result of detecting the line image using the distance from the start coordinate to the end coordinate of the line image. Yes. 47, in this case, the label count value of label 1 is 30, and the distance from the start coordinate to the end coordinate is 10.82.

  On the other hand, the lower part of FIG. 47 shows the result of detecting the line image using the label count value and the distance from the start coordinate to the end coordinate of the line image for the image obtained by thinning the entire line image portion in the image of FIG. The result of having detected the line image using is shown. 47, in this case, the label count value of label 1 is 20, and the distance from the start coordinate to the end coordinate is 10.82.

  When line image detection is performed using the label count value in this way, the label count value increases as the line thickness increases, so even if the line image length is the same, the label count value depends on the thickness. Increases or decreases. Therefore, for example, in the case of the example shown in FIG. 47, if the threshold value of the label count value is between 20 and 30 (for example, 25), the line detection result changes depending on the thickness of the line.

  On the other hand, when the line image is detected using the distance from the start coordinate to the end coordinate of the line image, from the start coordinate to the end coordinate if the length of the line image is the same even if the thickness of the line image is different. The distance is substantially constant.

  Therefore, when it is desired to determine the line image without depending on the thickness of the line image, it is preferable to detect the line image using the distance from the start coordinate to the end coordinate of the line image.

  In the above example, the intersection extraction unit 72 uses the center coordinates of the pixels determined to be a line image in both the scan from the upper left to the lower right and the scan from the upper right to the lower left as the intersection of the line images. However, the method of extracting the intersection is not limited to this.

  For example, as shown in FIG. 51, a straight line (line segment) expression of this line image is calculated from the start coordinates and end images of the line image detected in the upper left to lower right scanning, and the upper right to lower left scanning is performed. Similarly, for the line image detected in step 1, the straight line (line segment) formula of the line image is calculated from the start coordinate and the end image, and the coordinates of the intersection point are calculated based on the two straight line (line segment) formulas. You may do it.

Specifically, as shown in FIG. 51, in the case of the image of FIG. 23, the start coordinate of the label “2” determined as a line at the time of scanning from the upper left to the lower right is (2, 2), and the end coordinate is Since (10, 6), the formula of the straight line (line segment) between the two points is as follows.
y = x / 2 + 1 (2 ≦ x ≦ 10, 2 ≦ y ≦ 6)
Further, since the start coordinates of the label “1” determined as a line at the time of scanning from the upper right to the lower left are (10, 1) and the end coordinates are (1, 7), a straight line (line) between these two points The equation of (minute) is as follows.
y = -2x / 3 + 23/3 (where 1 ≦ x ≦ 10, 1 ≦ y ≦ 7)
When the intersection point is calculated from these two straight line (line segment) equations, (40/7, 27/7) ≈ (5.71,3.86), and when the decimal point is rounded off, the intersection point coordinates are (6.4). It becomes.

  In addition, when calculating the intersection of the horizontal line image and the vertical line image, as in the case of calculating the intersection of the diagonal line images, the straight line equations of both line images are calculated. The intersection of both lines may be calculated based on the formula of the two lines, and the coordinates corresponding to the calculated intersection may be used as the intersection of both line images.

  In the present embodiment, the intersection calculation processing is performed on all the pixels binarized by the binarization processing unit 44, but the present invention is not limited to this.

  For example, when the area of the connected area of black pixels is greater than or equal to a predetermined value, or when the vertical width and horizontal width of the connected area of black pixels are greater than or equal to a predetermined value, the intersection calculation is performed for this connected area. Processing may not be performed. Thus, for example, when an area of a predetermined range or more is painted black, this area is detected as, for example, a local background (when the size is large) or an isolated point (when the size is small), and the intersection It can be excluded from the calculation process.

  Further, an edge detection unit that performs an edge detection process on the input image data before the intersection calculation process may be provided, and the intersection calculation process may be performed on a pixel detected as an edge. In this case, for example, even when a region of a predetermined range or more is painted black, a portion of about several pixels from the peripheral portion in this region is detected as an edge, and line detection processing and intersection calculation are performed on this portion. Processing can be performed. As the edge detection means, for example, a method of calculating the absolute value of the pixel value of the target pixel and a pixel adjacent to the target pixel, and determining the edge when the calculated absolute value is equal to or greater than a predetermined threshold, A method of extracting an edge by performing threshold processing on the processing result of the Sobel filter or Laplacian filter can be used.

  In the present embodiment, the case where the present invention is applied to the digital color multifunction peripheral 1 has been described. However, the application target of the present invention is not limited to this. For example, the present invention may be applied to a monochrome multifunction device. Further, the present invention is not limited to a multifunction machine, and may be applied to an image processing apparatus such as a single facsimile communication apparatus, a copying machine, and an image reading apparatus.

  FIG. 26 is a block diagram showing a configuration example when the present invention is applied to a flatbed scanner (image reading apparatus, image processing apparatus) 1 ′.

  As shown in this figure, the flatbed scanner 1 'includes a color image input device 2 and a color image processing device 3'. The color image processing device 3 ′ includes an A / D conversion unit 11, a shading correction unit 12, a document collation processing unit 13, a control unit 7 (not shown in FIG. 26), and a memory 8 (not shown in FIG. 26). The color image input device 2 is connected to the image reading device 1 ′ as a whole. The functions of the A / D conversion unit 11, the shading correction unit 12, the document collation processing unit 13, the control unit 7, and the memory 8 in the color image input device (image reading unit) 2 are substantially the same as those of the digital color multifunction peripheral 1 described above. Since it is the same, description is abbreviate | omitted here.

  Further, the function of the document collation processing unit 13 may be realized by an image processing system including an image processing device and a server device that is communicably connected to the image processing device. 27 shows an image processing apparatus (MFPs A, B,..., Printers A, B,..., Facsimiles A, B,..., Computers A, B,. (A, B,..., Scanners A, B,...) And a server apparatus 50 are explanatory diagrams showing the configuration of an image processing system 100 that is communicably connected. The configuration of the image processing system 100 is not limited to this. For example, the server device 50, a multifunction device, a printer (image forming device), a facsimile, a computer, a digital camera (image reading device), a scanner (image reading device). ) May be included.

  The scanner includes a document table, an optical scanning unit, a CCD (charge coupled device), etc., and scans the document image placed on the document table by the optical scanning unit to read the document image and generate image data. To do. The digital camera includes an imaging lens, a CCD, and the like (image input device), and shoots a document image, a person, a landscape, and the like to generate image data. The scanner and the digital camera may have a function of performing predetermined image processing (for example, various correction processes) in order to appropriately reproduce the image. The printer prints an image based on image data generated by a computer, a scanner, and a digital camera on a sheet (recording paper). Further, the facsimile performs processing such as binarization processing, resolution conversion processing, and rotation on the image data read from the image input device, and transmits the image data compressed to a predetermined format to the other party. Then, the image data transmitted from the other party is decompressed, subjected to rotation processing, resolution conversion processing, and halftone processing according to the performance of the image output apparatus, and output as a page unit image. The multifunction machine has at least two of a scanner function, a facsimile transmission function, and a printing function (copying function, printer function). The computer edits image data read by a scanner or a digital camera, or creates a document using application software.

  In the image processing system 100, each unit of the document collation processing unit 13 described above is distributed and provided in a server device 50 and an image processing device connected to the server device 50 via a network. The image processing device and the server device 50 cooperate to realize the function of the document collation processing unit 13.

  FIG. 28 is a block diagram illustrating a configuration example in which the functions of the document matching processing unit 13 are provided in a distributed manner in the server device 50 and the digital color multifunction peripheral 1.

  As shown in FIG. 28, the color image processing apparatus 3 of the digital color multifunction peripheral 1 controls the operation of the document matching processing unit 13a including the feature point calculating unit 31 and the feature amount calculating unit 32, and the document matching processing unit 13a. Control unit 7a, a memory 8a for storing information necessary for processing of the document collation processing unit 13a, and a communication device 5 for communicating with an external device. The server device 50 includes a communication device 51 that performs communication with an external device, a voting processing unit 33, a similarity determination processing unit 34, and a document matching processing unit 13b that includes a registration processing unit 37, and a document matching processing unit. A control unit 7b for controlling 13b and a memory 8b for storing information necessary for processing of the document collation processing unit 13b. When data transmission / reception is required between each functional block provided in the digital color multifunction peripheral 1 and each functional block provided in the server device 50, the control unit 7a and the control unit 7b are connected to the communication devices 5 and 51, respectively. To transmit and receive data as appropriate. Other functions are the same as those described above.

  In the example of FIG. 28, all of the feature amount calculation unit 32 (feature point extraction unit 32a, invariant calculation unit 32b, and hash value calculation unit 32c) are provided in the digital color multifunction peripheral 1, but this is not limitative. For example, as shown in FIG. 29, the feature point extraction unit 32 a and the invariant calculation unit 32 b may be provided in the digital color multifunction peripheral 1, while the hash value calculation unit 32 c may be provided in the server device 50.

  Each unit of the feature amount calculation unit 32 is provided in the server device 50, and data related to the feature points calculated by the feature point calculation unit 31 is transmitted from the digital color multifunction peripheral 1 to the server device 50, and is provided in the server device 50. The feature amount calculation unit 32 may calculate a hash value based on the hash table 103 stored in the memory 8b and the received feature point data. Further, the feature point calculation unit 31 and the feature amount calculation unit 32 are provided in the server device 50, and input image data is transmitted from the digital color multifunction peripheral 1 to the server device 50 to calculate the feature points provided in the server device 50. The unit 31 and the feature amount calculation unit 32 may calculate the hash value based on the input image data received from the server device 50 and the hash table 103 stored in the memory 8b.

  Further, in the above description, an example in which the similarity determination process is performed has been described. However, in the case of performing the registration process, the registration processing unit 37 provided in the server device 50 has received from the digital color multifunction peripheral 1. The document ID and the hash value (or the hash value calculated by the hash value calculation unit 32c provided in the server device 50) may be registered in the hash table 103 provided in the memory 8b. Whether the similarity determination process or the registration process is performed is designated by the user of the digital color multifunction peripheral 1 via the operation panel 6, and a signal indicating which process is performed is transmitted to the server device 50. Alternatively, the registration process may be performed for the input image that the server device 50 has determined as not similar as a result of the similarity determination process.

  When the server device 50 includes the hash value calculation unit 32c, the hash value is calculated by using a method different from the method for calculating the hash value stored in the hash table 103 (using another hash function). The hash table 103 may be updated using a hash value. As a result, for example, an appropriate hash value referring to a feature value (invariant) according to the type of document image or the like can be registered (updated) in the hash table 103, and voting processing can be performed using it. The accuracy (similarity determination accuracy) can be improved.

[Embodiment 2]
Another embodiment of the present invention will be described. For convenience of explanation, members having the same functions as those of the first embodiment are denoted by the same reference numerals as those of the first embodiment, and description thereof is omitted.

  FIG. 32 is a block diagram illustrating a schematic configuration of the feature point calculation unit 31b provided in the digital color multifunction peripheral 1 according to the present embodiment. The feature point calculation unit 31b is provided in the digital color multifunction peripheral 1 instead of the feature point calculation unit 31 in the first embodiment. The configuration of the digital color MFP 1 according to the present embodiment is substantially the same as that of the digital color MFP 1 according to the first embodiment except that the configuration of the feature point calculation unit is different.

  As shown in FIG. 32, in addition to the configuration of the feature point calculation unit 31 in the first embodiment, the feature point calculation unit 31b includes an inclination detection unit 47 and an inclination in front of the signal conversion processing unit (achromatic processing unit) 41. A correction processing unit 48 is provided.

  The inclination detection unit 47 performs processing for detecting the inclination of the input image (the inclination of the original image in the input image data in the vertical direction and the horizontal direction with respect to the vertical and horizontal directions of the input image data). The tilt detection method is not particularly limited, and various conventionally known methods can be used.

  For example, as shown in FIG. 33, the edge of the document at two points separated by w / 2 on both sides in the x direction from the center in the x direction in the input image data (the center in the x direction in the area read by the scanner). e1 and e2 and end portions E1 and E2 of the input image data are detected. Then, a distance d1 along the y direction from the edge e1 of the document to E1 of the input image data, a distance d2 along the y direction from the edge e2 of the document to E2 of the input image data, and an edge e1 of the document. Tan θ = (d2−d1) / w is calculated based on the distance w along the x direction from to e2, and the tilt angle θ is calculated. The calculation method of the inclination angle θ is not particularly limited, but can be easily calculated by using, for example, a tangent-angle table as shown in FIG.

  As a method of extracting the edge of the document, for example, a method of calculating the luminance value in the edge measurement window as shown in FIG. 33 can be used. Specifically, as shown in FIG. 35, edge enhancement filter processing (edge enhancement processing) is performed on input image data in an edge measurement window having a predetermined size (here, 20 pixels in the x direction × 150 pixels in the y direction). To obtain an edge-enhanced image. FIG. 36 shows an example of an edge detection filter. Then, in order to detect the edge in the vertical direction (y direction) on the edge-enhanced image, a projection process for calculating the sum of the pixel values in the horizontal direction (x direction) is performed, and the position where the peak in the vertical direction is taken (for example, Find the minimum value. The reason for performing the projection processing (processing for summing up the luminance values of the images arranged in the x direction in the window) is to suppress the influence of noise. FIG. 37 is a graph showing the projection results.

  The tilt correction processing unit 48 corrects the tilt of the document in the input image data based on the tilt angle θ detected by the tilt detection unit 47. The method of the inclination correction process is not particularly limited, and for example, an affine transformation process using a rotation matrix can be used.

  In general, coordinates (x ′, y ′) obtained by rotating the coordinates (x, y) by θ can be expressed by the following rotation determinant.

  When outputting as a pixel value of an image, a decimal value (xs, ys) corresponding to an integer value (x ′, y ′) may be calculated, and this may be interpolated by bilinear or the like. The decimal value (xs, ys) is calculated by the following determinant, which is an inverse transformation formula of the above rotation determinant.

In the xy coordinate system, the coordinates of four pixels surrounding (xs, ys) are (x _i , y _i ), (x _{i + 1} , y _i ), (x _i , y) as shown in FIG. _{i + 1} ), (x _{i + 1} , y _{i + 1} ) (i and j are integers of 1 or more). Here, x _i ≦ xs <x _{i + 1} and y _j ≦ ys <y _{j + 1} . Then, the values of the four pixels are set to (x _i , y _i ): Z ₁ , (x _{i + 1} , y _i ): Z ₂ , (x _i , y _{i + 1} ): Z ₃ , (x _{i + 1} , y _{i + 1} ), respectively. : and _{Z 4,} the distance in the x direction between _{x i} and xs, the ratio of the distance for the x-direction between xs and _{x i + 1 u: 1-} u, the distance in the y direction between _{y i} and ys And the ratio of ys and the distance in the y direction between y _{i + 1} and v: 1−v, the coordinate value Z after bilinear interpolation is Z _{(x ′, y ′)} = Z _{(xs, ys)} = (1-v) {(1-u) Z ₁ + u · Z ₂ } + v {(i−u) Z ₃ + u · Z ₄ } Note that the method of calculating the trigonometric ratio is not particularly limited, but may be performed using a table, for example, as in the example of FIG. 34 described above.

  FIG. 39 is a flowchart showing the flow of the feature point calculation process in the digital color multifunction peripheral 1 according to the present embodiment.

  As shown in this figure, when image data is input from the shading correction unit 12 to the feature point calculation unit 31b, the tilt detection unit 47 performs a tilt (tilt angle) detection process of the input image (S20a), and the tilt detection unit. Based on the detection result 47, the tilt correction processing unit performs tilt (tilt angle) correction processing (S20b). Thereafter, the achromatization processing by the signal conversion processing unit (achromatic processing unit) 41, the resolution conversion processing by the resolution conversion unit 42, the MTF processing by the MTF processing unit 43, and the binarization processing by the binarization processing unit 44 are performed. After that (none of them are shown), the intersection calculation unit 45 performs intersection (feature point) detection processing by the same method as in FIG. 11 (S21 to S28).

  As described above, in the present embodiment, the tilt detection unit 47 and the tilt correction processing unit 48 are provided, and the intersection (feature point) is extracted after the tilt correction of the input image. Thereby, for example, when the document image in the input image data is tilted (for example, when the document is read while being tilted with respect to a predetermined arrangement angle at the reading position of the image reading apparatus) However, as in the case where there is no inclination, the feature points can be extracted with high accuracy. Therefore, the similarity between the input image and the registered image can be accurately determined by calculating the image similarity based on the feature amount calculated based on the extracted feature points.

[Embodiment 3]
Still another embodiment of the present invention will be described. For convenience of explanation, members having the same functions as those of the first and second embodiments are denoted by the same reference numerals as those of the first and second embodiments, and description thereof is omitted.

  FIG. 40 is a block diagram illustrating a schematic configuration of the feature point calculation unit 31c provided in the digital color multifunction peripheral 1 according to the present embodiment. The feature point calculation unit 31c is provided in the digital color multifunction peripheral 1 in place of the feature point calculation unit 31 in the first embodiment. The configuration of the digital color MFP 1 according to the present embodiment is substantially the same as that of the digital color MFP 1 according to the first embodiment except that the configuration of the feature point calculation unit is different.

  As illustrated in FIG. 40, the feature point calculation unit 31 c includes, in addition to the configuration of the feature point calculation unit 31 in the first embodiment, an expansion processing unit 52 and a degeneration processing unit 53, following the binarization processing unit 44. The missing correction unit 54 is provided.

  The expansion processing unit 52 counts the number of black pixels included in the pixel of interest and the peripheral pixels of the pixel of interest, sets the pixel of interest whose count value is equal to or greater than a threshold (first threshold) as a black pixel, and the count value is less than the threshold The pixel of interest is a white pixel (hereinafter referred to as expansion processing). The threshold value is not particularly limited, but in the present embodiment, the threshold value is 1.

  FIG. 41A is an explanatory diagram showing an example of image data before the expansion process is performed. FIG. 41B is an example in which the expansion process is performed on the image data of FIG. It is explanatory drawing which shows image data.

  The reduction processing unit 53 counts the number of white pixels included in the target pixel and the peripheral pixels of the target pixel in the image data after the expansion processing, and sets the target pixel whose count value is equal to or greater than a threshold (second threshold) as a white pixel. A pixel having a count value less than the threshold is defined as a black pixel (hereinafter referred to as a degeneration process). The threshold value is not particularly limited, but in the present embodiment, the threshold value is 1.

  Fig.42 (a) is explanatory drawing which shows an example (it is the same as the image data of FIG.41 (b)) before performing degeneration processing, FIG.42 (b) shows the image data of Fig.41 (a). On the other hand, it is explanatory drawing which shows the image data which performed the degeneracy process by setting the said threshold value to 1.

  FIG. 43 is a flowchart showing the flow of the feature point calculation process in the digital color multifunction peripheral 1 according to the present embodiment.

  As shown in this figure, when image data is input from the shading correction unit 12 to the feature point calculation unit 31c, the achromatization processing by the signal conversion processing unit (achromatic processing unit) 41 and the resolution by the resolution conversion unit 42 are performed. After performing the conversion processing, the MTF processing by the MTF processing unit 43, and the binarization processing by the binarization processing unit 44 (both not shown), the expansion processing unit 52 performs the expansion processing described above on the input image data. (S20c), the reduction processing unit 53 performs the reduction processing on the image data after the expansion processing (S20d). Thereafter, the intersection calculation unit 45 performs intersection (feature point) detection processing by the same method as that in FIG. 11 (S21 to S28).

  As described above, in the present embodiment, the expansion processing unit 52 counts the number of black pixels included in the target pixel and the peripheral pixels of the target pixel, and sets the target pixel whose count value is equal to or greater than the threshold as the black pixel. . Further, the reduction processing unit 53 counts the number of white pixels included in the target pixel and the peripheral pixels of the target pixel in the image data after the expansion process, and sets the target pixel whose count value is equal to or greater than the threshold as the white pixel.

  Thereby, even when a gap (missing image data) occurs in the line portion of the input image data, the line detection process can be performed after correcting the gap. Therefore, it is possible to prevent the line from being cut and detected by a gap (missing image data) and to accurately detect the feature points of the input image data.

  In addition, although this embodiment demonstrated the structure provided with the expansion process part 52 and the degeneration process part 53 in addition to each structure of the feature point calculation part 31 in Embodiment 1, it does not restrict to this. For example, as shown in FIG. 44, a feature point calculation unit 31d provided with an expansion processing unit 52 and a degeneration processing unit 53 in the feature point calculation unit 31b shown in the second embodiment may be used. In this case, as shown in FIG. 45, when image data is input from the shading correction unit 12 to the feature point calculation unit 31d, the inclination detection unit 47 performs an inclination (inclination angle) detection process of the input image (S20a). Based on the detection result of the tilt detection unit 47, the tilt correction processing unit performs tilt (tilt angle) correction processing (S20b). Thereafter, the achromatization processing by the signal conversion processing unit (achromatic processing unit) 41, the resolution conversion processing by the resolution conversion unit 42, the MTF processing by the MTF processing unit 43, and the binarization processing by the binarization processing unit 44 are performed. (Neither is shown), the expansion processing unit 52 performs the expansion processing described above on the input image data (S20c), and the compression processing unit 53 performs the compression processing on the image data after the expansion processing (S20c). S20d). Thereafter, the intersection calculation unit 45 performs intersection (feature point) detection processing by the same method as that in FIG. 11 (S21 to S28).

[Embodiment 4]
Still another embodiment of the present invention will be described. For convenience of explanation, members having the same functions as those in the above embodiments are denoted by the same reference numerals as those in the above embodiments, and description thereof is omitted.

  The digital color multifunction peripheral 1 according to the present embodiment is substantially the same as the digital color multifunction peripheral 1 according to the first embodiment except that the labeling method by the first labeling unit 63 and the second labeling unit 67 is different. It is the composition.

  In the first embodiment, the first labeling unit 63 scans each pixel from the upper left to the lower right, whether or not the pixel of interest is a black pixel, and labels for the pixels adjacent to the left and top of the pixel of interest. Based on the result of the labeling, the target pixel is labeled. In addition, the second labeling unit 67 scans each pixel from the upper right to the lower left, and determines whether or not the target pixel is a black pixel and the labeling result for the right adjacent and upper adjacent pixels of the target pixel. Accordingly, the target pixel is labeled.

  On the other hand, in the present embodiment, the first labeling unit 63 scans each pixel from the upper left to the lower right, whether or not the target pixel is a black pixel, and the left and upper sides of the target pixel. The pixel of interest is labeled based on the labeling result for the adjacent pixel and the upper left adjacent pixel. In addition, the second labeling unit 67 scans each pixel from the upper right to the lower left, whether or not the pixel of interest is a black pixel, and labels the pixels adjacent to the right, upper, and upper right of the pixel of interest. Based on the result, the target pixel is labeled.

  In the first label history table 66, each pixel on the right side from the upper left adjacent pixel of the target pixel in the upper adjacent line of the target pixel, and the pixel on the left side of the target pixel from the left end pixel in the line including the target pixel. A total of 1 horizontal line + 1 pixel label is stored. The first label history table 66 deletes the labeling result for the pixel on the upper left side of the target pixel every time the labeling for the target pixel is completed, and stores the labeling result for the target pixel. As a result, the storage capacity required for the first label history table 66 can be reduced without lowering the accuracy of line determination compared to storing labels assigned to all pixels. Further, the labels of the pixels on the left side, the upper side, and the upper left side of the target pixel can be easily and quickly extracted.

  Similarly, in the second label history table 70, each pixel on the left side from the pixel on the upper right side of the target pixel in the line on the upper side of the target pixel, and on the right side of the target pixel from the right end pixel in the line including the target pixel. A total of 1 horizontal line + 1 pixel label up to the pixel is stored. Then, every time the labeling for the target pixel is completed, the second label history table 70 deletes the labeling result for the pixel on the upper right side of the target pixel and stores the labeling result for the target pixel. As a result, the storage capacity required for the second label history table 70 can be reduced without lowering the accuracy of line determination compared to storing labels assigned to all pixels. In addition, the labels of the pixels adjacent to the right side, the upper side, and the upper right side of the target pixel can be easily and quickly extracted.

  Next, the labeling method by the first labeling unit 63 will be described in more detail with reference to the drawings. FIG. 48 is an explanatory diagram showing the reference order of adjacent pixels when the first labeling unit 63 labels the target pixel.

As shown in this figure, the first labeling unit 63 scans the image data from the upper left to the lower right (the process of scanning each line from the left end to the right end is performed from the upper end line to the lower end line. The labeling is performed based on the following rules (a) to (e).
(A) When the target pixel is a black pixel and is labeled with respect to the pixel adjacent to the left of the target pixel (when the label of the pixel adjacent to the left is not “0”), Assign the same label.
(B) The pixel of interest is a black pixel and is not labeled with respect to the pixel adjacent to the left of the pixel of interest (the label of the pixel adjacent to the left is “0”), and the pixel above the pixel of interest Is labeled, the same label as the adjacent pixel is assigned.
(C) The target pixel is a black pixel, and is not labeled with respect to the left and upper adjacent pixels of the target pixel (the labels of the left adjacent pixel and the upper adjacent pixel are “0”) If the pixel adjacent to the upper left of the target pixel is labeled, the same label as the pixel adjacent to the upper left is assigned.
(D) If the pixel of interest is a black pixel and is not labeled on any of the pixels adjacent to the left, top, or top left of the pixel of interest, a new label is assigned to the pixel of interest.
(E) When the target pixel is not a black pixel, the label of the target pixel is set to “0”.

  Note that the labeling method in the second labeling unit 67 changes the point of scanning each pixel from the upper right to the lower left, and changes the “left neighbor” in the above (a) to (e) to “right neighbor”. The labeling method in the first labeling unit 63 is the same as that in the first labeling unit 63, except that “upper left” is changed to “upper right”. FIG. 49 is an explanatory diagram showing the reference order of adjacent pixels when the second labeling unit 76 labels the target pixel.

  As described above, in the present embodiment, when labeling is performed by the first labeling unit 63, in addition to the labeling result for the left adjacent pixel and the upper adjacent pixel of the target pixel, the labeling result for the upper left adjacent pixel is considered. Then, the target pixel is labeled. In addition, when performing labeling by the second labeling unit 67, in addition to the labeling result for the right adjacent pixel and the upper adjacent pixel of the target pixel, the labeling for the target pixel is performed in consideration of the labeling result for the pixel adjacent to the upper right. Do.

  As a result, even if there is a shift in the position of the diagonal line due to variations caused by errors at the time of image reading, separate labels are attached to the lines that are originally the same. It is possible to prevent the detection from being lost and improve the line detection accuracy.

  This point will be described in more detail with reference to FIGS. 50 (a) to 50 (c). FIG. 50A is an explanatory diagram showing a result of labeling this image by the labeling method shown in the first embodiment when the image shown in FIG. 18 is appropriately read by the color image input apparatus 2. is there. FIG. 50B shows a case where a reading error occurs when the image shown in FIG. 18 is read by the color image input device 2, and the image in which this reading error has occurred is labeled by the labeling method shown in the first embodiment. It is explanatory drawing which shows the attached result. FIG. 50C shows a case where a reading error occurs when the image shown in FIG. 18 is read by the color image input device 2, and the image in which this reading error has occurred is labeled by the labeling method shown in the present embodiment. It is explanatory drawing which shows the attached result.

  In the example shown in FIG. 50B, a position shift due to a reading error occurs in the line image from the upper left to the lower right. Then, when the labeling method shown in the first embodiment is applied to an image in which such a positional deviation has occurred, a label different depending on the position is attached even though the label is originally the same line. Will not be detected as.

  On the other hand, as shown in FIG. 50 (c), according to the labeling method of the present embodiment, the labeling result for the upper left adjacent pixel is taken into account even if there is a positional shift due to the reading error. Thus, the same label is attached to the pixels that are originally the same line, and can be detected as a line. Therefore, even when a reading error occurs, the line can be detected with high accuracy.

  Note that the labeling method (line detection method) according to the present embodiment can be applied to any configuration of the above-described embodiments.

  In each of the above embodiments, each unit (each block) constituting the document collation processing unit and the control unit provided in the digital color multifunction peripheral 1 and / or the server device 50 is realized by software using a processor such as a CPU. The That is, the digital color multifunction peripheral 1 and / or the server device 50 develops a central processing unit (CPU) that executes instructions of a control program that realizes each function, a read only memory (ROM) that stores the program, and the program. A random access memory (RAM), and a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is to read the program code (execution format program, intermediate code program, source program) of the control program of the digital color multifunction peripheral 1 and / or the server device 50, which is software that realizes the functions described above, with a computer. This is achieved by supplying the recording medium recorded as possible to the digital color multifunction peripheral 1 and / or the server device 50, and reading out and executing the program code recorded on the recording medium by the computer (or CPU or MPU). .

  Examples of the recording medium include tapes such as magnetic tapes and cassette tapes, magnetic disks such as floppy (registered trademark) disks / hard disks, and disks including optical disks such as CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the digital color multifunction peripheral 1 and / or the server device 50 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  In addition, each block of the digital color multifunction peripheral 1 and / or the server device 50 is not limited to being realized using software, and may be configured by hardware logic, and performs a part of the processing. A combination of hardware and arithmetic means for executing software for controlling the hardware and performing the remaining processing may be used.

  The computer system of the present invention includes an image input device such as a flatbed scanner, a film scanner, and a digital camera, a computer in which various processes such as the similarity calculation process and the similarity determination process are performed by loading a predetermined program, An image display device such as a CRT display or a liquid crystal display that displays the processing results of the computer, and an image forming device such as a printer that outputs the processing results of the computer to paper or the like may be used. Furthermore, a network card, a modem, or the like as communication means for connecting to a server or the like via a network may be provided.

  In the above embodiment, the line image is detected by determining whether each pixel is a white pixel or a black pixel based on the input image data subjected to the binarization processing by the binarization processing unit 44. Although the case where the process is performed has been described, the present invention is not limited thereto, and the line image detection process may be performed by determining whether or not the pixel value of each pixel in the multi-valued image data is equal to or greater than a predetermined value.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, the technical aspects of the present invention also apply to embodiments obtained by appropriately combining technical means disclosed in different embodiments and embodiments obtained by combining technical means appropriately changed within the scope of the claims. Included in the range.

  The present invention can be applied to an image processing apparatus that extracts feature points for specifying input image data from the input image data.

It is a block diagram which shows schematic structure of the document collation process part with which the image processing apparatus concerning one Embodiment of this invention is equipped. 1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to an embodiment of the present invention. It is a block diagram which shows schematic structure of the feature point calculation part with which the document collation process part shown in FIG. 1 is equipped. It is explanatory drawing which shows an example of the filter coefficient of the mixing filter with which the MTF process part of the feature point calculation part shown in FIG. 3 is equipped. It is explanatory drawing which shows an example of the attention feature point and periphery feature point which are extracted when the feature-value calculation part with which the document collation process part shown in FIG. 1 is provided calculates a feature-value. (A)-(c) is explanatory drawing which shows an example of the combination of the attention feature point and peripheral feature point which are extracted when a feature-value calculation part calculates a feature-value. (A)-(c) is explanatory drawing which shows an example of the combination of the attention feature point and peripheral feature point which are extracted when a feature-value calculation part calculates a feature-value. (A) And (b) is explanatory drawing which shows an example of the index showing the hash value registered into a hash table, and input image data in the document collation process part shown in FIG. 3 is a graph showing an example of the number of votes for each registered image in a voting processing unit provided in the document matching processing unit shown in FIG. 1. It is a flowchart which shows the flow of a process in the document collation process part shown in FIG. It is a flowchart which shows the flow of a process in the intersection calculation part with which the document collation process part shown in FIG. 1 is equipped. It is a block diagram which shows schematic structure of the intersection calculation part with which the document collation process part shown in FIG. 1 is equipped. It is explanatory drawing which shows an example of input image data. It is explanatory drawing for demonstrating the process in the case of detecting the line image of a horizontal direction and a vertical direction with respect to the input image data shown in FIG. It is explanatory drawing which shows the detection result of the horizontal and vertical line image with respect to the input image data shown in FIG. 13, and the calculation result of an intersection. (A)-(c) is explanatory drawing which shows the allocation method of the label to each pixel in the case of detecting the line image extended in the diagonal direction from upper left to lower right in input image data. (A)-(c) is explanatory drawing which shows the allocation method of the label to each pixel in the case of detecting the line image extended in the diagonal direction from the upper right to the lower left in input image data. It is explanatory drawing which shows an example of input image data. It is explanatory drawing which shows the procedure of labeling in the case of performing the detection process of the line image extended from upper left to lower right with respect to the input image data shown in FIG. It is explanatory drawing which shows the procedure of the labeling in the case of performing the detection process of the line image extended from upper right to lower left with respect to the input image data shown in FIG. It is explanatory drawing which shows the detection result of the diagonal line image with respect to the input image data shown in FIG. 18, and the calculation result of an intersection. It is a block diagram which shows the modification of the intersection calculation part with which the document collation process part shown in FIG. 1 is equipped. It is explanatory drawing which shows an example of input image data. FIG. 24 is an explanatory diagram illustrating a labeling procedure in the case of performing detection processing of line images extending from upper left to lower right (using line start / end coordinates) for the input image data illustrated in FIG. 23. FIG. 24 is an explanatory diagram illustrating a labeling procedure in a case where detection processing of line images extending from upper right to lower left (using line start / end coordinates) is performed on the input image data illustrated in FIG. 23. It is a block diagram which shows the modification of the image processing apparatus concerning one Embodiment of this invention. It is explanatory drawing which shows the structure of the image processing system concerning one Embodiment of this invention. 1 is a block diagram illustrating a configuration example of an image processing system according to an embodiment of the present invention. It is a block diagram which shows the other structural example of the image processing system concerning one Embodiment of this invention. (A)-(d) is description which shows an example of the combination of the attention feature point and peripheral feature point which are extracted when the feature-value calculation part of the image processing apparatus concerning one Embodiment of this invention calculates a feature-value. FIG. (A)-(d) is description which shows an example of the combination of the attention feature point and peripheral feature point which are extracted when the feature-value calculation part of the image processing apparatus concerning one Embodiment of this invention calculates a feature-value. FIG. It is a block diagram which shows the modification of the feature point calculation part with which the document collation process part shown in FIG. 1 is equipped. FIG. 33 is an explanatory diagram for describing an example of edge detection processing performed by an inclination detection unit in the feature point calculation unit illustrated in FIG. 32. It is explanatory drawing which shows an example of the tangent-angle table used by the inclination detection part in the feature point calculation part shown in FIG. FIG. 33 is an explanatory diagram for describing an example of edge detection processing performed by an inclination detection unit in the feature point calculation unit illustrated in FIG. 32. FIG. 33 is an explanatory diagram illustrating an example of an edge detection filter used in edge detection processing performed by an inclination detection unit in the feature point calculation unit illustrated in FIG. 32. FIG. 33 is an explanatory diagram for describing an example of edge detection processing performed by an inclination detection unit in the feature point calculation unit illustrated in FIG. 32. FIG. 33 is an explanatory diagram for describing an example of an inclination correction process performed by an inclination correction processing unit in the feature point calculation unit illustrated in FIG. 32. It is a flowchart which shows the flow of a process in the feature point calculation part shown in FIG. It is a block diagram which shows the modification of the feature point calculation part with which the document collation process part shown in FIG. 1 is equipped. (A) And (b) is explanatory drawing for demonstrating the expansion process performed in the expansion process part in the feature point calculation part shown in FIG. (A) And (b) is explanatory drawing for demonstrating the reduction process performed in the reduction process part in the feature point calculation part shown in FIG. It is a flowchart which shows the flow of a process in the feature point calculation part shown in FIG. It is a block diagram which shows the modification of the feature point calculation part with which the document collation process part shown in FIG. 1 is equipped. It is a flowchart which shows the flow of a process in the feature point calculation part shown in FIG. It is explanatory drawing which shows the detection result of the diagonal line image with respect to the input image data shown in FIG. 23, and the calculation result of an intersection. Explanatory drawing which shows the difference in the result by each method (use of label count, distance use between the start and end coordinates of a line) regarding the detection of the line image in the diagonal direction, taking the detection process of the line image extending from the upper right to the lower left as an example It is. It is explanatory drawing which shows an example of the allocation method of the label to each pixel in the case of detecting the line image extended in the diagonal direction in the intersection calculation part with which the document collation process part shown in FIG. 1 is equipped. It is explanatory drawing which shows an example of the allocation method of the label to each pixel in the case of detecting the line image extended in the diagonal direction in the intersection calculation part with which the document collation process part shown in FIG. 1 is equipped. (A) is explanatory drawing which shows the result of having labeled this image with the labeling method shown in Embodiment 1 when the image shown in FIG. 18 is read appropriately. FIG. 19B is an explanatory diagram illustrating a result of labeling the image in which the reading error occurs when the image illustrated in FIG. 18 is read by the labeling method illustrated in the first embodiment. FIG. 19C is an explanatory diagram illustrating a result of labeling an image in which the reading error has occurred by the labeling method illustrated in the fourth embodiment when a reading error occurs when the image illustrated in FIG. 18 is read. . FIG. 47 is an explanatory diagram when a straight line (line segment) formula is used when calculating the intersection of lines in FIG. 46;

Explanation of symbols

1 Digital color MFP (image processing device, image forming device, image reading device, image processing device)
1 'Flatbed scanner (image processing device, image reading device)
2 Color image input device (input data acquisition unit, scanner device)
3,3 'color image processing device (image processing device)
4 color image output device (image output unit)
5, 51 Communication device (input data acquisition unit, transmission device)
6 Operation panel (Processing input section)
7, 7a, 7b Control unit 8, 8a, 8b Memory (storage unit)
13, 13a, 13b Document collation processing unit (similarity calculation unit)
31, 31b to 31f Feature point calculation unit 32 Feature amount calculation unit 33 Voting processing unit (similarity calculation unit)
34 similarity determination processing unit 37 registration processing unit 41 signal conversion processing unit (achromatic processing unit)
42 Resolution Conversion Unit 43 MTF Processing Unit 44 Binarization Processing Unit 45 Intersection Calculation Unit 46 Center of Gravity Calculation Unit 47 Inclination Detection Unit 48 Inclination Correction Processing Unit 50 Server Device 52 Expansion Processing Unit 53 Degeneration Processing Unit 54 Missing Correction Unit 100 Image Processing System 103 Hash table

Claims (14)

A feature point calculation unit that calculates a feature point of the input image data; and a feature amount calculation unit that calculates a feature amount of the input image data based on a relative position between the feature points calculated by the feature point calculation unit. An image processing apparatus,
The feature point calculation unit
A line image extending in a first direction inclined downward to the right with respect to the horizontal direction in the input image data from the input image data, and extending in a second direction inclined downward to the left with respect to the horizontal direction in the input image data. A line detection unit that detects a line image that extends, a line image that extends in a horizontal direction, and a line image that extends in a vertical direction ;
The intersection point between the line image extending in the first direction and the line image extending in the second direction, and the intersection point between the line image extending in the horizontal direction and the line image extending in the vertical direction are calculated as the feature points. A calculation unit ,
The line detector is
The detection process of the line image extending in the horizontal direction and the line image extending in the vertical direction is performed prior to the detection process of the line image extending in the first direction and the line image extending in the second direction.
The intersection calculation unit
Detection of the line image extending in the first direction and the line image extending in the second direction when the number of intersections between the line image extending in the horizontal direction and the line image extending in the vertical direction is equal to or greater than a predetermined value. An image processing apparatus characterized by not performing processing. A feature point calculation unit that calculates a feature point of the input image data; and a feature amount calculation unit that calculates a feature amount of the input image data based on a relative position between the feature points calculated by the feature point calculation unit. An image processing apparatus,
The feature point calculation unit
A line image extending in a first direction inclined downward to the right with respect to the horizontal direction in the input image data from the input image data, and extending in a second direction inclined downward to the left with respect to the horizontal direction in the input image data. A line detection unit that detects a line image that extends, a line image that extends in a horizontal direction, and a line image that extends in a vertical direction;
An intersection point between the line image extending in the first direction and the line image extending in the second direction, and the intersection point between the line image extending in the horizontal direction and the line image extending in the vertical direction are calculated as the feature points. A calculation unit;
An instruction input unit that receives an instruction input from the user,
The intersection calculation unit
An image processing apparatus that determines whether or not to perform detection processing of the line image extending in the first direction and the line image extending in the second direction in response to a user instruction input to the instruction input unit. . When the intersection of the line image extending in the first direction and the line image extending in the second direction is composed of a plurality of adjacent pixels, the intersection calculation unit, the center coordinates of the plurality of pixels, the barycentric coordinates, or the image processing apparatus according to the coordinates of the predetermined position in the region consisting of the plurality of pixels to claim 1 or 2, characterized in that calculated as a point above features. The intersection calculation unit
A straight line expression corresponding to the line image extending in the first direction and a straight line expression corresponding to the line image extending in the second direction are calculated, and an intersection of both straight lines is calculated based on the two straight line expressions. calculating the image processing apparatus according to coordinates corresponding to the calculated intersection point to claim 1 or 2, characterized in that calculated as a point above features. A storage unit that stores identification information for identifying the registered image in association with the feature amount of the registered image;
A similarity determination unit that determines the similarity between the input image and the registered image based on the feature amount calculated from the input image data by the feature amount calculation unit and the feature amount of the registered image stored in the storage unit; the image processing apparatus according to claim 1, any one of 4, characterized in that is. An inclination detection unit for detecting an inclination angle of the document image included in the input image data;
An inclination correction unit that corrects an inclination of the document image based on an inclination angle detected by the inclination detection unit, and the input image data after the inclination correction is input to the intersection calculation unit. The image processing apparatus according to any one of 1 to 5 . The number of black pixels included in the pixel of interest and the peripheral pixels of the pixel of interest in the input image data is counted, and a pixel of interest having a count value equal to or greater than the first threshold is set as a black pixel, and the count value is less than the first threshold. An expansion processing unit that performs expansion processing with the target pixel as a white pixel;
The number of white pixels included in the target pixel and the peripheral pixels of the target pixel in the input image data after the dilation processing is counted, and the target pixel whose count value is equal to or larger than the second threshold is set as the white pixel, and the count value is the second threshold. A reduction processing unit that performs a reduction process with the target pixel being less than
The image processing apparatus according to any one of the input image data after the reduction process of claims 1, wherein the input to the intersection calculation unit 6. An image processing apparatus according to any one of claims 1 to 7, the image forming apparatus characterized by comprising an image output section for forming on a recording material an image corresponding to input image data. An image processing method for calculating feature points of input image data and calculating feature amounts of the input image data based on relative positions of the calculated feature points,
A first line detecting step for detecting a line image extending in the horizontal direction and a line image extending in the vertical direction from the input image data;
A first intersection calculation step of calculating an intersection of the line image extending in the horizontal direction and the line image extending in the vertical direction as the feature point;
A line image extending in a first direction inclined downward to the right with respect to the horizontal direction in the input image data from the input image data , and the first direction inclined downward to the left with respect to the horizontal direction in the input image data. A second line detection step of detecting a line image extending in a second direction different from
Look including a second intersection point calculation step of calculating the intersection of the line image extending in the line image and the second direction extending in the first direction as a point above features,
The first line detection step and the first intersection calculation step are performed before the second line detection step, and the number of intersections between the line image extending in the horizontal direction and the line image extending in the vertical direction is equal to or greater than a predetermined value. In the case of the image processing method, the second line detection step and the second intersection calculation step are not performed . An image processing method for calculating feature points of input image data and calculating feature amounts of the input image data based on relative positions of the calculated feature points,
A first line detecting step for detecting a line image extending in the horizontal direction and a line image extending in the vertical direction from the input image data;
A first intersection calculation step of calculating an intersection of the line image extending in the horizontal direction and the line image extending in the vertical direction as the feature point;
A line image extending in a first direction inclined downward to the right with respect to the horizontal direction in the input image data from the input image data, and the first direction inclined downward to the left with respect to the horizontal direction in the input image data. A second line detection step of detecting a line image extending in a second direction different from
A second intersection calculation step of calculating an intersection of the line image extending in the first direction and the line image extending in the second direction as the feature point;
Including an instruction input process for receiving an instruction input from a user,
  An image processing method comprising: determining whether or not to perform the second line detection step and the second intersection calculation step in accordance with a user instruction input received in the instruction input step. A feature point calculation unit that includes an image processing device and a server device that is communicably connected to the image processing device, calculates feature points of input image data, and feature points calculated by the feature point calculation unit A feature amount calculation unit for calculating a feature amount of the input image data based on the relative position of the image processing device or the server device, or distributed in the image processing device and the server device. An image processing system,
The feature point calculation unit
A line image extending in a first direction inclined downward to the right with respect to the horizontal direction in the input image data from the input image data, and extending in a second direction inclined downward to the left with respect to the horizontal direction in the input image data. A line detection unit that detects a line image that extends, a line image that extends in a horizontal direction, and a line image that extends in a vertical direction ;
The intersection point between the line image extending in the first direction and the line image extending in the second direction, and the intersection point between the line image extending in the horizontal direction and the line image extending in the vertical direction are calculated as the feature points. A calculation unit ,
The line detector is
The detection process of the line image extending in the horizontal direction and the line image extending in the vertical direction is performed prior to the detection process of the line image extending in the first direction and the line image extending in the second direction.
The intersection calculation unit
Detection of the line image extending in the first direction and the line image extending in the second direction when the number of intersections between the line image extending in the horizontal direction and the line image extending in the vertical direction is equal to or greater than a predetermined value. An image processing system characterized by not performing processing. A feature point calculation unit that includes an image processing device and a server device that is communicably connected to the image processing device, calculates feature points of input image data, and feature points calculated by the feature point calculation unit A feature amount calculation unit for calculating a feature amount of the input image data based on the relative position of the image processing device or the server device, or distributed in the image processing device and the server device. An image processing system,
The feature point calculation unit
A line image extending in a first direction inclined downward to the right with respect to the horizontal direction in the input image data from the input image data, and extending in a second direction inclined downward to the left with respect to the horizontal direction in the input image data. A line detection unit that detects a line image that extends, a line image that extends in a horizontal direction, and a line image that extends in a vertical direction ;
The intersection point between the line image extending in the first direction and the line image extending in the second direction, and the intersection point between the line image extending in the horizontal direction and the line image extending in the vertical direction are calculated as the feature points. Calculation part ,
An instruction input unit that receives an instruction input from the user,
The intersection calculation unit
An image processing system for determining whether or not to perform detection processing of the line image extending in the first direction and the line image extending in the second direction in response to a user instruction input to the instruction input unit. . A program for operating an image processing apparatus according to any one of claims 1 to 7, a program for causing a computer to function as the feature point calculating unit. A computer-readable recording medium on which the program according to claim 13 is recorded.

Images