JP7310151B2 - Mark selection device and image processing device - Google Patents

Description

The present invention relates to a mark selection device and an image processing device.

There are scenes that require image registration. Here, as an example of a scene that requires alignment of images, a scene of automatically grading answers on an image obtained by reading an answer sheet with a scanner will be taken up. An image obtained by reading with a scanner has a positional deviation, and in some cases, the image is oblique. For this reason, if this is left as it is, there is a risk that the answer entry column will not be in the correct position and scoring will not be possible or that scoring will be erroneous. Therefore, in such a situation, it is necessary to align the images obtained by reading so that the answer entry columns are positioned correctly. In order to perform this alignment, it is conceivable to print a mark for alignment on an answer sheet in advance and use the mark as a clue to perform alignment.

For example, Patent Document 1 discloses recording marks at four corners of an image for barcode recognition.

Further, Japanese Patent Application Laid-Open No. 2002-200001 discloses printing out an image in which a reference mark is recorded at a position designated by a user, and reading the printed out image.

Further, Japanese Patent Application Laid-Open No. 2002-200002 discloses recording a distortion correction mark on an image and correcting the distortion of the image based on the mark.

JP-A-2003-271942 JP-A-10-091752 JP-A-6-311333

Pre-recording a mark on an image for image alignment means that something unrelated to the original content of the image is recorded, and the design of the image is affected. It can cause problems. Also, the mark may impose restrictions on the design of the image itself.

An object of the present invention is to provide a mark selection device that selects marks useful for alignment from an image without additional marks, and an image processing device that performs image alignment using the selected marks. .

Claim 1 is
Candidate extracting means for extracting a mark candidate consisting of a pattern existing in a candidate area of the patterns appearing on the first image or a set of such patterns without having pattern information as a mark;
Mark selection means for selecting a mark from among the mark candidates based on the distance from a predetermined base point on the first image to the mark candidate and the mark suitability of the mark candidate including at least the size of the mark. and
The candidate extraction means is
A cluster that extracts two patterns that are not spatially connected to one on the first image as a cluster consisting of the two patterns based on the spatial distance between the two patterns. extraction means;
cluster merging means for merging a plurality of adjacent clusters within a range not exceeding a predetermined dimension;
The mark selection apparatus is characterized in that the clusters merged by the cluster merging means are used as the mark candidates .

Claim 2 is
The cluster extraction means is
contour extraction means for performing contour extraction processing on the pattern on the first image;
2. The mark selecting apparatus according to claim 1 , further comprising noise removing means for removing contours as noise components less than a predetermined size from the contours extracted by said contour extracting means. .

Claim 3 is
When the cluster merging means searches a set of two clusters in order of shortest distance between them and merges the two clusters that make up one set into one cluster, one cluster after merging A process of merging two clusters constituting one set whose dimensions do not exceed a predetermined dimension as one cluster is repeated while changing the order of distances that change due to merging to shortest order. The mark selection device according to claim 1 .

Claim 4 is
The candidate extracting means extracts the mark candidates as a set of polygonal lines,
2. The mark selecting apparatus according to claim 1, wherein said mark selecting means selects said marks by considering the number of vertexes of polygonal lines forming said mark candidates as one of said mark aptitudes.

Claim 5 is
The candidate extracting means extracts a mark candidate consisting of a pattern or a set of patterns present in each of at least three candidate regions on the first image;
The mark selecting means selects at least three points on the first image based on the distance from the base point corresponding to the candidate area to the mark candidate in the candidate area corresponding to the base point and predetermined mark aptitude. 2. The mark selection device according to claim 1, wherein each mark in is selected.

Claim 6 is
When the candidate extracting means uses each of the vertices of the four corners of the first rectangular image as a base point, the pattern exists in each of four candidate areas predetermined corresponding to each of the base points, or the pattern exists in each of the four candidate areas. Extract mark candidates consisting of a set of
The mark selecting means selects marks at four locations on the first image based on the distance from the base point corresponding to the candidate area to the mark candidate in the candidate area corresponding to the base point and predetermined mark aptitude. 2. The mark selection device according to claim 1, wherein each mark is selected.

Claim 7 is the mark selection apparatus according to any one of Claims 1 to 6 , wherein the first image is an image read by a reading means for reading an image.

Claim 8 is
mark storage means for storing shape data of the mark selected by the mark selecting apparatus according to any one of claims 1 to 7 and position data of the mark on the first image;
shape detection means for searching a predetermined search area around the position represented by the position data on the second image to detect a shape corresponding to the shape of the mark represented by the shape data; ,
The first image and the first image are arranged such that the shape represented by the shape data stored in the mark storage means and the shape corresponding to the shape of the mark detected by the shape detection means overlap each other. positioning means for positioning two images.

(9 ) The image processing apparatus according to ( 8) , wherein the second image is an image read by reading means for reading an image.

Claim 10 is the image processing apparatus according to claim 8 or 9 , comprising the mark selection apparatus according to any one of claims 1 to 7 .

Claim 11 is executed in an information processing device that executes a program, and causes the information processing device to operate as the mark selection device according to any one of claims 1 to 7 . It's a program.

According to claim 12 , image processing is executed in an information processing apparatus that executes a program, and causes the information processing apparatus to operate as the image processing apparatus according to any one of claims 8 to 10 . It's a program.

According to the mark selection apparatus of claim 1 and the mark selection program of claim 11 , marks useful for alignment, for example, are selected from an image in which there are no marks having a predetermined pattern.

Further, according to the mark selection apparatus of claim 1 and the mark selection program of claim 11 , mark candidates whose dimensions are adjusted can be obtained as compared with the case where cluster merging means is not provided.

According to the mark selection apparatus of claim 2 , clusters of good quality that are the basis of mark candidates can be extracted as compared with the case where the noise removal means is not provided.

According to the mark selection apparatus of claim 3 , clusters suitable as mark candidates can be obtained as compared with the case where the search is not performed in order of shortest distance between each other.

According to the mark selection device of claim 4 , a proper mark is selected as compared with the case where the number of vertices of a polygonal line is not used as mark aptitude.

According to the mark selection apparatus of claim 5 , a mark that is more useful for alignment is selected than when two or less marks are selected.

According to the mark selection apparatus of claim 6 , marks that are more useful for alignment are selected than when three or less marks are selected in the case of a rectangular image.

As in claim 7 , the mark selection device of the present invention is also effective in selecting marks from images read by the reading means.

According to the image processing apparatus of claim 8 and the image processing program of claim 12 , image alignment is performed based on the mark selected by the mark selecting apparatus of the present invention.

As in claim 9, the image processing apparatus of the present invention is also effective in aligning images read by the reading means.

According to the image processing apparatus of claim 10 , consistent processing can be executed from mark selection to alignment using the selected mark.

1 is a schematic diagram showing an image processing system in which a personal computer (hereinafter abbreviated as “PC”) and a multi-function peripheral are connected; FIG. 1 is a block diagram showing an overview of a mark selection device and an image processing device as one embodiment of the present invention; FIG. FIG. 3 is a diagram showing an example of a blank answer sheet, which is an answer sheet with no answers. Figure 4 shows selected marks on the blank answer sheet shown in Figure 3; It is the figure which showed the completed answer sheet. FIG. 4 is a diagram showing a flowchart of a main program; 7 is a diagram showing a flowchart of mark candidate extraction processing in step S03 of FIG. 6. FIG. FIG. 8 is a diagram showing a flowchart of cluster merging processing in step S13 of FIG. 7; FIG. 10 is a diagram showing a flowchart of a process of merging a set of clusters in step S22 of FIG. 8;

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below.

FIG. 1 is a schematic diagram showing an image processing system connecting a multifunction machine and a personal computer (hereinafter abbreviated as "PC").

The MFP 1 functions as a printer that receives an image signal from the PC 2 and prints out an image based on the image signal on paper, reads an image on the paper to generate an image signal, and sends the image signal to the PC 2 . It has a function as a scanner for transmission and a function as a copier that reads an image on paper to generate an image signal and prints out an image based on the image signal on paper. The MFP 1 may include other functions such as a facsimile transmission/reception function. Here, among those functions of the multifunction machine 1, attention is paid to the function as a scanner.

The PC 2 implements various functions according to programs executed therein. Here, attention is focused on the function of processing an image signal transmitted from the multifunction device, which will be described below. In the following, when there is no particular need to distinguish between an image on paper and an image on a signal, and when it is clear from the context etc. whether it is an image on paper or an image on a signal In some cases, they may be expressed as "images" without distinguishing between them. The same applies to the "answer sheet" that will appear later. Here, the physical paper, the image expressed on the paper, and the image on the signal obtained by reading the image on the paper can be referred to as "answer sheet" without distinguishing between them. be.

FIG. 2 is a block diagram showing an outline of a mark selection device and an image processing device as one embodiment of the present invention. The mark selection device 10 and the image processing device 20 are devices as functions built inside the PC 2 , which are realized by combining the hardware and software of the PC 2 .

The mark selection device 10 is a device that selects marks near the four corners of the image read by the multifunction machine 1 and transmitted to be used for aligning the images. Also, the image processing device 20 is a device that aligns images using the marks selected by the mark selection device 10 .

Here, an example of alignment between a blank answer sheet, which is an answer sheet with no answers written, and a completed answer sheet, which is an answer sheet after answers have been written, will be described.

FIG. 3 is a diagram showing an example of a blank answer sheet, which is an answer sheet with no answers.

Hereinafter, the blank answer sheet will be referred to as blank answer sheet P1 or answer sheet P1, and the completed answer sheet to be described later will be referred to as completed answer sheet P2 or answer sheet P2. Also, when there is no need to distinguish between the blank answer sheet P1 and the completed answer sheet P2, the answer sheet P is referred to.

In the multi-function device 1, the blank answer sheet P1 shown in FIG. The mark selection device 10 selects marks to be used for alignment near the four corners of the blank answer sheet P1. However, the pattern intended to be used as a mark for alignment is not printed on this answer sheet P1. Marks are selected from the original print contents of the answer sheet P1.

Here, in the image of the answer sheet P read by the multifunction machine 1, the rectangular edge of the sheet is often reflected. However, the edges of the rectangles on the paper are not always reflected clearly. Attempting to use edges or corners as alignment marks can lead to large errors, and here we avoid using rectangle edges or corners as marks.

FIG. 4 is a diagram showing selected marks on the blank answer sheet shown in FIG.

In FIG. 4, selected marks 30 are indicated by enclosing them in hatched rectangles.

As described above, the answer sheet P1 is not printed with a pattern intended to be used as a mark for alignment. Corresponding to this, the mark selection device 10 does not have information of a predetermined pattern as a mark. The mark selection device 10 selects a mark for alignment from the original printed contents of the answer sheet P1.

Here, in the case of this answer sheet P1, a mark 31 consisting of a combination of numbers "1" and "0" is selected as the upper left mark 31 of the marks 30 near the four corners. As the upper right mark 32, a mark 32 consisting of a combination of the numbers "1" and "8" and the number "4" arranged below them is selected. As the lower left mark 33, a mark 33 consisting of a combination of the symbol "(", the number "5", and the symbol ")" is selected. Furthermore, as the lower right mark 34, a mark 34 consisting of a combination of the character "advance" and the character "me" is selected.

FIG. 5 shows a completed answer sheet. However, the dashed rectangular areas D1 to D4 shown on the completed answer sheet P2 indicate the search range of marks for alignment, and the printed content on the completed answer sheet P2 is do not have.

This completed answer sheet P2 is also an answer sheet that has been read and sent by the multifunction device 1, like the blank answer sheet P1. The filled-in answer sheet P2 after being read may be misaligned or tilted with respect to the blank answer sheet P1.

This completed answer sheet P2 is input to the image processing apparatus 20 shown in FIG. Then, the image processing apparatus 20 searches the rectangular regions D1 to D4 and searches for patterns having the same shape as the marks 31 to 34 shown in FIG. 4 from the rectangular regions D1 to D4 by pattern matching. Then, the completed answer sheet P2 is aligned with the blank answer sheet R1 so that each of the found patterns is at the same position as each of the marks 31 to 34 shown in FIG. There are a large number of completed answer sheets P2, and the multifunction device 1 sequentially reads the multiple completed answer sheets P2, and the image processing apparatus 20 sequentially aligns the multiple completed answer sheets P2. be done.

Returning to FIG. 2, the configuration of the mark selection device 10 that selects the mark 30 as shown in FIG. 4 and the image processing device 20 that positions the answer sheet using the selected mark 30 will be described. Here, with reference to FIG. 2, first, an overview of the configurations of the mark selection device 10 and the image processing device 20 will be described.

The mark selection device 10 comprises candidate extraction means 11 and mark selection means 12 .

The candidate extracting means 11 extracts a pattern or a set of patterns existing in a predetermined candidate area among the patterns appearing on the blank answer sheet P1 without having predetermined pattern information as a mark. to extract mark candidates. This blank answer sheet P1 corresponds to an example of the first image referred to in the present invention.

Here, as an example, the blank answer sheet P1 is divided vertically and horizontally into two equal parts to form a total of four candidate areas. to extract Although the edge of the answer sheet P is not clearly reflected in many cases, the outline of the edge can be estimated.

Here, the candidate extracting means 11 is composed of cluster extracting means 111 and cluster merging means 112 .

The cluster extracting means 111 extracts clusters present as clusters on the blank answer sheet P1. Existing as a mass initially means that, for example, characters, symbols, or parts thereof are connected in a single stroke. However, it is not necessary to be able to write with a single stroke as long as they are connected to one. Adjacent clusters may be merged in the next cluster merging means 112, and in the case of one merged cluster, it is not connected to one.

Cluster merging means 112 merges a plurality of adjacent clusters within a range not exceeding a predetermined size. In the processing by this cluster merging means 112, in this mark selection apparatus 10, the size suitable for a mark is predetermined, and the processing in this cluster merging means 112 is performed by merging adjacent clusters to obtain clusters of sizes suitable for marks. It is a process of

The clusters merged by the cluster merging means 112 are sent from the cluster merging means 112 to the mark selecting means 12 as mark candidates.

Furthermore, the cluster extraction means 111 is composed of a contour extraction means 1111 and a noise elimination means 1112 .

The contour extraction means 1111 performs contour extraction processing on the patterns of characters, symbols, etc. on the blank answer sheet P1. Further, the noise removing means 1112 removes contours as noise components less than a predetermined dimension from the contours extracted by the contour extracting means 1111 . In addition, the noise removing means 1112 in this embodiment also removes contours that are too large as unnecessary noise.

Further, the mark selection means 12 is based on the distance from a predetermined reference point on the blank answer sheet P1 to the mark candidate and the predetermined mark aptitude of the mark candidate, which includes at least the size of the mark, A mark is selected from the mark candidates.

Here, as an example, using each of the four corners of the answer sheet P as a base point, a mark is selected from each of four candidate regions obtained by dividing the answer sheet P1 into four.

As described above, although the edge of the answer sheet P is not clearly reflected in many cases, the outline of the edge and corner can be estimated. The four corners that are assumed to be base points for mark selection function well as base points even if they have positional errors.

The image processing device 20 has mark storage means 21 , shape detection means 22 and alignment means 23 . A large number of completed answer sheets P2 read by the multifunction machine 1 are sequentially input to the image processing apparatus 20 . Here, one of them, the completed answer sheet P2, will be described.

The shape data of the mark 30 (see FIG. 4) selected by the mark selection device 10 and the position data of the mark 30 on the answer sheet P1 are stored in the mark storage means 21. FIG. The shape data of the mark is, for example, "(5)" in the case of the lower left mark 33 in FIG. The position data on the answer sheet P1 is data representing the distance and direction from the base point (for example, the lower left corner of the answer sheet P in the case of the lower left mark 33 in FIG. 4).

The completed answer sheet P2 input to the image processing device 20 is referred to by the shape detection means 22 . The shape detection means 22 detects predetermined search areas (search areas D1 to D1 shown in FIG. D4) is searched, and a pattern of a shape corresponding to the shape of the mark represented by the shape data stored in the mark storage means 21 is detected by pattern matching processing. These search areas D1 to D4 are determined in consideration of positional deviations when the answer sheet P is read by the multifunction device 1, corner position errors associated with estimation of the positions of the corners of the sheet, and the like. This area is predetermined so that a pattern of a shape corresponding to the shape of the mark can be detected by searching. The pattern matching process is a widely known process and will not be described here.

Further, the alignment means 23 is arranged so that the mark having the shape represented by the shape data stored in the mark storage means 21 and the pattern of the shape corresponding to the shape of the mark detected by the shape detecting means 22 overlap each other. Then, the completed answer sheet P2 is aligned with the blank answer sheet P1.

The completed answer sheet P2 after alignment is input to an automatic scoring device (not shown) placed behind the image processing device 20 shown in FIG. 2, and filled in the entry frame A shown in FIG. The correct answer will be graded. This automatic scoring device is not the subject of this embodiment, and further explanation is omitted here.

Here, the mark selection device 10 for selecting marks and the image processing device 20 for alignment have been described separately, but the overall function of selecting marks and alignment is regarded as one image processing device. good too.

The mark selection device 10 shown in FIG. 2 will be further described below with reference to the flow chart. The flowchart described below represents processing by a program executed within the PC 2 shown in FIG. .

FIG. 6 is a diagram showing a flow chart of the main program.

Here, an image, that is, the blank answer sheet P1 read by the multifunction device 1 is loaded so that it can be referenced from this program (step S01). Then, the captured image (blank answer sheet P1) is divided into four (step S02). Here, as described above, the image is divided into four by dividing it into two at the vertical center and dividing it into two at the horizontal center. Next, mark candidates are extracted for each divided image (step S03). The details of the mark candidate extraction processing will be described later. Then, when the mark candidates are extracted, a mark for each divided image is selected from the mark candidates in each of the four divided images (step S04). The mark selection process in step S04 will be mentioned again later.

FIG. 7 is a diagram showing a flowchart of the mark candidate extraction process in step S03 of FIG.

Here, first, the outline of the pattern on the image including characters, symbols, etc. in the image is extracted (step S11). The contour extraction is a known process performed by, for example, a filtering process for contour extraction, and detailed description thereof will be omitted here. Here, the contours connected to one are regarded as one contour.

Next, unnecessary contours are removed from the extracted contours (step S12). The contour extraction process is a differentiation process itself or a process similar to the differentiation process, and often emphasizes fine noise and extracts the noise as a contour. Therefore, fine noise is first removed here. Also, contours that are too large are unsuitable as marks. So contours that are too large are also removed here. As a scale for measuring the size of the contour, for example, one or a combination of the perimeter of the contour, the area of the contour, the area of the circumscribing rectangle that circumscribes the contour, the length of the long side/short side of the circumscribing rectangle, etc. is used. Here, minimum/maximum thresholds are set for the scale used, and contours outside these thresholds are removed. Here, each of the remaining contours after removing unnecessary contours, that is, each of the connected contours is called a "cluster".

Next, cluster merging processing is performed (step S13). This is a process of merging a plurality of clusters and regarding the merged plurality of clusters as one new cluster. Mark candidates are extracted by this cluster merging process. Details of the cluster merging process will be described later, but clusters extracted as mark candidates are clusters that are not connected to one after the cluster merging process. However, depending on the size of the cluster, there may be clusters that remain connected as one without being merged.

When the cluster merging process ends, a mark is selected from among a plurality of clusters (step S14). In this mark selection process, if there is a mark candidate with a markedly high evaluation as a mark for one divided area, it may be narrowed down to only one mark. A plurality of marks may be selected from the candidates. When a plurality of marks are selected, at this stage, the selected marks remain in the mark candidate stage.

In this step S14, it is marked that the distance to the mark candidate from the base point of each divided area, that is, the corner of the sheet included in the divided area, is within the range of the maximum threshold distance and the minimum threshold distance. Marks are selected as one criterion for selection. The reason for setting the minimum threshold distance is that the position of the base point, i.e. the corner of the paper, may be a guess, and mark candidates too close to the base point may be noise, etc. Because there is also Also, the reason why the maximum threshold distance is provided is that the marks existing at the corners of the paper as much as possible can be expected to be more accurate in aligning the images. Also, when the aspect ratio of the paper is not 1:1, the vertical and horizontal distances are measured separately, and depending on the aspect ratio, different maximum threshold distances are measured for the vertical distance and the horizontal distance. and different minimum threshold distances.

The position of the mark candidate when measuring the distance from the base point to the mark candidate is determined arbitrarily. It may be the position of the center point of the rectangle that Here, the mark candidate is extracted through the contour extraction process (step S11), and in the contour extraction process of this embodiment, the contour is extracted as a set of multiple polygonal lines. Therefore, mark candidates are also represented by a set of polygonal lines. Therefore, using this, the average value of a plurality of distances between each of the vertexes of the polygonal line as a mark candidate and the base point may be used as the distance to the mark candidate.

Alternatively, instead of setting a threshold for the distance, a distance reference value may be set, and several mark candidates may be selected in order of proximity to the reference value.

In addition to the distance, there are other criteria for selecting mark candidates. In this embodiment, dimensions and the number of vertices are adopted as criteria in addition to the distance. If the size is too small or too large, it is not suitable as a mark. Therefore, here, mark candidates are selected using dimensions as one of the criteria. As the size of the mark, for example, the perimeter or area of the circumscribed rectangle can be used. Alternatively, the size of the mark may be the length of the long side of the circumscribing rectangle or the distance between the two most distant points in the mark. When selecting mark candidates based on dimensions, an upper threshold value and a lower threshold value for dimensions may be set, and mark candidates between these thresholds may be selected, or a reference value for dimensions may be set. , a predetermined number of mark candidates may be selected in order of size close to the reference value.

In addition, the reason why the number of vertices is used as a criterion is that a small number of vertices means that the shape is often simple. This is because there is a risk of erroneous recognition when On the other hand, if the number of vertices is too large, the shape of the mark is often too complicated, which may lead to erroneous recognition. As for the number of vertices, a threshold value may be set to select mark candidates, or a reference value may be set, and mark candidates may be selected in order of closeness to the reference value.

Although the number of vertices is used here, other than the number of vertices, for example, the circumference length of the mark candidate may be used. When the dimensions are close to the reference value, a short perimeter means a simple shape, and a long perimeter means a complex shape, similar to the number of vertices. , has the meaning of an index for the complexity of the mark candidate. As an index of complexity, (perimeter/dimension) may be used instead of perimeter. Alternatively, other parameters representing complexity may be adopted as criteria. However, it is also possible to use only the size of the mark candidate as a criterion for selection in addition to the distance, without adopting the index of complexity such as the number of vertices or the perimeter.

In step S14 in this embodiment, first, the mark candidates are selected by distance, the size is selected from among the selected mark candidates, and the remaining candidates are further selected by the number of vertices. However, if no mark is selected when selecting by the number of vertices, the selection by the number of vertices is stopped, and mark candidates are selected by distance and size.

The mark candidates selected in this manner are evaluated again in step S04 of FIG. 6, and one mark is selected for each of the four divided regions.

When proceeding to step S04 in FIG. 6, the mark candidates for each of the four divided regions are all present. Therefore, in this step S04, a total of four marks, one for each divided area, are selected by comprehensively reviewing the available mark candidates. As a mark selection criterion in step S04, for example, a mark candidate having the longest perimeter of a quadrilateral having four vertices corresponding to the mark candidates of each of the four areas is selected as a mark, or the quadrilateral or the mark candidate with the smallest deviation from 90° among the four corner angles of the quadrilateral (i.e., the quadrilateral that is as close to a rectangle as possible is It is possible to adopt the method of selecting the obtained mark candidate) as the mark.

Note that step S04 may be deleted by selecting only one mark for each divided area in step S14, or step S14 may be deleted and a mark for each divided area is selected in step S04. may

FIG. 8 is a diagram showing a flow chart of the cluster merging process in step S13 of FIG.

Here, a process of merging clusters composed of remaining contours after removing unnecessary contours in step S12 of FIG. 7 is performed. Note that the cluster merging process itself described below is merely an example, and various known algorithms related to the cluster merging process can be employed.

Each of the clusters when proceeding to the process of step S13 in FIG. 7, that is, the process shown in FIG. 8, is a cluster made up of connected contours. Here, first, a distance matrix is created. This distance matrix is obtained by calculating the distance between two clusters for all combinations of clusters, and arranging sets of two clusters in ascending order of the distance. The distance between two clusters is calculated by applying the representative point of the cluster in the calculation of the distance between the base point and the cluster described above to both of the two clusters.

That is, the distance between the centroid positions of the clusters may be calculated, or the distance between the positions of the center points of rectangles circumscribing the clusters may be calculated. Alternatively, the average value of the distances between the vertices of clusters as polygonal lines may be used as the distance between two clusters.

After the distance matrix is created, a set of clusters is merged (step S22). If the clusters are merged in step S22 (step S23), the process of merging a set of clusters in step S22 is performed again, and this process is repeated until clusters are no longer merged (step S23). . Here, the distance matrix created in step S21 is the first distance matrix, and this distance matrix is updated each time step S22 is executed (see step S33 in FIG. 9).

FIG. 9 is a diagram showing a flow chart of the process of merging a set of clusters in step S22 of FIG.

Here, based on the distance matrix created in step S21, a search is performed in ascending order of distance, and it is determined whether or not two clusters arranged in the distance matrix can be merged (step S31). The criterion here is the size of the cluster after merging. As the size of the cluster, for example, the perimeter or area of the circumscribing rectangle, the length of the rectangle, or the like can be used. Alternatively, the distance between the two furthest points in the cluster may be the dimension of the cluster. In this step S31, it is determined that merging is possible when the size of the cluster after merging does not exceed a predetermined maximum size. Then, two clusters forming a set with the shortest distance among those determined to be mergeable are merged into one cluster (step S22). Then, the distances between the two clusters are recalculated for all cluster combinations after the one set of clusters are merged, and the distance matrix created in step S21 of FIG. 8 is updated (step S33). After updating the distance matrix, returning to FIG. 8, it is recognized in step S23 that the clusters have been merged, and the process proceeds again to step S22, that is, the process of merging a set of clusters shown in FIG.

This process is repeated until it is determined that any two clusters lined up in the distance matrix cannot be merged (step S31). Then, if it is determined in step S31 that any two clusters cannot be merged, the process returns to FIG. 8, and in step S23 it is recognized that the clusters have not been merged. When it is recognized in step S23 that the clusters have not been merged, the cluster merging process in step S13 of FIG. 7 is completed, and the mark selection process in step S14 is performed. This mark selection process has already been explained, and redundant explanation will be omitted. When the mark selection process of step S14 ends, the mark candidate extraction process for one divided image in step S03 of FIG. 6 is completed. The above mark candidate extraction processing is sequentially repeated for each of the four divided images. Then, when the mark selection process of step S03 is completed for all four divided images, the mark selection process of step S04 is performed. This mark selection process has already been explained, and redundant explanation is omitted here.

In the embodiment described above, an example is shown in which the answer sheet P read by the multifunction device 1 is divided into four areas and a mark is selected for each divided area. If selected, it is possible to align the answer sheets P with each other, including inclination and expansion/contraction. If marks are selected for three locations that are not on a straight line, alignment can be performed by affine transformation (rotation, parallelogram, and movement transformation). Also, if marks are selected for four points that are not on a straight line, as in the example of the answer sheet above, alignment can be performed by projective transformation (trapezoidal transformation: projecting a three-dimensional object into two dimensions). . Alternatively, the answer sheet P may be divided into 6 or 8 sections and a mark may be selected for each divided area to improve the alignment accuracy. Alternatively, if the tilt and expansion/contraction are negligible, marks may be selected only at two diagonal points or one point at the center, for example.

In the above embodiment, the mark is selected from the answer sheet P read using the scanner function of the multifunction machine 1. However, the answer sheet P read by a scanner with a single function other than the multifunction machine 1, for example, is selected. may be targeted. Furthermore, an answer sheet P photographed by a camera, for example, may be used instead of a scanner.

Further, in the above-described embodiments, the selection of the mark from the printed pattern on the answer sheet P was described as an example, but the answer sheet P is only an example, and the present invention does not require alignment. Nevertheless, it can be widely applied to alignment of printed materials on which alignment marks are not printed.

1 MFP 2 PC (personal computer)
10 Mark Selection Device 11 Candidate Extraction Means 111 Cluster Extraction Means 1111 Contour Extraction Means 1112 Noise Removal Means 112 Cluster Merging Means 12 Mark Selection Means 20 Image Processing Device 21 Mark Storage Means 22 Shape Detection Means 23 Alignment Means

Claims (12)

Candidate extracting means for extracting a mark candidate consisting of a pattern existing in a candidate area of the patterns appearing on the first image or a set of such patterns without having pattern information as a mark;
Mark selection means for selecting a mark from among the mark candidates based on the distance from a predetermined base point on the first image to the mark candidate and the mark suitability of the mark candidate including at least the size of the mark. and
The candidate extraction means is
A cluster that extracts two patterns that are not spatially connected to one on the first image as a cluster consisting of the two patterns based on the spatial distance between the two patterns. extraction means;
cluster merging means for merging a plurality of adjacent clusters within a range not exceeding a predetermined dimension;
A mark selection device , wherein clusters merged by said cluster merging means are used as said mark candidates . The cluster extraction means is
contour extraction means for performing contour extraction processing on the pattern on the first image;
2. The mark selecting apparatus according to claim 1 , further comprising noise removing means for removing contours as noise components less than a predetermined dimension from the contours extracted by said contour extracting means. When the cluster merging means searches a set of two clusters in order of shortest distance between them and merges the two clusters that make up one set into one cluster, one cluster after merging A process of merging two clusters constituting one set whose dimensions do not exceed a predetermined dimension as one cluster is repeated while changing the order of distances that change due to merging to shortest order. The mark selection device according to claim 1 . The candidate extracting means extracts the mark candidates as a set of polygonal lines,
2. The mark selection apparatus according to claim 1 , wherein said mark selection means selects said marks by considering the number of vertexes of polygonal lines forming said mark candidates as one of said mark aptitudes. The candidate extracting means extracts a mark candidate consisting of a pattern or a set of patterns present in each of at least three candidate regions on the first image;
The mark selecting means selects at least three points on the first image based on the distance from the base point corresponding to the candidate area to the mark candidate in the candidate area corresponding to the base point and predetermined mark aptitude. 2. The mark selection device according to claim 1 , wherein each mark in is selected . When the candidate extracting means uses each of the vertices of the four corners of the first rectangular image as a base point, the pattern exists in each of four candidate areas predetermined corresponding to each of the base points, or the pattern exists in each of the four candidate areas. Extract mark candidates consisting of a set of
The mark selecting means selects marks at four locations on the first image based on the distance from the base point corresponding to the candidate area to the mark candidate in the candidate area corresponding to the base point and predetermined mark aptitude. 2. The mark selection device according to claim 1, wherein each mark is selected . 7. The mark selection device according to any one of claims 1 to 6 , wherein the first image is an image read by reading means for reading an image . mark storage means for storing shape data of the mark selected by the mark selecting apparatus according to any one of claims 1 to 7 and position data of the mark on the first image;
shape detection means for searching a predetermined search area around the position represented by the position data on the second image to detect a shape corresponding to the shape of the mark represented by the shape data; ,
The first image and the first image are arranged such that the shape represented by the shape data stored in the mark storage means and the shape corresponding to the shape of the mark detected by the shape detection means overlap each other. 2. An image processing apparatus characterized by comprising positioning means for performing positioning of the images of the second image processing method . 9. An image processing apparatus according to claim 8 , wherein said second image is an image read by a reading means for reading an image . 10. An image processing apparatus according to claim 8, comprising the mark selection apparatus according to any one of claims 1 to 7 . 8. A mark selection program, which is executed in an information processing device that executes the program, and causes the information processing device to operate as the mark selection device according to any one of claims 1 to 7. 11. An image processing program, which is executed in an information processing apparatus that executes the program, and causes the information processing apparatus to operate as the image processing apparatus according to any one of claims 8 to 10 .

Images