JP4543675B2 - How to recognize characters and figures - Google Patents

Description

  The present invention reads characters / graphics printed or engraved on an object with an image input device such as a TV camera or a scanner, and recognizes the characters / graphics by performing image processing on the image obtained by the image input device. The present invention relates to a method for recognizing characters and figures to be performed.

  Conventionally, a character / graphic (“character / graphic” means a character or graphic that can be generated by a character generator, and hereinafter simply referred to as “character”) is an image like a TV camera or a scanner. Various techniques for recognizing characters by reading with an input device and applying image processing to an image obtained with the image input device have been proposed. Since the technology for recognizing characters is a pattern recognition technology that associates with categories specified in advance, description of the structure of characters that makes it easy to associate categories, correction for deformation of characters, written on the object A technique such as segmentation that cuts out characters one by one is required.

  For example, in order to recognize a complicatedly deformed character such as a handwritten character, it has been proposed to describe the character structure in the following form (for example, refer to Patent Document 1 and Patent Document 2). In the techniques described in Patent Documents 1 and 2, a character image is thinned and a branch point or an intersection is extracted as a singular point, and the structure around the singular point is an intersection such as “X”, such as “K”. By dividing into three types of contact and branch such as “T”, a line connected at a singular point is divided into a plurality of strokes, and a primitive series that is a connected structure of monotone curves called primitives is used. Each stroke is represented, and a character structure is described by a combination of a primitive sequence and a singular point. A stroke corresponds to a part of a hand-written character (almost identical to a part written with a single stroke), and a primitive means a monotone curve in which x and y coordinates change monotonically in a two-dimensional image. Yes. The primitive connection structure is expressed by the convex direction formed by connecting two primitives and the arrangement order of primitives around the connection point in the right-handed system. It becomes a set of linked structure of the primitives that make up.

  In Patent Documents 1 and 2, a bounding box (the smallest upright rectangle surrounding the character) is used to correct the deformation of the character, and the longer side is set to the unit length without changing the aspect ratio of the bounding box. A technique is also disclosed in which the scale is normalized and correction is further performed by affine transformation.

  That is, in the techniques described in Patent Documents 1 and 2, after segmentation of characters is performed on the input image, the thinned line is analyzed to analyze the structure represented by the primitive sequence and the singular point, and then cut out from the input image Category candidates are extracted by matching the structure of the image and the model, and the partial images are normalized by calculating the distance to the model and evaluating the similarity, thereby associating the partial images with the model character categories. ing. Thus, robust pattern recognition is possible by using the structural information and quantitative information of characters.

By the way, although the segmentation technique is not specifically described in Patent Documents 1 and 2, as a segmentation technique, a skeleton line figure obtained by thinning an input character string image is converted to a simple arc or a simple closed curve. After dividing into partial curves, a technique for estimating the possibility of touching a character using the presence or absence of a branch point or an intersection in the partial curve, the horizontal width of the partial curve, and the height of the character string has been proposed (for example, And Patent Document 3). Further, Patent Document 3 recognizes a character using a technique similar to Patent Documents 1 and 2 for a character string generated by combining partial curves obtained by dividing a skeleton line figure, and results of recognition. There is also disclosed a technique of forming a network having a partial figure recognized as a character among the combinations of nodes and outputting a combination of recognized characters of the node by selecting an optimum route from the network. Further, it is described that when the number of characters is known, an optimum route is selected from a route having the number of nodes that matches the number of characters from the network.
JP-A-7-21317 Japanese Patent No. 3183949 JP-A-6-76114

  By the way, in the techniques described in Patent Documents 1 and 2 described above, although the category of the input image is narrowed down using the structural information of characters, the structural information is used on the basis of branch points and intersections of line figures. It is assumed that the lines constituting the characters are continuous. The same applies to the technique described in Patent Document 3.

  Therefore, even for characters with relatively little deformation, such as characters printed or engraved on the object, dot characters widely used for displaying the expiration date or expiry date of food, defects in the printing device For characters that are faint due to the above (hereinafter referred to as “faint characters”), characters that are discontinuously separated from the lines that constitute the characters (hereinafter referred to as “divided characters”), etc. Information cannot be extracted accurately, and even if the techniques described in Patent Documents 1 to 3 are adopted, there is a possibility that characters cannot be recognized accurately.

  The present invention has been made in view of the above reasons, and an object of the present invention is to provide a method for recognizing characters and figures that can be recognized even for dot characters, faint characters, and divided characters.

The invention of claim 1 extracts a skeleton line of a character / figure from a character / figure image input as a recognition target, and then approximates the skeleton line as a set of approximate lines that are simple shapes, and continuously When a minimum distance between approximate lines belonging to each of a pair of approximate line groups that are not continuous is equal to or less than a predetermined threshold value, and target data extraction step of extracting as a handling songs over target cluster one character or more approximate line group as a unit by integrating as one an approximate line group, approximates the skeleton line of the shape made to the standard characters and graphics A shape similarity calculation process for evaluating the similarity of the shape of the target cluster and the master cluster consisting of approximate line groups approximated as a set of lines, and the master cluster and the target cluster are approximate lines. Approximate line data specifying the position and shape is included, and in the shape similarity calculation process, after the circumscribed rectangle set for the master cluster is aligned with the circumscribed rectangle set for the target cluster, the target cluster and the master cluster The shape similarity is obtained from the distance between the skeleton lines included in and the master cluster having a high shape similarity is associated with the target cluster category.

According to this method, a character / graphic to be recognized and a character / graphic to be matched with the recognition target are expressed as a target cluster and a master cluster that are a set of approximate lines of simple shapes, and the character input as the recognition target -For graphic images , approximate the skeleton line as a set of approximate lines, add continuous approximate lines to one approximate line group, and for non-continuous approximate line groups, the minimum between approximate lines belonging to each Since the target cluster is an approximate line group of one character or more by integrating when the distance is less than the specified threshold , for example, characters / figures represented by dots, blurred characters / figures, divided characters・ Even if it is a figure, it can be handled as one unit of the target cluster, and it is possible to recognize these characters and figures that could not be recognized by the conventional method. It made. Also, since the shape similarity is obtained by aligning the circumscribed rectangle set for the master cluster with the circumscribed rectangle set for the target cluster, it is compared with the case where the template is shifted by one pixel as in normal pattern matching. Then, the area to be collated in the image is limited, and the processing load can be greatly reduced as compared with the conventional configuration, and as a result, high-speed processing can be expected.

  The invention of claim 2 is characterized in that, in the invention of claim 1, a straight line and an elliptical arc are used as the approximate line.

  According to this method, straight lines and elliptical arcs are used as approximate lines for characters and figures without using complex curves such as Bezier curves and spline curves, so the parameters for expressing approximate lines are Bezier curves. In addition to reducing the amount of data required to represent approximate lines, compared to spline curves, the processing load in collation processing such as computation for calculating shape similarity from the master cluster and target cluster can be reduced. As a result, high-speed processing can be expected.

  According to a third aspect of the present invention, in the first or second aspect of the invention, in the shape similarity calculation process, height dimensions and width dimensions of a circumscribed rectangle set for the target cluster and a circumscribed rectangle set for the master cluster are set. By determining the size ratio, the number of characters and figures in the height and width directions included in the target cluster is estimated, and for each circumscribed rectangle set for the target cluster, the master cluster for each position corresponding to the estimated number The circumscribing rectangle set for is aligned, and the shape similarity is obtained at each position.

  According to this method, even when a plurality of characters / graphics are included in the target cluster, the shape similarity can be obtained by cutting out each unit of characters / graphics in the master cluster. That is, a set of approximate lines fused as a target cluster combines a plurality of characters that can be treated as a character / graphic composed of dots, a blurred character / graphic, and a divided character / graphic as one character. On the other hand, the number of characters / graphics included in the target cluster can be evaluated by this method, and when a plurality of characters / graphics are included in the target cluster, it can be separated one by one. .

  According to a fourth aspect of the present invention, in the first to third aspects of the invention, in the master cluster and the target cluster, an isolated point to which no approximate line is connected, an end point that is an open end of the approximate line, and 2 At least one kind of a line connection point that connects two approximate lines on a straight line, a bending point that connects two approximate lines at an angle within 180 degrees, and a branch point that connects three or more approximate lines The master cluster and the target cluster used for the node include node data for specifying the position of the node in addition to the approximate line data for specifying the position and shape of the approximate line. Before determining the distance between each other, the type of each node of the master cluster and the type of each node of the target cluster existing within a prescribed distance range from the node Cross, and at least one of the nodes included in the master cluster does not match the type of the target cluster node, characterized in that it does not seek the shape similarity between the master cluster and target cluster.

  According to this method, when the node type does not match between the target cluster and the master cluster, it is judged that the topology of the character / figure is different, and the shape similarity is not calculated, so that they can be matched. Preprocessing that avoids calculation of the shape similarity with the inferior master cluster is performed, and the processing load can be greatly reduced.

  According to a fifth aspect of the present invention, in the first to third aspects of the invention, in the master cluster and the target cluster, an isolated point to which no approximate line is connected, an end point that is an open end of the approximate line, and 2 At least one kind of a line connection point that connects two approximate lines on a straight line, a bending point that connects two approximate lines at an angle within 180 degrees, and a branch point that connects three or more approximate lines The master cluster and the target cluster used for the node include node data for specifying the position of the node in addition to the approximate line data for specifying the position and shape of the approximate line. Before determining the distance between each other, by using the approximate line data of the surrounding approximate line and the node data of the surrounding node for each node of the target cluster, The Bayes probability of the node type of the target cluster existing within the specified distance range from each node of the host cluster is obtained, and the node similarity, which is the similarity between the node type of the master cluster and the node type of the target cluster, is bayed. If the node similarity of the node of the target cluster corresponding to at least one of the nodes included in the master cluster is not equal to or greater than the prescribed threshold, the shape similarity between the master cluster and the target cluster is calculated. It is characterized by not seeking.

  According to this method, by expressing the node type as a Bayesian probability, even when the topology of the character / graphic changes when extracting the skeleton line, the node type in the original character / graphic is relatively It can be obtained with high reliability. By using the node type obtained in this way, as in the method of claim 5, when the node type does not match between the target cluster and the master cluster, it is determined that the topologies of the characters and figures are different. Thus, by not obtaining the shape similarity, preprocessing for avoiding the calculation of the shape similarity with the master cluster that has no possibility of matching is performed, and the processing load can be greatly reduced.

  According to a sixth aspect of the present invention, in the first to fifth aspects of the invention, in the shape similarity calculation process, the shape similarity is measured from an approximate line included in the master cluster to an approximate line included in the target cluster. The ratio of the total length of the approximate lines whose distance is within the specified distance to the total length of the approximate lines included in the master cluster, and the value obtained by subtracting the average distance between the approximate lines within the distance from the distance It is proportional to a value obtained by multiplying an evaluation value of the degree of approach between approximate lines obtained by dividing by a distance.

  According to this method, the degree of shape similarity increases as the distance between the approximate lines of the target cluster and the master cluster is close and the portion where the approximate lines are close to each other occupies the total length of the approximate line of the master cluster. Therefore, it is possible to appropriately evaluate a similar shape. Further, even when noise is superimposed on characters / graphics, similarity can be evaluated only for the portion of the skeleton line.

  According to a seventh aspect of the present invention, in the first to sixth aspects of the invention, in the shape similarity calculation process, after the master cluster is aligned with the target cluster using the circumscribed rectangle, the approximation included in the master cluster The least square method is used to obtain parameters of a transformation equation that transforms at least one element of position, rotation, scale, and linear distortion so that the line is superimposed on the approximate line included in the target cluster, and the shape similarity is obtained after the transformation is applied. It is characterized by that.

  According to this method, the skeleton lines of the characters and figures included in the target cluster and the master cluster are misaligned, rotated relative to each other, have different scales, or are linear to the target cluster with respect to the master cluster. Even if distortion occurs, it can be corrected so that they match, reducing the amount of error that cannot be absorbed by just aligning the circumscribed rectangles, and improving the accuracy of character / graphic category matching Can do. In other words, categories can be associated with deformed characters / graphics such as italics and unmodified characters / graphics using the same master cluster.

  The invention of claim 8 is different in the shape similarity calculation process in the case of the invention of claim 1 when there are a plurality of characters / graphics inputted as recognition targets and the number and arrangement are known. One corner position is selected as a representative point from at least one circumscribed rectangle, and the position of the circumscribed rectangle is aligned by superimposing the position of the representative point of the master cluster on the position of the representative point of the target cluster. .

  According to this method, when the recognition target includes a plurality of characters / graphics, the master cluster is aligned at a plurality of points relatively distant from each other using the character / graphic arrangement in the target cluster. It is possible to accurately match the position of the circumscribed rectangle to the position of the circumscribed rectangle of the target cluster. In particular, if the distance between the representative points is increased, the rotation angle of the target cluster can be accurately detected, which makes it possible to align the master cluster more accurately.

  According to a ninth aspect of the present invention, in the first aspect of the invention, when there are a plurality of characters / figures input as recognition targets, and the number and arrangement are known, each target is calculated in the shape similarity calculation process. Obtain a circumscribed rectangle whose rotation position is adjusted so that the area of each cluster is minimized, and use the angle of the axis that maximizes the frequency of the angle when the corner of the circumscribed rectangle is mapped on one axis. The circumscribed rectangle is aligned by rotating the master cluster at this angle.

  According to this method, since the circumscribed rectangle is set so that the area is minimized, it is possible to set the circumscribed rectangle that matches the direction in which the characters / graphics are upright, and in addition, a plurality of target clusters can be set. Since the angle of the circumscribed rectangle is detected based on the frequency when the corners of the circumscribed rectangle are mapped on one axis, the inclination of the character string can be accurately obtained with relatively simple processing.

  According to a tenth aspect of the present invention, in extracting the skeleton line in the first to ninth aspects of the present invention, a contour line of a character / figure is extracted using a grayscale image, and the origin edge point selected from the contour line is extracted. Find the brightness change direction in the surrounding area, and find another pixel on the contour line obtained by searching on the straight line of the brightness change direction inward from a known character or figure from the start edge point as the end point edge. A middle point between the edge point and the end point edge point is a pixel on the skeleton line.

  According to this method, since the skeleton line is extracted using the contour line obtained from the grayscale image, if the image input as a recognition target is binarized as in the case of uneven brightness, the character / graphic information is normal. Even if it is impossible to extract the correct skeleton line, the correct skeleton line can be extracted.

  In the invention of claim 11, in extracting the skeleton line in the inventions of claims 1 to 9, the outline of a character / figure is extracted using a grayscale image, and the origin edge point selected from the outline is selected. Find the direction of brightness change in the surrounding area, search for a straight line on the direction of brightness change from the starting edge point to the inside of a known character or figure, and end other pixels on the contour line within the specified distance range It is obtained as an edge point, and the middle point between the starting edge point and the ending edge point is a pixel on the skeleton line.

  According to this method, since the skeleton line is extracted using the contour line obtained from the grayscale image, if the image input as a recognition target is binarized as in the case of uneven brightness, the character / graphic information is normal. Even if it is impossible to extract the correct skeleton line, the correct skeleton line can be extracted. Moreover, since skeleton lines are extracted by selecting only line figures within a specified distance range (that is, a specified width range), skeletal lines are not extracted from wide areas that cannot be considered as characters and figures. Skeletal lines can be accurately extracted.

  According to a twelfth aspect of the present invention, in extracting the skeleton line according to the first to ninth aspects of the present invention, a contour line of a character / figure is extracted using a grayscale image, and an origin edge point selected from the contour line is extracted. Find the brightness change direction in the surrounding area, search the straight line in both directions in the brightness change direction from the starting edge point, and find other pixels on the contour line within the specified distance range as the end edge point, and start edge The midpoint between the point and the end point edge point is a pixel on the skeleton line.

  According to this method, since the skeleton line is extracted using the contour line obtained from the grayscale image, if the image input as a recognition target is binarized as in the case of uneven brightness, the character / graphic information is normal. Even if it is impossible to extract the correct skeleton line, the correct skeleton line can be extracted. In addition, since the skeleton line is searched on both sides of the contour line, the skeleton line can be extracted regardless of whether the character is white or black.

  According to a thirteenth aspect of the present invention, in extracting the skeletal line according to the first to ninth aspects of the present invention, a contour line of a character / figure is extracted using a grayscale image, and the brightness around the pixels on the contour line is extracted. The change direction is obtained, and an expansion process is performed to expand the contour line in at least one of the two directions where the gradient of the brightness is maximum, and pixels that collide with pixels expanded from different two directions are determined as pixels on the skeleton line. It is characterized by.

  According to this method, since the skeleton line is extracted by expanding the pixels on the contour line obtained from the grayscale image, the continuous skeleton line can be extracted. That is, when the starting edge point selected from the contour line is selected as in claims 10 to 12, the obtained skeleton line is discontinuous, so interpolation processing is required, but the skeleton line is obtained by dilation processing. Thus, a continuous skeleton line can be obtained without interpolation. In addition, even when the image input as a recognition target has a nonuniform brightness, binarization makes it possible to extract the correct skeleton line even if character / graphic information cannot be extracted normally.

According to the present invention, characters / graphics to be recognized and characters / graphics to be compared with the recognition target are expressed as a target cluster and a master cluster that are a set of approximate lines of simple shapes, and characters input as recognition targets -For graphic images , approximate the skeleton line as a set of approximate lines, add continuous approximate lines to one approximate line group, and for non-continuous approximate line groups, the minimum between approximate lines belonging to each Since the target cluster is an approximate line group of one character or more by integrating when the distance is less than the specified threshold , for example, characters / figures represented by dots, blurred characters / figures, divided characters・ Even if it is a figure, it can be handled as one unit of the target cluster, and it is possible to recognize these characters and figures that could not be recognized by the conventional method. There is an advantage of that. Also, since the shape similarity is obtained by aligning the circumscribed rectangle set for the master cluster with the circumscribed rectangle set for the target cluster, it is compared with the case where the template is shifted by one pixel as in normal pattern matching. Then, the region to be collated in the image is limited, and the processing load can be greatly reduced as compared with the conventional configuration. As a result, there is an advantage that high-speed processing can be expected.

  An example of an apparatus used in this embodiment is shown in FIG. In the illustrated example, a TV camera 12 is provided as an image input device for inputting an image of a sheet-like or three-dimensional object 11. Characters (meaning characters / graphics as in the conventional configuration) are printed or stamped on the object 11 (the surface of the object 11 is depressed or raised in an embossed shape), and the TV camera 12 captures an image including the characters. To do. The present embodiment mainly recognizes characters by printing or engraving, and does not assume recognition of handwritten characters. An analog signal image captured by the TV camera 12 is input to the digital image generation device 13 and converted into a digital signal. The digital image generation device 13 generates a grayscale image whose pixel value is a density value. The grayscale image is stored in the storage device 14, and the grayscale image stored in the storage device 14 is subjected to image processing described below in the image processing device 15.

  The basic processing procedure of the image processing performed in the image processing apparatus 15 is as shown in FIG. 1, and first, in the master data generation registration process (S1), all the characters described in the object 11 for character recognition are shown. As for the character, a character having a standard shape is imaged, the attribute of the character is extracted from the image including the character, and master data in which the attribute is associated with the character is registered. When the master data is registered, the character can be recognized by collating with the master data. First, in the target data extraction process (S2), the object 11 for character recognition is imaged, and the master data and Character attributes to be compared are extracted as target data. In the conventional technique, segmentation is performed as a pre-processing for extracting target data, and each character is segmented by segmentation. However, in this embodiment, segmentation is not performed in the target data extraction process, and dot characters, divided characters, By handling characters including a continuous part as a whole in the target data extraction process, division into units smaller than one character is suppressed. That is, if the distance between the separated parts is within a prescribed threshold value, the separated parts are handled as one unit. Such units are hereinafter referred to as clusters. A cluster extracted from the target data is called a target cluster, and the target cluster usually includes one or more characters, and may include a plurality of characters. A cluster in the master data is in units of one character, and the master data cluster is called a master cluster.

  When master data is registered in the master data generation registration process and target data divided into target clusters is obtained in the target data extraction process, a shape similarity calculation process (S3) for collating the master cluster with the target cluster is performed. . In the shape similarity calculation process, the position of the master cluster is aligned with the target cluster, and the similarity between the target cluster and the master cluster is calculated. For the alignment of the target cluster and the master cluster, the positions of the circumscribed rectangles set in the target cluster and the master cluster are aligned. However, since the target cluster may contain multiple characters, the number of characters included in the target cluster is estimated using the ratio of the height dimension to the width dimension of the master cluster and the target cluster, and the estimated number of characters is used. Accordingly, the master cluster is moved in the target cluster, and the shape similarity is obtained at each position. By this process, characters are separated in the target cluster including a plurality of characters. Each process described above is realized by executing an appropriate program on a computer. In this way, by obtaining the shape similarity by matching the positions of the circumscribed rectangles of the master cluster and the target cluster, the area to be collated is significantly larger than when collating by shifting the template by one pixel in the image. As a result, the processing load can be reduced.

  Below, the process sequence in each process mentioned above is demonstrated concretely.

  In the master data generation / registration process, a sheet-like object 11 on which characters having a standard shape as a recognition object are printed, that is, a master sheet 11a as shown in FIG. The master sheet 11a is printed with characters to be recognized. In the illustrated example, the numbers “0” to “9” and alphabets “A”, “B”, “C”, and “R” are used as recognition targets. All characters to be recognized are arranged on the master sheet 11a at appropriate intervals. However, the number of master sheets 11a is not limited to one, and a plurality of master sheets 11a may be sequentially imaged. From an image obtained by capturing the master sheet 11a, a cluster (hereinafter referred to as a “master cluster”), which is a unit graphic having the characteristic amount related to the structure of each character as an attribute, is generated by the following procedure. Master data as an attribute is associated with the master cluster.

  In the master data generation / registration process, as shown in FIG. 2, an area of one character unit is designated for the image of the master sheet 11a (S11). The area is specified by displaying an image of the master sheet 11a on the screen of a display device such as a CRT and manually by an operator using a pointing device such as a mouse. That is, when creating the master data, the image processing device 15 is made to recognize the range of one character according to an instruction from the operator. After designating an area in units of one character, the density value is binarized with an appropriate threshold value for each area (S12), and a skeleton line is extracted from the binary image (S13). A known technique such as the Hilditch method is used to extract the skeleton line. When a skeleton line is extracted from the “R” image at the lower right end of the master sheet 11a shown in FIG. 3, a binary skeleton line image as shown in FIG. 5 is obtained.

  When one character unit skeleton line is extracted, master data as a feature amount is extracted for each character unit skeleton line. In creating the master data, first, raster scanning is performed within an area of one character unit in the skeleton line image, and end points and branch points (including intersections) are extracted from the skeleton line as nodes (S14). For example, when nodes are extracted from the skeleton line image of FIG. 5A, end points t1 and t2 and branch points j1 and j2 are extracted as shown in FIG. 5B. After extracting the nodes, line elements connecting the nodes are extracted (S15). That is, the line is traced from the end points t1 and t2 and the branch points j1 and j2 to the other end points t1 and t2 or the other branch points j1 and j2 (S15b). e4 is extracted and recorded (S15c). When one end of the line elements e1 to e4 is open, the tracking path to reach the other end points t1 and t2 or the branch points j1 and j2 is one line element (for example, line elements e3 and e4). When both ends are closed, the tracking path to reach the end points t1 and t2 or the branch points j1 and j2 that are the start points is set as one line element (for example, line elements e1 and e2). If the node that has finished tracking is the end points t1 and t2, the tracking is terminated, and the tracking is performed from the end points t1 and t2 that are not being tracked. Further tracking is performed from the branch points t1 and t2 (S15a, S15d).

  When the line elements e1 to e4 are extracted as described above, the line elements e1 to e4 are approximated as a set of “approximate lines” which are simple shapes (geometric shapes such as straight lines and ellipses). Here, an approximate line represented by a simple mathematical expression is used as a simple shape. Two types of approximate lines are used: a straight line and a part of an ellipse (hereinafter referred to as an “elliptical arc”). By using straight lines and elliptical arcs as approximate lines in this way, parameters for expressing approximate lines can be reduced as compared to the case of using Bezier curves and spline curves, and the processing load for various operations is also reduced. . In the shape shown in FIG. 5B, the line element e1 is approximated by two straight lines that form the upper left corner of the character “R” and one elliptical arc from the right end of the upper straight line to the branch point j2. can do. That is, as shown in FIG. 5C, the line element e1 is approximated by the approximate lines 11 and 12 that are two straight lines and the approximate line l3 that is one elliptical arc, and the two approximate lines 11 that are straight lines. , L2 and a line connection point d1 connected to one approximate line l2 that is a straight line and the other approximate line l3 that is an elliptical arc are introduced as nodes. Since the line elements e2 to e4 are straight lines, they are used as they are for the approximate lines l4 to l6.

The cluster in the present invention is a figure made up of skeleton lines for each character unit, and the structure of the cluster consists of approximate lines 11 to 16 and nodes (end points t1, t2, branch points j1, j2, bend points c1, line connection points d1). ) And the attribute. That is, the master cluster mc relating to the character “R” is a set of six approximate lines 11 to 16 and six nodes (end points t1, t2, branch points j1, j2, bending point c1, line connection point d1). In the master cluster mc, the coordinates of each node expressed by using relative coordinates based on one node (for example, the bending point c1 which is the upper left node) and the approximate lines l1 to l6. Are associated with each other. If an elliptical arc approximate line l3 exists, the center of the ellipse is also included in the node, and if an isolated point (for example, a point above the letter “i”) is included in the region, the isolated point is also included in the node. Become. Node coordinates are associated with node identification codes (node names), and approximate line parameters are associated with approximate line identification codes (approximate line names). Further, the approximate line name of the approximate line connected to the node is associated with the node name. For example, the approximated lines l1, l4, and l5 connected to the branch point j1 that is a node are associated with the branch point j1 shown in FIG. Further, node names at both ends of the approximate line are associated with the approximate line name. For example, the approximate line l2 shown in FIG. 5C is associated with the bending point c1 and the line connection point d1, which are nodes at both ends of the approximate line l2. The above approximate line parameters mean coefficients α and β when the approximate line is a straight line represented by αx + βy−1 = 0, and an elliptical arc whose approximate line is represented by ax ² + by ² + cxy + dx + ey−1 = 0. Means the coefficients a to e.

  As described above, the master cluster mc includes information of {node name (node coordinates, approximate line name connected to the node), approximate line name (approximate line parameters, node names at both ends of the approximate line)}. Are associated. In the following, node names (node coordinates, approximate line names connected to nodes) are referred to as “node data”, and approximate line names (approximate line parameters, node names at both ends of the approximate line) are referred to as “approximate line data”. A set of node data and approximate line data associated with the master cluster mc is called master data. FIG. 6 shows nodes (white circles) in the master clusters mc1 to mc14 generated from the characters written on the master sheet 11a shown in FIG. In FIG. 6, the node at the center of the ellipse is omitted.

  As described above, the line elements e1 to e4 obtained in step S15 in FIG. 4 are approximated by straight lines and elliptical arcs, and the bend points, line connection points, isolated points, and ellipse centers are extracted as necessary. (S16) Further, a cluster composed of a set of node data and approximate line data is generated, and this cluster is set as a master cluster, and master data corresponding to the master cluster is registered (S17). The above-described processing is performed for each character, and when a master cluster is generated for all characters obtained by imaging the master sheet 11a and master data is registered (S18), the master data generation / registration process is terminated.

  As described above, nodes and approximate lines are extracted from the skeleton line, and the above-described set of node data and approximate line data is used as master data, so the character structure can be described with a relatively small amount of data. In addition, since the approximate line is represented by a mathematical expression, when correcting for rotation, scale, and deformation in the shape similarity calculation process described later, it is possible to cope with the problem by simply correcting the parameters of the approximate line. It is not necessary to perform correction on the pixels, and the processing is simplified.

  Next, a description will be given of a target data extraction process for extracting a cluster from characters to be recognized and extracting target data to be collated with master data. In the target data extraction process, the TV camera 12 images a sheet-like object 11 on which characters to be recognized are printed, that is, a target sheet 11b as shown in FIG. In the illustrated example, each character string of “03 06 25” and “3A87B” is shown in the upper and lower two stages on the target sheet 11b. Also, “030”, which is the upper left three characters, is a faint character and a divided character that is divided into left and right. That is, these three characters are all characters having one element consisting of continuous lines in the master sheet 11a, but the portion to be continuous is discontinuously divided in the target sheet 11b, It is a letter. Furthermore, “06”, which is the two characters at the center of the upper row, is a touched character (hereinafter referred to as “contact character”). That is, although these characters are two characters, there is a possibility of being mistaken as one character by contact.

  In the target extraction process, there is no manual operation for the operator to specify the range of one character. As shown in FIG. 8, the entire image captured by the TV camera 12 is binarized (S21), and the skeleton is converted from this binary image. A line is extracted (S22). As with the master data generation and registration process, end points and branch points are extracted from this skeleton line image (S23), and line elements are extracted by the same procedure (S24a to S24d) (S24). Further, an approximate line obtained by approximating the line element with a straight line and an elliptic arc is extracted, and further a node including a bending point, an isolated point, and the center of the ellipse is extracted (S25). That is, the nodes and approximate lines that are elements of the cluster are extracted without performing segmentation. For the target sheet 11b shown in FIG. 7, nodes (represented by white circles) and approximate lines can be extracted as shown in FIG. 9A. Similar to the master data, coordinates and approximate line names are associated with nodes, and parameters and node names are associated with approximate lines. That is, node data and approximate line data are generated.

  However, since the characters written on the target sheet 11b shown in FIG. 7 include a blurred character, a divided character, and a contact character, the node data and the approximate line data obtained as described above are 1 Instead of character units, one character may be divided into a plurality of characters, or a plurality of characters may be combined into one. That is, since segmentation is not performed, it is not guaranteed that the set of node data and approximate line data is for one character, and even if it is collated with the master data as it is, a corresponding one cannot be found. Will be associated with the wrong category. However, since the contact character can be separated into each character in a separate process described later, it is not set as a processing target in the target data extraction process.

  Accordingly, in order to enable collation with the master data, node data and approximate line data handled as one unit are generated using the node data and approximate line data obtained from the target sheet 11b, and a target cluster is generated. This process is performed in the procedure represented by step S26 and step S27 in FIG. First, an appropriately selected node is set as a starting point node (S26b), and a node at the other end of the approximate line having the starting point node as one end is detected (S26d). The type of the node at the other end is classified into an end point, a branch point, and others (S26e). If it is an end point, the same processing is performed for another approximate line connected to the origin node (S26c). The branch point is set as a new start node (S26g), and the same processing is performed. If the node at the other end is neither an end point nor a branch point, the node is set as a new origin node (S26f), and the type is classified for the other end node of the approximate line where the node is connected to one end. When this processing is performed, all nodes connected to the first selected starting node via the approximate line can be extracted. However, any number of approximate lines may be interposed between the starting node selected first and another node. Therefore, when there is no approximate line that has not been processed (S26c), a set of node data and approximate line data that includes the first selected origin node and is regarded as one character is generated. The set of approximate line data is temporarily stored as cluster candidates (S26h). The cluster candidate generation and storage operations are repeated until all nodes in the image are included in any cluster candidate (S26a). When nodes and approximate lines are extracted as shown in FIG. 9A, cluster candidates pc1 to pc14 are extracted as shown in FIG. 9B. In FIG. 9B, the cluster candidates pc1 to pc14 are surrounded by a circumscribed rectangle.

  The cluster candidates pc1 to pc14 obtained as described above do not necessarily form a cluster for each character, as is apparent from FIG. 9B. Therefore, in step S27 shown in FIG. 8, processing for combining cluster candidates corresponding to one character is performed. Whether or not the cluster candidates pc1 to pc14 are to be combined is evaluated based on the distance between the nodes and approximate lines included in each pair of cluster candidates pc1 to pc14. That is, two cluster candidates pc1 to pc14 are extracted from each cluster candidate pc1 to pc14, and the shortest distance between the distances between all the nodes included in the other cluster candidates pc1 to pc14 and the distance between approximate lines is obtained. (S27b). When the shortest distance is equal to or less than the prescribed threshold (S27c), it is determined that both cluster candidates pc1 to pc14 should be treated as one unit, and both cluster candidates pc1 to pc14 are treated as one cluster as target data. Register (S27e). When the shortest distance is larger than the prescribed threshold, both cluster candidates pc1 to pc14 are determined as different characters, and are treated as different clusters and target data is registered (S27d). For example, by setting a threshold value so that the shortest distance between nodes in the cluster candidates pc3 and pc5 in FIG. 9B is smaller than the threshold value, both cluster candidates pc3 and pc5 are changed as shown in FIG. 9C. It will be handled as one unit. However, when the distance between the node and the approximate line is short like the cluster candidates pc2 and pc3, a plurality of characters may be treated as one character, and the evaluation of the cluster candidates pc1 to pc4 as shown in FIG. 9C. In some cases, the target clusters tc1 to tc9 obtained by the above include a plurality of characters.

  By combining the cluster candidates by the above-described processing, it is widely used for printing characters including discontinuously separated parts such as “i” and “ha”, food expiry date or expiry date. Even in the case where one character is composed of two or more adjacent parts, such as a dot character, it can be grouped into a cluster of one character. This is the same even if it is a faint character or a divided character.

  Next, in the shape similarity calculation process, the process of associating each cluster included in the target data with the category of the master cluster by collating the master data with the target data and evaluating the similarity between the shapes of both skeleton lines. Done. That is, the character included in the target data is recognized by associating the target data with any character included in the master data.

  In the shape similarity calculation process, as shown in FIG. 10, the target cluster and the master cluster are aligned, and the target data corresponding to the target cluster and the master data corresponding to the master cluster are collated. That is, after the alignment of the target cluster and the master cluster, the distance between the approximate lines is calculated using the target data and the master data, and this distance is used as the evaluation value of the similarity regarding the shape of the target cluster and the master cluster. Use.

  Since the master cluster is a figure in units of one character and the target cluster is a figure presumed to be in units of one character, when aligning the target cluster and the master cluster, first the target cluster and the master cluster are overlapped Align the position to maximize the overlap area. Therefore, in order to define the positions and sizes of the target cluster and the master cluster two-dimensionally and compare the sizes, circumscribed rectangles are obtained for the target cluster and the master cluster (S31). To set a circumscribed rectangle for the target cluster and the master cluster, the maximum value and the minimum value in the X direction and the Y direction are set for each node data and approximate line data included in each target cluster or each master cluster. The coordinate position represented by (minimum value in the X direction, minimum value in the Y direction) is the upper left point, and the coordinate position represented by (maximum value in the X direction, maximum value in the Y direction) is the lower right line. Generate a rectangle. Further, when the circumscribed rectangles set for the target cluster and the master cluster are overlapped, the positions are aligned so that the overlapping area is maximized. Alignment is possible with this method. For the target cluster and master cluster, the height and width dimensions are almost the same for the same character, and there is no significant difference in the typeface between the target cluster and the master cluster. When the condition is satisfied. The latter condition can be easily satisfied as long as it is a character printed or stamped on the object 11 in a factory product.

  However, as shown in FIG. 9C, since one target cluster tc1, tc2 may include a plurality of characters (two characters in the illustrated example), the former condition among the above conditions is not satisfied. There is. Therefore, a ratio is obtained for each of the height dimension and the width dimension of the target cluster and the master cluster, and the obtained ratio is regarded as the number of characters in the height direction or the width direction (S32). However, since the number of characters is an integer value, the calculated ratio is rounded off. For example, as shown in FIG. 11, when the target cluster tc1 shown in FIG. 9C and the master cluster mc4 shown in FIG. 6 are collated, the width dimension (left-right dimension) of the target cluster tc1 is the master cluster. Since it is approximately twice mc4, it is estimated that the target cluster tc1 contains two characters.

  The value obtained by dividing the height dimension and width dimension of the target cluster by the height dimension and width dimension of the master cluster as described above is the alignment of the master cluster with respect to one target class of interest. Shows how many times are needed. Next, the shape similarity between the target cluster and the master cluster is obtained (S34). In order to obtain the similarity, first, the position of the master cluster is aligned with the target cluster (S34a). Here, if the value obtained as described above is 2 or more, after evaluating the similarity of the shape as follows, the master cluster is moved by one character relative to the target cluster, and the similarity of the shape The process of evaluating is repeated. That is, the similarity is evaluated as many times as the number of characters estimated to be included in the target cluster (the number of characters represented by the above-described ratio).

  For example, in the example shown in FIG. 11, the characters “0” and “3” are included in one target cluster tc1, the height dimension is substantially equal to the master cluster mc4, but the width dimension is the same as that of the master cluster mc4. Since it is approximately double, it is estimated that the data of two characters is included in the cluster of the target cluster tc1. Therefore, the master cluster mc4 is aligned with the left half of the target cluster tc1, and the similarity is evaluated, and the master cluster mc4 is aligned with the right half of the target cluster tc1, and the similarity is evaluated. In the illustrated example, when the master cluster mc4 is aligned with the right half of the target cluster tc1, the degree of similarity increases. Therefore, if an appropriate threshold is set for the degree of similarity, the target cluster tc1 is set to “0” and “ 3 "can be divided.

  By the way, the target cluster is usually an area of one character or more, but the target cluster for one character may be divided into a plurality of parts. In this case, since the ratio is smaller than 1 when the ratio is obtained as described above, when the ratio is smaller than 1, the ratio does not become an almost integer value around the target cluster of interest. If the target cluster is found, the target cluster whose ratio is approximately an integer value is found. If such a target cluster exists, it is merged with the target cluster of interest and collated with the master cluster. .

As shown in FIG. 12, each part of the approximate line (broken line) included in the master cluster and the approximate line (two-dot chain line) included in the target cluster is used to evaluate the similarity in shape between the target cluster and the master cluster. And the total distance T1 of the approximate line whose distance is equal to or less than the specified value d (that is, the distance that can be regarded as substantially overlapping), the approximate line included in the target cluster, and the master cluster The average value Da of the distance between the included approximate lines is used. Here, if the total extension of the approximate lines included in the master cluster is M1, the shape similarity Sa can be expressed by the following equations (S34b, S34c).
Sa = (1-Da / d) × (T1 / M1)
As is clear from the above equation, the shape similarity Sa increases as the ratio of the average distance of the adjacent parts where the approximate line included in the master cluster and the approximate line of the target cluster are within the distance d to the distance d decreases (above In the equation, the value obtained by subtracting the ratio of the average distance Da to the distance d from 1 is used as the evaluation value of the approach degree), and the ratio of the total length of the adjacent portion within the distance d to the total length of the approximate line of the master cluster The larger the value, the larger. Further, the value of the shape similarity Sa is normalized so as to take a value between 0 and 1 regardless of the number and length of the approximate lines constituting the character. The recognition result can be judged based on the standard. Here, the shape similarity Sa having the maximum value (score) obtained by repeating the above-described processing for each master cluster with respect to the target cluster of interest is obtained (S33, S35 to S38).

By obtaining the shape similarity Sa as described above, even if a line other than a line constituting a character (that is, a line not in the master cluster) is included in the target cluster, the distance of the approximate line is usually obtained. Since the distance between each other increases and does not affect the shape similarity Sa, it is possible to accurately determine missing or different characters even when noise is superimposed on the characters. However, for example, when used in an optical character reader (OCR) or the like, if it is viewed graphically as “C” and “O” or “3” and “8”, it is part of other characters. In such a case, the shape similarity Sa may be relatively large, and correct determination may not be possible. In such a case, it is preferable to use the following equation obtained by multiplying the above equation by the total extension T1 of the approximate line less than the distance d with respect to the total extension Ta of the target cluster approximate line.
Sa = (1-Da / d) × (T1 / M1) × (T1 / Ta)
By the way, in the above procedure, since it is necessary to calculate the distance between the approximate lines in order to obtain the similarity of the shape between the target cluster and the master cluster, the approximate lines are expressed by mathematical formulas. However, if the number of types of master clusters is small, the processing load will not increase significantly. However, there are many types of master clusters that require similarities, such as optical character readers. If there are many, it is considered that the processing load becomes very large. Therefore, after performing alignment using the circumscribed rectangle in step S34a of FIG. 10, the similarity between the nodes of the target cluster and the master cluster is evaluated as preprocessing without proceeding to step S34b. That is, the number of nodes included in the master cluster is Mn, the number of nodes in the target cluster existing within a distance d from the nodes included in the master cluster is Tn, and the average distance between the nodes of the target cluster and the nodes included in the master cluster The node similarity Sd represented by the following equation is obtained with the value Dn, and when the node similarity Sd is equal to or less than the prescribed threshold value, the process for obtaining the shape similarity Sa is not performed for the master cluster.
Sd = (1-Dn / d) × (Tn / Mn)
Since the average value of the distances Dn used when determining the node similarity Sd is the distance between nodes (points), the processing load is significantly reduced as compared with the calculation for determining the distance between approximate lines. In addition, for a master cluster having a small shape similarity Sa in advance, calculation of the distance between approximate lines that increases the processing load is avoided, so that the processing load in the entire shape similarity calculation process shown in FIG. Become.

  Further, when determining the node similarity Sd, the parameters related to the rotation, scale, and deformation of the master cluster are changed and collation is performed, thereby correcting the rotation, scale, and deformation between the master cluster and the target cluster. In this case, when the shape similarity Sa is obtained, it can be correctly recognized even if the characters are rotated in the target cluster, the scale of the master cluster is different, or the master cluster is deformed. .

  In the above-described embodiment, the node type (end point, bending point, branch point, line connection point, isolated point, ellipse center) is not considered, but if it is not a division character or a contact character, it is included in the target cluster. The characters of the master cluster and the characters of the master cluster should have many common parts in the topology. Therefore, when obtaining the node similarity Sd, the same type as the nodes of the target cluster existing within the distance d from the nodes included in the master cluster. If attention is paid only to the nodes, the reliability of the node similarity Sd can be improved. In other words, before obtaining the shape similarity, determine the type of each node of the master cluster and the type of each node of the target cluster that is within a specified distance range from the node, and at least of the nodes included in the master cluster If one does not match the node type of the target cluster, the shape similarity between the master cluster and the target cluster is not obtained. By performing such preprocessing, the processing load for obtaining the shape similarity is greatly reduced.

  On the other hand, if the topology of the character included in the target cluster and the character of the master cluster is different, such as a split character or contact character, the surrounding approximate line (shape, length, direction), The node information may be used to obtain a Bayes probability for the node type, and the node type may be evaluated using the Bayes probability when the nodes are collated. Here, the Bayesian probability relating to the node type means a probability that the total probability for each node type is set to 100%. Hereinafter, a method for estimating the node type using the Bayes probability will be described.

  In FIG. 13A, a part of the character “3” included in the target cluster is divided, end points t3 and t4 are generated in the middle of the continuous line, and the end point t5 and the bend are bent near the branch point. An example in which the point c2 occurs is shown. In FIG. 13B, the character “4” written on the target sheet 11b is bold, and when the skeleton line is extracted, a beard-like skeleton line is formed at a portion to be a bending point at the upper end. In the example, a node that appears and becomes a bending point is a branch point j3, and an end point t6 is generated at the tip of the beard-like skeleton line.

  Now, paying attention to the example of FIG. 13A, the two end points t3 and t4 are close to each other, and the tangential directions of the approximate lines l7 and l8 respectively connected to the end points t3 and t4 are substantially the same. From the above, it can be estimated that if both end points t3 and t4 are continuous, the node does not exist or if the node exists, the type of the node becomes a line connection point. Further, regarding the end point t5 and the bending point c2, the end point t5 connecting the relatively long approximate line l9 is close to the bending point c2, and therefore it is estimated that the end point t5 and the bending point c2 become a branch point. it can. The Bayes probability is obtained for each type of node according to the procedure shown in FIG. 14 with the topology obtained as a result of the estimation as described above being regarded as the original topology of the characters. For estimation as described above, a rule for estimation is determined in advance.

  As shown in FIG. 14, the category probability calculation for determining the probability of the node type is performed by first setting a group in which the distance between two nodes is within a prescribed threshold D in the target cluster (S41). A temporary node is set at the position, and the length dimension of the approximate line connected to each node in the group is measured (S42). If an approximate line other than the approximate line connected to the node is present within the distance D at the end point of the node (S43), this node is set as a temporary node, and the approximate line connected to the temporary node The length dimension and the length dimension of the approximate line close to the temporary node are measured (S44).

  Next, the number of approximate lines equal to or greater than the defined length dimension L is obtained from the length dimensions of each approximate line (S45), and if it is 1, the probability that the temporary node is an end point is set to 100% (S46). . If there are two, the probability that the temporary node is a line connection point is ζ / 2π, and the probability that the temporary node is a bending point is 1-ζ / 2π (S47). Further, if there are three or more, the probability that the temporary node is a branch point is set to 100%. Here, ζ is an angle between two approximate lines, and is set in the range of 0 <ζ <2π in the right-handed system, and the probability of the line connection point is set as ζ / 2π.

  After determining the probabilities of end points, bending points, line connection points, and branch points for the temporary node, the distance d from the temporary node is determined for each node in the group (S49). Using this distance d and distance D, the Bayes probability of each node is set as follows. That is, the probability for each type of node is obtained by the calculation of the probability that the node in the group is the type × (d / D) + the probability that the temporary node is the type × (1−d / D). For example, the probability of an isolated point is the probability that a node in the group is an isolated point × (d / D) + the probability that a temporary node is an isolated point × (1−d / D). By this calculation, in the example shown in FIG. 13A, the probabilities that the end points t3 and t4 are line connection points are high, and the probabilities that t5 and c2 are branch points are high. Similarly, in the example shown in FIG. 13B, there is a high probability that the branch point j3 and the end point t3 are bending points. That is, the original topology of the character is restored.

As described above, when the Bayes probability regarding the node type is obtained for each node, the node similarity Sd is obtained by the following equation. That is, the number of nodes included in the master cluster is Mn, the number of nodes of the target cluster existing within a distance d from the node included in the master cluster is Tn, and the distance between the node of the target cluster and the node included in the master cluster is ds. The node similarity Sd is expressed by the following equation, where ps is the Bayes probability of the target cluster node belonging to the same type as the node type of the master cluster.
Sd = Σ [{(1-ds / d) × ps} / Tn] × (Tn / Mn)
When the node similarity Sd is obtained using the above equation, the topology is incompletely matched in the master cluster and the target cluster even though the shape is recognized as the same character by humans. Even so, it is possible to estimate the correct node type.

  By obtaining the node similarity, it is possible to obtain the node similarity with the target cluster nodes existing within the prescribed distance range ds from the master cluster node, and therefore corresponds to at least one of the nodes included in the master cluster. When the node similarity of the target cluster is not equal to or greater than a predetermined threshold, the shape similarity between the master cluster and the target cluster is not obtained. Such pre-processing can greatly reduce the number of times the shape similarity is obtained, thereby reducing the processing load.

  By the way, in the calculation of the shape similarity Sa described above, the position of the circumscribed rectangle set for each of the master cluster and the target cluster is matched, but the circumscribed rectangle is set for each of the master cluster and the target cluster. Even if the positions of the circumscribed rectangles of the master cluster and the target cluster are aligned, the characters included in both may be shifted without completely overlapping (the broken line in FIG. 15 indicates the master cluster). Character, solid line is the character of the target cluster). When the positional deviation between the characters is large, the distance between the two approximate lines becomes relatively large. Therefore, the value (score) of the shape similarity Sa obtained in the shape similarity calculation process is small, and the shape similarity Sa is small. There is a possibility. In addition, as described above, characters included in the target cluster may be rotated relative to the characters included in the master cluster, may have different scales, or may be deformed. There is a possibility that the shape similarity Sa is reduced only by aligning the circumscribed rectangles.

  To solve this problem, the circumscribed rectangle is used for rough alignment, and the shape similarity Sa is maximized by obtaining the shape similarity Sa a plurality of times while relatively moving the circumscribed rectangle up and down and left and right. It is conceivable to adopt a position that becomes. However, since it is necessary to obtain the shape similarity Sa for a single character a plurality of times, the number of times the shape similarity Sa is obtained increases explosively, resulting in a problem that the processing load increases.

  Therefore, as shown in FIG. 23, at least six sample points sp1 to sp6 are set on the approximate line of the characters of the master cluster, and the closest distance in the normal direction of the approximate line from each sample point sp1 to sp6. Corresponding points tp1 to tp6 on the approximate line of the target cluster character are obtained, and the coordinates of the sample points sp1 to sp6 and the coordinates of the corresponding points tp1 to tp6 are applied to the conversion formula of the affine transformation expressed by Equation 1 to obtain the master cluster The parameters α1, β1, α2, β2, dx, and dy of the affine transformation are determined by the method of least squares so that the characters included in are superimposed on the characters included in the target cluster. Since the tangent direction of the approximate line cannot be defined at the node, the sample points sp1 to sp6 are set at appropriate intervals on the approximate line. In the following formulas 1 and 2, X and Y indicate coordinates on the target cluster, and x and y indicate coordinates on the master cluster.

  The conversion formula of the affine transformation shown in Equation 1 is a total of 6 degrees of freedom of rotation around the X axis and around the Y axis, enlargement / reduction (scale) in the X axis direction and the Y axis direction, and positions in the X axis direction and the Y axis direction. Since this is a coordinate transformation for the parameters, the six parameters α1, β1, α2, β2, dx, dy in Equation 1 are obtained by the least square method, in addition to the rotation correction, scale correction, and character position. It is also possible to correct linear distortion such that the characters are italicized. If linear distortion such as italic characters is not taken into consideration, the parameters of Helmat transform with a total of 4 degrees of freedom of scale α, rotation angle θ, and movement amount (dx, dy) are set as shown in Equation 2. You may ask for it. When performing Helmert conversion, it is sufficient that there are four or more sample points.

  In the above-described method, if a sample point is set for each approximate line included in the master cluster, it is possible to cope with deformation of a part of characters included in the target cluster. For example, for the character “8”, if the upper and lower “o” are set to the same size in the master cluster, and the upper “o” is smaller than the lower “o” in the target cluster, the entire master cluster The scale cannot be overlaid on the target cluster. In such a case, if the sample points are set for the approximate line of the upper “○” and the approximate line of the lower “○”, the upper “○” and the lower “○” are enlarged separately. It becomes possible to reduce. However, affine transformation (or Helmart transformation) will be performed for each approximate line constituting the character, and nodes connecting different approximate lines will not match (although the nodes of the same position will not match) In such a case, it is necessary to correct the coordinate data of the node by performing an interpolation process. Note that, after correction by affine transformation (or Helmart transformation), the shape similarity Sa is naturally recalculated.

  Next, a technique for recognizing characters included in an image obtained by capturing the target sheet 11b using the shape similarity Sa obtained as described above will be described. In this type of character recognition, each character included in an image obtained by imaging the target sheet 11b is associated with a category of master data, and whether a known character contained in an image obtained by imaging the target sheet 11b is correctly described. There are times when it is inspected. The former is a technique used for a character reader (for example, OCR). In this case, a plurality of master clusters are aligned with respect to one target cluster to obtain shape similarities, respectively. The largest master cluster can be associated with the category at the position of the target cluster image. On the other hand, when checking the notation of a character, the master cluster corresponding to the character that must be written at the position is aligned with one target cluster to obtain the shape similarity, and the threshold value defined by the shape similarity By evaluating whether or not the above is true, it is possible to check whether or not the character is correctly written.

  By the way, in the case of checking whether the character notation is correct, when there are a plurality of characters and the character arrangement is known, the master cluster is set as the target cluster by the following method. If the positions of these are matched, the positioning becomes easy.

  Now, as shown in FIG. 16, characters are written in the upper and lower rows on the object 11 to be imaged by the TV camera 12, a character string indicating the date “2003.10.02” is written in the upper row, and the lower row. It is assumed that the character string “AB8560” is written. This type of notation has been made in industrial products and the like, and it is considered that information on character arrangement such as character type, character size, character spacing, line spacing, and number of characters is known. In this case, first, a master cluster in which an operator cuts out one character at a time from an image obtained by capturing the target object 11 as shown in FIG. A character string master in which node data and approximate line data are associated with each other is generated. In the character string master, information on the character arrangement is also stored.

  In the above example, the position of each character included in the character string is constrained. Therefore, by associating the positions of multiple characters included in the character string master and the character string to be inspected, Can be attached. For example, in the character string master shown in FIG. 18A, representative points P1 and P2 are selected one by one from the corner positions of circumscribed rectangles of two or more different characters (two characters in the illustrated example). As shown in 18 (b) to (d), for all combinations for selecting two target clusters, the representative points P1 and P2 of the character string master are overlapped at the corner positions of each target cluster, and the character string master and target The character string master is rotated, the scale is adjusted, or the position is changed so that the overlapping area of the circumscribed rectangle with the cluster increases. That is, affine transformation (or Helmart transformation) is performed on the character string master. The position where the overlapping area of circumscribed rectangles set for the character string master and the target cluster in this way (the circumscribed rectangle of the character string master is indicated by a broken line and the circumscribed rectangle of the target cluster is indicated by a solid line) is maximized. If (the position in FIG. 18C) is detected, the target cluster and the character string master can be easily collated.

  As the representative points P1 and P2 of the character string master, it is desirable to select two points at the corners of the two master clusters located on the diagonal line among the circumscribed rectangles including all the characters included in the character string master. This is because the representative points P1 and P2 selected in this way are the maximum distance among the two points that can be selected from the character string cluster included in the character string master, and the alignment accuracy is increased.

  When aligning the character string master to the target cluster, if the entire character string to be inspected is rotated with respect to the character string master, correction for rotation is unnecessary by obtaining the rotation angle in advance. Thus, the process of aligning the character string master with the target cluster is simplified. Therefore, as shown in FIGS. 19A and 19B, a circumscribed rectangle having a minimum area is extracted for each target cluster. That is, when the direction of the circumscribed rectangle is changed, the area of the circumscribed rectangle changes, and it is considered that the area of the circumscribed rectangle is minimized when the length and width of the character coincide with the length and width of the circumscribed rectangle. Therefore, the circumscribed rectangle having the smallest area is extracted. Since the circumscribed rectangles obtained in this way are considered to be oriented in the same direction, when the corners of the circumscribed rectangles of the respective target clusters are mapped in an appropriate direction, the frequency is obtained as shown in FIG. The maximum value of the frequency becomes small when the direction does not match the direction of each side of the circumscribed rectangle, whereas the mapped direction as shown in FIG. In the case of matching with the (character alignment direction), the maximum value of the frequency increases, and a plurality of distinct peaks appear. Therefore, as shown in FIG. 19B, the direction in which the maximum value of the frequency is large and a clear peak appears can be obtained as the direction of the target cluster. In short, the rotation angle of the target cluster with respect to the character string master can be known, and the processing load in the process of aligning the character string master with the target cluster is reduced. Note that the direction of mapping may be changed at a smaller angle interval in the vicinity of the angle at which the guideline is obtained after the guideline of the angle range is obtained, for example, by changing at an interval of 10 degrees. As described above, when the character string is known, the processing load can be further reduced by using the fact that the positions of a plurality of characters are constrained as compared with the case where the characters are individually recognized. Can do.

  In the above-described embodiment, the use of a technique such as the Hilditch method has been described. However, when the brightness of the background is not uniform, it is difficult to set a threshold value used for binarization for separating characters from the background. If the set threshold value is inappropriate, there is a possibility that a part of the character cannot be distinguished from the background. Therefore, if the skeleton line is extracted by the following procedure, the skeleton line can be extracted even when the difference between the lightness of the background and the lightness of the characters is small. That is, in a line figure having a width about the character, the change direction of lightness (direction in which the lightness gradient is maximized) between the contour line on one side and the contour line on the other side in the width direction of the lines constituting the character. The skeleton line is extracted by utilizing the fact that the direction is substantially opposite, and the skeleton line is extracted from the grayscale image instead of the binary image.

  In this method, by using an edge extraction filter such as a Sobel filter on an image obtained by imaging the target sheet 11b, a width of one line of characters (referred to as “character line”) as shown in FIG. It is possible to extract two contour lines k1 and k2 separated in the direction. After obtaining the contour lines k1, k2, sample points as starting edge points Ps1, Ps2,... Are appropriately extracted from the pixels on the contour lines k1, k2, and around the starting edge points Ps1, Ps2,. Find the direction of brightness change. Here, it is assumed that white characters are used on a black background. In this case, since the direction from the starting edge points Ps1, Ps2,... To the inside of the character line is the direction in which the brightness increases, the direction in which the lightness gradient is maximized around the starting edge points Ps1, Ps2,. , And the points that have reached the contour line on the opposite side of the character line are defined as the end point edge points Pe1, Pe2,. Here, in the case of a black character on a white background, the direction toward the inside of the character line is a direction in which the lightness decreases. In short, the search from the starting edge point to the ending edge point is performed with respect to the known direction of change in lightness inward of the character.

  The starting edge points Ps1, Ps2,... And the ending edge points Pe1, Pe2,... Are obtained by a change in brightness, and are not binarized using a threshold value. If there is a difference, it can be detected. As shown in FIG. 20B, the midpoints of the starting edge points Ps1, Ps2,... And the ending edge points Pe1, Pe2,. By using this, the possibility that the background and the characters can be separated is higher than in the case of binarization. Further, when the obtained skeleton points Pm1, Pm2,... Are large in distance (greater than a prescribed threshold), the skeleton lines are obtained by performing an interpolation process connecting the skeleton points Pm1, Pm2,. Continue.

  If the distance range between the starting edge points Ps1, Ps2,... And the ending edge points Pe1, Pe2,... Is limited, the width of the recognized character line can be limited. For example, when a character written with a thin line on a circuit pattern is read, it is possible to prevent the circuit pattern from being mistaken as a character line.

  Furthermore, the above procedure assumes that the direction of change in the brightness of the character line is constant, that is, the case where the white background is black or the white background is white. If the background of the black background is mixed and the lightness of the character is an intermediate value between the background of the white background and the black background, the lightness change direction of the character line partially changes and the above procedure is applied. Can not do it.

  In such a case, the direction inside the character line with respect to the starting edge points Ps1, Ps2,... Cannot be specified by the change in brightness, so the direction in which the brightness increases from the starting edge points Ps1, Ps2,. Searching for the end point edge points Pe1, Pe2,... In both the decreasing direction and limiting the range of distances from the start point edge points Ps1, Ps2,... To the end point edge points Pe1, Pe2,. By extracting the skeletal line in the background, the black and white characters on the white background, such as when the relationship of light and darkness is reversed in the background where the character overlaps, or when the character is engraved on a shiny object such as metal The skeleton line can be correctly extracted even if it cannot be specified whether the character is black or white.

  As a technique for extracting a skeleton line, after extracting the contour line, the contour line is searched for the direction of lightness change (direction in which the lightness gradient is maximum), and the contour line is expanded in two directions where the lightness gradient is maximum. The pixel position on which the pixel which expanded from two different directions collided may be used as the pixel position on the skeleton line.

  In the above-described embodiment, an example is shown in which master data is generated from an image obtained by capturing the master sheet 11a representing a reference character with the TV camera 12, but master data is generated from character font data, It is also possible to generate master data from character data created by CAD or the like. If master data is created using font data, character data created by CAD, etc., unnecessary noise is not mixed as compared to the case where master data is created from an image, and master data can be easily generated.

It is operation | movement explanatory drawing of embodiment of this invention. It is a schematic block diagram of the apparatus used for the same as the above. It is a figure which shows the master sheet | seat used for the same as the above. It is operation | movement explanatory drawing of the master data production | generation registration process in the same as the above. It is operation | movement explanatory drawing same as the above. It is a figure which shows the master data in the same as the above. It is a figure which shows the target sheet | seat used for the same as the above. It is operation | movement explanatory drawing of the target data extraction process in the same as the above. It is operation | movement explanatory drawing which shows the production | generation process of the target cluster in the same as the above. It is operation | movement explanatory drawing which shows the shape similarity calculation process in the same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing which shows the category probability calculation process in the same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is a figure which shows the character string master used for the same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Target object 11a Master sheet 11b Target sheet 12 TV camera 13 Digital image generation apparatus 14 Storage apparatus 15 Image processing apparatus

Claims (13)

Extracts the skeleton line of the character / graphic from the image of the character / graphic input as the recognition target, then approximates the skeleton line as a set of approximate lines that are simple shapes, and approximates the continuous approximate line as one approximation In addition to the line group, when the minimum distance between the approximate lines belonging to each of the pair of approximate line groups that are not continuous is equal to or less than a specified threshold, the pair of approximate line groups is integrated as one approximate line group. approximation approximated as a set of approximate line and the target data extraction step of extracting as a handling songs over target cluster one character or more approximate line group as a unit, a skeleton line shape to be a standard characters and graphics by A shape similarity calculation process for evaluating the similarity of the shape between the master cluster composed of line groups and the target cluster, and the master cluster and the target cluster determine the position and shape of the approximate line. In the shape similarity calculation process, the circumscribed rectangle set for the master cluster is aligned with the circumscribed rectangle set for the target cluster, and then included in the target cluster and the master cluster. A character / figure recognition method characterized in that a shape similarity is obtained from a distance between skeleton lines and a master cluster having a high shape similarity is associated with a target cluster category of interest.   2. The character / graphic recognition method according to claim 1, wherein a straight line and an elliptical arc are used as the approximate line.   In the shape similarity calculation process, the height direction and the width direction included in the target cluster are obtained by determining the ratio of the height dimension and the width dimension between the circumscribed rectangle set for the target cluster and the circumscribed rectangle set for the master cluster. Estimate the number of characters / figures in, align the circumscribed rectangle set for the master cluster for each position corresponding to the estimated number with the circumscribed rectangle set for the target cluster, and obtain the shape similarity at each position 3. The method for recognizing characters / graphics according to claim 1 or 2.   In the master cluster and the target cluster, an isolated point to which no approximate line is connected, an end point that is an open end of the approximate line, a line connection point that connects two approximate lines in a straight line, and two At least one kind of bending point that connects the approximate lines at an angle of 180 degrees or less and a branch point that connects three or more approximate lines is used as a node. In addition to the approximate line data to be specified, node data that specifies the position of the node is included, and in the shape similarity calculation process, before obtaining the distance between the skeleton lines, Determining the type of each node of the target cluster existing within a prescribed distance range from the node, and at least one of the nodes included in the master cluster is 4. If the type of node of the target cluster does not match, the shape similarity between the master cluster and the target cluster is not obtained. How to recognize shapes.   In the master cluster and the target cluster, an isolated point to which no approximate line is connected, an end point that is an open end of the approximate line, a line connection point that connects two approximate lines in a straight line, and two At least one kind of bending point that connects the approximate lines at an angle of 180 degrees or less and a branch point that connects three or more approximate lines is used as a node. In addition to the approximate line data to be specified, node data for specifying the position of the node is included, and in the shape similarity calculation process, before obtaining the distance between the skeleton lines, each node of the target cluster By using the approximate line data of the approximate line and the node data of the surrounding nodes, the data existing within a specified distance range from each node of the master cluster. Obtain the Bayes probability of the node type of the get cluster, obtain the node similarity, which is the similarity between the node type of the master cluster and the node type of the target cluster, using the Bayes probability. 4. The shape similarity between the master cluster and the target cluster is not obtained when the node similarity of a node of the target cluster corresponding to at least one is not equal to or greater than a predetermined threshold value. The method for recognizing characters / graphics according to any one of the above.   In the shape similarity calculation process, the shape similarity is obtained by determining the total extension of approximate lines in which the distance from the approximate line included in the master cluster to the approximate line included in the target cluster is within a specified distance as the master cluster. An evaluation value of the degree of approach between approximate lines obtained by dividing the ratio of the approximate lines included in the total length and the value obtained by subtracting the average distance between approximate lines within the distance from the distance by the distance; The character / graphic recognition method according to claim 1, wherein the character / graphic recognition method is proportional to a value obtained by multiplying.   In the shape similarity calculation process, after aligning the position of the master cluster with the target cluster using the circumscribed rectangle, the position, rotation, and scale so that the approximate line included in the master cluster overlaps the approximate line included in the target cluster. 7. The method according to claim 1, wherein a parameter of a conversion equation for converting at least one element of the linear distortion and the linear distortion is obtained by a least square method, and the shape similarity is obtained after the transformation is applied. Recognition method of written characters and figures.   When there are a plurality of characters / figures input as recognition targets and the number and arrangement are known, in the shape similarity calculation process, one corner position is represented from two or more different circumscribed rectangles. 2. The character / graphic recognition method according to claim 1, wherein the position of the circumscribed rectangle is matched by selecting as a point and overlapping the position of the representative point of the master cluster with the position of the representative point of the target cluster.   When there are multiple characters / figures input as recognition targets and the number and arrangement are known, the rotation position is adjusted so that the area of each target cluster is minimized in the shape similarity calculation process. The circumscribed rectangle is obtained, the angle of the target cluster is detected by using the angle of the axis that maximizes the angle when the circumscribed rectangle corner is mapped onto one axis, and the master cluster is rotated by the angle. The character / graphic recognition method according to claim 1, wherein the position of the circumscribed rectangle is matched.   In extracting the skeleton line, the outline of the character / figure is extracted using a grayscale image, the change direction of the brightness around the starting edge point selected on the outline is obtained, and the known character / Find other pixels on the contour line obtained by searching on a straight line in the direction of brightness change inward of the figure as the end point edge point, and use the midpoint of the start point and end point point as the pixel on the skeleton line The character / figure recognition method according to claim 1, wherein:   In extracting the skeleton line, the outline of the character / figure is extracted using a grayscale image, the change direction of the brightness around the starting edge point selected on the outline is obtained, and the known character / Searches on a straight line in the direction of lightness change toward the inside of the figure, finds other pixels on the outline obtained within the specified distance range as the end point edge point, and skeletons the midpoint of the start point and end point point The character / graphic recognition method according to claim 1, wherein the pixel is a pixel on a line.   In extracting the skeleton line, the outline of the character / graphic is extracted using a grayscale image, the change direction of the brightness around the starting edge point selected from the outline is obtained, and the change direction of the brightness from the starting edge point is determined. Search on a straight line in both directions to find other pixels on the contour line within the specified distance range as the end point edge point, and set the midpoint between the start point and end point point as the pixel on the skeleton line The character / graphic recognition method according to claim 1, wherein the character / figure recognition method is a feature.   In extracting the skeleton line, a contour line of characters / graphics is extracted using a grayscale image, a change direction of lightness around pixels on the contour line is obtained, and at least one of the two directions having the maximum lightness gradient is obtained. The expansion processing for expanding a contour line on one side is performed, and a pixel on which a pixel expanded from two different directions collides is set as a pixel on the skeleton line. Recognition method of characters and figures.

Images