JPH0321811A - Method of determining text azimuth - Google Patents

Abstract

PURPOSE: To hold a required sensitivity to posture while maintaining insensitivity to changes of contents, by determining which of 'upward', 'inverted' and 'uncertain' postures a text has after a predetermined count of symbols are processed. CONSTITUTION: At a step (a), an image of at least one feature on an object or at lest one symbol (symbol or the like) of a text of a predetermined size is extracted. At a step (b), when the extracted symbol or the like does not correspond to a symbol or the like in an image of a second predetermined size, the symbol or the like is normalized in the image. At a step (c), a similar measurement value of vectors is determined and a probability for each symbol or the like to have a predetermined 'up' or 'down' posture is determined. At a step (d), which of the 'up', 'down' and 'uncertain' postures the text or the like has is determined from the probability. At a step (e), which of the 'up', 'down' and 'uncertain' postures the object has is determined from an index of the posture of the text or the like.

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention describes the orientation of features on an object, such as printed text, using a network including a determination device in the form of a forward neural network for determining the orientation of the object. Concerning technology for detection. The C process for assembling 11 views in the 1st ward of departure is “just in time” (Ju
As we move closer to production, automatic inspection becomes a more important technology. For example, a tight loop is created between manufacturing operations and inspection, which eliminates errors caused by manufacturing settings. If a defect occurs, a minimum number of defective items are guaranteed not to be manufactured. This is in contrast to traditional batch manufacturing, where errors are detected only after the loft or batch of products has been manufactured. Rapid detection has other advantages as well, for example, in surface mount assembly of circuit packets, which are first held in place with an element or adhesive before being soldered, this Inspection systems placed in-line after the installation operation can detect errors before the soldering process is performed, thus minimizing repair costs.

(Various configurations for detecting board defects have been proposed. For example, in June 7, 1977, B.H.
．． U.S. Pat. No. 4,028,827 to B.H. Sbarp discloses the presence or absence of at least two different types of light-reflecting surfaces on article-L;
and a video to selectively achieve discrimination between them.
Disclose the system. These systems can be used to inspect circuit paths and solder connections. 6. Another device or J. W. MacFarlane (J.W.MacF
arlane et al., U.S. Pat. No. 4,578,810 issued May 25, 1986;
printed wiring board (printed *iringboar
d, P W B ) defects are detected. These detectors include an array of optical sensors forming a binary image pattern of the PWB for optically detecting printed wiring circuits.

s. p. S.P. Denker et al., Philadelphia, Benichirhania, 1984.
InL
International Test Conference
e). ;ii. The paper, published on pages 558-563, uses machine vision technology to create printed circuit boards.
An automatic vision tester for detecting assembly errors (Ard, PCB) is disclosed. This tester is used to capture an image of a camera or PCB and send an electrical representation of this image to a computer. The computer is. Compares the features of the PCB image with the ideal image stored in memory to detect assembly errors. Another structure is the hybrid circuit.
its. GB). No. 13. In 1987, D.B. J. Svetkov (D.J.Sve
Here, a three-dimensional map of the circuit board under test and a Gray-Scale vision map are disclosed in a paper published by Tkoff et al.
A technique for dynamically inspecting element substrates using internal data is disclosed. This technique is described as being useful for detecting the amount and measuring the orientation of elements, such as half [11 paste.

The prior art detects the various positions of the elements, as well as the connections and whether all elements are placed correctly on the circuit board, or whether the item has no defects or is always identical to the stored image. limited to visible situations. What is missing in the prior art is the ability of the elements themselves to be integrated at various stages of manufacturing, such as loading the hopper, mounting the element on the board before soldering, and mounting the chip on the circuit board. There is no way to obtain a low-cost, reliable, and complete inspection system for determining orientation. Furthermore, another problem with prior art systems is that these systems do not have stored search images to compare detected elements, whereas they do not have marks on these elements, such as date stamps. or due to variations in printing position and style.

The above-mentioned deficiencies and problems of the prior art are solved according to the present invention; and then the orientation of the object itself. In the system according to the invention, it is used to extract images of all or part of an object, or line symbols or individual symbols. The thus separated symbols are then each suitably trimmed and scaled to iE normalized symbols. Optimal Rays decision methods, for example in the form of forward foot neural networks, are then used.
ian DecisionIle thod) determines whether the text orientation is “right-side up”, “upside-down”, or “indeterminate” after a predetermined number of symbols have been processed. intervinate)
Because the present invention determines the orientation of a feature or marking without using a stored ideal image, this may be insensitive to changes in content, e.g., date stamps and type style. on the other hand, retaining the required sensitivity to orientation.

FIG. 1 shows a block diagram of a preferred configuration of an object force position detection system according to the present invention, which system includes the following elements:
For example, reading information on an electrical device or chip, such as printed text or marks, and determining the orientation of this information. This system can be used, for example, in electronic assembly lines where hundreds of elements are often placed on one circuit pack. Within such assemblies, the elements are initially placed into their hoppers, eg, manually, or during set-up or replenishment, orientation errors may occur due to the symmetry of the elements. Many articles have orientation marks, such as notches, diagonals, or dots, or there is currently no standardization of these marks, and when marks are used, they are usually It is very difficult to detect it with a vision system.

Some essential requirements of the front orientation detection problem are: (1) there is no prior knowledge of the exact features or content of the features, e.g., type style or size; To read 〈〈, for example,
Prints are usually of low quality; (3) detection must be accomplished quickly, while (a) the probability of misorientation detection must be as high as possible; and (b) "false discard" ( and (4) many features that are constant or nearly constant over 180 degrees of rotation and can be read correctly from either orientation, e.g. Regarding this latter section,
Approximately 40% of the set of letters A-Z and numbers 0-9 (
In other words, BHIMOQSWXZO l 689) is rotationally invariant or rotationally conjugate-invariant.
invariant). That is, the symbol after a 180 degree rotation or, depending on the typeface used, appears to be rotated with other symbols in a set. For illustrative purposes, the system of the present invention is used to determine the orientation of a chip, or the system of the present invention is used to determine the orientation of a chip, or a system of the present invention is used to determine the orientation of a chip, or a system of the present invention, that
It can also be modified to determine the orientation of any object labeled bottle L. Furthermore, this system
It can be similarly applied to determining the orientation of elements or marks other than text. This system relies on having an arbitrary set of reference images as input.

In the direction detection system shown in Figure 1, (1) video
Camera: (2) a frame grabber for capturing frames of a video picture and associated picture signals, and (3) a real object scale (re
al object scale) 5l2x5
A video imaging device 10 is prepared that includes analog/digital (A/D) hardware to provide a 12 pixel frame. Standard design database 1l
or for storing and providing the exact footprint or position of an individual interrogated device, e.g., a chip assembled on a substrate, with respect to that substrate, and the feature orientation for the precisely positioned or inserted device. used for. Image as a result of an example chip from Video Image Dehys 10 or Adaptive Threshold module
Module, ATM) 12 is provided as one input.

Within ATM 1 2, the footprint of the chip from design database 11 is used to direct attention to a particular element or region of interest on the image of the chip from video imaging device 10, and
The resulting image of the element or region is now, for example, instead of the 256x256 pixel imaging provided by the video imaging device IO. ATM1, containing only 200xl30 pixels
2 converts the resulting element or region image into a binary image, where each pixel contains only O or 1 levels instead of one of 256 play levels as an example. It is also possible to omit this binarization step and process the image as a tally scale data, or in the preferred embodiment, it is possible to omit this binarization step and process the image as a tower scale data. step or used. In general, the automatic evaluation of the appropriate threshold between the original 256 levels and the two binary levels is a very difficult problem; however, in this system, in most cases the text appears as a single color on a contrasting background, and the clay level
In order to have a histogram with two fairly easily determined peaks, a simple histogram method can be used. These peaks are determined by two parameters: radial circumvection
ction ) and mass thresho
ld) interactions adaptively until a satisfactory solution is obtained. Radial circumvection is the radius of approximation in the histogram that is examined to verify that this or some value is a local maximum. In order to constitute a peak, the gray level must be at a local maximum and have a quality that exceeds the mass threshold. This technique can also be modified so that the print is represented as the foreground or "l" and the background as "0", regardless of whether the print is a black clay light or black on light, etc.

The binary image generated by ATM12 is processed by a framer module (Framer Molcle).
It is provided as human power to l3, where the symbols (letters, numbers, etc.) of the image are captured. More specifically, the capture of these symbols can be accomplished by, for example, first taking a horizontal summation of the O's and 1's bitmap matrix of the binary image to extract the line symbols, and then extracting the individual symbols. This is accomplished by taking the vertical summation within the line. For this purpose, quantile thresholds according to the invention are used. More specifically, for each scan line i of the binary image, a sum r[il of a sequence of "l" bits is formed. Thus, r [il includes the peaks of the text columns and the valleys or gaps between the text columns. These peaks and gaps are Te? Sometimes people are jealous of the noise and vibrations of strikes. Therefore, the threshold d is used to separate peaks from gaps. here,
For example, d=δγ■, δ is a predetermined constant, for example, 0.
07, γ1~ is the largest item in the column summation vector [ ]. The process essentially begins at the top scan line, steps through the scan lines sequentially to find the beginning of the peak above a given castle value, and then searches for the valley below a certain area value. Continuing through the peak area,
Extract line symbols. Lines or gaps that appear too small are discarded as being due to noise. These processes are repeated using the summation of the rows of the binary image to extract individual symbols. Note that these horizontal and vertical histogram techniques expose a 90 degree rotation and can be corrected immediately.
In order to filter the tots separated through the process of 2, the framer module 13 or the “
connected dot mass
This process runs the risk of losing some of the valid symbols that are crushed or no negative impact on the recognizer procedure is seen.

The framer module l3 can also be omitted if the symbol is always in place. However, in a preferred embodiment, this module is used to provide IRF capability to inspect electronic elements marked in a conventional manner where the position of the print varies considerably. The framer mosule 13 therefore has a displacement-invariant capability for the entire system.
e@eM-invariance capability
Understand y).

Normalization module
dule, N M ) 14 receives the output from the framer module l3 and "trims" and "scales" the individual extracted symbols. This trimming is performed because the extracted symbols may have associated white or nearly white spaces on the sides, L and/or bottom. Scaling is performed to scale up the symbol image to occupy a predetermined standard size, for example, 24 columns by 16 rows. In some cases, this normalization process
The original image is distorted. For example, a thin character, such as the character “■”, or a “bloated” character (f) that occupies 16 lines.
However, this is not a problem since both the input data and the reference vector described below are transformed in the same way. Alternative methods have the potential advantage of being able to tolerate misalignment due to vertical misalignment of the reference and sample images.Normalization module l4
can be omitted if the symbol is always a given size. However, in a preferred embodiment,
NMl4 provides scale-invariant ability (sca
le-invariance capability
) is used to give. The signal representing the normalized symbol is then sent to a decision module l5, which uses the design database 11 information to determine whether the symbol orientation is "L" or not.
A determination is made whether it is facing "down" or "indeterminate". As will be explained later, its particular form is ``cededforward.
Optimal DeLecLi+, also called F F )'' neural network method,
A preferred configuration for the determination module l5 using the O D ) method is to measure the "similarity measurements" between the captured symbols and the reference image bitmap using, for example, a look-up table.
．． For example, Hamming distance (Ilammingdis
tance), it is possible to determine whether the direction of the symbol is "up", "down", or
Calculate the probability of “uncertainty”. A symbol, eg, a letter, is said to be oriented "up" when it is in its normal orientation, and is said to be oriented "down" when it is in an inverted position.

The object, or chip, has normally oriented text pointing in the "up" or "down" direction. A normally oriented chip is here “right-side up”
A chip with an incorrect orientation is called a “reverse orientation (
upside down)''.

More specifically, the observed image is a bitmap of length N. In other words, Ω={0.1}'? is the vector of When the direction is “-L” U, u2, U■
, and when facing downward, d1, d2, . ．． ．．
It is assumed that there is a set of reference images or symbols (i.e., letters, numbers, l°11 logs of various type sizes) that appear as d. It is also assumed that there is a distortion process that represents both noise inherent in the image, noise due to the image capture process, and variations due to the use of type styles, sizes, etc. that are not within the reference set.

These distortion processes will hereinafter be denoted by p(xly), which indicates that the reference vector y is distorted into the observed vector X with probability p (x I y). Also, p(xlu) or p(x I d) is
The probability that vector X will be observed is given by 5. Here, the vectors X are respectively ul-. '2,. ．． u3
Or d,,d2, . ．． -d. is the distortion of a randomly selected symbol from the reference set of . Therefore, the probabilities p(xlu) - Σp(xlu,) p(uI)+-1 and p(xld) − : p(xld;) p(d:). +
−1, where p (u+) or p (dt) is the symbol u. when the symbol has an “up” or “down” orientation, respectively. Alternatively, d+ is the app slip probability used. The symbols in the above reference set, for example, consist of letters AZ and numbers 0-9 of various sizes, type styles, and orientations. Whether the scaling operation is performed within the normalization module l4, the presence of multiple sizes of the reference set, or the typeface, typically scale invariant (
Note that this is required because it is (scale-invariant). One formulation of this problem is therefore the decision of Ω "-h (1111)", "down" or "indeterminate".
The problem is to find a partition into regions Ω1, Ω4, Ωi that represent “false discards (false discards)”.
The objective is to maximize the probability of an IEL/I determination of orientation within the limits of the probability of "reject)". This leads to the following practical technique for optimal determination of orientation of a single symbol.

This technique outputs:

If p (xld)/p (xlu)≧in, then d(“
(lower'), if p (x l d)/p (x l u)≦in-1, then U (“Lx”), otherwise, l (“uncertain”) Next, the parameter input It is adjusted to be ≧1 or as small as possible from analysis or experiment, and to avoid an excessive number of erroneous discards.

Multiple symbols: x (1). x (2), . ．．
．． It is necessary to determine the orientation for the chip consisting of x(L). Therefore, the following conditional independence assumption is adopted. This optimal test then gives the following output:

d(“bottom”) [ p(x(1) l d)/ p(x(
1) l u)≧λ(L),++1 In other cases, l (uncertain).

If the samurai assigned to inspect the board are sufficiently available, best results are obtained by examining all symbols found on object E. However, it is usually possible to obtain a very high degree of certainty about orientation without reading every symbol.

This is from the book Sequential Analysis by A. Wald.
l^nalysis) 1, Dower Publications, 1
947 suggests using the sequential test procedure described for Page 34-243. In this method, symbols are read until the cumulative product of probability ratios exceeds some upper bound or falls below a lower bound - 1, at which point a determination of orientation is made. This potentially achieves a significant speed-up of the process when measured in average time per chip. However, if there is a bias regarding the previous distribution or lack of knowledge about the distribution, It is necessary to use a rapidly growing input function or to restrict this distribution to the product of any one observation. An exemplary configuration for implementing a forward neural network to perform the process described above in decision module l5 is shown in FIG. In the second V, the input from the normalization mosule l4 has separate elements of the bitmap (matrix) for the normalized symbols, which include from Xt to x. The symbol of 6 is given
Each bitmap element X, has as input an individual input unit (Inputυnip. I U) 20+
Sent from 20H. More formally, if the individual normalized symbols are arranged to be placed in a bitmap matrix as an example of 24x l6 elements,
N will be equal to 384 elements, so the decision module l5 includes 384 input units 20. Individual input units 20. The output from M pattern units 21. It has been distributed by 2l. Here, it is used, for example, to determine the probability of the first half or less of M pattern units 2l. and these values P (u, ) and p (d.) can be obtained, for example, from field experiments, or by using uniform probabilities throughout, i.e. by making all symbols identical in any orientation. This is the a priori probability that can be obtained by taking the probability of . These values P(Xl
In order to obtain u1) and p(xld.), it is necessary to make some assumptions about the strain process. As a simple example, Motel has (1) probability g (k),
The k bits of the bitmap are changed, and (2) we make a decision that these bits change according to independent and co-partitioning trials. Therefore, h (x, u,) is the vector X and U. represents the Hamming distance between The neural network diagram for this procedure is shown in Figure 2. The first active layer consisting of pattern units 2l has p(xlu.) according to the above formula.
and p(x1d.). The next layer consisting of summing units 221 and 222 has probabilities p(xlu) and p(
xld) and the sequential decision unit 23 provides the final decision for the chip. The parts of FIG. 2 and the terminology used are as follows: D. F. D.F.Spe
chL), IEEE
A- (The ProceediBs of
the IEEE International Co.
nferenceOn Neural Network
s), June 1988, San Diego, Caliphate, Vol. 1, similar to that used in papers published in Basis 1-525 to I-532. However, the internal structure of this pattern unit is very different. Figure 2 immediately suggests a parallel realization. Note that the algebraic quantities shown on the interconnects in FIG. 2 are the values bused between these elements, and the quantities enclosed in parentheses are multiple weights.

The internal structure of this pattern unit can be either serial or parallel. In the remainder of this paragraph, we refer to “silarity sea
pattern such that ``sure'' is the Hamming distance.
We will explain how to fully realize unit 2l.
In the absence of dedicated hardware to calculate Hamming distances, there are techniques that can be employed to quickly calculate them. In this method, the Hamming distance between bitmaps is determined using a precomputed table lookup that takes a bitwise "exclusive OR" and then returns the number of 1 bits in the resulting word. is calculated one word at a time. This reduces Hamming distance calculations to a few machine instructions per word, allowing modern microprocessors to calculate distances on the order of millions of words per second.

After the operations in elements 20 to 22 of FIG. 2 are completed, output probability values are obtained for each symbol. These are combined using the FF method using a multiplier in the sequential H constant unit 23. This result is then compared to the upper and lower thresholds in the sequential decision unit 23 to generate an overall decision or indeterminacy for that chip. This procedure proceeds according to the "Stottsbing" rule. In other words, the processing of the symbols on the chip is completed when a predetermined certainty is obtained. @Later, the result for this chip is stored in the comparator 24,
It is compared with the correct orientation stored in the “Design Database” and the settings, e.g.
on errors plus instructions
), an output signal is generated to, for example, activate or deactivate an alarm.

The above description of forward foot neural network methods is for illustrative purposes only and is not intended to be limiting; other suitable methods may be used to produce a given decision. Ru. For example, T. Koenen (T. Ko
hnen) in his book ``Self-Organize and Associative Memory (Self-Organize Lion and
Associative Mexxory) J.
72nd edition, Springer Harleg
r-Verlag). A learning vector algorithm similar to the method described in Pages 199-209
ing Vector Quantization,
It is also possible to use a It is not an explicit probability value as in the FF method.

[Brief explanation of the drawing]

Figure r5l is a block diagram of a preferred implementation of the object orientation detection system according to the present invention;
1 is a block diagram of an exemplary architecture for a decision module used in the illustrated system; FIG. <Explanation of symbols of main parts> Video imaging device adaptive agony value module framer module normalization module judgment model setting module l O l 2 l 3 l 4 l 5 l1

Claims (10)

[Claims] 1. A method for determining the orientation of a feature or text on an object, the method comprising: (a) a predetermined size of at least one symbol of the feature or of the text located on the object; (b) extracting the at least one feature or symbol extracted in step (a) into a normalized feature or symbol in a second predetermined sized image; (c) normalizing each of the at least one feature or symbol obtained in step (a) in a second predetermined sized image if they do not already correspond; (c) step (a); processes the predetermined sized image for each of the at least one feature or symbol obtained in (b) to determine a similarity measure between vectors, and from this the at least one feature or (d) determining the probability that each of the symbols has a predetermined "up" or "down"orientation; (d) determining the probability from the probability for each of the at least one feature or symbol obtained in step (c); Is the feature or text facing “up”?
(e) determining whether the orientation of the object is "upward" from the feature or text orientation indicator obtained in step (d); The method includes the step of determining whether the object is "inverted,""inverted," or "indeterminate." 2. Step (a) further comprises: (a1) summing the binary values for the individual columns of the bitmap of the feature or text and adding a threshold value to the summation of the columns to determine the position of the feature or text on the line; and (a2) summing the binary values for each row of the bitmap of the feature or text and adding a second threshold value to the summation of the rows; and (a1)
A method according to claim 1, characterized in that it includes the substep of determining the position of individual features or symbols of text in relation to the results of the process. 3. Step (c) further comprises (c1) bitwise "exclusive OR" of the individual normalized extracted feature or text symbol bitmaps obtained in step (a) or (b). and a table lookup operation by measuring the Hamming distance between the input vector containing the distortion from a randomly selected reference set of features or symbols and each reference set of features or symbols. a sub-step of determining the probability that the vector is in a particular "up" or "down"orientation; and (c2) in response to step (c1), determining the probability that the extracted symbol is in an "up"orientation; 3. Method according to claim 1, characterized in that it comprises a sub-step of determining the probability of being "downward" or "uncertain". 4. In carrying out step (d), the process of multiplying the probability determined for each of the sequential symbols processed in step (c) by the product of the probabilities of all other previously processed symbols or text The method further includes a substep that continues until a certain threshold value indicating whether the orientation is "up", "down", or "uncertain" is achieved. The method according to claim 1 or 2. 5. In carrying out step (d): (d1) determining whether the orientation of each symbol of the text is "up", "down", or "indeterminate"; and ( d2) Step (d1)
Is the direction of the text based on the consensus of decisions made in “?
Is it facing “up”, facing “down”, or “determined”
3. A method according to claim 1, further comprising the substep of determining whether the . 6. An apparatus for determining the orientation of features or text on an object, the apparatus comprising: extracting symbols of features or text from an image and generating a bitmap of standardized size for each extracted symbol; means for generating an output signal representing the normalized size for each extracted feature or symbol; and a determining device, the determining device responsive to the output signal from the extracting and generating means. The similarity value between the bitmap vectors is determined, and for each extracted feature or symbol, whether their orientation is "up", "down",
or, in response to the output signal of the first means, the orientation of the feature or text as a whole is "up"; Apparatus characterized in that it comprises second means for producing an output signal indicative of the probability of orientation, "down" orientation or "indeterminate". 7. 7. Apparatus according to claim 6, characterized in that the determining apparatus further comprises third means for determining the orientation of an object from the output signal of the second means. 8. The extraction and generation means generates a sum of binary values for each column of the bitmap of the image of the feature or text, and determines the position of the line within the feature or text by adding a threshold value to the sum of the columns. a first means for generating an output signal indicative of the feature or text; 8. The method according to claim 6, further comprising second means for determining the position of an individual feature or symbol within the text from both the output signal from the first means and the summation of the rows. The device described. 9. The second means of the determining device is further responsive to the output signal from the first means of the determining device;
The probability determined for each sequential signal is multiplied by the probabilities of all other previously processed features or text symbols, depending on whether the orientation of the feature or text is "up" or "down"; or multiplying until a certain threshold value indicating "uncertainty" is achieved, at which point it includes means for producing an output signal representative of the determination of the orientation. 7. The device according to 7. 10. The second means of the determining device further determines whether the orientation of the individual features or symbols of the text is “
means for generating an output signal indicating whether the feature or text is oriented "up", oriented "down", or "indeterminate", and whether the orientation of the feature or text is "normal"; , "inverted" or "indeterminate" means for consolidating the output signal from the generating means of the second means to determine whether it is "inverted" or "indeterminate". Section 6
8. The device according to 7.