JP3053512B2 - Image processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention digitizes an image input from a two-dimensional imaging device such as a television camera,
The present invention relates to an image processing apparatus for recognizing and outputting the number, type, normality / abnormality, and the like of an object existing in a captured scene.

[0002]

2. Description of the Related Art An image processing apparatus that digitizes an image input from a television camera or the like, recognizes the number, type, normality / abnormality, and the like of an object present in a captured scene and outputs the image is provided by: It is used for discriminating and inspecting the type of work on a production line, and also for automating remote monitoring of a plant or the like.

FIG. 40 shows a spatial band-pass filter and 2
FIG. 3 is a block diagram illustrating a configuration of an image processing apparatus that generates a ternary image based on three thresholds, recognizes a target existing in a captured scene, and outputs the recognition. In the figure, reference numeral 10 denotes image digitizing means for digitizing an image input from a television camera or the like, and 11 denotes a digitized original image. Numeral 12 is an image reducing means for reducing the digitized original image, and 13 is a switching means for switching to either the digitized original image or the original image reduced by the image reducing means 12 and outputting to the filtering means 20. .

The filtering means 20 performs a filtering process on the original image output from the switching means 13. 21 is an optimum threshold value determining means for determining an optimum threshold value, 3
0 is a ternary processing means for converting the output of the filtering means 20 into ternary data by two thresholds, and 31 is a ternary processing means.
A ternary image 32, which is a multi-resolution scene representation data represented by a value, represents a 3D image based on the cutout address specified by the cutout position scanning means and its control means 33.
A partial image cutout means for cutting out a partial image from the multi-resolution scene expression data expressed by a value; 56, a ternary standard pattern created and registered in advance; 58, a partial image and a standard pattern 56 cut out by the partial image cutout means 32; The two-dimensional pattern coincidence counting means 59 for counting the two-dimensional pattern coincidence with the above-described two-dimensional pattern coincidence counting means 59 calculates the best coincidence result by using a similarity map created based on the coincidence counted by the two-dimensional pattern coincidence counting means 58. This is a peak evaluation unit that outputs the obtained position and the similarity at that position as a recognition result.

Next, the operation of the image processing apparatus will be briefly described. From the two-dimensional image digitized by the image digitizing means 10, only a certain spatial frequency component is extracted by the filtering means 20, and then converted to a ternary image 31 by the ternary processing means 30. Then, in the same manner as the ternary image 31, a map relating to the similarity created and registered in advance is sent to the peak evaluation means 59, and the position at which the best matching degree is obtained and the similarity at that position are output as a determination result.

The above-described image processing apparatus is configured to use a ternary image, but there is also an image discriminating technique that does not use such a ternary image. Examples of such prior art include magazine movie information (Nobuyuki Otsu, adaptive learning type general-purpose image measurement and recognition system, movie information vol. 121, p.
p. 41-46, 1989), there is an image discrimination technique based on the appearance frequency of a local pattern.

The image discrimination technique will be described with reference to FIG. 41, the same or corresponding parts as those in FIG. 40 are denoted by the same reference numerals, and description thereof will be omitted. In the figure, a digitized original image 11 as an input image is binarized by a binarization processing unit 35 and a feature vector extraction unit 40
25 local feature detecting means 41 operating in parallel
1, 41-2, ..., 41-25. The local feature detecting means 41-1 to 41-25 generate an output when a white pixel as shown in FIG. 42 is generated, and this output is output from the local feature counting means 42-1 and 42-2.
.. Are counted by a counter provided in 42-25.
This results in a 25-dimensional feature vector 49 specific to a particular screen.

At the time of standard pattern registration, a teacher signal as a judgment result is input to the feature vector discriminating means 50 together with the feature vector 49. The feature vector discriminating means 50 uses these learning vectors (that is, the local feature detecting means 41 and the local feature counting means). 42 are subjected to statistical processing by multiple regression analysis to generate multiple regression coefficients that can provide a desired determination output with respect to an arbitrary feature input. At the time of performing the recognition, a discrimination result is obtained from the feature vector 49 and the generated multiple regression coefficient.

[0009]

Since the conventional image processing apparatus is configured as described above, in the prior art 1, an object having a pattern similar to a standard pattern registered in advance is extracted from an image. However, when the extracted object has a pattern that is locally different from the reference pattern, an algorithm that is suitable for classification and inspection, such that the difference cannot be distinguished from the rough pattern difference, is used. Since it is not provided, there is a problem that it is not suitable for applications such as product inspection on a production line and abnormal monitoring of a plant.

In the prior art 2, the discriminant function can be automatically determined without requiring procedural programming or the like. On the other hand, since the feature input relies on local features of the binary image, When binarizing with a single threshold, a characteristic pattern does not appear, or a pattern necessary for discrimination is lost by binarization. That is, when an image having a plurality of gradations as shown in FIG. 43 (a) is simply processed with one threshold value, the threshold value may be selected no matter how it is devised. ), It is difficult to simultaneously extract a black circular area inside the right object and a white circular area inside the left object. As described above, there is a problem that the processing ability is low and not practical in a general general scene.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to filter a digitized original image so as to be able to describe local features of a ternary image. Thus, it is possible to obtain an image processing apparatus capable of reliably determining and recognizing a local difference of an object, and improving the ability to perform processing for identifying and recognizing the object in an image of a general scene having a plurality of gradations. The purpose is to:

[0012]

According to a first aspect of the present invention, there is provided an image processing apparatus comprising: a filtering means for applying one or more kinds of spatial bandpass filters to a digitized input image; Ternary processing means for processing the output with two threshold values to obtain a ternary image; local feature detecting means for detecting the similarity between the ternary image and the local pattern; when the similarity is equal to or greater than a certain value Feature vector extracting means having a plurality of local feature counting means for counting the number of points at which the output of the threshold means occurs in an image, and storing the output of the feature vector extracting means in advance. The degree of similarity to one or more standard vectors that have been
Feature vector discriminating means for outputting as a final discrimination result.

[0013] The image processing apparatus according to the second aspect of the present invention comprises:
The feature vector extracting means includes local feature detecting means for detecting the similarity between the value image and the local pattern, and a plurality of feature accumulating means for accumulating the similarity between the local pattern and the image in the image.

An image processing apparatus according to a third aspect of the present invention uses a linear discriminant function represented by a linear sum of the feature vectors when the feature vectors can be linearly separated, and determines in advance to generate the linear discriminant function. A regression analysis means for performing a multiple regression analysis using a plurality of learning image data whose results are known and the discrimination result, and using the result of the multiple regression analysis as a weighting coefficient of a linear discriminant function, and a discrimination result from the weighting coefficient and the feature vector And a determination unit that outputs

According to a fourth aspect of the present invention, a group to be determined is expressed as a union of several groups,
A super-group forming means for forming an individual group while sequentially updating the individual group by using a plurality of pieces of learning image data whose discrimination results are known in advance and the discrimination result is provided.

In the image processing apparatus according to the present invention, when the monitor determines that the discrimination result is incorrect during the recognition processing, the standard pattern is recalculated by inputting a desired discrimination result, and the operation is performed. While improving the reliability of the judgment.

In the image processing apparatus according to the present invention, the data is normalized in each dimension of the multidimensional feature vector at the time of learning the standard pattern, and the variance of the center vector representing the group is set to any dimension. It is provided with a normalization processing means for obtaining the same value.

An image processing apparatus according to a seventh aspect of the present invention performs learning by using a feature vector obtained from other sensor information or a global feature of an image in addition to a feature vector obtained from a local feature of an input image. The determination is performed by configuring a feature space.

An image processing apparatus according to an eighth aspect of the present invention classifies features obtained from other sensor information or global features of an image into a plurality of classes in addition to feature vectors obtained from local features of an input image during learning. It has a large classification class generating means, selects the corresponding class from the learning data using the same features at the time of recognition processing, and performs recognition using only the learning data stored in the selected class.

An image processing apparatus according to a ninth aspect of the present invention has partial image division scanning means for dividing an image into a plurality of sections in advance, and the feature vector determination means prepares learning data for each of the divided sections. , Learning and recognition are performed for each section.

According to a tenth aspect of the present invention, there is provided an image processing apparatus comprising:
Weighting factor generating means for setting a lower weighting factor than the center of the image area at the periphery of the image area divided into a plurality of areas; and a local feature counter for counting the output obtained by adding the weighting factor to the output of the local feature detecting means. Means and a feature vector determining means to which the output of the local feature counting means is supplied as a feature vector.

An image processing apparatus according to an eleventh aspect of the present invention
Position detection means for performing registration with a standard image at a plurality of measurement points on the image, and transformation parameter generation means for obtaining a geometric transformation parameter for bringing the image closest to the standard image from the output of the position detection means And a geometric transformation unit that geometrically transforms the image based on the geometric transformation parameter generated by the transformation parameter generation unit and uses the image after the geometric transformation as an input image.

According to a twelfth aspect of the present invention, there is provided an image processing apparatus comprising:
A local feature detecting means for detecting a similarity between a ternary image obtained from an input image and a plurality of local patterns registered in advance, a local feature counting means for counting the output thereof, and A partial image cutout scanning means for limiting the position of the limited area on the screen,
The output of the local feature counting means is an input feature vector,
A feature vector discriminating means for evaluating a similarity between a pre-stored standard pattern and outputting a class to which the input feature vector belongs and a position on the image of the input feature vector belonging to the class; and outputting a feature map. Feature selection means, a combination between the feature map and the standard local feature map that can be associated with each other, and a feature map correspondence search means for obtaining a deformation parameter required at that time, and a standard object pattern selected from the deformation parameters. And a geometric conversion parameter generating means for generating and outputting a geometric conversion parameter between object patterns on an image.

[0024]

According to the first aspect of the present invention, the image processing apparatus detects the similarity between the ternary image and the local pattern and extracts the similarity based on the result of counting the number of occurrences of portions having a similarity equal to or greater than a certain value. It works so as to obtain the final discrimination result based on the similarity between the vector and the previously stored standard vector.

The image processing apparatus according to the second aspect of the present invention
It works so as to obtain the object discrimination result based on the similarity between a feature vector extracted based on the similarity of the local feature in the image or the accumulation result thereof and a standard vector stored in advance.

According to a third aspect of the present invention, there is provided an image processing apparatus comprising:
In order to generate a linear discriminant function, a multiple regression analysis is performed using a plurality of learning image data whose judgment results are known in advance and the discrimination result, and the result of the multiple regression analysis is used as a weight coefficient of the linear discriminant function. The analysis is performed, and acts so as to obtain a discrimination result from the weight coefficient and the feature vector.

According to the fourth aspect of the present invention, the super group forming means forms a plurality of regions in the feature vector space and determines the union of the regions as one group. Even when it is not possible, the result of the determination can be obtained by the unified determination criterion without performing a procedural conditional branch, and the ability to perform the determination and the recognition of the object is improved.

According to a fifth aspect of the present invention, there is provided an image processing apparatus comprising:
If it is determined that the discrimination result is incorrect during the discrimination recognition process, the standard pattern is recalculated by inputting a desired discrimination result without re-reading all the learned data, and the correction of the standard pattern is performed. Since it becomes possible, it acts to improve the reliability at the time of discrimination.

According to a sixth aspect of the present invention, there is provided an image processing apparatus comprising:
During standard pattern learning, data is normalized in each dimension of the multidimensional feature vector, and the variance of the central vector representing the group becomes the same value in any dimension, which simplifies the calculation, , The recognition processing ability for the object is improved.

According to the seventh aspect of the present invention, the comprehensive feature vector generating means of the image processing apparatus performs learning using not only local image features but also features obtained simultaneously from other sensor information and global image features. Then, a multidimensional feature space is constructed using these features and discrimination is performed, and a highly reliable discrimination of the object in the image is performed in accordance with a more complicated situation, thereby improving the recognition processing capability.

An image processing apparatus according to the invention of claim 8 is:
At the time of learning, features obtained from other sensor information other than local features of the input image or global features of the image are classified into a plurality of classes and learning data is obtained. Since the recognition process is performed using the selected and the learning data stored in the selected class, highly reliable determination of the object in the image is performed to adapt to more complicated situations, and the recognition processing capability is improved. Will do.

According to a ninth aspect of the present invention, there is provided an image processing apparatus comprising:
The image is divided into a plurality of sections, and learning and recognition are performed for each of the divided sections. By focusing on a partial area in the image, the local features of the surrounding object image are included in the determination result. This has no effect, and as a result, a highly reliable determination of the object in the image can be made.

In the image processing apparatus according to the tenth aspect of the present invention, the weight count given to the peripheral portion of the partial area is made lower than the weight count given to the central portion, so that the shape of the object is slightly changed or the position is changed. The misalignment causes the corresponding feature to be included or not included in the region at the time of learning and at the time of recognition, and acts to prevent a reduction in recognition reliability.

In the image processing apparatus according to the eleventh aspect of the present invention, the displacement of the object between the image at the time of learning and the image at the time of recognition is measured at a plurality of measurement points. Since the image is geometrically transformed and determined using geometric transformation parameters such as movement, rotation, and enlargement / reduction, fluctuations in the feature vector due to image overlay errors are suppressed, and highly reliable determination is possible. Become.

The image processing apparatus according to the twelfth aspect performs the correspondence between the standard pattern and the object in the picked-up image at the local feature level of the ternary image, and after the object in the partial image is determined, A combination between the feature map and the standard local feature map that can be associated with the feature map and the standard local feature map without any inconsistency is obtained, and the geometric transformation parameters and the above determination result are output as image processing recognition results. Thus, local feature pattern collation can be performed in a state in which the influence of peripheral objects in the partial image is eliminated, and a highly reliable determination result can be obtained.

[0036]

【Example】

Embodiment 1 FIG. An embodiment of the first aspect of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram illustrating the configuration of the image processing apparatus according to the present embodiment. 40 or 41 in FIG.
The same or corresponding parts are denoted by the same reference numerals and description thereof is omitted.

The image processing apparatus of this embodiment is shown by the block diagram of FIG. 1. As is apparent from comparison with FIGS. 40 and 41, the image digitizing means 10 in FIG. It has a configuration up to the ternary image 31, which is the multi-resolution scene expression data, and a configuration from the feature vector extracting means 40 to the determination result 59 shown in FIG. The image processing apparatus of the present embodiment compensates for the drawbacks of the conventional image processing apparatus, reliably discriminates and recognizes a local difference of an object, and detects an object in an image of a general scene having a plurality of gradations. An object of the present invention is to improve the discrimination and recognition processing ability for an object.

Next, the operation will be described. An image captured by an image capturing unit such as a television camera (not shown) is converted into an original image 11 digitized by the image digitizing unit 10. The original low-frequency component and high-frequency component of the original image 11 are removed by the filtering unit 20. Next, -1 / 0 /
It is converted into a ternary image 31 represented by 1.

FIG. 2 shows how the ternary image 31 is generated. In FIG. 2, for the sake of simplicity, the spatial distance of the image is shown only in the X-axis direction. If a certain X-axis section of the two-dimensional original image 11 is an image including four stepwise edges as shown in FIG. 2A, the output of the filtering means 20 is shown in FIG. The waveform becomes as shown. Next, this waveform is divided into two thresholds + δ and −
By performing the threshold processing with δ, it is possible to obtain a ternary output as an edge expression having a wide ternary width as shown in FIG.

On the other hand, FIG. 2D shows an original image in a conventional image processing apparatus, and FIG. 2E shows a binarized image obtained by subjecting the original image to threshold processing with one threshold value δ. In image processing using such a binarization method, FIG.
(E), only a specific part of the luminance change in the image is extracted.

Returning to FIG. 1, the processing of the ternary image 31 will be described. This ternary image 31 is input to the feature vector extracting means 40. The feature vector extracting unit 40 includes a local image cutout position scanning unit 39 and a local feature detecting unit 4 that operates in parallel according to the number N of elements of the feature vector.
1-i and the local feature counting means 42-i function in combination. This parallelization may actually be configured by operating a plurality of circuits in parallel as hardware,
The processing may be performed in time series while sequentially switching the features detected by the local feature detection unit. Of course, if the processing contents are the same, the filtering means 20 and the ternary processing means 3
It may be realized by computer program processing including the zero part.

Next, the combined operation of the local feature detecting means 41-i and the local feature counting means 42-i will be described. The local feature detecting means 41-i generates an output only when the ternary pattern around the pixel of interest in the image becomes a unique combination. Then, based on the output of the local image cutout position scanning means 39, the frequency of a pattern having a certain characteristic generated by scanning the target pixel over the entire image is determined by the counter of the local characteristic counting means 42-i. Count. By changing the ternary local pattern to which the local feature detecting unit 41-i reacts, an N-dimensional feature vector 49 corresponding to the number N of local patterns is obtained as an output of the feature vector extracting unit 40.

Next, the local feature detecting means 41 will be described with reference to FIGS. As shown in FIG. 7, the registered local pattern g (x, y) and the partial image f (x, y) cut out from the periphery of the pixel of interest (i, j) of the ternary intermediate image f (x, y) i + δi, j + δj), and a pattern detection output is determined based on the following equation (1).

[0044]

(Equation 1)

However, as shown in FIG. 8, the operation ○ is “+1” when the pixel value of the intermediate image f represented by the ternary value as the input pixel value matches the pixel value of the local pattern g, and “0” when the difference is 1 "-1" if the difference is 2, and "0" regardless of the pixel value of the intermediate image f if the pixel value of the local pattern g is undefined.
Is output. Regarding the coincidence determination operation, not only the operation as shown in FIG. 8 but also the operation as shown in FIG. 9 can obtain similar results. Also, t in the above equation (1)
threshold means a threshold for local pattern detection,
For example, if a value equal to or more than a certain ratio is obtained by subtracting the area of the pixel having an indefinite value included in the local pattern from the area of the local pattern, the local pattern is determined to be present. However, it is desirable to select an appropriate value such as “100” when the local pattern is small and “50” when the local pattern is large.

Next, FIG. 3 will be described as a specific embodiment of the local pattern. In the figure, the local pattern g
1 indicates that the target pixel is [white] and is "+1", and the other pixels are [gray] and is "undefined". Accordingly, the detection result of the local pattern g1 is stored in the counter 44 of FIG.
The output obtained as a result of counting is "+1" in the entire screen.
Is equal to the number of pixels having a value of

Similarly, the count result of the local pattern g2 is the number of pixels having a value of "-1" in the entire screen.
The local patterns g3 to g18 are local patterns for examining + 1 / -1 combinations of adjacent pixels, respectively, and are equivalent to measuring the number of edges for each direction.

FIG. 4 shows another example of the local pattern.
This will be described based on FIG. In the figure, the local pattern g101 is
This means that there is a pair of vertically consecutive “+1” pixels and “−1” pixels, which corresponds to detecting a vertically continuous contour. Similarly, local patterns g102 to g104 are modifications of local pattern g101, and local patterns g101 to local pattern 1
Using the four local patterns of 04 and the similar four local patterns obtained by exchanging + 1 / -1 is equivalent to measuring the length of a continuous contour in each direction.

Next, still another embodiment of the local pattern will be described with reference to FIG. In the figure, the local pattern g
Reference numeral 201 can detect an isolated point whose center is bright and whose periphery is dark. Conversely, the local pattern g202 can detect an isolated point whose center is dark and whose periphery is bright.

Next, another embodiment of the local pattern will be described with reference to FIG. In the figure, the local pattern g
Reference numeral 301 can detect a vertical line element having a width “3” whose center is bright and its periphery is dark, and conversely, the local pattern g302 detects a vertical line element whose center is dark and its periphery is bright. Similarly, the local pattern g303 can detect a horizontal line element having a width of “3” whose center is bright and whose periphery is dark, and conversely, the local pattern g30
Reference numeral 4 denotes a horizontal line element having a width "3" in which the center is dark and the periphery is bright.

As described above, various features of an image can be described in three values using the local patterns shown in FIG. 3, FIG. 4, FIG. 5, FIG.

Here, returning to FIG. 1 again, the feature vector 4 which is the total of the local features in the image generated in this way is shown.
9 is input to the feature vector determining means 50. The feature vector determining means 50 performs matching between the feature vector 49 and the standard pattern 56. For example, at the time of pattern learning, if m learning data, that is, the feature vectors ci at the time of learning belong to the class k after discrimination, an average vector c * k shown in the following equation (2) is created, and 58 are stored as k standard patterns 56.

[0053]

(Equation 2)

At the time of recognition, this standard pattern is read out from the newly input feature vector c by the path 57, and the distance between the feature vectors is measured for each of the k standard patterns by the following equation (3). Then, the class k of the closest vector is output as the final judgment result 59.

[0055]

(Equation 3)

As shown in FIGS. 3, 4, 5, and 6, a local pattern is detected by using a local feature having a certain large size. There is a need. However, the comparison with a local feature having a large size requires a large number of repetitive operations in the above equation (1), uses a large-scale hardware, and requires a long detection time when executed by software. There is a problem.

For this reason, in this embodiment, the image reducing means 12
Is used, it is possible to detect a large coherent local pattern even when only a small-sized local feature detecting means is provided. For example, the original image 11 is input to the feature vector extracting unit 40 after averaging and resampling the original image 11 by 1/2 using the image reducing unit 12. Here, when the local pattern is a 3 × 3 pattern as shown in FIG. 3, it is equivalent to a 6 × 6 local pattern when represented by the resolution at the time of returning to the original image. Accordingly, a local feature having a large size can be detected by a local feature detecting means having a small size, and a complex image feature vector is created by switching an intermediate image when the image is reduced or not, thereby creating a more complex pattern. And a highly reliable determination result can be obtained. Further, the ternary threshold value δ can be automatically determined from the output histogram of the filtering means 20 by, for example, setting it to a constant value of the maximum value, and can more flexibly follow the brightness change of the entire image. become.

Embodiment 2 FIG. Hereinafter, an embodiment of the second aspect of the present invention will be described with reference to the drawings. FIG. 10 is a block diagram showing the configuration of the image processing apparatus according to the present embodiment. In FIG. 10, the same or corresponding parts as those in FIG. In the figure, 43-1, 43-2, 43
43 ... 43-N is local feature accumulation means. The local feature accumulating means accumulates the similarity between the ternary image detected by the local feature detecting means 41-i and the local pattern in the image.

Next, based on FIG.
0 will be described. As shown in the figure, the registered local pattern g (x, y) and the intermediate image f expressed in three values
(X, y) is overlapped with a partial image f (i + δi, j + δj) cut out from the periphery of the pixel of interest (i, j), and the pattern detection output E (i , J).

[0060]

(Equation 4)

However, the operation ○ is a coincidence degree judgment operation as shown in FIG. 8 or FIG. In this way, the local features in the image are accumulated in the adder 45 and the feature vector 49
Is generated as The feature vector 49, which is the sum of the local features in the generated image, is calculated by a feature vector determination unit 50.
Is input to The feature vector discriminating means 50 performs matching between the feature vector 49 and the standard pattern, and calculates k
The distance between the feature vectors is measured for each of the standard patterns, and the class k of the closest vector is output as the final determination result 59 based on the measured distance between the feature vectors.

Embodiment 3 FIG. Hereinafter, one embodiment of the third aspect of the present invention will be described with reference to the drawings. FIGS. 12 and 13 are block diagrams showing the configuration of the image processing apparatus according to the present embodiment.
13, the same or corresponding parts as those in FIGS. 1 and 10 are denoted by the same reference numerals, and description thereof will be omitted. In this embodiment, the feature vector determination means 50 includes a regression analysis means (determination reliability improving means) 55, a standard pattern 56, and a determination unit 54.

The feature vector discriminating means 50 performs the same operation as the statistical discriminating feature extraction based on the multiple regression analysis in the conventional image processing apparatus shown in FIG. 41. To learn, i
For the second example, the input of the N-dimensional feature vector 49 is the vector xi = (xi, 0, xi, 1,..., Xi, N
-1), and when the output of the desired discrimination result is vector zi = (zi, 0,..., Zi, M-1), a multiple regression coefficient represented by the following equation (5): A matrix A is obtained and registered as a standard pattern 56.

[0064]

(Equation 5)

Then, at the time of executing the recognition, the discrimination unit 54 performs an operation on the newly input feature vector 49 by the following equation (6), and outputs the result vector y as the discrimination result 59.

[0066]

(Equation 6)

Here, ^AT indicates a transposed matrix of the multiple regression coefficient matrix A, and A ^-1 indicates an inverse matrix of the multiple regression coefficient matrix A.

Embodiment 4 FIG. An embodiment of the invention according to claim 4 will be described below with reference to the drawings. FIGS. 14 and 15 are block diagrams showing the configuration of the image processing apparatus according to the present embodiment.
In FIG. 15, the same or corresponding parts as those in FIGS. 1 and 10 are denoted by the same reference numerals and description thereof is omitted. In this embodiment, the feature vector discriminating means 50 includes a super-population forming means 60, a super-population standard pattern 56, and a multidimensional space class discriminating means 61.

The supergroup forming means 60 learns a total of L examples at the time of learning, and inputs the N-dimensional feature vector 49 to the vector xi for the i-th example.
= (Xi, 0, xi, 1,..., Xi, N−1), and the output of the desired discrimination result is the vector z
When i = (zi, 0,... zi, M−1), one element zi, j (j = 0,.
, M−1), the supergroup Cj, k = (cj, k0)
U, cj, k1U... Ug, ks). Here, cj, kg (g = 0,..., S−1) is the super-group C
j, k are individual groups, and the super-group Cj, k is expressed as a union of the individual groups. Further, one group cj, kg is the center vector γj, kg of the group.
And a variance vector σj, kg indicating a variation of learned feature vectors included in the group.

Updating of the super group is performed in the following procedure. That is, as shown in FIG. 16, when the new input is included in any of the groups constituting the super-group (the distance of the vector is smaller than the constant multiple of σ), the new input is used to change the existing group. Modify center and variance. On the other hand, when the distance of the vector is larger than the constant times σ, that is, when the vector is far from any group, a new group is formed. In this way, a group is formed such that all feature vectors belonging to one determination result are included in the union of several groups.

FIGS. 17 and 18 are flowcharts showing the procedure for updating the supergroup described above. The operation when the supergroup is updated will be described based on the flowchart. First, in step ST1, an i-th feature vector xi at the time of learning and an output (ie, teacher input) vector zi of a desired determination result corresponding thereto are input. In the following step ST2, a supergroup Cj, k of k = zi, j is selected based on the output vector zi of the desired determination result. In step ST3, it is determined whether or not the selected super-group Cj, k is an empty set. If the selected super-group Cj, k is an empty set (ie, when a feature whose discrimination output is to be k for the first time is learned), the process proceeds to step ST4. Proceed, and otherwise, execute step ST5.

In step ST4, a new supergroup Cj,
k = (Cj, k0) is generated, and the feature vector center of Cj, k0 is set to vector γj, ks = vector xi, and C
Let the feature vector variance of j, k0 be σj, k0 = σ0. Here, σ0 is a constant.

In step ST5, the super group Cj, k
The absolute value d of the distance from the center vector rj, kg of each of the element groups cj, kg (g = 0,..., S-1) is measured. Then, in step ST6, d <Aσj, k
It is determined whether or not the above g that satisfies g (where A is a constant) exists, and if so, the next step ST7 is executed,
If not, the process proceeds to step ST8.

In step ST7, for the groups cj, kg indicated by g satisfying d <Aσj, kg, the central vectors rj, kg are calculated by the following equations (7) and (8).
And the variance δj, kg of the center vector is updated.

[0075]

(Equation 7)

[0076]

(Equation 8)

In step ST8, a new group cj, k
s is generated and added to the super-group Cj, k, and the group cj, ks
Γj, ks = vector x
i, and the variance of the feature vector is σj, ks = σ0
I do.

Then, at the time of recognition execution, the multi-dimensional space class discriminating means 61 performs the test of step ST5 on all super-groups Cj (j = 0,..., M) without specifying the identification number of the super-group. Then, the identification number of the supergroup to which the closest group belongs is output as the determination result.

Embodiment 5 FIG. An embodiment of the invention will be described below with reference to the drawings. 19 and 20 are block diagrams showing the configuration of the image processing apparatus according to the present embodiment.
20, parts that are the same as or correspond to those in FIGS. 12 and 13 are given the same reference numerals, and descriptions thereof will be omitted. In the figure, reference numeral 70 denotes a result display means, which is a means for displaying a determination result to the operator 72. Reference numeral 71 denotes correction input means, and an operator 72 manually inputs correction data. In this image processing apparatus, the operator 72 monitors the discrimination output 59 at the time of performing the recognition, and when the discrimination output is determined to be undesirable, the correction input means 71 inputs a desired output (correction teacher input). This corrected teacher input is sent to the regression analysis means 55.
And the multiple regression coefficient A is recalculated according to the equation (5). Regression analysis means 5 to recalculate multiple regression coefficient A
In step 5, the following equations (9) and (10) are stored as intermediate data, and recalculation is performed based on this. Therefore, it is not necessary to store all past learning data.

[0080]

(Equation 9)

In the embodiment described above, the operator 7
Although the description has been made assuming that the correction input is manually performed, the monitoring means for monitoring the inconsistency of the discrimination output may be provided, and the correction may be automatically performed based on the monitoring result of the monitoring means. Also, when there is no correction input, it is determined that the discrimination output is correct. In this case, too, a large amount of learning can be performed by correcting the regression coefficient A by performing regression analysis, thereby improving the recognition accuracy. Can be done.

Although the above-described embodiment has been described with reference to FIGS. 12 and 13, the image processing apparatus shown in FIGS. 14 and 15 may be used. To play. That is, FIG. 14, FIG.
In order to apply to the group formation type classification method of the image processing apparatus shown in FIG. 5, a calculation formula for recalculation may be executed according to the procedure based on steps ST2 to ST8 in FIGS.

Embodiment 6 FIG. An embodiment of the invention according to claim 6 will be described below with reference to the drawings. FIGS. 21 and 22 are block diagrams showing the configuration of the image processing apparatus of the present embodiment.
In FIG. 22, the same or corresponding parts as those in FIGS. 14 and 15 are denoted by the same reference numerals and description thereof is omitted. In the figure, reference numeral 81 denotes a normalization parameter extracting means for analyzing the feature vector 49. Reference numeral 82 denotes a normalization processing unit for performing a normalization operation.

In this embodiment, learning is performed according to two steps, a first step and a second step.
In the first step, only the feature vector 49 is evaluated and the teacher input is ignored. Then, the input of the feature vector is analyzed by the normalization parameter extracting means 81, and an average value xj and a standard deviation sj are calculated for each dimension of the feature vector.

In the second step, a supergroup is formed as described in the fourth embodiment. Before the supergroup is sent to the supergroup forming means 55, a normalization operation is performed in accordance with the following equation (11). Is performed in the normalization processing means 82.

[0086]

(Equation 10)

Therefore, the feature vector element j of any dimension
, The variance of the data is constant, and it can be approximated that the variance value of the group constituting the super-group is the same in any dimension direction. , Kg are all the same value for one g, and the calculation is simplified.

Embodiment 7 FIG. An embodiment of the present invention will be described below with reference to the drawings. FIGS. 23 and 24 are block diagrams showing the configuration of the image processing apparatus of the present embodiment.
In FIG. 24, the same or corresponding parts as in FIGS. 14 and 15 are denoted by the same reference numerals, and description thereof is omitted. In the figure, reference numeral 90 denotes a color measurement optical system for measuring the color of an object, reference numeral 91 denotes a local average luminance detecting means for detecting the local average luminance of the digitized original image 11, and reference numeral 92 denotes an output of the color measurement optical system 90. This is a color measuring unit that measures the color of an object and detects color information including, for example, R, G, and B primary color signals.

The image processing apparatus of this embodiment is not limited to the local features of the image, but the luminance of a point on the image may significantly change the situation of the photographed object, or the image processing apparatus may not be able to determine from the monochrome image. In such a case, comprehensive discrimination can be performed using other sensor information such as the color information.

The outputs of the local average luminance detecting means 91 and the color measuring means 92 are combined at the stage of the feature vector 49, and are comprehensively determined by the comprehensive feature vector determining means 51. Therefore, for example, "I want to distinguish between a painted product that is entirely whitish and a painted product that is darkish" or "I want to distinguish between a triangular red painted part and a green painted part." Discrimination work under complicated conditions such as the above can be performed with high reliability.

In a situation where the light source or the background greatly changes depending on the weather, season, or time zone, such as outdoors, in an environment where the output changes from a sensor for detecting the weather condition or the like or time information is used as external information, the environment changes complicatedly. However, it is possible to perform the determination with high reliability.

Embodiment 8 FIG. An embodiment of the invention according to claim 8 will be described below with reference to the drawings. FIGS. 25 and 26 are block diagrams showing the configuration of the image processing apparatus of this embodiment.
In FIG. 26, the same or corresponding parts as those in FIGS. 5 in the figure
Reference numeral 6 denotes a standard pattern, which is provided in plurality. 93 classifies the outputs of the local average luminance detecting means 91 and the color measuring means 92 for measuring the color of the object into a plurality of classes, and generates a large classification class used when switching a standard pattern used for discrimination. Means.

In the image processing apparatus of this embodiment, as in the case of the seventh embodiment, not only the local features of the image but also the brightness of a certain point on the image greatly changes the situation of the photographed object, or In the case where it cannot be determined from the image of the above, a comprehensive determination can be made by using other sensor information together. Then, the local average luminance detecting means 91
The output of the object color measuring means 92 is the feature vector 4
Large classification class generating means 9 without being synthesized at the stage of 9
3, the integrated feature vector discriminating means 51 switches and uses the standard pattern 56 thus classified at the time of recognition.

Therefore, for example, “the judgment criteria for the inspection differ between a painted product that is entirely whitish and a painted product that is blackish” or “the number of holes is measured for a red painted triangular part. I want to measure the area of a green painted round part. "

In a situation where the light source and the background greatly change depending on the weather, season, and time zone, such as outdoors, the output and time information of a sensor for detecting the weather state and the like, and information such as temperature and weight are used as external information. By combining them as needed, it is possible to perform the determination with high reliability.

Embodiment 9 FIG. An embodiment of the ninth aspect of the present invention will be described below with reference to the drawings. FIGS. 27 and 28 are block diagrams showing the configuration of the image processing apparatus of this embodiment.
In FIG. 8, the same or corresponding parts as those in FIG. In the figure, reference numeral 101 denotes a partial image dividing operation unit that divides a ternary image 31 represented by ternary into rectangular partial areas as shown in FIG. 29B and operates the inside thereof.

In the image processing apparatus of this embodiment, the corresponding standard pattern is selected from the plurality of standard patterns 56 by using the identification numbers of the partial images divided at the same time as the scanning by the partial image dividing operation means 101. Perform learning and recognition for standard pattern numbers. In FIG. 29, the image is divided into 30 partial areas a0 to a29, and FIG. 29C shows a state where the partial area with the area number a7 is selected. Such selection is repeated for all the partial areas from the area numbers a0 to a29, and the entire scanning is completed.

As a result, in the image processing apparatus of this embodiment,
Since only a local feature in the area of interest has an effect on a certain standard pattern and features of other areas are not added, the feature vector of the peripheral area and the feature vector of the selected area are not added. Even if the patterns are different, the determination does not become difficult, and the reliability of the determination is improved.

In the embodiment described above, the image is divided into rectangular partial images. However, the shape of the object occupying the screen is freely defined, and the inside is defined as the partial image. Is also good.

Embodiment 10 FIG. An embodiment of the invention will be described below with reference to the drawings. FIGS. 30 and 31 are block diagrams showing the configuration of the image processing apparatus of the present embodiment.
In FIGS. 27 and 28, the same or corresponding parts in FIGS. 27 and 28 are denoted by the same reference numerals and description thereof is omitted. In the figure, reference numeral 104 denotes a weight coefficient generating means,
When the value image 31 is divided into rectangular partial areas as shown in FIG. 29B and the inside is scanned, when the peripheral area of the partial area is scanned, the central area is scanned. A lower weighting factor is generated as compared to the case. This weighting factor is calculated from the local feature counting unit 42-1 to the local feature counting unit 4
2-N, and a value corresponding to the weight is counted.

As described above, in the present embodiment, the weight of the local feature is low in the peripheral portion of the partial region. It is possible to prevent phenomena such as the fact that the feature to be performed is included or not included during learning and during recognition and the reliability of recognition is reduced.

Embodiment 11 FIG. An embodiment of the present invention will be described below with reference to the drawings. FIGS. 32 and 33 are block diagrams showing the configuration of the image processing apparatus of this embodiment.
In FIGS. 2 and 33, the same or corresponding parts as those in FIGS. 27 and 28 are denoted by the same reference numerals and description thereof is omitted. In the figure, reference numeral 106 designates the position alignment performed in the same manner as described in the three-dimensional image alignment technique by the two-dimensional pattern coincidence counting means 58 and the peak evaluation means 59 of the prior art 1 and executes the alignment. Position detecting means for outputting the coordinate data of the position, 107 is the position detecting means 10
This is a geometric transformation parameter generating means for calculating how to geometrically transform the image from the coordinate data output from 6 so that the image can be superimposed on the standard image better. 1
Reference numeral 01a denotes a geometric transformation unit that performs geometric transformation based on the calculation result by the geometric transformation parameter generation unit.

In the image processing apparatus of this embodiment, after the ternary image 31 represented by the ternary image is created, the image first learned at several points in the image using the position detecting means 106 (standard Image) and the currently input image.

From the position detecting means 106, the coordinate data of each of the positions where the positioning has been performed is sent to the geometric transformation parameter generating means 107. The geometric transformation parameter generation means 107 calculates how to geometrically transform the image from the plurality of coordinate data so that the image can be superimposed best on the standard image.

The simplest geometric transformation for superposition is translation, and this transformation is represented by two parameters (xd, yd). For example, M on the screen
, And the result is (xi, yi), i = 0,..., M−1, and the corresponding position on the image stored first is (xi0, yi).
0), i = 0,..., M−1, the following equations (12) and (13) are operated to determine the alignment parameters (xd, yd).

[0106]

[Equation 11]

[0107]

(Equation 12)

Similarly, the geometric transformation can be expressed by three parameters of translation (x, y) and rotation θ, or the translation (x, y), rotation θ and enlargement /
It is also possible to express by the parameter of reduction s. Alternatively, assuming a general affine transformation, the following equation (14),
As shown in (15), it is also possible to represent by six parameters a, b, c, d, e, and f, and to estimate what causes the movement or distortion of the image. , An optimal value can be selected.

[0109]

(Equation 13)

[0110]

[Equation 14]

By taking out the partial image by the partial image division scanning means 101 while performing the geometric conversion by the period conversion unit 101a in consideration of the geometric conversion parameter thus obtained, the standard pattern learned at the first time is obtained. Then, it is possible to extract a partial region having good overlap with and vectorize the feature.

As a result, in the present embodiment, learning and recognition can be always performed with an image having no positional deviation, the signal-to-noise ratio of the feature vector is improved, and highly reliable recognition is possible.

Embodiment 12 FIG. An embodiment of the invention will be described below with reference to the drawings. FIGS. 34 and 35 are block diagrams showing the configuration of the image processing apparatus of this embodiment.
4 and 35, the same or corresponding parts as those in FIG. In the figure, 130
Is a feature selection unit to which the outputs of all the local feature detection units 41-i are given, 111 is a partial image cutout scanning unit that cuts out a specified partial image and scans the entire cutout partial image, and 112 is a local feature. A feature map correspondence search unit 115 for searching for a correspondence between features between the map (feature map) 113 and the standard local feature map 114 selected by the output of the feature vector determination unit 50. A geometric transformation 115 for generating geometric transformation parameters It is a parameter generation means.

Next, the operation will be described. The digitized original image is converted into a ternary ternary image and output to the feature vector extracting means 40. The local feature detecting means 41 generates a large output when the ternary pattern around the pixel of interest in the image becomes a unique combination. Scanning means 11 for extracting partial image from target pixel
By scanning over the entire partial image designated by 1, the frequency at which a certain feature occurs is counted by the local feature counting unit 42-i. The ternary local pattern to which the local feature detecting means 41-i reacts is shown in FIG.
The local patterns shown in FIG. 4, FIG. 5, and FIG. 6 are appropriately combined and set so that each part of the object to be searched can be distinguished and expressed.

The feature vector 49 which is the sum of the local features in the partial image generated in this way is input to the feature vector discriminating means 50. Feature vector discriminating means 50
Then, matching between the feature vector and the standard pattern is performed. The operation of the feature vector determining means may use any of the methods described in the first to fourth embodiments. The k standard patterns 56 stored in the standard feature vector are compared with the input feature vector 49, and the class k of the closest vector among them is output as the result of class discrimination. The local feature map 114 is switched to the corresponding class.

In the feature map correspondence search means 112, the local feature map 113 of the input image generated by using the local features in the local feature detection means 41, and the feature vector discrimination means 5
A local feature association is searched for with the standard local map 114 selected by the output of 0. FIG. 38 is an explanatory view showing an example of the feature map. As shown in FIGS. 38A and 38B, even if the local patterns are different, the local features (refer to FIG. 39) that can be collated by rotation can be grouped. Then, a possible combination with less inconsistency in labeling of adjacent local features on the map is searched.

That is, in the feature map shown in FIG. 38, the local features m8, m32, m12, m3 shown in FIG.
6, m0 and the local feature m in FIG.
The combination of 11, m35, m15, m39, and m3 is determined to be a combination that can be collated by rotating clockwise by 60 °.

The feature map correspondence search means 112 searches for such a possible combination, and finally outputs a result which can be superimposed most consistently. That is, in the example shown in FIG. 38, the local feature m32 at the coordinates (x1, y1) on the standard pattern shown in FIG. 38A matches the local feature m35 at the coordinates (x1 ′, y1 ′) on the input pattern. Information on the result sequence that has been matched (ie, rotated by 60 ° clockwise and matched) is output from the feature map correspondence search unit 112, and is input to the geometric transformation parameter generation unit 115.

In the geometric transformation parameter generation means 115,
From the combination of these (x, y, θ) conversions for each coordinate, the coordinate conversion parameters (a,
b, c, d, e, f). At this time, the cutout position information output from the partial image cutout scanning unit 111 and used to generate the partial image area is added to the coordinate conversion parameters (a, e) and output. Then, the class (that is, the type of the object included in the partial area) that is the output of the feature vector determination unit 50 and the output of the geometric conversion parameter generation unit 115 are output as the recognition result of the image processing.

Embodiment 13 FIG. Next, another embodiment of the present invention will be described. FIGS. 36 and 37 are block diagrams showing the configuration of the image processing apparatus of this embodiment.
In FIG. 37, the same or corresponding parts as those in FIGS. 34 and 35 are denoted by the same reference numerals and description thereof is omitted. In this embodiment, the local feature vector extracting unit 40 is replaced with a local feature detecting unit 41 and a feature selecting unit 130, and a part of the local feature map 113 is cut out and an identifier (feature number) of a local feature included therein is extracted. The histogram will be used as the feature vector. For this reason, the feature map histogram forming means 121 is newly added, and the partial local feature map cut out by the partial image cut-out scanning means 111 is input and the output is supplied to the feature vector determining means 50. Subsequent operations are the same as those shown in FIGS.

The twelfth embodiment and the first embodiment described above are used.
In the image processing apparatus of the third embodiment, the feature vector discriminating means 50 has been described as having the function as the feature vector discriminating means shown in FIG. 1 of the first embodiment. It may be a vector discriminating means or a feature vector discriminating means having the super-population forming means in the fourth embodiment.

As described above, in this embodiment, the posture and the distance of the imaged object are different from those at the time of learning or registration, and they are not changed in the rough image but cannot be completely overlapped. Even in such a situation, the approximate position and the type of object are first identified using the local feature vector, and then the local feature pattern is precisely matched in the next stage, and finally the consistent identification results are obtained. And position adjustment including a change in posture.

As still another embodiment, when the feature map correspondence search means 112 has a large number of inconsistencies in the correspondence search and cannot obtain a good correspondence result, the unsuccessful information is fed back to the feature vector discrimination means 50. The feature vector discriminating means 50 may discard the class of the feature vector adopted this time and select the next closest candidate again. In such a configuration, the histogram of the feature map is similar but the pattern is similar. Can be prevented from giving an erroneous determination result to a case where a plurality of standard patterns having different arrangements are provided.

Similarly, using the output of the geometric transformation parameter generation means 107, the standard image or the ternary standard image stored corresponding to the selected standard local feature map 114 is actually converted, and By calculating the state of superposition of the input image and the standard image on the plane, it is possible to re-evaluate the quality of the recognition result and output data indicating the degree of reliability of recognition.

[0125]

As described above, according to the first aspect of the present invention, a ternary image is obtained from an input image, and the similarity between the ternary image and the local feature pattern is subjected to threshold processing to obtain a local feature count. As a feature vector is obtained based on the count value counted by the means, and the discrimination result is obtained from the feature vector and one or a plurality of standard patterns stored in advance. There is an effect that the ability to determine and recognize a target object in an image of a general scene can be improved.

According to the second aspect of the present invention, since the similarity between the ternary image and the local pattern is configured to be accumulated in the image, the object in the image of a general scene having a plurality of gradations is obtained. This has the effect of improving the discrimination and recognition processing ability for.

According to the third aspect of the present invention, since the discrimination result is obtained by discriminating the feature vectors based on the multiple regression analysis, the object in the image of the general scene having a plurality of gradations is obtained. In addition to the improved discrimination processing capability, a highly reliable discrimination result is obtained, and the recognition processing capability can be improved.

According to the fourth aspect of the present invention, a group to be determined is expressed as a union of several groups, and each group is divided into a plurality of learning image data whose determination results are known in advance and a multidimensional space class determination. By using the results determined by the means, the individual groups are formed while being sequentially updated, and the determination results are output based on the super-group which is the union of the groups. This has the effect of improving not only the discrimination processing capability for the target in an image of a general scene having the above, but also the recognition processing capability.

According to the fifth aspect of the present invention, when the monitor determines that the discrimination result is incorrect during the discrimination recognition processing, the standard pattern is recalculated by inputting a desired discrimination result, and the operation is performed. In addition, it is possible to improve the reliability of the judgment while improving not only the discrimination processing ability for the object in the image of the general scene having multiple gradations but also the recognition processing ability There is an effect that can be made.

According to the invention of claim 6, data is normalized in each dimension of the multidimensional feature vector at the time of learning the standard pattern, so that the variance of the center vector representing the group becomes the same value in any dimension. Because it was configured to be
There is an effect that not only the discrimination processing capability for the target in the image of a general scene having a plurality of gradations but also the recognition processing capability can be improved.

According to the seventh aspect of the present invention, the comprehensive feature vector discriminating means combines features obtained from other sensor information or global features of the image in addition to the feature vectors obtained from the local features of the input image during learning. The multi-dimensional feature space is configured to perform discrimination and perform discrimination, so that not only the discrimination processing capability for an object in an image of a general scene having a plurality of gradations is improved, but also recognition processing is performed. There is an effect that the ability can be improved.

According to the eighth aspect of the present invention, in addition to the feature vector obtained from the local feature of the input image at the time of learning, a feature obtained from other sensor information and classified into a plurality of classes or a global feature of the image is used. A corresponding class is selected from the learning data, and the recognition is performed using only the learning data stored in the selected class. Therefore, the object in the image of the general scene having a plurality of gradations is selected. This has the effect of improving not only the discrimination processing capability but also the recognition processing capability.

According to the ninth aspect of the present invention, the image is divided into a plurality of sections in advance, learning data is prepared for each of the divided sections, and learning and recognition are performed for each section. This has the effect of improving not only the discrimination processing capability for an object in an image of a general scene having a plurality of gradations but also the recognition processing capability such as reliability.

According to the tenth aspect, a lower weighting factor is set at the periphery of the image section divided into a plurality of sections than at the center of the section, and the output obtained by adding the weighting factor is used as a feature vector as a feature vector. Since it is configured to be supplied to the discriminating means, it is possible to prevent phenomena such as a change in the shape of the object or a decrease in the reliability in recognition due to misalignment or the like, and a general method having a plurality of gradations. This has the effect of improving not only the discrimination processing capability for the object in the scene image but also the recognition processing capability such as reliability.

According to the eleventh aspect of the present invention, registration with a standard image is performed at a plurality of measurement points on the image, and a geometric transformation parameter for making the image closest to the standard image is obtained. Since the image geometrically transformed based on the transformation parameters is configured as the input image, not only the discrimination processing capability for the object in the image of the general scene having a plurality of gradations is improved, but also the reliability is improved. This has the effect of improving recognition processing ability such as sex.

According to the twelfth aspect of the present invention, after the object in the partial image is determined, the combination between the feature map and the standard local feature map can be consistently associated with the local feature map and the standard local feature map. Is obtained, and the geometric transformation parameter and the result of the above determination are output as a recognition result of the image processing. Therefore, it is possible to cope with a case where the posture of the recognition target is slightly changed. As a result, not only is the discrimination processing capability for an object in an image of a general scene having a plurality of gradations improved,
This has the effect of improving recognition processing capability such as reliability.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention;

FIG. 2 is an explanatory diagram showing a process of ternarizing an image in the image processing apparatus according to one embodiment of the present invention;

FIG. 3 is an explanatory diagram showing an example of a local pattern in the image processing apparatus according to one embodiment of the present invention;

FIG. 4 is an explanatory diagram showing an example of a local pattern in the image processing apparatus according to one embodiment of the present invention;

FIG. 5 is an explanatory diagram showing an example of a local pattern in the image processing apparatus according to one embodiment of the present invention;

FIG. 6 is an explanatory diagram showing an example of a local pattern in the image processing apparatus according to one embodiment of the present invention;

FIG. 7 is a conceptual diagram showing a configuration of a local feature detecting means in the image processing apparatus according to one embodiment of the present invention.

FIG. 8 is an output calculation diagram showing a coincidence determination operation of the local feature detection means in the image processing apparatus according to the first embodiment of the present invention;

FIG. 9 is an output calculation diagram showing a matching degree determination operation of the local feature detection means in the image processing apparatus according to one embodiment of the present invention;

FIG. 10 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention;

FIG. 11 is a conceptual diagram showing a configuration of a feature vector extracting means of the image processing apparatus according to one embodiment of the present invention.

FIG. 12 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention;

FIG. 13 is a block diagram showing a configuration of an image processing apparatus according to one embodiment of the third invention.

FIG. 14 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 15 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 16 is an explanatory diagram showing a multidimensional feature space in a method for forming a super-population in the image processing apparatus according to an embodiment of the present invention;

FIG. 17 is a flowchart for explaining a super-population update procedure in the image processing apparatus according to an embodiment of the present invention;

FIG. 18 is a flowchart for explaining a super-population update procedure in the image processing apparatus according to one embodiment of the fourth invention;

FIG. 19 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 20 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention;

FIG. 21 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 22 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 23 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention;

FIG. 24 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention;

FIG. 25 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 26 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention;

FIG. 27 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 28 is a block diagram showing the configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 29 is an explanatory diagram for explaining the operation of the partial image division scanning means of the image processing apparatus according to one embodiment of the ninth invention;

FIG. 30 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 31 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 32 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 33 is a block diagram showing a configuration of an image processing apparatus according to one embodiment of the present invention.

FIG. 34 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 35 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 36 is a block diagram showing a configuration of an image processing apparatus according to another embodiment of the present invention.

FIG. 37 is a block diagram showing a configuration of an image processing apparatus according to another embodiment of the present invention.

FIG. 38 is an explanatory diagram showing a feature map in the image processing apparatus according to the embodiment of the twelfth aspect of the present invention.

FIG. 39 is an explanatory diagram showing local features in the feature map of FIG. 38;

FIG. 40 is a block diagram illustrating a configuration of a conventional image processing apparatus.

FIG. 41 is a block diagram illustrating a configuration of a conventional image processing apparatus.

FIG. 42 is an explanatory diagram showing a local pattern in a conventional image processing apparatus.

FIG. 43 is an explanatory diagram showing a problem in a conventional image processing apparatus.

[Explanation of symbols]

Reference Signs List 20 filtering means 30 ternarization processing means 40 feature vector extraction means 41-1 to 41-N local feature detection means 42-1 to 42-N local feature counting means 43-1 to 43-N local feature accumulation means 46 threshold means Reference Signs List 50 feature vector discriminating means 51 comprehensive feature vector discriminating means 54 discriminating unit 55 regression analyzing means 60 super-population forming means 61 multidimensional space class discriminating means 55 regression analyzing means 70 result display means 71 correction input means (decision reliability improving means) 82 Normalization processing means 93 Large classification class generation means 101 Partial image division scanning means 101a Geometric conversion unit 104 Weight coefficient generation means 106 Position detection means 107, 115 Geometric conversion parameter generation means 111 Partial image cutout scanning means 112 Feature map correspondence search means 113 Local feature map (feature map) 14 standard local feature maps 130, wherein selection means g1~g18, g101~g104, g201, g20
2, g301-g304 local pattern

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Yoshio Izui 8-1-1 Tsukaguchi Honcho, Amagasaki City Mitsubishi Electric Corporation Industrial Systems Research Institute (56) References JP-A-4-128976 (JP, A) 58-130030 (JP, A) JP-A-6-36034 (JP, A) JP-A-6-76062 (JP, A) (58) Fields studied (Int. Cl. ⁷ , DB name) G06T 7 / 00 G01B 11/24 G06T 5/20

Claims (12)

(57) [Claims] An image processing apparatus for processing a two-dimensional image input from an imaging device such as a television camera to determine the type, number, classification, normality / abnormality, etc. of an object present in the image. Filtering means for applying one or more kinds of spatial bandpass filters to the image, ternary processing means for processing the output of the spatial bandpass filter with two threshold values to obtain a ternary image, Local feature detection means for detecting the similarity between an image and a local pattern, threshold means for generating an output when the similarity with the local pattern is equal to or more than a certain value, and the number of outputs of the threshold means in the image Feature vector extraction means having a plurality of local feature counting means for counting whether or not occurrence has occurred in one of a plurality of local feature counting means; An image processing apparatus comprising: a feature vector determining unit that obtains a similarity with a standard vector and outputs a final determination result. 2. An image processing apparatus for processing a two-dimensional image input from an imaging device such as a television camera to determine the type, number, classification, normality / abnormality, etc. of an object present in the image. Filtering means for applying one or more kinds of spatial band-pass filters to the image; ternary processing means for processing the output of the spatial band-pass filter with two threshold values to obtain a ternary image; A feature vector extracting means comprising a local feature detecting means for detecting the similarity between an image and a local pattern, and a plurality of local feature accumulating means for accumulating the similarity between the local pattern in the image; And a feature vector discriminating means for obtaining a similarity between the output of the above and one or more standard vectors stored in advance and outputting the similarity as a final discrimination result. An image processing apparatus comprising: 3. The feature vector discriminating means uses a linear discriminant function represented by a linear sum of feature vectors, and generates a linear discriminant function by combining a plurality of learning image data whose discrimination results are known in advance with discrimination results. A regression analysis unit that performs a multiple regression analysis using the results and uses the result as a weight coefficient of the linear discriminant function; and a discrimination unit that outputs a discrimination result from the weight coefficient and the feature vector obtained by the regression analysis unit. The image processing apparatus according to claim 1, wherein: 4. The feature vector discriminating means stores a center vector of a group included in a standard pattern super-group in a multidimensional space defined by a dimension of the feature vector and a parameter representing a variance from the center vector of the corresponding group. In addition, when the distance between the input feature vector and the center vector of any of the groups belonging to the standard pattern supergroup is within a constant multiple of the corresponding standard pattern variance, the input feature vector is the standard pattern variance. Multi-dimensional space class discriminating means for judging to belong to a super-group of a plurality of learning image data and a discrimination result to determine a standard pattern vector and a standard pattern variance, and a discrimination result in multi-dimensional space. And the training vector data belonging to the same group is the central vector of the provisional standard pattern group being trained. When it appears at a position that is more than a constant multiple of the variance of the standard group pattern, a new standard pattern group is formed, the center vector and the variance are set, and the set of these groups is interpreted as one supergroup. And, when appearing at a position closer than a constant multiple of the variance of the standard group pattern from the center vector of one of the groups, the center vector and the variance of the corresponding group pattern are sequentially updated by considering the new learning pattern. The image processing apparatus according to claim 1, further comprising: a supergroup forming unit configured to form a supergroup of a standard pattern that is periodically updated. 5. When the monitor determines that the discrimination result is incorrect during the recognition processing, the standard pattern is recalculated by inputting a desired discrimination result, and the reliability of the discrimination is improved while operating. The image processing apparatus according to claim 3, further comprising a determination reliability improving unit for measuring. 6. A normalization method for normalizing data for each dimension of a multidimensional feature vector at the time of learning a standard pattern so that the variance of a center vector representing a group has the same value for any dimension. The image processing apparatus according to claim 4, further comprising a conversion processing unit. 7. A feature vector discriminating means performs learning by simultaneously using features obtained from other sensor information other than feature vectors obtained from local features of an input image or global features of an image at the time of learning, and performing recognition processing. 3. The image processing apparatus according to claim 1, wherein the image processing apparatus is a comprehensive feature vector discriminating unit that occasionally configures a multidimensional feature space using the same feature to perform discrimination. 8. A feature vector discriminating means classifies a feature obtained from other sensor information or a global feature of an image into a plurality of classes in addition to a feature vector obtained from a local feature of an input image during learning and classifies the learning data. Classifying means for updating only the class to be performed, and at the time of recognition processing, a corresponding class is selected from learning data using the same feature, and recognition is performed using only learning data stored in the selected class. 3. The image processing apparatus according to claim 1, wherein the image processing apparatus is a comprehensive feature vector determination unit. 9. A partial image division scanning unit for dividing an image into a plurality of sections in advance, wherein the feature vector determination unit prepares learning data for each of the divided sections, and performs learning and recognition for each of the sections. 3. The image processing device according to claim 1, wherein the image processing device is a feature vector determining unit. 10. A partial image division scanning unit that divides an image into a plurality of sections in advance, a weighting factor generating unit that sets a lower weighting factor in the periphery of the divided section than in the center of the section, and a local feature detecting unit. Local feature counting means for counting an output obtained by adding the weighting factor generated by the weighting factor generating means to the output of the above, and feature vector discriminating means for supplying the output of the local feature counting means as a feature vector. The image processing device according to claim 9, wherein: 11. A position detecting means for performing registration with a standard image at a plurality of measurement points on the image, and a geometric conversion parameter for making the image closest to the standard image from an output of the position detecting means. A geometric transformation parameter generation unit to be obtained, and a geometric transformation unit that geometrically transforms the image based on the geometric transformation parameter generated by the geometric transformation parameter generation unit and uses the image after the geometric transformation as an input image. The image processing device according to claim 1 or 2, wherein 12. An image processing apparatus for processing a two-dimensional image input from an imaging device such as a television camera to determine the type, position, orientation, and the like of an object present in the image. A filtering means for applying a plurality of types of spatial band-pass filters; a ternary processing means for processing the output of the spatial band-pass filter with two threshold values to obtain a ternary image; Local feature detection means for detecting the similarity with a plurality of detected local patterns, and local feature counting for counting the number of times that the local feature detection means detects a similarity of a certain value or more within a certain area of a certain size Means, a partial image cut-out scanning means for scanning the position of the partial area counted by the local feature counting means on the screen, and an output of the local feature counting means as an input feature vector. The class to which the input feature vector belongs (that is, the presence or absence or type of an object included in the ternary partial image) is determined by evaluating the similarity between the standard feature vector stored in advance and the above. A feature vector discriminating means for outputting the position of the input feature vector belonging to the class on the image and the class, an identifier such as a local feature number of the local feature detected most strongly from the output of the local feature detecting means, and its strength And a feature selection means for outputting a feature map in which the identifier or the strength of the identifier and the strength are described in a two-dimensional manner, and the feature map and a standard local feature registered in advance based on the determination result by the feature vector determination means. The map is compared with the map, and the correspondence between the input image and the registered image is tested in consideration of the deformation. A feature map correspondence search means for obtaining a combination between the feature map and the standard local feature map, which can be attached without contradiction, and a deformation parameter required at that time; and a standard object pattern selected from the obtained deformation parameter and An image processing apparatus comprising: a geometric conversion parameter generation unit that generates and outputs a geometric conversion parameter between object patterns.