JPS63284687A - Image processor - Google Patents

Abstract

PURPOSE:To obtain the titled processor which surely extracts a feature pattern without requiring an unnecessary time for window setting, by labeling each extracted feature pattern by using the attribute of the feature pattern as a key. CONSTITUTION:Before the processing of a image to be processed, the image of a standard model is picked up and plural feature patterns of this image are obtained and one or more attributes are generated for each feature pattern and are preliminarily stored in an attribute storage part 5. Thereafter, a plural feature patterns are obtained in a CPU 1 with respect to the image to be processed generated by an image pickup device 8 and attributes of respective feature patterns are generated in the CPU 1. The CPU 1 collates attributes of respective feature patterns of the image to be processed with those of the standard label to label each feature pattern of the image to be processed.

Description

DETAILED DESCRIPTION OF THE INVENTION <Industrial Application Field> The present invention relates to visual recognition technology in industrial robots and the like. The present invention relates to an image processing apparatus for extracting points, lines, etc. (these are referred to as "one feature pattern") constituting the features of a computer, and labeling each feature pattern.

<Prior Art> Conventionally, as shown in FIG. 17(1), in this type of image processing apparatus, a plurality of windows 51 to 53 having a predetermined size are set on an image to be processed 50, and each window 51 is ~
53, for example, bending points of the image to be processed 50 are extracted as characteristic patterns 50A to 50C. In this case, usually 1
Since one feature pattern corresponds to each window, it is possible to identify each feature pattern extracted from the image to be processed 50 depending on which window 51 to 53 the feature pattern was extracted from. , and no special labeling or processing is required.

<Problem to be solved by the invention> However, when extracting a large number of feature patterns from an image to be processed, it is necessary to set a number of windows corresponding to the number of feature patterns, so the window settings The problem is that it takes time.

Also, if the number of windows increases, the size of each window must be set smaller accordingly, but as shown in Figure 17 (
2), there is no problem if the amount of positional deviation ΔS from the predetermined position of the image to be processed 50 is small, but if
As shown in 3), when the positional deviation amount ΔS becomes large, each of the characteristic patterns 50A to 50C moves to the window 51.
Characteristic patterns 50A to 50C protrude outside of ~53
It becomes difficult to extract.

This invention was made with attention to the above problem, and by making it possible to label each extracted feature pattern using the attributes of each feature pattern as clues, unnecessary time for setting windows can be saved. It is an object of the present invention to provide a novel image processing device that can reliably extract feature patterns without the need for a feature pattern.

<Means for Solving the Problems> In order to achieve the above object, the image processing apparatus of the present invention includes an image generation means for generating an image to be processed, and a plurality of image generating means constituting the characteristics of the image from the image to be processed. Feature pattern extraction means for extracting feature patterns; attribute generation means for producing one or more attributes for each feature pattern of the image to be processed; and one or more attributes for each feature pattern of the standard model are stored in advance. an attribute storage means for storing the image, and an attribute of each feature pattern for the processed image generated by the attribute generation means and an attribute of each feature pattern for the standard model stored in the attribute storage means. , and attribute matching means for labeling each feature pattern of the image to be processed.

<Operation> Prior to processing the image to be processed, a standard model is imaged, a plurality of characteristic patterns of the image are obtained, and one or more attributes are generated for each characteristic pattern and stored in advance in the attribute storage means.

After that, a plurality of characteristic patterns are obtained from the image to be processed generated by the image generation means by the characteristic pattern extraction means, and an attribute of each characteristic pattern is generated by the attribute generation means. Next, the attribute matching means compares the attributes of each feature pattern of the image to be processed with the attributes of each feature pattern of the standard model, and labels each feature pattern of the image to be processed.

According to this invention, there is no need to set many small windows for extracting feature patterns, so extra time for window settings is not required, and feature patterns may be extracted outside of the window due to positional shift of the image. There is no risk that it will be difficult.

<Embodiment> FIG. 1 shows an example of a circuit configuration of an image processing apparatus according to an embodiment of the present invention. In the illustrated example, the CPU is the main control body, and the CPU is connected to the R
OM2. A first RAM 3 used for calculations. In addition to a second RAM as the image storage section 4 and a third RAM backed up by a battery as the attribute storage section 5, a display section 6 consisting of a CRT and an operation section 7 consisting of a keyboard are connected. An imaging device 8 is connected to the image storage section 4 via an interface circuit 9, and images generated by this imaging device 8 are stored in the image storage section 4. In the case of this embodiment, a plate-shaped slit light is irradiated onto the object using a light projection device (not shown), and the intersection line of the slit light formed on the surface of the object is imaged by the imaging device 8, and the slit is used as an image to be processed. I'm getting a statue.

FIG. 2 shows the segments 21 to 23 of the slit image, and by appropriately selecting the direction of the light projecting device, the slit image becomes approximately horizontal. Each segment 21 to 23 of this slit image includes end points PA-P and bending points QA to be extracted as a feature pattern, and each of these points is extracted within one large window 24. The same figure also shows the small-area windows 25 to 29 in the conventional feature pattern extraction method, and FIG. 2 (2) shows a state in which each slit image is slightly shifted, and FIG. 3(3) shows a state in which each slit image is largely misaligned. In the case of the conventional method, each end point PA-P or bending point QA is located within the corresponding window 25 to 29 for a slight positional deviation, but for a large positional deviation, each end point PA-P or bending point QA is located within the corresponding window 25 to 29.
You can see that it sticks out from 9. On the other hand, if a large window 24 is used, the end points PA to PD and the bending point Q will not protrude beyond the window 24 regardless of the size of the positional shift, and no problem will occur in extracting the characteristic pattern.

Returning to FIG. 1, the attribute storage unit 5 stores data indicating the attributes of each characteristic pattern (end points and bending points in this embodiment) for the standard model, that is, data obtained by irradiating the standard model with plate-shaped slit light. For each feature pattern of the slit image, where in the image area is it located? At what height in three-dimensional space is each feature pattern? Is each feature pattern an end point or an inflection point? , the number of each feature pattern in a predetermined image area, and which segment each feature pattern belongs to are stored.

The CPU decodes and executes the program in ROM2, and
M3. Image storage unit 4. It executes necessary calculations and processing while reading and writing data to the attribute storage unit 5.
In this case, cpul is a function for extracting a predetermined feature pattern from an image to be processed, a function for generating an attribute for each feature pattern, and a function for storing attributes of each feature pattern for an image to be processed and a standard stored in the attribute storage unit 5. It has various functions such as a function of collating the attributes of each characteristic pattern of the model and labeling each characteristic pattern of the image to be processed.

Note that extraction of feature patterns, collation of attributes, and labeling are performed by software processing using a program as in the above embodiment, and also by the feature pattern extraction unit 10 as shown in FIG.
Alternatively, a dedicated circuit such as the attribute matching section 11 may be provided to make this hardware.

Furthermore, in each of the embodiments shown in FIGS. 1 and 3, the attribute storage unit 5
To set the attribute data of the characteristic pattern for the object, the operator operates the operation unit 7 and confirms the setting contents on the display unit 6. However, it is possible to set the attribute data of the characteristic pattern in advance by using a ROM as the attribute storage unit 5, for example. If it is fixed, as shown in FIG. 4, the configuration corresponding to the operation section 7 and display section 6 of the embodiment described above becomes unnecessary.

FIG. 5 shows a procedure for setting attribute data in the attribute storage section 5. As shown in FIG. At the start point in the figure, a standard model is irradiated with a plate-shaped slit light to form an intersection line of the slit lights on its surface.

This intersection line is imaged by the imaging device 8 in step 1 (indicated by rsT 1 j in the figure), and the slit image is stored in the image storage unit 4.

In the next step 2, it is determined whether or not to partition the region of interest for this slit image, and the determination is "YES".
If the result is ``S'', data for partitioning the area is input using the operation unit 7 in step 3, but if the determination is ``NO''.
”, the entire image is defined as one region of interest (step 4). In this embodiment, as shown in FIG.
A plurality of regions of interest are defined as r4, and in the next step 3, the section data of the first region of interest arl is input.

Next, in step 5, the CPUI extracts feature patterns (end points and bending points in this embodiment) for the slit image in the first region of interest arl.

7 and 8 show a method for extracting end points, and FIGS. 9 and 10 show a method for extracting bending points, respectively.

In FIG. 7, ar(i) is the i-th region of interest, and a small window 31 for image tracking is set within this region of interest ar(i) and moved in the direction of the arrow in the figure. As a result, the slit image 3 in the region of interest ar(i)
2 is performed, and the pixel position immediately before the brightness within this window 31 becomes equal to or less than a predetermined threshold value is extracted as the end point P of the slit image 32. In this case, the tracking starting point 34. Although the tracking direction, brightness threshold, etc. are given from the outside, the tracking starting point 34 may be determined by scanning the image in a fixed direction.

Now, the coordinates of the lower left point of the tracking window 31 are (+w,
jw), each axis of the window 31 in one direction and the J direction is di, dj, and the sum of the brightness within the 9-into 31 is sg.
i, this window 31 is determined to be located at a position outside the end point P (the position indicated by So in FIG. 8) when the following equation 0 is satisfied, and when the following equation 0 is satisfied,
It is determined that the position is located inside the end point P (the position indicated by S in FIG. 8).

Note that in formula 00, th is a brightness threshold.

As a result, if it is determined that the window 31 is at the 81st position, the next window position (i Set ヮ ′, ji ′).

In the above equation, 5g1j is the brightness inside the window 31 and J
It is the sum of products with the coordinate value, and 5kip is 1 of the window 31.
This is the step movement distance.

Again, window 31 is S. When it is determined that the tracking direction is in the forward direction, the current S is moved from the previous position 81. Two paired windows 35.
36 and move it one pixel at a time. At this time, the brightness within both windows 35 and 36 is determined at each pixel position, and the position where the value obtained by subtracting the brightness within one window 35 from the brightness within the other window 36 is maximum (
Let Iw, Iw) be the position of the end point P. When the tracking direction is in the negative direction, the moving directions of the pair windows 35 and 36 are reversed, and similar calculations are performed to find the end point positions.

Next, to extract the bending point, as shown in FIG. 9, set a small window 31 for image tracking within the region of interest ar(i) in the same way as above, and move it in the direction of the arrow in the figure. Accordingly, the image 32 within the region of interest ar(i) is tracked. In this case, at each movement position of the window 31, the center point 38 of the image 32 within the window 31 is calculated, and the difference in displacement of the J coordinate value with the center points on both sides of the center point of interest is determined as the threshold value. When the above conditions occur, the bending point Q is extracted.

Now, the coordinates of the lower left point of the tracking window 31 are (tt, l
, jw), each axis of the window 31 in the positive direction and the J direction is d++dJ, and the total brightness within the window 31 is s
gi, the sum of products of the brightness in the window 31 and the J coordinate value is sgi j, and when the tracking direction is the positive direction, the next window position (jw'.

jw ′) is obtained by the following ■■2.

In the above equation, 5 kip is the distance that the window 31 moves in one step.

This tracking shows the position Nw' of the window 31.

jw') goes outside the region of interest arci) or collides with an end point, and as a result, the I coordinate data series 1c(k) (k - 0,
1,...+, n c-+ ) and a J coordinate data series jc(k) are obtained. Note that nC is the number of central points extracted.

The degree of curvature β(k) at the center point is now 0
Defined by the formula (O<k<'Hc-,).

1 (k) −abs(2Jc (k) jc (k
-1)-jc (k+1)) ...■ In the above formula, abs indicates an absolute value.

As a result of executing this 0 expression calculation, ! (k)>th2
(However, when th2 is a threshold value), the second center point is set as the mth bent portion lβ [m]. However, m=o, 1. .......

nn-1, and this n7! indicates the number of bent parts.

FIG. 10 shows that as a result of calculating the degree of curvature N [kl at the 0th to 12th center points, it was determined that bent portions exist at each of the 3rd, 5th, and 9th center points.

Next, the m-th bending portion βjl! To find the coordinates of the bending point Q corresponding to (m), first the m-1th bending part (
m-th bending portion β from the center point data of 11cm-1)
A straight line is applied to the center point data of A(m) by the least squares method to obtain a straight line L1 shown by the following linear equation.

al ・I +bl ・J +cl= O・---(m
In the above formula, al, bl, and cl are parameters of this linear equation.

Next, from the center point data of the m-th bending part β/(m), m+1-th bending part 7! When a straight line is applied to the center point data of A[m+1) by the least squares method, a2
．． b2. A straight line L2 expressed by the following equation with c2 as a parameter is obtained.

a2 ・I b2 ・J +c2= O----・■Thus, the coordinates (iβ (m), jβ [m]) of the bending point at the m-th bending portion lβ[m] are the above two straight lines L
It is found as the intersection of I and L2.

In addition, when applying the above least squares method, if the m-1th bent portion βN (m-1) does not exist, the coordinate data of the tracking start point of the image 32 is used, and n1+
1st bending part molecule fj! If (m+l) does not exist, the coordinate data of the tracking end point of image 32 is used.

Returning to FIG. 5, in step 5, the CPUI extracts end points and bending points as feature patterns for the image in the first region of interest ar(i) by implementing the feature pattern extraction method described above. Each feature pattern extracted here is displayed on the display unit 6 in the next step 6. Next, in step 7, when any characteristic pattern is selected by operating the operating section 7, several attributes of that characteristic pattern are generated and stored in the attribute storage section 5.

FIG. 6 shows four areas of interest a set in the image area 30.
rl to ar4, and one of the attributes of the feature pattern is which region of interest the feature pattern is included in among these regions of interest. For this purpose, the number ar of the region of interest is used as the attribute data, but if the data value is negative, it is assumed that there is no region specification.

FIG. 11 shows the height ph of the feature pattern 40 in three-dimensional space, and this height ph can be obtained by stereoscopic viewing using a camera model, slit light, and image point coordinates. This height ph is constant as long as the object to be imaged is in a stable posture, so it is treated as one of the attributes of the characteristic pattern, but if the data value is negative, it is assumed that there is no height specification.

Figure 12 shows the format of attribute data for the feature pattern type ps, including whether the feature pattern is an end point or an inflection point, if there is an end point, whether it is a left end point or a right end point, and the inflection point. If so, information such as what the model number is is included. Here, the model number of the bending point is a number set by classifying the bending point according to its shape and degree of bending, and an example thereof is shown in FIG. In FIG. 13, aLbLa2. b2 is a parameter of the linear equation. Note that when the type ps is 0, it is assumed that there is no type specification.

FIG. 14 shows the order of the feature patterns 40 in the region of interest ar(i).
Ordinal number pp as attribute data in descending order of coordinate data
is given. Note that here too, when the ordinal number pp is negative,
It is assumed that there is no ordinal specification.

FIG. 15 shows the structure of attribute data for each feature pattern to which labels from 0 to mp-1 are assigned. For example, for a feature pattern assigned a label of i, the region of interest is ar(i), The height in three-dimensional space is ph[
i), the type of feature pattern is pSC1], the ordinal number of the region of interest is ppC1], and the segment number is pi(i). Note that the segment number here indicates which segment of the slit image the feature pattern belongs to, and if this value is negative, it is assumed that this number is not specified.

Returning to FIG. 5, in step 9, each of the above attribute data for one feature pattern is generated and stored in the attribute storage unit 5.
, the next feature pattern is selected and a similar attribute data generation procedure is executed. When the generation of attribute data for all feature patterns in the first region of interest is completed, the determination in step 10 becomes "YES" and the process proceeds to step 11. In this step 11, it is determined whether or not to set another region of interest. If the determination is "YES", return to step 2, specify a new region of interest, and extract the feature pattern in the same way. Attributes for each feature pattern are generated sequentially.

FIG. 16 shows the processing procedure for the image to be processed. At the start point in the figure, when a plate-shaped slit light beam is irradiated onto the imaging target to form an intersection line of the slit lights on its surface, this intersection line is imaged by the imaging device 8 in step 21, and the slit image is The image is stored in the image storage unit 4.

In the next step 22, it is determined whether or not a region of interest has been set for this slit image, and the determination is "
If the answer is "YES", one of the regions of interest is set using the operation unit 7 in step 23, but if the determination is "NO", the entire image is determined as one region of interest (step 24). . In this embodiment, as described above, a plurality of regions of interest arl to ar4 are sectioned, so step 22 is "YES", and step 23 is
First, the first region of interest arl is set.

Next, in step 25, the CPU extracts end points and bending points as feature patterns from the image in the region of interest arl in the same manner as described above, and then in step 26, extracts the attribute data for each feature pattern, that is, the region of interest,
Attribute data such as the height in the three-dimensional space, the type of feature pattern, the ordinal number of the region of interest, and the segment number are generated.

Next, in step 27, the CPUI compares these attribute data of each feature pattern with the attribute data of each feature pattern for the standard model stored in the attribute storage section 5, and labels each feature pattern. conduct.

Now, attribute data of a certain feature pattern generated in step 26 is defined as: the region of interest is ar', the height in three-dimensional space is ph', the type of feature pattern is ps', the ordinal number of the region of interest is pp', and the segment number. is p1', whether or not to give the standard model feature pattern label i to this feature pattern is determined by the function v (i
) is determined by whether the value is 11 or 0. In other words, when v (i) - 1, it is determined that the label i is a label candidate for that feature pattern, and when v (i) - 0, it is determined that label i is not a label candidate for that feature pattern. be.

v (i) = v, , (ar', ar (t) )
&&Vh (ph', ph (i)) &&
Vs (ps', ps (i)) && Vp (pp', pp (il) && VL (p
l', pi (i)) 1.・・・ @ During the above ceremony,
vllr + vh + v, + vll +
``L indicates a logical function to be compared with that of the standard model for each attribute of the region of interest, height in three-dimensional space, type of feature pattern, ordinal number of the second region of interest, and segment number, and && is the logical product of the two. It shows.

Among the above logical functions, the logical functions v, r(
ar', ar (i)) is "1" when ar' - ar (i) or ar (i) < Q, otherwise "
0".

Logical function of height Vh (ph', ph (i))
is abs (ph' - ph Ci) ) <th
h (where thh is the threshold) or phc
+)<0, it is "1", and otherwise it is "0".

The logical function Vs (ps', ps C1)) of type is p
s'-ps(i) or ps(i)=O or ps'・
When pS(i)=1, it is "1", otherwise it is "0"
It is.

The ordinal logical function Vp (pp', pp [i)) is
pp'-ppci] or "1" when pp(i)<O
”, and the others are “0”.

9 Logical function ML of segment number (Ill', Ill
[i]) is "1" when pH-pl[i) or pl(i)<0, and is "0" otherwise.

When, as a result of performing the above calculation, there is only one label i that gives v (i) −1 for the characteristic pattern of interest, the CPUI sets this unique label as the label of the characteristic pattern.

Note that if there is no label that gives this v (i) −1 or if there are multiple labels, labeling is determined to be a failure.

Returning to FIG. 16, when labeling for one feature pattern is completed in steps 27 to 29, collation and labeling of attribute data of other feature patterns is completed, and a determination as to whether "verification is complete" is made in step 31. This process is repeated until "YES" is returned. Note that if no match is found as a result of comparing the attribute data, it is checked in step 30 whether there is any unprocessed data, and step 2 is performed.
7 or proceed to step 32.

In step 32, it is determined whether the processing for all regions of interest has been completed, and if the determination is "NO", the process returns to step 23, and a new region of interest is designated.
Similarly, feature pattern extraction, attribute generation for each feature pattern, and attribute data matching are sequentially executed. Then, in step 32, the judgment of “Is the whole area covered?” is “YES”.
S'', the process advances to step 33, where system control processing such as recognition of the object to be imaged is executed, and the process waits for the next object to be imaged.

<Effects of the Invention> As described above, the present invention does not require setting a large number of small windows for extracting feature patterns, and thus does not require extra time for window setting. Also, due to the positional shift of the image, the feature pattern may protrude beyond the window.
There is no possibility that extraction will become difficult, and the invention has a remarkable effect of achieving the purpose of the invention.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the circuit configuration of an image processing apparatus according to an embodiment of the present invention, FIG. 2 is an explanatory diagram showing each segment of a slit image and its positional deviation state, and FIGS. 3 and 4 are A block diagram showing another embodiment of this invention,
FIG. 5 is a flowchart showing the procedure for setting attribute data in the attribute storage unit, FIG. 6 is an explanatory diagram showing a plurality of regions of interest divided within an image area, and FIGS. 7 and 8 show an end point extraction method. Explanatory diagrams, Figures 9 and 10 are explanatory diagrams showing the bending point extraction method, Figure 11 is an explanatory diagram showing the height of the feature pattern in three-dimensional space, and Figure 12 is attribute data about the type of feature pattern. Figure 13 is an explanatory diagram showing the format of the bending point, Figure 14 is an explanatory diagram showing the model number of the bending point.
FIG. 15 is an explanatory diagram showing the order of feature patterns in the region of interest; FIG. 15 is an explanatory diagram showing the attribute data structure of each labeled feature pattern; FIG. The figure is an explanatory diagram showing a positional shift state between a window set for an image to be processed and the image to be processed in a conventional method.

Claims (3)

[Claims] (1) Image generation means for generating an image to be processed; feature pattern extraction means for extracting from the image to be processed a plurality of feature patterns constituting features of the image; and one feature pattern for each feature pattern of the image to be processed. Attribute generation means for generating the above-mentioned attributes; Attribute storage means for previously storing one or more attributes for each feature pattern of the standard model; An image comprising attribute matching means for labeling each feature pattern of the image to be processed by comparing the attributes of each feature pattern with the attributes of each feature pattern for the standard model stored in the attribute storage means. Processing equipment. (2) The image processing device according to claim 1, wherein the characteristic pattern is an end point and a bending point of an image. (3) A patent claim in which the attributes include the image area where each feature pattern is located, the height in three-dimensional space, the type of feature pattern, the number within the image area, and the number of the image to be processed to which the feature pattern belongs. The image processing device according to item 1.