JPH07181011A - Method for recognizing hole through visual sensor - Google Patents

Abstract

PURPOSE:To recognize a hole accurately using a visual sensor. CONSTITUTION:A hole 3 having an outline 3' is made in a work 2 mounted on a work table 1. When the image thereof is picked up by means of a CCD camera 4, the work 2 is illuminated from different directions by lighting the light sources 5A, 5B sequentially thus picking up the image twice. The intensity of each image signal is then normalized and the lower image signal value is selected for each pixel thus forming an image where the shaded parts are collected and emphasized. In general, the images of objective holes are picked up sequentially under a plurality of illumination geometry conditions for shading the different positions thus acquiring the image signal under each illumination geometry condition. An image signal representative of minimum brightness is then selected for each pixel and the hole is recognized based on the selected image signals.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for recognizing a hole (meaning a concave portion or a through hole, the same applies hereinafter) using a visual sensor.

[0002]

2. Description of the Related Art In recent years, a visual sensor having a camera means such as a CCD camera has been widely used for the purpose of automating work in a factory. As one of the usage modes of such a visual sensor, there is a usage method of detecting the position, size, shape, and the like of a hole. For example, when the robot is used for automating the fitting operation, the position of the hole is detected by the visual sensor and the robot position is corrected based on the result. Thus, when accurate information about a hole is needed, the contour of the hole must of course be clearly recognized. However, conventionally, no special method has been established for recognizing holes by a visual sensor,
Under normal lighting conditions, take a normal image with the camera,
A method of analyzing the obtained image information and recognizing the contour of the hole has been adopted.

[0003]

However, in the above-mentioned conventional method, the object seen through the hole interferes with the accurate recognition of the contour of the hole, or the side surface of the hole is illuminated by the illumination to recognize the contour of the hole. There was a problem that it would be difficult to do.

The present invention overcomes such problems and
It is an object of the present invention to provide a method of recognizing a hole by a visual sensor that does not cause recognition failure due to the effect of an object seen through the hole and the brightness of the side surface of the hole.

[0005]

According to the invention of the present application, "a camera means of a visual sensor is used to sequentially image a target hole under a plurality of illumination geometry conditions that form shadows on different portions, and Obtaining an image signal under illumination geometry conditions, selecting one of the stored image signals that exhibits the minimum brightness for each pixel, and forming the different image parts on the basis of the selected image signal. The above technical problem is solved by a method of recognizing a hole by a visual sensor, characterized in that it includes a step of forming an image in which the formed shadows are collected and emphasized.

[0006]

The principle of the present invention will be described with reference to FIGS. 1 and 2, taking as an example the case of recognizing a hole of a perforated work placed on a work table by a visual sensor including a CCD camera. In FIG. 1, reference numeral 1 denotes a work table, on which a perforated work 2 is placed, and a hole 3 formed in the central portion of the work 2 is imaged by a CCD camera 4. 4'represents the field of view. 5A and 5B are illumination light sources that illuminate the work 2 from different directions. The hole 3 is drawn here as a circular recess having a contour 3 ', but the contour may be polygonal and may penetrate the work 2. Also, another object may be visible through the bottom of the hole or through the hole.

Under such an arrangement, the images obtained when the illumination light sources are individually turned on and the image is taken by the camera 4 are as shown in FIGS. 2 (1) and 2 (2). Become. That is, when only the illuminating light source 5A is turned on, the outline 3'C of the hole is reflected over almost the right half of the hole in the figure, and the contrast is clear and the width is substantially wide over the left half of the hole. While the image in which the shaded portion 3A ′ with a shadow is reflected is obtained, when only the illumination light source 5B is turned on, the contour line 3′C of the hole is reflected over almost the half circumference on the left side of the hole, and about half the circumference on the right side. Therefore, it is possible to obtain an image in which the shadow portion 3B ′ having a clear contrast and a wide width is reflected. Then, each shaded portion 3A ′,
The outer edge of 3B 'is characterized in that it corresponds to the contour 3'of the hole.

2 (1) and 2 (2), since the image of the hole is composed of a wide shadow line and a thin contour line with a light and dark contrast, these images were analyzed as they were. It is difficult for the visual sensor to accurately recognize the hole 3. Especially when another object is visible at the bottom of the hole or through the hole, the possibility of misidentifying the hole increases.

Therefore, according to the present invention, if the illumination conditions are changed so that the holes to be recognized are irradiated with light from different directions as in the above example, clear shadows having a wide width are formed in different portions, and Paying attention to the fact that the outer edge of this shadow matches the contour of the hole, the shadow parts were gathered by performing a process equivalent to extracting the shadow parts from the images obtained under each lighting condition and overlapping them. An image is acquired and a hole is recognized based on the image.

In the above example, from the images of FIGS. 2 (1) and 2 (2), FIG. 2 having an outer edge corresponding to the contour 3'of the hole 3 is obtained.
The image 3D 'of (3) is obtained, and the hole 3 can be easily recognized by analyzing the image 3D'. In order to obtain the image 3D ′ of FIG. 2 (3) from the images of FIG. 2 (1) and (2), the darker one of FIG. 2 (1) and FIG. 2 (2) is selected for each pixel, An image composed of a set of selected pixels may be constructed.

If there is a possibility that the brightness level between the picked-up images may be totally changed by changing the illumination geometry condition, the standardization process between the images may be performed as necessary. A region S indicated by a broken line in FIGS. 2A to 2C is a reference region for this standardization process (for an example of the standardization process using the reference region S, see the embodiment).

Since the shadow part set emphasizing process including such standardization is a simple process that can be executed by a simple software process utilizing the image processing function of a normal visual sensor system, it is particularly expensive. There is no need to newly prepare a complicated device.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 3 is a block diagram showing the essential parts of an example of a visual sensor system used in carrying out the method of the present invention. To explain this, 10 is an image processing apparatus, which has a central processing unit (hereinafter referred to as CPU) 11, and the CPU 11 has a camera interface 12 connected to the camera 4 in FIG. A frame memory 13, an image processor 14, and a CPU, which store the image signal captured via 12
A program memory 15 storing a program for controlling the operation of each part of the image processing system via 11, and a monitor 30 for displaying an image before or after image processing.
A monitor interface 16 connected to (for example, a monitor CRT) stores data such as various setting values, and
External data such as a data memory 17, which is also used as a data temporary storage means when the CPU 11 executes an operation, and an illumination light source switching device 20 for switching ON / OFF of the illumination light source devices 5A and 5B shown in FIG. 1 and a process controller. General purpose signal interfaces 18 connected to the sources are each connected via a bus 19.

The configuration described above corresponds to a conventional visual sensor system to which the illumination light source switching device 20 is added. In the present embodiment, an image processing program for executing the image processing described below and Related setting values are stored in the program memory 15 and the data memory 17.

Image processing in this embodiment will be described below with reference to the flow chart of FIG. Figure 4
The flowchart of FIG. 1 is a CPU of the image processing apparatus 10 in order to apply the present invention to the cases shown in FIGS.
The outline of the content of the image processing for emphasizing by gathering the shaded areas executed by 11 is described. The number of pixels of the camera used in this example is 512 × 512,
2 (1) and 2 (2), the image signal intensity at the i-th row and the j-th column is Z (1) ij, Z (2) ij (i, j = 1, 2, ...
512). Further, a method is adopted in which the final image is constructed by performing the standardization so that the average output of the image signals in the reference area S is made equal and then selecting the minimum signal value for each pixel.

The series of processes described in the flowchart of FIG. 4 is started when a processing start command is transmitted from an external signal source such as a process controller to the CPU of the image processing apparatus 10.

When the process is started, first, a command is sent to the illumination light source switching device 20 via the general-purpose signal interface 18 to turn on the illumination light source 5A independently (step S).
1), the work 2 is imaged through the camera interface 12 at a timing when the light power of the light source is stable, and the image (1) is captured in the frame memory 13 (# 1) (step S2).

Next, similarly, the illumination light source 5B is independently turned on (step S3), and the frame memory 13 for the image (2).
Import to (# 2) (step S4) Sequentially execute the same.

When the two types of images (1) and (2) are acquired, the average values Zav (S) 1 and Zav (S) 2 of the average image signal intensities for the reference area S in both images are calculated. (Step S5). Then, the correction coefficient γ12 is set to γ12 = Z
It is calculated by av (S) 1 / Zav (S) 2 (step S6). Depending on the characteristics of the image signal, it may be appropriate to normalize γ12 as the shift correction amount Zav (S) 1 -Zav (S) 2 instead of the correction coefficient.

Next, the pixel position indexes i and j are reset to 1 (step S7), and the process proceeds to step S8. In step S8, the minimum value Min. Of the video signals Z (1) ij and γ12Z (2) ij standardized with respect to the pixel in the i-th row and the j-th column. Zij
To decide. When a correction shift amount is used instead of a correction coefficient, Z (1) ij and Z (2) ij + γ12 are compared to determine a value that is not larger. Determined Min. The value of Zij is written in the address corresponding to pixel (i, j) in the frame memory 13 (# 3).

Since the processing for one pixel is completed as described above, the same processing may be repeated for each pixel thereafter. That is, in step S9, 1 is added to the column index j, and after confirming that j does not exceed 512, the process of returning to step S8 is repeated 511 times, a yes determination is made in step S10, and step S11 is performed. move on. In step S11, 1 is added to the row index i, i = 5
After confirming that it is not 13 (step S12), only the column index j is reset to 1 (step S13), and the process returns to step S8 again. Hereinafter, by repeating the same process, the minimum value Min. When Zij is written in the address corresponding to the pixel (i, j) of the frame memory 13 (# 3), step S
The image processing for assembling and emphasizing the shaded portions through 9 → step S10 (yes) → step S11 → step S12 (yes) ends.

The image obtained from the image signal written in the frame memory 13 (# 3) is an image in which the shaded parts under each illumination condition are gathered and emphasized, and the unevenness of brightness is suppressed by the standardization. It has become an image. Once such an image is acquired, for example, from the line image 3C 'and the shadow inner edge image 3C "corresponding to the contour 3'of the hole 3 as shown in FIG. Image is obtained, and image processing for removing the inner line image 3C ″ is performed to obtain only the line image 3C ′ corresponding to the contour 3 ′ of the hole 3 as shown in FIG. 5B. Obtaining the image can be easily performed using well-known software processing. And from this line image 3C ',
For example, it goes without saying that the center position O and the radius R of the circular hole 3 can be obtained based on the coordinate values of the three points P1 to P3 on the pixel surface.

It should be noted that the image based on the image signal stored in the frame memory 13 (# 3) is periodically displayed on the screen of the monitor 30 so that it is possible to visually visually confirm that each of the above-mentioned image processing is being executed accurately. It is also possible to add processing for displaying.

Although the above embodiments have been described under the condition that two illumination devices are switched for a circular hole, any shape such as a quadrangle or an ellipse can be used as appropriate. The holes can be recognized by selecting the number of lighting devices to be operated, the number of switching times, and the like. For example, 6 lighting devices are lit up from the 3 directions for each of the triangular holes, and 3 lighting devices are turned on.
It is possible to acquire three types of images under different types of illumination geometry conditions and perform the same image processing as described above.

[0025]

According to the present invention, holes can be recognized accurately by a simple process using a visual sensor.

Therefore, in an application such as automation of fitting work, a system using a visual sensor is adopted without worrying about malfunction due to erroneous recognition of hole position, size, type, etc., and work with high accuracy can be performed. It becomes possible.

[Brief description of drawings]

FIG. 1 is a diagram for explaining the principle of the present invention by exemplifying a case in which a hole of a perforated work placed on a work table is recognized by a visual sensor including a CCD camera.

FIG. 2 shows images (1) and (2) obtained when the illumination light sources are individually turned on under the arrangement shown in FIG. It is the figure which illustrated the image (3) which was set and emphasized.

FIG. 3 is a principal block diagram showing an example of a visual sensor system used when performing the method of the present invention.

4 is a flowchart outlining the contents of processing executed by the CPU of the image processing apparatus shown in FIG. 3 in order to apply the present invention to the cases shown in FIGS. 1 and 2.

5A and 5B are views for explaining that a contour image of a hole is obtained by further performing necessary processing on the image obtained by the processing shown in FIG. 4, where (1) is edge enhancement processing and (2) is Further, it represents an image obtained by performing a process of removing the inner line image.

[Explanation of symbols]

 1 Work Table 2 Work 3 Hole 3'Hole Outline 4 CCD Camera 4 CCD Camera Field of View 5A, 5B Illumination Device 10 Image Processing Device 11 Central Processing Unit (CPU) 12 Camera Interface 13 Frame Memory 14 Image Processing Processor 15 Program Memory 16 Monitor Interface 17 Data Memory 18 General Purpose Signal Interface 19 Bus 20 Illumination Light Source Switching Device 30 Monitor

Claims (1)

[Claims] 1. A camera means of a visual sensor is used to sequentially image a target hole under a plurality of illumination geometry conditions that form shadows at different portions, and an image signal under each illumination geometry condition is acquired. And an image in which shadows formed in the different parts are aggregated / emphasized based on the selected image signal are selected based on the selected image signal. A method for recognizing a hole by a visual sensor, comprising the steps of configuring.