JPH0696193A - Method and device for displaying component identification result - Google Patents

Abstract

PURPOSE:To accurately perform teaching without operating an industrial robot actually and to easily limit a range of malfunction when the supply of components is performed erroneously by performing a feature value extraction processing based on the detection image of the components, identifying the components based on an extracted feature value, and displaying an identification result with the detection image. CONSTITUTION:This method and device is equipped with an image detecting part 4 consisting of a CCD camera, etc., and an image processing part 5 which performs binarization processing, the feature value extraction processing, component identification processing. etc., setting an image detection signal from the image detecting part 4 as input and displays a binary image and a component identification result on a display 5a visually, etc. In the component identification processing, the binarization processing is applied to a fetched image, and the binary image at every component is extracted by performing connectivity analysis based on the binary image, and the feature values of an area value, peripheral length, circumscribing square length, and the number of holes, etc., can be obtained basing on the binary image, then, it is discriminated whether or not the component is the one of identification target basing on the obtained feature value and corresponding threshold value.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for displaying a component identification result, and more specifically, it detects an image of at least one component located within a field of view by an image detection device such as a CCD camera. The present invention relates to a novel method and apparatus for visually displaying a component identification result obtained based on a detected image.

[0002]

2. Description of the Related Art Conventionally, in a component feeder having a bowl feeder as a main part, instead of providing a mechanical component aligning mechanism to align the components, the component transport provided around the bowl for storing the components. An image of a predetermined area of the truck is detected by an image detection device such as a CCD camera, the type and position of the parts are identified based on the detected image, and the industrial robot or the like is operated based on the identification result. It has been proposed to supply the above parts to a desired position. However, the component conveying track is configured so that the overlapped components can be returned to the bowl again (see JP-A-60-118479).

If such a configuration is adopted, since the image on the component conveying track is captured by the image detecting device, the type of the component can be accurately identified and the desired component can be extracted by the industrial robot. Therefore, tooling for each part is not required, and various kinds of parts can be easily dealt with.

[0004]

When the component supplying apparatus having the above-mentioned configuration is adopted, the image of a predetermined area of the component conveying truck is simply detected, and the identification of the component based on the detected image is performed. Since the control signal including the operation command is supplied to the industrial robot based on the identification result, whether the instruction for identifying the component is accurate, or whether the teaching of the gripping position of the identification target component is accurate. If it is attempted to discriminate whether or not it is necessary to operate the industrial robot and actually carry out the component extraction process in addition to the image detection process and the component identification process, the above-mentioned discrimination work is complicated. There is an inconvenience of becoming

Further, if a component supply error occurs during the actual component supply operation, whether a malfunction has occurred on the side that performs image detection and component identification based on the detected image, it is Since it is not possible to determine whether or not a malfunction has occurred on the side of the industrial robot that takes out the product, there is the inconvenience that a great deal of labor and time are required to investigate the cause of component supply error. .

[0006]

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and it is possible to achieve accurate teaching without actually operating an industrial robot and to prevent malfunctions when a component supply error occurs. It is an object of the present invention to provide a component identification result display method and device that can easily limit the range of the operation.

[0007]

In order to achieve the above object, the method for displaying a component identification result according to claim 1 detects an image of a component located in the visual field by an image detecting means, and based on the detected image. It is a method of performing a feature value extraction process, identifying a component based on the extracted feature value, and visually displaying a component identification result together with a detected image.

In the component identification result display method according to the second aspect, a rectangle surrounding the image of the identified component is adopted as a visual display of the component identification result. In the component identification result display method according to the third aspect, the number of identified components in the field of view is employed as a visual display of the component identification result. The component identification result display method according to claim 4 further includes coordinates of a predetermined position of the identified component as a visual display of the component identification result.

The component identification result display method according to the fifth aspect further includes a predetermined reference axis of the identified component as a visual display of the component identification result. According to a sixth aspect of the present invention, there is provided a component identification result display device which includes image detection means for detecting an image of a component located within a field of view, feature value extraction means for performing a feature value extraction process based on the detected image, and the extracted feature values. It includes a component identifying means for identifying the component on the basis of the component and a visual display means for visually displaying the component identification result together with the detected image.

[0010]

According to the component identification result display method of the first aspect, the image of the component located within the field of view is detected by the image detecting means, the characteristic value extraction processing is performed based on the detected image, and the extracted characteristic value is obtained. Since the parts are identified based on the result and the result of the parts identification is visually displayed together with the detected image, the teaching data for identifying the parts is accurate without actually performing the parts picking operation by the industrial robot. Whether or not the teaching data is valid can be simplified. Also, if a component supply error occurs while the industrial robot is actually taking out components, is the image processing side malfunctioning by observing the component identification result visually displayed along with the detected image? It is possible to easily determine whether the operating robot side is malfunctioning, reduce the labor required for investigating the cause of component supply error, and significantly reduce the time required.

According to the component identification result display method of claim 2, since a rectangle surrounding the image of the identified component is used as a visual display of the component identification result,
In addition to the above action, the visibility of the component identification can be significantly improved.
In this case, it is preferable to use a circumscribing rectangle as the rectangle, and when a value that defines the circumscribing rectangle is used as the feature value, the process of generating a rectangle for display is not required. The visibility can be improved without being changed.

According to the component identification result display method of claim 3, since the number of identified components in the visual field is employed as a visual display of the component identification result, for example, when the component supply device is started up. In the test run, if the number of parts is different for each type, it is possible to easily determine which part was identified by referring to only the number, and if there is an equal number of parts for all types It is possible to determine whether or not the component identification has been accurately achieved.

According to the component identification result display method of claim 4, since the visual indication of the component identification result further includes the coordinates of the predetermined position of the identified component, the component extraction result is extracted. Not only can you easily and accurately grasp the data when creating an operation program for an industrial robot to improve workability for creating an operation program, but also the coordinate system of the industrial robot and the coordinate system of the image processing device side. It is possible to easily determine whether or not the coordinate alignment with and is properly achieved.

According to the component identification result display method of claim 5, since the predetermined reference axis of the identified component is further included as a visual display of the component identification result, the visibility of the component identification is remarkably increased. Can be improved. In this case,
It is preferable to use the inertial spindle as the reference axis, and when adopting a value that defines the relationship between the inertial spindle and other reference axes etc. as the feature value, there is no need to specially generate the inertial spindle for display. Therefore, the visibility can be improved without complicating the processing. Further, it is possible to easily grasp the value based on the inertial spindle as the data at the time of creating the operation program of the industrial robot for taking out parts, and it is possible to enhance the workability for creating the operation program.

According to the sixth aspect of the component identification result display device, the image of the component located in the field of view is detected by the image detecting means, and the characteristic value extracting means performs the characteristic value extraction processing based on the detected image and extracts the characteristic value. Since the parts are identified by the parts identification means based on the identified feature values and the parts identification result is visually displayed together with the detected image by the visual display means, the parts removal operation by the industrial robot is not actually performed. However, it is possible to determine whether the teaching data for identifying the component is accurate, and it is possible to simplify the work for determining the validity of the teaching data. Also, if a component supply error occurs while the industrial robot is actually taking out components, is the image processing side malfunctioning by observing the component identification result visually displayed along with the detected image? It is possible to easily determine whether or not the operating robot side is malfunctioning, and it is possible to reduce the labor required for elucidating the cause of the component supply error and to significantly reduce the time required.

[0016]

Embodiments will be described in detail below with reference to the accompanying drawings showing embodiments. FIG. 1 is a partially transparent perspective view showing an embodiment of a component supply system to which the component identification result display method of the present invention is applied, and FIG. 2 is a schematic vertical sectional view showing a casing 1 made of a non-translucent material. Above and below a translucent plate 2c that accommodates the bowl feeder 2 inside and covers the opening formed at a predetermined position of the component transport track 2b arranged so as to surround the bowl 2a of the bowl feeder 2. The light sources 3a and 3b are arranged respectively in the. The half mirror 4a is provided between the translucent plate 2c and the light source 3b.
And an image detection unit 4 including a CCD camera for detecting an image of a region including the translucent plate 2c via the half mirror 4a and the reflection mirror 4b. Further, the image detection signal from the image detection unit 4 is input to perform binarization processing, feature value extraction processing, component identification processing, and the like, and the binarized image and the component identification result are visually displayed on the display 5a. The image processing unit 5 is provided. Further, the casing 1 has an opening 1a directly above the translucent plate 2c, and the opening 1a
It has a door member 1b capable of reciprocating between a closed position and an open position. The image processing unit 5 also generates an industrial robot control signal based on the component identification result, an image detection control signal for controlling the operations of the light sources 3a and 3b, and the door member 1b. Further, an industrial robot 6 and a robot controller 6a for taking out parts while maintaining a predetermined relative position with respect to the casing 1 are arranged.

Both of the light sources 3a and 3b may be fluorescent lamps, but only one light source or both light sources are electroluminescence (hereinafter abbreviated as EL) surface light emitters, LEDs and the like. Good. The operation of the component supply system having the above configuration will be described with reference to the flowcharts shown in FIGS.

First, the flow chart of FIG. 3 showing the part identifying process will be described. In step SP1, the light source 3a is turned on, in step SP2, an image of a predetermined area including the translucent plate 2c is captured by the image detection unit 4, and in step SP3, the captured image is binarized, and then step SP4. In step SP5, the connectivity analysis based on the binarized image is performed to extract the binarized image for each part, and in step SP5, the features such as the area value, the perimeter, the circumscribed rectangle length, and the number of holes are based on the binarized image. A value is obtained, and in step SP6, it is determined whether or not the component is an identification target based on the obtained feature value and the corresponding threshold value. If it is determined that the parts are the identification target parts, the inertial spindle, the center of gravity position, the holding position, etc. are calculated in step SP7, and the number of the identification target parts is increased by 1, and all the parts are identified in step SP8. It is determined whether or not the processing based on the binarized image is performed, and when it is determined that the processing based on all the binarized images is performed, the series of processing is ended as it is. When it is determined in step SP6 that the component is not the identification target, the determination in step SP8 is performed as it is. When it is determined in step SP8 that there is a binarized image that has not been processed, the process of step SP5 is performed again. Although not shown in this flowchart, if the component identification accuracy based on the silhouette image is not high, the light source 3b is turned on and the same processing is performed.

Next, FIG. 4 showing the part identification result display processing
The flowchart will be described. In step SP1, it is determined whether or not the display of the component identification result is requested, and if the display of the component identification result is not requested, only the binarized image is visually displayed in step SP2, and the process is ended as it is. To do. On the contrary, if the display of the component identification result is requested, it is determined in step SP3 whether or not the display of the component identification result is based on the already obtained information, and the already obtained information is obtained. If it is based on the display, the binarized image and the image based on the already obtained information are combined and visually displayed in step SP4, and a series of processing is ended. When it is determined in step SP3 that the display of the component identification result is a display other than the display based on the already obtained information, the information for the necessary display is generated in step SP5, and in step SP6. The binarized image and the image based on the information generated in step SP5 are combined and visually displayed, and a series of processing ends.

Although the flowchart of FIG. 4 is based on the premise that the display of the identification result can be freely selected, if the display of the identification result is predetermined, step SP
The determination process of 3 may be omitted, and the processes of step SP4 or steps SP5 and SP6 may be performed according to a predetermined display. FIG. 5 is a diagram showing an example of the display of the identification result, which visually displays the circumscribed rectangle of the identified component together with the binarized image.

Therefore, the part in which the circumscribed rectangle is displayed can be immediately recognized as the identified part, and whether or not the parts are accurately identified can be determined by the industrial robot 6
Can be easily identified without operating. FIG. 6 is a diagram showing another example of the display of the identification result, which visually displays the number of identified components together with the binarized image.

Therefore, when the numbers of the respective types of parts displayed as the binarized image are different from each other, the types of the identified parts can be recognized based on only the number, and conversely, the types of the parts can be recognized. When the numbers are equal, it is possible to easily determine whether or not the parts are correctly identified based on whether the displayed number is equal to the actual number without operating the industrial robot 6.

FIG. 7 is a diagram showing still another example of the display of the identification result, which visually displays the binarized image, the circumscribed rectangle of the identified part, and the principal axis of inertia of the identified part. Therefore, the same effect as in the case of FIG. 5 can be achieved due to the circumscribed rectangle being visually displayed, and at the time of creating a robot program for operating the industrial robot 6 with the inertial spindle being displayed. It is possible to simplify the work of creating a robot program by adopting it as the data of.

FIG. 8 is a view showing still another example of the display of the identification result, and the binarized image, the circumscribed rectangle of the identified part and the coordinate value indicating the holding position of the identified part are visually displayed. is doing. Therefore, the same effect as in the case of FIG. 5 can be achieved due to the visible display of the circumscribed rectangle, and at the time of creating the robot program for operating the industrial robot 6 with the displayed coordinate values. It is possible to simplify the work of creating a robot program by adopting it as the data of. Further, since the displayed coordinate value is a value in the coordinate system of the industrial robot 6, for example, by operating the industrial robot 6 based on the corresponding coordinate value, the coordinate system of the image detection unit 4 and the industrial coordinate system are changed. It is possible to determine whether or not the coordinate system of the robot 6 is the same, and it is possible to perform coordinate adjustment between both coordinate systems based on this determination result.

In addition, it is possible to add a display such as a part name when performing each of the above-mentioned displays, and it is possible to display all of these parts identification results at the same time.

[0026]

[Embodiment 2] FIG. 9 is a block diagram schematically showing the configuration of a component supply system including a component identification result display device of the present invention. The image obtained by the image detection unit 4 is binarized and binarized. A binarization processing unit 71 that obtains an image, a binarized image extraction unit 72 that extracts a binarized image for each part by performing connectivity analysis or the like based on the binarized image, and the extracted binarization A feature value calculation unit 73 that calculates a feature value based on the image, a component determination unit 74 that determines whether the component is an identification target component based on the calculated feature value, and a component identification target component Visual display for synthesizing and visually displaying a component identification result display image data output unit 75 for outputting image data for component identification result display and a binarized image and an image for component identification result display And a portion 76.

The operation of each component described above is the same as the process of the corresponding step in the flow charts of FIGS. 3 and 4, so a detailed description will be omitted. The operation of the above component supply system is as follows. If an image of a predetermined area including the translucent plate 2c is captured by the image detection unit 4, the binarization process is performed in the binarization process 71 to obtain a binarized image in which the influence of ambient light is eliminated. . Then, based on the binarized image, the binarized image extraction unit 72 performs connectivity analysis or the like to extract the binarized image for each component. After that, at least one type of feature value is calculated in the feature value calculation unit 73 based on each extracted binarized image, and the component determination unit 74
In, the calculated feature value is compared with a predetermined threshold value or the like to determine whether or not the component is an identification target.

After the discrimination results corresponding to the binarized images of all the parts are obtained, the image for displaying the part identification result is output from the part identification result display image data output unit 75 in correspondence with the part to be identified. The data is output, and all the binarized images and the image for displaying the component identification result are combined and visually displayed on the visual display unit 76. Therefore, the visibility of the component identification result can be significantly improved due to the visual display including the image for displaying the component identification result. Further, as is clear from the above description, since it is not necessary to actually operate the industrial robot 6, it is possible to easily determine whether or not the component identification is accurately achieved, and the image for displaying the component identification result is displayed. Depending on the type, it can contribute to simplification of the work of creating a robot program for operating the industrial robot 6. Furthermore, if a component supply error occurs while components are actually being supplied, it is possible to easily determine whether or not the component identification has been accurately achieved as described above. The work for investigating the cause can be simplified and the required time can be shortened.

The present invention is not limited to the above-described embodiment, and can be applied to, for example, a component supply system having only one light source, and component separation other than the bowl feeder. The present invention can be applied to a parts supply system incorporating a transfer mechanism, and various design changes can be made without departing from the scope of the present invention.

[0030]

As described above, according to the first aspect of the present invention, it is possible to determine whether or not teaching data for identifying a part is accurate without actually performing a part picking operation by an industrial robot. Not only can the work to determine the validity of the data be simplified, but if a part supply error occurs while the part is actually being picked up by an industrial robot, it will be visually displayed along with the detected image. It is possible to easily determine whether the image processing side is malfunctioning or the industrial robot side is malfunctioning by looking at the component identification result, and it is possible to reduce the labor required for investigating the cause of the component supply error and to significantly reduce the time required. It has a unique effect that it can be shortened to.

In addition to the effect of the first aspect, the invention of the second aspect has a unique effect that the visibility of the component identification can be remarkably improved. According to the invention of claim 3, if the number of parts is different for each type, it is possible to easily determine which part is identified by referring to only the number, and the same number of parts exists for all types. If so, there is a unique effect that it is possible to determine whether or not the component identification has been accurately achieved.

According to the fourth aspect of the invention, the data at the time of creating the operation program of the industrial robot for taking out parts can be easily and accurately grasped, and only the workability for creating the operation program can be improved. In addition, there is a unique effect that it is possible to easily determine whether or not the coordinate system of the industrial robot and the coordinate system of the image processing apparatus side are properly achieved.

In addition to the effect of claim 2 or claim 3, the invention of claim 5 has a unique effect that the visibility of component identification can be further improved. According to the sixth aspect of the present invention, it is possible to determine whether or not the teaching data for identifying the part is accurate without actually performing the part taking out operation by the industrial robot, and to determine the validity of the teaching data. Not only can the work be simplified, but even if a part supply error occurs while the part is actually being picked up by an industrial robot, you can see the part identification result that is visually displayed along with the detected image. It is possible to easily determine whether the processing side is malfunctioning or the industrial robot side is malfunctioning, and it is possible to reduce the labor required for investigating the cause of the component supply error and to significantly reduce the required time. .

[Brief description of drawings]

FIG. 1 is a partially transparent perspective view showing an embodiment of a component supply system to which a component identification result display method of the present invention is applied.

FIG. 2 is a schematic vertical sectional view showing a main part of the component supply system.

FIG. 3 is a flowchart illustrating a component identification process.

FIG. 4 is a flowchart illustrating a component identification result display process.

FIG. 5 is a diagram showing an example of display of an identification result.

FIG. 6 is a diagram showing another example of display of identification results.

FIG. 7 is a diagram showing still another example of display of identification results.

FIG. 8 is a diagram showing still another example of display of identification results.

FIG. 9 is a block diagram schematically showing a configuration of a component supply system including the component identification result display device of the present invention.

[Explanation of symbols]

4 Image Detection Section 73 Feature Value Calculation Section 74 Component Discrimination Section 75 Component Identification Result Display Image Data Output Section 76
Visual display

Claims (6)

[Claims] 1. An image detecting means (4) detects an image of a component located within the field of view, performs feature value extraction processing based on the detected image, and identifies the component based on the extracted feature value. A method for displaying a component identification result, which visually displays a component identification result together with a detected image. 2. The component identification result display method according to claim 1, wherein the visual display of the component identification result is a rectangle surrounding the image of the identified component. 3. The component identification result display method according to claim 1, wherein the visual display of the component identification result is the number of identified components in the field of view. 4. The component identification result display method according to claim 2, wherein the visual display of the component identification result further includes coordinates of a predetermined position of the identified component. 5. The component identification result display method according to claim 2, wherein the visual display of the component identification result further includes a predetermined reference axis of the identified component. 6. An image detecting means (4) for detecting an image of a component located within the field of view, a characteristic value extracting means (73) for performing a characteristic value extraction process based on the detected image, and an extracted characteristic value. Component identification means (7 for identifying the component based on
4) and a visual display means (75) (76) for visually displaying the component identification result together with the detected image, the component identification result display device.