JP2020187016A - Positional deviation detection method - Google Patents

Abstract

To provide a positional deviation detection method with which it is possible to improve the accuracy of detecting the outline lines of a workpiece.SOLUTION: Provided is a positional deviation detection method for detecting the amount of positional deviation between the position of a workpiece that serves as a reference taught to a robot and the actual position of the workpiece, said method comprising: (a) step of acquiring, using imaging means, images of a background display part and the workpiece placed on the background display part; (b) step of acquiring the data of outline line data of the workpiece; (c) step of comparing the outline line data against outline line data that serves as a reference, and calculating the percentage of the outline lines acquired; (d) step of changing the color of the background display part stepwise; (e) step of repeating the steps (a) through (d) n times (n=integer 1 or greater); (f) step of extracting main data that the percentage of the outline lines acquired is highest; and (g) step of comparing the main data with a calibration image and calculating the positional deviation of the workpiece as a correction amount.SELECTED DRAWING: Figure 2

Description

The present invention relates to a misalignment detection method.

The image obtained by imaging the work using the imaging means includes an image of the background on which the work is placed in addition to the work. Therefore, a process for distinguishing the work from the background is required. Patent Document 1 discloses a foreign matter detection method in which a plurality of background materials having different optical characteristics are used to irradiate an inspection target with measurement light, and a measurement step, a background determination step, and a non-defective product determination step are repeated.

Japanese Unexamined Patent Publication No. 2016-090476

The inventor has found that it is difficult to apply a method using a plurality of background materials having different optical characteristics to improve the accuracy of detecting an outer line using an imaging device.

The present invention has been made in view of the above problems, and provides a misalignment detection method capable of improving the detection accuracy of the outer line of the work.

The misalignment detection method according to the present invention
It is a position deviation detection method that detects the amount of deviation between the reference work position taught to the robot and the actual work position.
(A) A step of acquiring an image of the background display unit and the work placed on the background display unit using an imaging means, and
(B) The step of acquiring the data of the outer line of the work and
(C) A step of comparing the data of the outer shape line with the data of the reference outer shape line and calculating the ratio of obtaining the outer shape line.
(D) A step of gradually changing the color of the background display unit and
(E) A step of repeating steps (a) to (d) n times (n is an integer of 1 or more) and
(F) A step of extracting the main data having the highest ratio of acquiring the outline, and
(G) A step of comparing the main data with a proof image and calculating the positional deviation of the work as a correction amount is provided.

In the misalignment detection method according to the present invention,
The step of gradually changing the color of the background display unit and
The step of extracting the main data with the highest rate of acquisition of the outline, and
The main data is compared with the proof image, and the step of calculating the positional deviation of the work as a correction amount is provided.
Therefore, the detection accuracy of the outer line of the work can be improved.

According to the present invention, the detection accuracy of the outer line of the work can be improved.

It is a schematic diagram which shows the image processing apparatus used for the misalignment detection method which concerns on embodiment. It is a flowchart which shows the position deviation detection method which concerns on embodiment. It is a top view which showed the position of the boundary part by placing the work on the background display part which concerns on embodiment. It is a top view which showed an example of the outline line of the work which concerns on embodiment. It is a graph which shows the change of the detection accuracy (score) of the outer line of a workpiece when the background color is changed by using the misalignment detection method which concerns on Example. 6 is a graph showing the relationship between the detection accuracy (score) of the outer line of the work and the variation in the correction amount in the x direction when the background color is changed by using the misalignment detection method according to the embodiment. 6 is a graph showing the relationship between the detection accuracy (score) of the outer line of the work and the variation in the correction amount in the y direction when the background color is changed by using the misalignment detection method according to the embodiment.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings.

<Embodiment>
First, the image processing apparatus 1 used in the misalignment detection method according to the present embodiment will be described with reference to FIG.
FIG. 1 is a schematic view showing an image processing apparatus used in the misalignment detection method according to the embodiment. As shown in FIG. 1, the image processing device 1 includes a background display unit 10, an imaging means 20, an image processing unit 30, and a robot arm 40. For example, the image pickup means 20, the image processing unit 30, and the robot arm 40 may be separated from each other, or a robot vision (machine vision) which is an integrated structure may be used.

The work W in the present embodiment is an object that can be imaged by the imaging means 20 described later. For example, engine blocks, housing cases, suspension towers of body parts, suspension members, rear side members, etc., which are cast by cast aluminum die casting. The work W is placed on the background display unit 10. In other words, when the work W and the background display unit 10 are viewed in a plan view, the background display unit 10 is located in the background of the work W.

The background display unit 10 is a screen in which the background color to be displayed can be gradually changed to an arbitrary color. As the background display unit 10, for example, a liquid crystal display or an organic EL display can be used.

The image pickup means 20 is an image sensor (camera) that acquires images of the work W and the background display unit 10. It is preferable that the imaging means 20 is arranged at a position where the work W and the background display unit 10 can be imaged at one time. In particular, it is preferable that the imaging means 20 is arranged above the work W and the background display unit 10. As the imaging means 20, a fixed type fixed above the work W and the background display unit 10 can be used, but the imaging means 20 is not limited thereto. For example, the image pickup means 20 may be a movable one that is used by connecting to a robot arm 40 or another device. As the image pickup device included in the image pickup means 20, for example, a CCD (Charge-Coupled Device) sensor, a CMOS (Complementary Metal-Oxide Semiconductor) sensor, or the like can be used. It is also possible to detect the three-dimensional position of the work by combining, for example, a laser beam in addition to the imaging means.

The image processing unit 30 controls the operation of the image processing device 1, and includes a processor including a CPU (Central Processing Unit), a memory, a communication module, a camera port, and a robot teaching operation panel. The robot teaching operation panel is an operation panel for teaching the area of the image to be acquired. More specifically, the image processing unit 30 processes the image acquired by the imaging means 20, and controls the background display unit 10 or the robot arm 40 based on the processing result. A plurality of camera ports may be provided. It may have a function of controlling a robot separate from the robot arm 40 included in the image processing device 1 by using a communication module. Details regarding the operation of the image processing unit 30 will be described later with reference to FIGS. 2 and 3.

The robot arm 40 holds the work W and corrects the misalignment based on the processing result of the image processing unit 30. The type of the robot arm 40 can be appropriately selected from, for example, a vertical articulated robot, a horizontal articulated robot (SCARA robot), a parallel link robot, a orthogonal robot, and the like, depending on the shape of the work W. The work W may be held as long as it can hold the work W. More specifically, for example, the work W can be appropriately changed according to the type of the work W, such as gripping, holding, and using a magnet.

In the misalignment detection method according to the present embodiment, the above-mentioned image processing device 1 can be used. The misalignment detection method according to the present embodiment is a misalignment detection method for detecting the amount of misalignment between the reference work position taught to the robot and the actual work position.
FIG. 2 is a flowchart showing a misalignment detection method according to the embodiment. The misalignment detection method according to the present embodiment includes at least the following steps (a) to (g) among steps S1 to S14 shown in FIG.
(A) A step of acquiring an image of a background display unit and a work placed on the background display unit using an imaging means (step S2).
(B) A step (step S5) of acquiring data on the outline of the work.
(C) A step (step S6) of comparing the data of the outer line with the data of the reference outer line and calculating the ratio of the outer line being acquired.
(D) A step of gradually changing the color of the background display unit (step S8).
(E) A step (step S7) in which steps (a) to (d) are repeated n times (n is an integer of 1 or more).
(F) A step (step S9) of extracting the main data having the highest ratio of acquiring the outline.
(G) A step (step S11) of comparing the main data with the proof image and calculating the displacement of the work as a correction amount.

Hereinafter, a series of flows of the misalignment detection method according to the present embodiment will be described with reference to FIGS. 1 and 2 and FIGS. 3 and 4 showing a part of the steps of the flowchart of FIG.

<Step S1: Workset>
FIG. 3 is a plan view showing the position of the boundary portion by placing the work on the background display portion according to the embodiment.
In step S1, as shown in FIG. 3, the work W is placed and set on the background display unit 10.
As a method of placing the work W on the background display unit 10, for example, a robot or a machine can be used. An operator may place it. Further, in FIG. 3, as an example, the background display unit 10 having the same shape as the work W in a plan view is used, but the present invention is not limited to this. The shape of the background display unit 10 may be a shape that is larger than the outer shape of the work W when viewed in a plane. More specifically, the shape of the background display unit 10 may be, for example, a rectangular shape or the like, and can be appropriately changed to a desired shape.

As shown in FIG. 3, the background color 10a displayed by the background display unit 10 is displayed at least in a part of the background of the work W. More specifically, it is preferable to display the background color 10a (indicated by the hatching of dots) on the background display unit 10 in the range in which the image pickup means 20 can acquire an image. In other words, in FIG. 3, the portion of the background display unit 10 on which the background color 10a is displayed is the range in which the image pickup means 20 can acquire an image. The range for displaying the background color 10a can be appropriately changed according to the range in which the image can be acquired by the imaging means 20. Further, the display range of the background color 10a is not limited to the range in which the image can be acquired by the imaging means 20, and may be displayed on the entire background display unit 10. The background color 10a may be displayed by the start of step S2, may be displayed at the time when the work W is placed, or may be displayed after the work W is placed.

In the present embodiment, the background color 10a displayed during the series of flows is preferably a single color. As the background color 10a, for example, a color having a complementary color relationship with the color of the work W can be used. When the color of the work W is achromatic, any one of white, black, and blue, which easily contrasts with the achromatic color, can be used.

<Step S2: Image acquisition (a)>
In step S2, the image capturing means 20 shown in FIG. 1 is used to acquire an image of the work W placed on the background display unit 10 and the background display unit 10. It is assumed that the background color 10a is displayed on the background display unit 10.
In the present embodiment, the imaging means 20 is arranged above the work W and the background display unit 10. Therefore, it is possible to acquire a two-dimensional image in which the work W and the background display unit 10 are viewed in a plan view.

Hereinafter, in steps S3 to S14, the image processing unit 30 shown in FIG. 1 performs processing.

<Step S3: Calculate the contrast of the boundary between the work and the background display>
In step S3, the contrast of the boundary portion is calculated.
As shown in FIG. 3, the boundary portion B is a range in which the image pickup means 20 can acquire an image, and is a boundary portion between the work W and the background color 10a displayed by the background display unit 10 when viewed in a plan view. Point to that. In FIG. 3, the boundary portion B is shown by a thick black line. As a method for calculating the contrast of the boundary portion, a difference in brightness, saturation, density, or the like may be used, and a general method for calculating the contrast can be used.

<Step S4: Judgment of the outer line of the work>
In step S4, it is determined whether or not the outer line EL of the work W can be determined based on the acquired contrast of the boundary portion B.

First, the term "outer shape line" in the present embodiment will be described.
FIG. 4 is a plan view showing an example of the outer line of the work according to the embodiment. In FIG. 4, an example of the outer line EL of the work W is shown by a thick black line. As shown in FIG. 4, of the portion acquired as the boundary portion B, at least a part of the designated range is referred to as "outer shape line EL". More specifically, for example, in the boundary portion B shown in FIG. 3, a right-up diagonal line, a right-down diagonal line, and a semi-elliptical arc can be designated as the outer line EL for which data is to be acquired. The outer line EL of the data acquisition target may be specified so as to coincide with the boundary portion B.

The portion designated as the outer line EL of the data acquisition target can be used as an index for correcting the displacement amount of the work W in a later step. That is, if there is a deviation between the reference position of the outer shape line EL taught to the robot and the position of the acquired outer shape line EL, the position of the work W can be corrected based on the deviation amount. ..

Next, it will be described whether or not the outer line of the work performed in this step can be determined.
The case where the outer line of the work W can be determined (step S4: YES) is, for example, the case where the contrast of the boundary portion B calculated in step S3 is high. That is, the color of the outer line EL of the work W and the color of the background color 10a displayed by the background display unit 10 have a higher contrast.

On the other hand, the case where the outer line of the work W cannot be determined (step S4: NO) is, for example, the case where the contrast of the boundary portion B calculated in step S3 is low. That is, the color of the outer line EL of the work W and the color of the background color 10a displayed by the background display unit 10 are close to each other, and the contrast is low.

As shown in FIG. 2, the acquired data is based on the threshold data that the external line EL stored in the image processing unit 30 can be determined and the contrast data of the boundary portion B calculated from the acquired image. If is equal to or greater than the threshold value (step S4: YES), the process proceeds to step S5. On the other hand, if the acquired data is below the threshold value (step S4: NO), the process proceeds to step S13, and the acquired data is removed from the data list of the outer line. If the process proceeds to step S13, the flow may be restarted by returning to step S2.

<Step S5: Acquire external line data (b)>
In step S5, the data determined in step S4 that the outer line EL can be determined is acquired. That is, the portion designated as the outer line EL is acquired as the acquisition data from the data of the boundary portion B for which the outer line EL can be determined.

<Step S6: Calculate the ratio of acquired data to reference data (c)>
In step S6, the reference data that is the reference of the outer line EL and the acquired data of the outer line EL acquired in step S5 are compared, and the ratio of the outer line EL that can be acquired is calculated as a numerical value. Numerical values can be calculated, for example, in percentage (%) notation. The "reference data" that serves as the reference for the outer line EL refers to the actual measurement data of the outer line of the work W.

Here, a specific example of "the rate at which the outer line EL can be acquired" will be described.
For example, the case where the ratio of the external line EL that can be acquired is 100% means that when the reference data and the acquired data are compared, they match 100%. That is, it means that the outer line EL of the work W could be completely read from the data acquired from the image captured by the imaging means 20. In this case, the detection accuracy (score) of the outer line EL is 100.

For example, the case where the ratio of the external line EL that can be acquired is 80% means that when the reference data and the acquired data are compared, they match by 80%. That is, it means that 80% of the outer line EL of the work W could be read from the data acquired from the image captured by the imaging means 20 with respect to the reference data, but 20% could not be read. .. In this case, the detection accuracy (score) of the outer line EL is 80.

<Step S7: Judgment of completion of n repetitions (e)>
In step S7, it is determined whether or not steps S2 to S6 are repeated n times. n is an integer of 1 or more and can be set to any value. If the repetition is not completed n times (step S7: NO), the process proceeds to step S8. When the repetition is completed n times (step S7: YES), the process proceeds to step S9.

<Step S8: Change the background color (d)>
In step S8, the background color 10a of the background display unit 10 is changed stepwise. To change in stages is to change to a so-called gradation. More specifically, the color is changed stepwise for each series of flows in which steps S2 to S8 are repeated. That is, when the colors of each time from the first flow to the nth flow are arranged, they are changed so as to form a gradation. The degree of change in gradation can be set arbitrarily.

<Step S9: Extract the data having the highest ratio of acquired external lines (f)>
In step S9, the data having the highest ratio of acquiring the outer line EL is extracted as the main data. By changing the background color 10a n times in a stepwise manner, a maximum of n data can be acquired for the same work W. Of the n data, when the contrast is the largest at the boundary portion B between the work W and the background color 10a, the outer line EL of the work W can be read more accurately. That is, the data when the contrast is the largest and the ratio at which the outer line EL can be acquired becomes a value closer to 100% is extracted as the main data, and the process proceeds to step S10.

<Step S10: Determining whether the main data is above the threshold value>
In step S10, it is determined whether or not the main data extracted in step S9 is equal to or greater than the threshold value. Whether or not the value of the ratio at which the outer line EL can be acquired in step S9 is equal to or greater than a predetermined value is set as a threshold value. Any value (%) can be set for the threshold value, and the value can be changed as appropriate. If the main data is equal to or greater than the threshold value (step S10: YES), the process proceeds to step S11. If the main data is less than the threshold value (step S10: NO), the process proceeds to step S14 and the flow is stopped. If the process proceeds to step S14, the flow may be restarted by returning to step S2.

<Step S11: Collate with the proof image and calculate the correction amount (g)>
In step S11, the main data is collated with the proofreading image prepared for proofreading, and the displacement of the work W is calculated as the correction amount. The correction amount can be calculated in the x-axis direction, the y-axis direction, and the r direction when the work W is viewed in a plan view.

<Step S12: Send the correction amount to the robot>
In step S12, the correction amount calculated in step S11 is transmitted to the robot. Even if there is a deviation between the position where the work W is placed and the reference position, the robot that has received the correction amount can perform the transfer chuck or the like at an accurate position.

The above is a series of steps of the misalignment detection method according to the present embodiment.
The misalignment detection method according to the present embodiment can also be used in the following cases. For example, it can also be used when identifying the position of a product or work or the silhouette of the outer shape by using a vision or a camera. It can also be used when a rough material, a work, a product, or the like placed on a jig, a relay stand, a temporary stand, or the like is subjected to a transfer chuck by using a robot or a machine. Further, when workpieces and products of different colors are mixed, it can also be used when the workpieces and products are conveyed and chucked by using a robot or a machine. It can also be used when the transfer chuck is performed by using a robot or a machine before and after painting the work, before and after quenching, etc., both before and after the color change.

The inventor has found that it is difficult to apply a method using a plurality of background materials having different optical characteristics to improve the accuracy of detecting an outer line using an imaging device.

In the present embodiment, the background color of the background display unit on which the work is placed is changed stepwise for each repeating unit of a series of flows. Even when the same material is used, the color of the work may differ depending on the molding conditions, but even when the position shift is detected for the work with different colors, the background color of the background display portion is appropriately used. It can be changed step by step. That is, in the present embodiment, the background color can be changed stepwise, and then the correction amount of the misalignment can be determined based on the main data having the highest ratio of acquiring the outline of the work. Therefore, the detection accuracy of the outer line can be improved.

Hereinafter, the present invention will be specifically described with reference to FIGS. 5 to 7, but the present invention is not limited to these examples.

In this embodiment, misalignment was detected using a rough aluminum material as the work. The color of the work was silvery white.
FIG. 5 is a graph showing a change in the detection accuracy (score) of the outer line of the work when the background color is changed by using the misalignment detection method according to the embodiment. Here, the “detection accuracy (score)” refers to the external line by comparing the reference data that is the reference of the external line with the acquired data of the acquired external line, as described in step S6 of the embodiment. Is calculated as a numerical value (percentage,%).

The horizontal axis of the graph of FIG. 5 indicates the background color, and the vertical axis indicates the detection accuracy (score). Each plot shows the number of iterations. Specifically, for example, the leftmost plot shows the detection accuracy (score) when the first plot is repeated and the second plot shows the second repetition.

In this embodiment, as shown in FIG. 5, the background color is changed from white to black at the timing of the upward black arrow on the horizontal axis. Specifically, in this example, the detection using a white background was repeated 64 times. The average detection accuracy (score) of the total number of detections of 64 on a white background was 94.0%. On the other hand, after changing to a black background, detection using the black background was repeated 55 times. The average detection accuracy (score) of 55 detections on a black background was 99.2%. In this way, by changing the background color from white to black, the detection accuracy (score) could be improved. Since the work used in this example is silver-white, the contrast is higher when a black background is used than when a white background is used for the work, and the detection accuracy (score) can be improved. Conceivable.

FIG. 6 is a graph showing the relationship between the detection accuracy (score) of the outer line of the work and the variation in the correction amount in the x direction when the background color is changed by using the misalignment detection method according to the embodiment. The correction amount in the x direction means the correction amount calculated in step S11. That is, it is the value of the difference between the reference position data taught to the robot and the acquired position data. Each data shows variability within the range of Six Sigma (6σ).

As shown in FIG. 6, when the detection accuracy (score) of the outer line of the work is low, the correction amount in the x direction is 1.09 mm, especially when it is 79.2%. On the other hand, when the detection accuracy of the outer line was 99.9%, the correction amount in the x direction was 0.04 mm. Therefore, it was possible to obtain the result that the higher the detection accuracy (score) of the outer line of the work, the smaller the variation in the correction amount in the x direction.

FIG. 7 is a graph showing the relationship between the detection accuracy (score) of the outer line of the work and the variation in the correction amount in the y direction when the background color is changed by using the misalignment detection method according to the embodiment. The correction amount in the y direction means the correction amount calculated in step S11, as in the case of the correction amount in the x direction. That is, it is the value of the difference between the reference position data taught to the robot and the acquired position data. Each data shows variability within the range of Six Sigma (6σ).

As shown in FIG. 7, when the detection accuracy (score) of the outer line of the work is low, the correction amount in the y direction is 0.15 mm, particularly when it is 79.2%. On the other hand, when the detection accuracy of the outer line was 99.9%, the correction amount in the x direction was 0.02 mm. Therefore, it was possible to obtain the result that the higher the detection accuracy (score) of the outer line of the work, the smaller the variation in the correction amount in the y direction.

From the above results, the higher the detection accuracy (score) of the outer line of the work, the smaller the variation in the correction amount in the x-direction and the y-direction. That is, it is considered that there is a correlation between the detection accuracy (score) of the outer line of the work and the variation in the correction amount. Further, in this embodiment, the detection accuracy (score) of the outer line of the work can be improved by changing the background color to black, which has a high contrast with the silver white of the work. Therefore, by using the misalignment detection method according to this embodiment, the detection accuracy of the outer line of the work can be improved.

As described above, the work used in this embodiment was silver-white, but even a rough aluminum material having the same shape and dimensions may exhibit a color different from silver-white. For example, depending on the temperature at which casting is performed, even the same rough material made of aluminum may exhibit a blackish color. In that case, the detection accuracy (score) of the outer line of the work can be improved by appropriately changing the background color. More specifically, when a rough aluminum material showing a blackish color is used as the work, if a background color of white or a color close to white is used, the detection accuracy (score) of the outer line of the work can be further improved. It is thought that it can be done.

The present invention is not limited to the above embodiment, and can be appropriately modified without departing from the spirit.

1 Image processing device 10 Background display unit 10a Background color 20 Imaging means 30 Image processing unit 40 Robot arm B Boundary part EL External line W Work

Claims (1)

It is a position deviation detection method that detects the amount of deviation between the reference work position taught to the robot and the actual work position.
(A) A step of acquiring an image of the background display unit and the work placed on the background display unit using an imaging means, and
(B) The step of acquiring the data of the outer line of the work and
(C) A step of comparing the data of the outer shape line with the data of the reference outer shape line and calculating the ratio of obtaining the outer shape line.
(D) A step of gradually changing the color of the background display unit and
(E) A step of repeating steps (a) to (d) n times (n is an integer of 1 or more) and
(F) A step of extracting the main data having the highest ratio of acquiring the outline, and
(G) A step of comparing the main data with a proof image and calculating the positional deviation of the work as a correction amount is provided.
Misalignment detection method.

Images