JPH06333025A - Window size deciding method - Google Patents

Abstract

PURPOSE:To provide a window size deciding method which can surely catch a moving object in a window regardless of the moving speed of a subject to be held or the irregularity of the moving path. CONSTITUTION:When a window is set in an image obtained by photographing a holding subject 6 put on a belt conveyor 5 by a CCD camera 2, a visual information processor 1 adds the magnifying component accordant with the moving speed of the subject 6 and the magnifying component accordant with the sample correlation coefficient set between the time and a position to a basic size accordant with the size of the subject 6. Thus a window size is calculated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention sets an appropriate window size when setting a window to be image-processed for a moving object in an image obtained by photographing a moving object with a CCD camera or the like. It is about how to make a decision.

[0002]

2. Description of the Related Art In an assembly line of an automated factory,
Parts on the belt conveyor are gripped by a robot hand and automatically assembled. In this case, for example,
The CCD camera takes a picture of the parts on the belt conveyor, and the position of the parts is recognized by performing image processing.
The gripping operation of the robot hand is controlled based on the recognition.

By the way, since image processing generally requires a long calculation time, conventionally, a window of an appropriate size is set in a captured image of a moving object, and the window is targeted for image processing. This reduces the calculation time.

[0004]

However, the window size has to be individually and specifically determined according to the shape and moving state of the moving object (JSME Robotics.
Mechatronics Lecture '92 Proceedings vol.B, p613-61
8) There is no known universally applicable window size determination method.

It is an object of the present invention to clarify a universally applicable window size determining method regardless of the shape and moving state of a moving object.

[0006]

In a window size determining method according to the present invention, a basic size corresponding to the size of a moving object, an enlargement component corresponding to the moving speed of the moving object, and
The window size is calculated by adding the expansion component corresponding to the sample correlation coefficient between time and position.

Further, another window size determining method according to the present invention is, after calculating the window size as described above,
The feature amount is calculated from the image of the moving object captured within the window size, and the window size is corrected according to the deviation of the feature amount from the reference value.

[0008]

In the first window size determining method described above, the basic size is determined based on the shape of the moving object, if the shape is known in advance. When the shape of the moving object is unknown, the basic size is calculated by image processing. The basic size represents the minimum window size required to capture the entire moving object within the window.

Then, the speed of the moving object is recognized based on the captured image, and the larger the recognized speed is, the larger the enlargement size is set. Furthermore, the sample correlation coefficient between the time and the position of the moving object is calculated, and the smaller the sample correlation coefficient, the larger the enlargement size is set. By adding both enlargement sizes to the basic size, an appropriate window size for reliably capturing a moving object in the window can be obtained regardless of the irregularity of the moving speed and the moving path.

In the second window size determining method, if an error occurs and the window size determined by the first method includes an error, the moving object captured in the window is A part of it protrudes out of the window, and the feature amount of the moving object calculated in the window deviates from the feature amount (reference value) when the whole is caught in the window. Therefore, in this case, the moving object can be entirely accommodated in the window by correcting the window size according to the deviation.

[0011]

According to the window size determining method of the present invention, the window size is determined according to not only the size of the moving object but also the moving speed of the moving object and the sample correlation coefficient between time and position. Therefore, it can be applied more universally than before.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment FIG. 1 shows a device configuration of an object gripping system to which the window size determining method of the present invention is applied. The system grips an object to be gripped (6) on a belt conveyor (5) with an indirect hand (3) of three fingers attached to a manipulator (3).

A monocular C is located above the belt conveyor (5).
A CD camera (2) is provided, and an image signal from the CCD camera (2) is supplied to the visual information processing device (1) to be subjected to window size determination processing, image processing and the like according to the present invention.

In the above object grasping system, the monocular C
Based on the image of the gripping target (6) taken by the CD camera (2), the manipulator (3) is controlled so that the gripping target (6) on the belt conveyor (5) is tracked, and three fingers and three indirect joints are made. The hand (4) is controlled to grip the gripping target (6).

When processing an image of the object to be grasped (6), the window size determining procedure shown in FIG. 2 or 3 is executed to shorten the calculation time by setting the window of an appropriate size. .

In the window size determining method of the present invention, first, the basic window size is determined according to the size of the object to be gripped (6). For example, when the planar shape of the gripping target (6) is a circle, the circumscribed square of the circle is set as a basic window. Here, if S is the area of the circle,
The size W _{dx in} the X-axis direction and the size W _{dy in the} Y-axis direction of the window are represented by the following formulas 1 and 2.

[0017]

[Formula 1] W _dx = W _dy = 2√ (π / S)

Next, the enlargement component of the window size corresponding to the moving speed of the object to be grasped (6) is calculated. The moving speed of the X-axis direction component V _x of gripping target, gain G _vx in the X-axis direction on the velocity, the moving speed of the Y-axis direction component V _y of gripping target, the gain of the Y-axis direction on the velocity _Is G _vy , the enlargement component W _{vx in} the X-axis direction and the enlargement component W _{vy in the} Y-axis direction of the window size according to the speed are represented by the following _formulas 2 and 3.

[0019]

(2) W _vx = G _vx × V _x

(3) W _vy = G _vy × V _y

Further, the enlargement component is calculated according to the sample correlation coefficient between the time and the position in the movement process of the grip target (6). Here, assuming that the sample correlation coefficient in the X-axis direction is r _xt , the sample correlation coefficient in the Y-axis direction is r _yt , and the gains thereof are G _rx and G _ry , the expansion component W _{rx in the} X-axis direction and the Y-axis are shown. The directional expansion component W _ry is represented by the following Equations 4 and 5.

[0021]

(4) W _rx = G _rx × (1- | r _xt |)

## _EQU5 ## _Wry = _Gry * (1- | _ryt |)

When calculating the sample correlation coefficient between the time and the position of the gripping target (6) on the belt conveyor (5) as shown in FIG. 1, the position of the gripping target is predicted by, for example, linear approximation. Here, x is the coordinate value (predicted value) of the X axis, t
Is a time, the prediction model can be expressed by the following equation 6.

[0023]

## EQU6 ## x = a × t + b

The coefficients a and b are determined by the method of least squares. A similar prediction model can be set for the Y-axis direction. In this case, the sample correlation coefficient r _xt in the X-axis direction is calculated by the following equation 7.

[0025]

[Equation 7] _{r xt = s xt / s x} s t

Here, s _xt is the sample covariance of coordinate value x (measured data) and time t, s _x ² is the sample variance of coordinate value x, s _t ² is the sample variance of time t, and n is The number of samples is calculated by the following equations 8, 9, and 10, respectively.

[0027]

S _xt = {Σxt−ΣxΣt / n} / (n−1)

S _x ² = {Σx ² − (Σx) ² / n} / (n−1)

S _t ² = {Σt ² − (Σt) ² / n} / (n−1)

When the object to be grasped moves without error according to the prediction model of the above equation 6, the value of | r _xt | in the above equation 7 becomes 1, and the value of | r _xt | becomes smaller as it deviates from the prediction model. Therefore, the enlargement component of the window size is set large. Sample correlation coefficient r _{yt in the} Y-axis direction
Can be similarly calculated.

When the expansion component is calculated in this way, the size W _{x in} the X-axis direction and the size W _{y in the} Y-axis direction of the window can be calculated by the following formulas 11 and 12. The gains of the equations 2 to 5 can be adjusted to optimum values by experiments or the like.

[0030]

[Expression 11] W _x = W _dx + W _vx + W _rx

[ _Expression 12] W _y = W _dy + W _vy + W _ry

FIG. 2 shows a specific procedure when the above window size determining method is applied to the object gripping system shown in FIG. In the image signal obtained from the CCD camera, the data in the window is sampled for every plurality of fields and is input to the image processing unit of the visual information processing device (1).

First, in step S1, the first four images obtained from the CCD camera are input, and in step S2
Then, the image data for the four images is subjected to image processing for the entire area of the screen without setting a window,
Position information (X and Y coordinate values) representing the position of the object to be grasped is calculated.

Next, in step S3, the least squares method is applied to the calculated four position information to determine the prediction model of the above-mentioned equation 6, and the sample correlation coefficients r _xt and r _yt are calculated.

Then, in step S4, the setting position of the window at the time of inputting the next image is calculated using the determined prediction model, and the window size is calculated by the equations (11) and (12). After that, in step S5, it is determined whether or not the object to be grasped can be grasped by the three-finger, three-degree-of-freedom hand (4).

If gripping is impossible, the next image is input in step S7, and in step S8, the above-mentioned step S8 is input.
Image processing is performed on the window having the size calculated in step 4 to calculate the position information of the grasped object, and then the process returns to step S3.

By the above procedure, the object to be gripped on the belt conveyor is gripped by the robot hand. At this time, since a window having an appropriate size is set at an appropriate position, the calculation time for image processing is short and a reliable gripping operation is realized.

Second Embodiment In the method of the first embodiment described above, when a sudden change occurs in the data due to an abnormal situation such as an object to be grasped falling down on the way, the calculated window size moves the window. There is a risk that the entire object cannot be captured.

Therefore, in the present embodiment, after the window size is calculated in the same manner as in the first embodiment, the predetermined feature amount is calculated from the image of the gripping target captured within the window size, and the feature amount is calculated. Correct the window size according to the deviation from the reference value of.

The window size correction amount W _s can be expressed by the following equation 13 using T _ref as the reference value of the feature amount of the object to be gripped, T _now as the current value of the feature amount, and G _s as the gain.

[0040]

[Expression 13] W _s = | G _s × (T _ref −T _now ) |

Therefore, the corrected window size W _xs ,
_Wys can be calculated by the following _equations 14 and 15.

[0042]

[ _Expression 14] W _xs = W _x + W _s

(15) W _ys = W _y + W _s

As the feature quantity, for example, the circularity e defined by the following equation 16 can be adopted. It is also possible to adopt other known feature amounts.

[0044]

[Equation 16] e = (perimeter of planar shape) ² / (area of planar shape) In this case, the circularity of a perfect circle becomes the minimum value (= reference value), and the edge of the circle protrudes from the window. , The current value of circularity becomes large.

FIG. 3 shows a specific procedure of the window size determining method. First, in step S11, the first four images obtained from the CCD camera are input, and in step S12, the image data for the four images is subjected to image processing for the entire area of the screen without setting a window. Position information (X and Y
Calculate the coordinate values).

Then, in step S13, the least squares method is applied to the calculated four position information to determine the prediction model of the above-mentioned equation 6, and the sample correlation coefficients r _xt and r _yt are calculated.

Then, in step S14, the position of the window at the time of inputting the next image is calculated using the determined prediction model, and the initial value of the window size is calculated by the equations (11) and (12). At S15,
The circularity is calculated by the above equation 16.

Then, in step S16, the circularity is compared with a reference value to determine whether the difference is less than a certain value. If NO, in step S17,
After correcting the window size based on the equations (14) and (15), in step S18, it is determined whether or not the gripping target can be gripped by the robot hand, and if possible, the gripping operation in step S19 is performed. Transition.

If gripping is impossible, the next image is input in step S20, image processing is performed on the window of the size corrected in step S17, and the position information of the gripping target is calculated. After that, the process returns to step S14.

By executing the above procedure, the object to be gripped on the belt conveyor is gripped by the robot hand. At this time, since the window size is corrected based on the circularity of the object to be grasped, the object to be grasped can be more reliably caught in the window.

The above description of the embodiments is for explaining the present invention and should not be construed as limiting the invention described in the claims or reducing the scope. The configuration of each part of the present invention is not limited to the above-mentioned embodiment, and it goes without saying that various modifications can be made within the technical scope described in the claims.

[Brief description of drawings]

FIG. 1 is a perspective view showing a device configuration of an object gripping system in which the present invention is implemented.

FIG. 2 is a flowchart showing a first embodiment.

FIG. 3 is a flowchart showing a second embodiment.

[Explanation of symbols]

 (1) Visual information processing device (2) CCD camera (3) Manipulator (4) Hand with three fingers and three degrees of freedom (5) Belt conveyor (6) Object to be gripped

Claims (2)

[Claims] 1. A method for determining a window size to be set in an image of a moving object, wherein a basic size corresponding to the size of the moving object, an expansion component corresponding to the moving speed of the moving object, a time and a position are set. A window size determination method, characterized in that a window size is calculated by adding an expansion component corresponding to a sample correlation coefficient between the two. 2. A method of determining a window size to be set in a captured image of a moving object, wherein a basic size corresponding to the size of the moving object, an expansion component corresponding to the moving speed of the moving object, a time and a position are set. After calculating the window size by adding the expansion component according to the sample correlation coefficient between
A window size determining method, characterized in that a feature amount is calculated from an image of a moving object captured within the window size, and the window size is corrected according to a deviation of the feature amount from a reference value.