JPH05314257A - Image processor and robot controller - Google Patents

Abstract

PURPOSE:To exactly calculate the measuring information of an object by flexibly setting a computation object frame to an objective image concerning arithmetic for the centroid coordinate value of the object without fixing that frame and to recognize the object at high speed. CONSTITUTION:This image processor is provided with a range setting means 11 to set the computation object frame to the image acquiring object, a range moving means 12 to move the computation object frame set to the image acquiring object, an arithmetic means 13 to calculate the centroid coordinate or standard deviation of the objective image inside the computation object frame, and a range changing means 14 to enlarge or reduce the computation object frame, and changes the position or size of the computation object frame based on the centroid coordinate value or the standard deviation value of the objective image inside the computation object frame in the past. And the robot controller is provided with a robot work part 17, an image acquiring means 18, a control means 20 or an image processing means 19 composed of the image processor.

Description

Detailed Description of the Invention

[Table of Contents] Industrial Application Field of the Related Art (FIG. 10) Problem to be Solved by the Invention (FIG. 11) Means for Solving the Problem (FIGS. 1 and 2) Action Embodiment (1) First Example 1 (FIGS. 3 to 5) (2) Second Example (FIGS. 6 and 7) (3) Third Example (FIG. 8) (4) Fourth Example (FIG. 9) effect

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing device and a robot control device, and more specifically,
The present invention relates to improvements in an apparatus that calculates various measurement amounts from image acquisition information and an application apparatus thereof.

In recent years, along with automation of manufacturing processes of various electronic devices and precision machines, a robot control device for positioning a robot hand on a target object and performing various teaching work is used. In addition, a control method has been developed in which a television camera is mounted as the eyes of the robot and the teaching work of the target object is performed based on the position information measured from the image acquisition data of the target object.

By the way, according to the robot controller of the conventional example, when calculating various measurement amounts such as the position of the center of gravity, the area, and the direction of the object using the television camera, the image measuring section of the robot control device performs high-speed calculation processing. In order to achieve the realization, the calculation target frame is fixed for the target image involved in the calculation of the barycentric coordinate value.

Therefore, when an image of a target object and an object other than the target object are included in the calculation target frame, even in the calculation of the barycentric coordinate value for which the internal screen is calculated, the image processing is not performed. There may be a limit, and the barycentric coordinate calculated without accuracy may not match the barycentric position of the object.

In such a state, the control system may erroneously recognize the position of the center of gravity of the object, causing a control error or the like in the subsequent teaching work. Therefore, the calculation target frame is not fixed to the target image, which is involved in the calculation of the barycentric coordinate value of the target object, and it is fluidly set to accurately calculate the measurement information of the target object, and the target object is calculated at high speed. A device capable of recognizing an object and its application device are desired.

[0007]

2. Description of the Related Art FIGS. 10 and 11 are explanatory views of a conventional example. 10 is an explanatory diagram of an object recognition method of a robot controller according to a conventional example, FIG. 10 (a) is its configuration diagram, and FIG. 10 (b) is its image processing flowchart. ing.

For example, a robot controller for positioning the robot hand 1 on a target object 10 for various teaching operations is shown in FIG. 10 (a), which is a robot hand 1, a television camera 2, an image measuring section 3 and a robot. It consists of the control unit 4.

The function of the device is that when the robot hand 1 having the television camera 2 is moved to the vicinity of the area of the object 10, the image of the area is captured by the television camera 2. Further, the image measuring unit 3 converts the image signal into binarized data relating to black and white gradation, and the data is arithmetically processed by the image measuring unit 3 to recognize the object 10. As a result, the robot hand 1 is positioned on the object 10 and teaching work such as gripping work is performed.

11 (a) and 11 (b) are acquired image diagrams of the object 10 for explaining the problems in the conventional example, and the image data of the area of the object 10 is binarized by a certain slice level. The figure shows the changed bright parts in white and the dark parts in black on the monitor TV.

For example, when the robot hand 1 is positioned on the object 10, the image measuring unit 3 uses the object 1
The calculation target frame F is set in the region image in which 0 is considered to exist, and the barycentric coordinates of the target image inside thereof are measured.

That is, in FIG. 10B, first, in step P1, the camera input image is binarized by an appropriate level. Then, in step P2, the calculation target frame F is set at a position where the object 10 is most likely to exist.

For example, as shown in FIG.
In consideration of the separation distance between 0 and the object 9 other than the target object and the size of the target object 10, the size of the calculation target frame F related to the target image area in which the barycentric coordinates are calculated with respect to the image measurement unit 3 in advance. Is fixed. This is because the target object 10 and the object 9 outside the target object can be separated by setting the calculation target frame F in a certain range. As a result, the calculation time can be shortened by not including the area outside the calculation target frame in the calculation of the barycentric coordinates and executing the calculation only within the frame.

Next, in step P3, the barycentric coordinates are obtained for the inside of the calculation target frame F, and thereafter, the barycentric coordinates obtained in step P4 are used as the barycentric position of the object 10 for positioning and the like.

[0015]

By the way, according to the conventional example, in the image measuring unit 3, in order to speed up the calculation process, the calculation target frame F is fixed to the target image involved in the calculation of the barycentric coordinate value. ing.

Therefore, as shown in FIG. 11B, the image of the object 10 and the object 9 other than the object 9 are placed inside the calculation object frame F.
Even if the calculation of the barycentric coordinate value for which the internal screen is to be calculated is included, the calculated barycentric coordinate of the target object 10 is not accurate because the calculation process has a limit. It may not match the position of the center of gravity.

This is because the difference in area is smaller than the error in the coordinate value of the center of gravity, which makes it difficult to reliably detect the object 10 from the difference in area. That is, in the method in which the calculation target frame F is fixed, even if the calculation target frame F includes only a part of the non-target object 9 in FIG. Since the distance from the barycentric coordinates of the object 10 to the object 9 is a long component, the error of the calculated value of the barycentric coordinates becomes large, which greatly deviates from the true barycentric coordinate position of the object 10. ..

As a result, since the calculated area value of the object 10 in the calculation object frame is different from the actual area value of the object 11 that has been planned, measurement is impossible or an error in the acquisition area of the television camera 2, that is, It is processed as if the displacement of the robot hand 1 to which is attached is too large.

In such a case, the center of the calculation target frame F may happen to coincide with the calculated value of the barycentric coordinates of the internal screen. However, in the calculated barycentric coordinates including the pixels (noise image) of the object 9 other than the target object, the true calculation target frame F
The center of does not always match the image centroid coordinates of the object 10.

In such a state, when the control system mistakenly recognizes the position of the center of gravity of the object 10 and shifts to the teaching work thereafter, the robot hand 1 collides with the object 10 or falls. Control error occurs.

As a result, in the image measuring unit 3 which adopts the method of fixing the calculation target frame F, there is a possibility that an incorrect barycentric coordinate value may be calculated, and the reliability of the image processing is lowered. Further, when the gravity center coordinate value cannot be measured, the television camera 2 needs to be moved again, that is, the robot hand 1 needs to be moved again, which hinders the speeding up of the positioning process.

The present invention has been created in view of the problems of the conventional example, and it is not necessary to fix the calculation target frame to the target image, which is involved in the calculation of the barycentric coordinate value of the target, and it can be fluidized. An object of the present invention is to provide an image processing device and a robot control device that can set and accurately calculate measurement information of an object and can recognize the object at high speed.

[0023]

1 and 2 show principle diagrams (Nos. 1 and 2) of an image processing apparatus and a robot control apparatus according to the present invention, respectively.

The first image processing apparatus of the present invention is shown in FIG.
As shown in FIG. 2A and FIG. 2A, range setting means 11 for setting the calculation target frame F in the image acquisition target 16 and range moving means 12 for moving the calculation target frame F set in the image acquisition target 16. And a calculation means 13 for calculating the barycentric coordinates and standard deviation of the target image inside the calculation target frame F, and a range changing means 14 for reducing or enlarging the calculation target frame F,
The present invention is characterized in that the position and size of the calculation target frame F are changed based on the barycentric coordinate value and standard deviation value of the target image inside the past calculation target frame F.

The second image processing apparatus of the present invention is characterized in that, in the first image processing apparatus, the arithmetic processing is repeated based on a preset control target value of the calculation range frame F.

Further, the third image processing apparatus of the present invention is
In the first image processing apparatus, the range setting means 11 sets a new calculation target frame F that is a constant multiple of the standard deviation value.

A fourth image processing apparatus according to the present invention is the same as the first image processing apparatus, except that the calculating means 13 is the same as that shown in FIG.
As shown in (b), the distance information between the centers of gravity is calculated based on the center of gravity coordinate values of the target image inside the calculation target frame F and the center of gravity coordinate values of the target image inside the calculation target frame F. Is characterized by.

Further, the fourth image processing apparatus of the present invention is characterized in that a comparison means 15 for comparing the control target value and distance information between the centers of gravity is provided. The robot control device of the present invention, as shown in FIG. 1B, is a robot working unit 17 that performs various teaching operations related to the object 10.
And an image acquisition means 18 for acquiring an image of the object 10.
And an image measuring unit 19 for measuring the position information of the object 10, and a control unit 20 for controlling input / output of the robot working unit 17, the image acquiring unit 18, and the image measuring unit 19, and the image measuring unit. Means 19 are characterized by comprising first to fourth image processing devices to achieve the above object.

[0029]

[Operation] According to the first image processing apparatus of the present invention, as shown in FIG.
As shown in (a), a range setting means 11, a range moving means 12, a calculating means 13 and a range changing means 14 are provided,
The position and size of the calculation target frame F can be changed based on the barycentric coordinate value and standard deviation value of the target image inside the calculation target frame F in the past.

For example, when the calculation target frame F is set in the image acquisition target 16 as shown in FIG. 2A by the range setting means 11, the calculation target frame F set in the image acquisition target 16 is set.
Are moved by the range moving means 12. Further, the barycentric coordinates and the standard deviation of the target image inside the calculation target frame F are calculated by the calculating means 13, and the calculation target frame F is changed to the range changing means 1
It is reduced or enlarged by 4 (see FIG. 2A).

This makes it possible to freely change the position and size of the calculation target frame F at high speed based on the barycentric coordinate value and standard deviation value of the target image inside the calculation target frame F in the past. Becomes

Therefore, as shown in FIG. 2B, in the case where the image of the object 10 and the object 9 other than the object are included in the initially set calculation target frame F, Even when the barycentric coordinate value is calculated for the internal screen, the position and size are automatically set so that the images of the object 9 other than the target object are sequentially excluded from the calculation target frame F. Since the change can be changed, the limit of the arithmetic processing as in the conventional example is reduced, and the barycentric coordinate value can be measured from the area of the object 10 with good reproducibility.

As a result, it becomes possible to apply the barycentric coordinates calculated based on the function of dynamically changing the position and size of the calculation target frame F as the true barycentric position of the object 10. As a result, it is possible to improve the reliability of the image measurement processing related to the calculation of various measurement information.

Further, the calculation target frame is not fixed to the target image due to the calculation of the barycentric coordinate value of the target object 10,
Since it is set fluidly, it is possible to accurately calculate the measurement information of the object.

Further, according to the second image processing apparatus of the present invention, the calculation processing is repeated based on the preset control target value of the calculation range frame F. Therefore, as shown in FIG. 2A, when the first calculation target frame F that is initially set includes only a part of the non-target object 9,
Even if the distance from the barycentric coordinates of the object 10 to the object 9 becomes a long component and the error of the barycentric coordinate calculation value becomes large, the calculation processing is performed based on the control target value of the first calculation range frame F. By repeating the above, for example, the non-target object 9 is excluded from the target image by the second calculation target frame F that is set next, so that the barycentric coordinates of the target object 10 are the center of the calculation target frame F. Since it converges (asymptotically), it does not erroneously largely deviate from the true barycentric coordinate position of the object 10.

Further, the preset control target value of the calculation range frame F, for example, the planned actual area value of the object 10 and the calculated area value of the object 10 within the calculation object frame are extremely close to each other. As a result, it is possible to reduce the possibility that measurement cannot be performed as in the conventional example.

When the barycentric coordinate value cannot be measured, an alarm or the like is issued to notify the upper control system that an abnormality has occurred in the image measurement system, so that a control error or the like can be made in advance. It becomes possible to prevent it.

Furthermore, according to the third image processing apparatus of the present invention, the range setting means 11 sets a new calculation target frame F that is a constant multiple of the standard deviation value. For example, inside the calculation target frame F, the standard deviation is obtained in the X and Y directions, and the magnification is set.

Therefore, when there are many objects 9 other than the objects having different shapes near the object 10 or the object 10
When the shape of is not determined, the optimum calculation target frame F can be set at high speed.

As a result, the measurement information of the object can be accurately calculated, as in the first and second image processing apparatuses. Further, according to the fourth image processing apparatus of the present invention,
As shown in FIG. 2B, the calculating means 13 calculates the barycentric coordinate values Xgp, Ygp of the target image inside the past calculation target frame F and the barycentric coordinate values Xgr, Ygr of the target image inside the calculation target frame F. The distance information between the centers of gravity is calculated based on At this time, the control target value and the distance information between the centers of gravity are compared by the comparison unit 1.
5 compared.

For this reason, when the barycentric coordinates of the object 10 are repeatedly processed in relation to the calculation target frame F like the first and second image processing apparatuses, the center of the target calculation target frame F is calculated. The position can be quickly brought close to the barycentric coordinate value of the object 10.

As a result, the dead time associated with the image measurement processing is suppressed as much as possible, and the measurement information of the object can be calculated at high speed. According to the robot controller of the present invention, as shown in FIG.
Image acquisition means 18, image measurement means 19, and control means 20
Is provided, and the image measuring means 19 is composed of first to fourth image processing devices.

For example, when performing various teaching operations related to the object 10, the robot working unit 1 is controlled through the control means 20.
7 is moved onto the area of the object 10, and the image of the object 10 is acquired by the image acquisition means 18. Further, the position information of the object 10 is measured by the image measuring means 19. At this time, the position and size of the calculation target frame F are calculated based on the barycentric coordinate values and standard deviation values of the past calculation target frame F by the image measuring means 19 including the first to fourth image processing devices of the present invention. Is automatically changed.

Therefore, by adopting the image processing device having the function of automatically changing the position and size of the calculation target frame F in a fluid manner, the accurate barycentric coordinate value of the object 10 can be calculated. .. From this, the calculation target frame F of the conventional example
Compared to the image measuring unit in which is fixed, an error in the acquisition area of the image acquiring unit 18, that is, the positional deviation of the robot working unit 17 to which it is attached is less likely to be processed.

As a result, again, the movement of the image acquisition means 18, that is, the number of adjustments of the robot working unit 17 is reduced, and the positioning processing can be speeded up. In addition, erroneous recognition of the position of the center of gravity of the object 10 in the control system becomes almost zero, and a control error such as colliding the robot working unit 11 with the object 10 or causing it to tip over is avoided during subsequent teaching work. It becomes possible to prevent it as much as possible.

[0046]

Embodiments Next, embodiments of the present invention will be described with reference to the drawings. 3 to 9 are views for explaining the image processing device and the robot control device according to the embodiment of the present invention.

(1) Description of First Embodiment FIG. 3 is a block diagram of a robot control device to which the image processing device according to each embodiment of the present invention is applied, and FIG. 4 is an image for specifying an object thereof. Each of the processing flowcharts is shown.

For example, in FIG. 3, a robot controller that combines the functions of the first to fifth image processing apparatuses of the present invention and applies them is shown in FIG. 3 as a robot hand 27, a television camera 28, an image measuring system 29 and a CPU. (Central processing unit) 30, drive control unit 31, alarm unit 35 and the like.

That is, the robot hand 27 is an example of the robot working unit 17, and is for carrying out various teaching work in relation to the object 10. For example, the object 10 is gripped.

The television camera 28 is an embodiment of the image acquisition means 18, and acquires an image of the object 10 and outputs its analog video signal S to the image processing system 29. For example, the TV camera 28 is the robot hand 27.
It is fixed to the side of the work area or installed on the ceiling surface of the work area.

The image measuring system 29 is the image measuring means 19
This is an example of measuring the position information of the object 10. The image measuring system 29 is the first aspect of the present invention.
~ It comprises a fourth image processing device.

For example, in the image measuring system 29, the calculation frame setting editor 21, the calculation frame moving editor 22, the arithmetic unit 23, and the calculation frame reduction connected to the data bus 34 are shown in the part surrounded by the one-dot chain line in FIG. / Enlargement editor 24, data comparison unit 25, signal processing unit 32, memory unit 33 and the like.

That is, the calculation frame setting editor 21 is an embodiment of the range setting means 11 and sets the calculation target frame F to the image acquisition target 16. For example, the calculation frame setting editor 21 sets a new calculation target frame F that is a constant multiple of the standard deviation value of the target image inside the calculation target frame F, and constitutes the first and third image processing apparatuses of the present invention.

The calculation frame moving editor 22 is the range moving means 1
This is an example of No. 2 and moves the calculation target frame F set in the image acquisition target 16. The calculation unit 23 is an embodiment of the calculation unit 13, and calculates the barycentric coordinates and standard deviation of the target image inside the calculation target frame F. For example, the calculation unit 23 calculates the distance between the centers of gravity based on the past center-of-gravity coordinate values Xgp and Ygp of the target image inside the calculation target frame F and the center-of-gravity coordinate values Xgr and Ygr of the target image inside the calculation target frame F. Compute information.

The calculation frame reduction / enlargement editor 24 is an embodiment of the range changing means 14 and reduces or enlarges the calculation target frame F. The data comparison unit 25 is an embodiment of the comparison unit 15 and compares the control target value with the distance information between the centers of gravity. The fourth image processing apparatus of the present invention is configured by the functions of the calculation unit 23 that calculates the distance information between the centers of gravity related to the target image and the data comparison unit 25.

Further, the signal processing unit 32 is the television camera 28.
The analog video signal S output from the binarization circuit is binarized, for example, it is composed of an A / D conversion circuit and its threshold circuit, and as a result, image data Din relating to black and white gradation is obtained.

The memory unit 33 temporarily stores the image data Din, and stores various control data relating to the setting, reduction, enlargement, movement, etc. of the calculation target frame F. As a result, in the image measurement system 29, the position and size of the calculation target frame F can be changed based on the barycentric coordinate value and standard deviation value of the target image inside the calculation target frame F in the past. For example, it is possible to repeat the arithmetic processing for converging the calculation target frame F on the object 10 based on a preset control target value (corresponding to the function of the second image processing apparatus of the present invention).

The CPU 30 is an embodiment of the control means 20 and controls the input / output of the robot hand 27, the television camera 28 and the image measuring system 29.
For example, the drive control data Dm is output to the drive control unit 31, and the alarm data Da is output to the alarm unit 35.

The drive control unit 31 uses the drive control data D
The robot hand 27 is driven and controlled in the X, Y, and Z directions based on m. The alarm unit 35 is the object 1
When 0 cannot be recognized or the recognition processing cannot be performed, an alarm is issued based on alarm data Da output from the image measuring system 29 via the CPU 30.

In this way, according to the robot control device to which the image processing device according to each embodiment of the present invention is applied,
As shown in FIG. 3, a robot frame 27, a television camera 28, an image processing system 29, a CPU 30, a drive control unit 31, and an alarm unit 35 are provided, and the image processing system 29 is connected to a data bus 34 to set a calculation frame. An editor 21, a calculation frame moving editor 22, a calculation unit 23, a calculation frame reduction / enlargement editor 24, a data comparison unit 25, a signal processing unit 32, a memory unit 33 and the like.

For example, when performing various teaching operations related to the object 10, the robot hand 27 is executed via the CPU 30.
Is moved onto the area of the object 10, and an image of the object 10 is acquired by the television camera 28. Further, the position information of the object 10 is measured by the image processing system 29.

At this time, the image processing system 29 including the first to fourth image processing apparatuses of the present invention uses the barycentric coordinate values Xgp and Ygp of the past calculation target frame F and the standard deviation value to calculate the calculation target frame. The position and size of F are automatically changed.

Therefore, the image processing system 2 having a function of automatically changing the position and size of the calculation target frame F dynamically.
By adopting 9, it is possible to accurately calculate the barycentric coordinate value of the object 10. From this, as compared with the image measuring unit in which the calculation target frame F is fixed in the conventional example, the error in the acquisition area of the TV camera 28, that is, the positional deviation of the robot hand 27 to which it is attached is processed as too large. Less often.

As a result, the movement of the television camera 28, that is, the number of times the robot hand 27 is adjusted is reduced again, and the positioning process can be speeded up. In addition, erroneous recognition of the position of the center of gravity of the object 10 in the control system becomes almost zero, and a control error such as colliding the robot working unit 11 with the object 10 or causing it to tip over is avoided during subsequent teaching work. It becomes possible to prevent it as much as possible.

Next, a robot controller for recognizing an object and gripping the object according to each embodiment of the present invention will be described, supplementing the operation of the image measuring system 29 of the apparatus.

FIG. 4 is a flow chart of image processing for recognizing an object according to the first embodiment of the present invention, and FIG. 5 is a supplementary explanatory view of the flow chart. For example, recognizing the object 10 as shown in FIG.
When gripping it, first in step P in FIG.
At 1, the robot hand 27 is moved onto the area of the object 10. At this time, when the drive control data Dm is output from the CPU 30 to the drive control unit 31, the drive control unit 31 causes the drive unit 31 to move the X, X, and X based on the control data Dm.
Drive control is performed in the Y and Z directions. As a result, the television camera 28 fixed to the side surface of the robot hand 27 is moved and moved onto the area of the object 10.

Next, in step P2, an image acquisition process of the object 10 under the attachment area of the television camera 28 is performed. At this time, the image of the object 10 is acquired by the television camera 28, and the analog video signal S is acquired by the image processing system 29.
Is output to.

Then, in step P3, the camera input image is binarized by an appropriate level. At this time, the signal processing unit 32 binarizes the analog video signal S output from the television camera 28, and outputs image data Din related to the black and white gradation. This image data Din is stored in the memory unit 3
Temporarily stored in 3.

For example, in FIG. 5A, when a monitor display is performed based on the binarized image data Din,
An image acquisition target 16 including an image of the object 10 and an image of the object 9 other than the object is obtained.

Further, in step P4, the first calculation object frame F is provisionally set at the position where the object 10 is most likely to exist. At this time, the image processing system 2
The calculation frame setting editor 21 of 9 sets the calculation target frame F in the image acquisition target 16 as shown in FIG.
Note that in FIG. 5B, the calculation target frame F indicated by a thin line
Was initially set.

Next, in step P5, the first calculation target frame F
The barycentric coordinates Xg, Yg are obtained for the inside of the. At this time, for example, black, which is the color of the object 10, is weighted with “1” and white is weighted with “0” based on the image data Din, and (1) and ( The average is calculated by the formula 2).

[0072]

[Equation 1]

As a result, the object 1
The barycentric coordinates Xg and Yg of 0 can be obtained. Here, fi is “1” when the i-th pixel is black, “0” when it is white, xi, yi are the coordinate values of the X and Y axes of the pixel, and n is the total number of pixels in the screen. Is represented. The value of the denominator represents the number of pixels of the object, that is, the area.

Thereafter, in step P6, the second calculation target frames F having the same size centered on the barycentric coordinates Xg, Yg previously obtained are set. At this time, the calculation frame moving editor 22
As a result, the calculation target frame F set as the image acquisition target 16 is moved, and the calculation target frame F indicated by the dotted line in FIG. 5B is set for the second time.

Further, in step P7, the barycentric coordinates Xg, Yg are obtained for the inside of the second calculation object frame F. At this time, as in step P5, the center of gravity coordinates Xg and Yg are obtained by the calculation unit 23 for the inside of the second calculation target frame F.

Then, in step P8, the third calculation target frame F having the same size with the center of gravity coordinates Xg, Yg previously obtained as the center is set. At this time, the calculation frame moving editor 22
As a result, the calculation target frame F set as the image acquisition target 16 is moved, and the calculation target frame F indicated by the thick line in FIG. 5B is set for the third time.

Further, in step P9, the barycentric coordinates Xg, Yg are obtained for the inside of the third calculation object frame F. At this time, as in steps P5 and P7, the center of gravity coordinates Xg and Yg are obtained by the arithmetic unit 23 for the inside of the second calculation target frame F.

After that, in step P8, the barycentric coordinates Xg, Yg previously obtained are recognized as the barycentric positions XG, YG of the object, and various operations thereafter are performed. At this time, various teaching operations are performed by the robot hand 27 in relation to the object 10. For example, the robot hand 27 is operated by the television camera 28 via the drive control unit 31 based on the drive control data Dm.
Is moved in the X and Y directions by the mounting offset of, and thereafter, it is driven and controlled in the Z direction, and the object 10 is gripped.

As described above, according to the image measuring method of the robot controller for recognizing the object according to the first embodiment of the present invention, as shown in the processing flowchart of FIG. 4, steps P4, P6 and P8 are performed. Then, the position or size of the calculation target frame F is changed based on the barycentric coordinate value of the target image inside the calculation target frame F in the past.

Therefore, as shown in FIG. 5B, when the image of the object 10 and the object 9 other than the object are included in the initially set calculation target frame F, Even when the center-of-gravity coordinate values Xg and Yg are calculated for the internal screen, the positions and sizes of the objects 9 other than the object are sequentially excluded from the calculation target frame F. Can be changed automatically.

As a result, the occurrence of the limit of the arithmetic processing as in the conventional example is reduced, and the barycentric coordinate values Xg and Yg of the object 10 can be measured with good reproducibility in step P9.

As a result, the barycentric coordinate values Xg, Yg calculated based on the function of dynamically changing the position and size of the calculation target frame F are used as the barycentric position XG, Y of the true object 10.
It can be applied as G. Also, the object 10
Since the calculation target frame F is not fixed with respect to the target image due to the calculation of the center-of-gravity coordinate value of the target image, but is set in a fluid manner, it is possible to accurately calculate the measurement information of the target object.

As a result, it is possible to improve the reliability of the image measurement processing relating to the calculation of various measurement information. (2) Description of Second Embodiment FIG. 6 is a flowchart of image processing for recognizing an object according to the second embodiment of the present invention, and FIG. 7 is a supplementary explanatory diagram of the flowcharts according to other embodiments. Shown respectively.

In FIG. 6, the difference from the first embodiment is that in the second embodiment, the convergence value between the barycentric coordinates of the target image inside the calculation target frame F is set to the control variable i, for example, the control variable in advance. i = 5 is set, and the calculation process is repeated until it becomes less than or equal to the target.

That is, as in the first embodiment, the robot hand 27 is moved onto the area of the object 10 in step P1, and the image acquisition processing of the object 10 is performed in step P2. In step P2, the camera input image is binarized by an appropriate level. So far, this is the same as the first embodiment.

Next, in step P3, the first calculation target frame F is set at the place where the object is most likely to exist. At this time, when the control variable i = 1 is set in the calculation frame setting editor 21, the first calculation target frame F is set as the image acquisition target 16 as shown by the thin line in FIG.

Then, in step P4, the barycentric coordinates Xg, Yg are obtained for the inside of the first calculation target frame F. At this time, as in the first embodiment, the barycentric coordinates Xg, Yg of the object 10 are obtained via the calculation unit 23.

Next, in step P5, 2 is substituted for the control variable i. At this time, the control variable i = 2 is the data comparison unit 25.
When set to, the calculation frame moving editor 22 moves the calculation target frame F set in the image acquisition target 16.

Then, in step P6, the i-th calculation target frame F having the same size with the center of gravity coordinates Xg, Yg previously obtained as the center is set. Here, as shown by the dotted line in FIG. 7A, the second calculation target frame F is set as the image acquisition target 16.

Further, in step P7, the barycentric coordinates Xg, Yg are obtained for the inside of the i-th calculation target frame F. At this time, the barycentric coordinates Xg, Yg of the object 10 are obtained via the calculation unit 23, as in step P5.

Then, in step P8, the distance between the i-th calculated barycentric coordinates Xg and Yg and the i-1th calculated barycentric coordinates is determined. Here, the calculation unit 23 is used as shown in FIG.
As shown in (b), between the centroids based on the centroid coordinate values Xgp, Ygp of the target image inside the calculation target frame F and the centroid coordinate values Xgr, Ygr of the target image inside the calculation target frame F in the past. Distance information is calculated.

Further, in step P9, 1 is added to the control variable i. Here, when i = 1 is added to the control variable i = 2 in the calculation frame setting editor 21, the calculation frame moving editor 22 moves the calculation target frame F set in the image acquisition target 16. Here, as shown by the thick line in FIG. 7A, the calculation target frame F for the third time is set as the image acquisition target 16.

Next, in step P10, it is determined whether or not the distance between the barycentric coordinates is below a certain amount. At this time, it is less than a certain amount, for example, a preset control variable i = 5
If it becomes (YES), the process proceeds to step P11.
If it does not fall below a certain amount (NO),
It returns to step P6. Here, the data comparison unit 25 compares the distance information between the barycentric coordinates of the target images inside the old and new calculation target frames F with the control target value.

Here, when the object 10 cannot be recognized or its recognition processing is impossible, an alarm is issued by the alarm unit 35 based on the alarm data Da output from the image measuring system 29 via the CPU 30.

Therefore, when it becomes equal to or less than the predetermined amount, the barycentric coordinates Xg, Yg obtained in step P11 are recognized as the barycentric positions XG, YG of the object 10 and various teaching operations thereafter are performed. Since the other processing is the same as that of the first embodiment, the description thereof will be omitted.

As described above, according to the image measuring method of the robot controller for recognizing the object according to the second embodiment of the present invention, the control is performed in steps P5 and P9 as shown in the processing flowchart of FIG. Variable i is set and step P
At 10, it is determined whether or not the distance between the positions of the barycentric coordinates Xg and Yg is below a certain amount.

Therefore, as shown in FIG. 7A, the first set calculation target frame F is the object 9 which is not the target.
Even when only a small part of the object is included, and the distance from the barycentric coordinates of the object 10 to the object 9 becomes a long component and the error of the barycentric coordinate calculation value becomes large, the first value is determined in step P10. When the non-target object 9 is excluded from the target image by repeating the calculation process based on the control target value of the second calculation range frame F, for example, by the second calculation target frame F set next. In addition, since the barycentric coordinates of the object 10 converge (asymptote) to the center of the calculation target frame F, the barycentric coordinates of the true object 10 are prevented from being erroneously largely separated.

As a result, even if the object 10 is smaller than the object 9 other than the object, the control target value of the preset calculation range frame F, for example, the planned object 1
Since the actual area value of 0 and the calculated area value of the object 10 in the calculation target frame become extremely close to each other, it is possible to reduce the possibility of measurement failure as in the conventional example.

When the gravity center coordinate value cannot be measured, an alarm or the like is issued to notify the upper control system that an abnormality has occurred in the image measurement system, and control as in the conventional example. It becomes possible to prevent mistakes in advance.

(3) Description of Third Embodiment FIG. 8 shows a flowchart of image processing for recognizing an object according to the third embodiment of the present invention.

In FIG. 8, the difference from the first and second embodiments is that in the third embodiment, the range setting editor 21 sets a new calculation target frame F that is a constant multiple of the standard deviation value.

That is, as in the first and second embodiments,
The robot hand 27 is moved onto the area of the object 10 and an image acquisition process for the object 10 is performed. Further, in FIG. 8, the camera input image is binarized at an appropriate level in step P1, and further, in step P2, the first calculation target frame F is set at a position where an object is most likely to exist. Here, as shown by the thin line in FIG. 7A, the first calculation target frame F is set as the image acquisition target 16.

Next, in step P3, the first calculation target frame F
The barycentric coordinates Xg, Yg are obtained for the inside of the. So far, this is the same as the first embodiment. After that, step P4
Then, the standard deviation is calculated for the inside of the first calculation target frame F. Here, the calculation frame setting editor 21 sets a constant multiple of the standard deviation value of the target image inside the calculation target frame F, for example, its magnification k to 3. An appropriate value is selected as the magnification k depending on the complexity of the shape of the object 10.

Further, standard deviation values sx, sy in the X and Y directions
Is calculated by the calculation unit 23 according to equations (3) and (4).

[0105]

[Equation 2]

Note that N is the number of pixels for the gradation color (black = “1” here) of the object 10 and is given by the equation (5).

[0107]

[Equation 3]

The ranges of the calculation target frame F in the X-axis direction and the Y-axis direction are defined by the equations (6) and (7).

[0109]

[Equation 4]

Next, in step P5, the second calculation target frame F is set with the center of gravity coordinates Xg, Yg obtained previously as the center and the size of the calculation target frame F set to three times the standard deviation values sx, sy. .. Here, the calculation frame reduction / enlargement editor 24 reduces the calculation target frame F as shown by the dotted line in FIG. 7B.

Further, in step P6, the barycentric coordinates Xg, Yg are obtained for the inside of the second calculation object frame F, and thereafter,
In step P7, the standard deviation is calculated for the inside of the second calculation target frame F. Here, similarly to step P4, the calculation frame setting editor 21 sets the magnification k of the standard deviation values sx and sy of the target image inside the calculation target frame F to 3 times.

Next, the size is standard deviation values sx, sy centered on the barycentric coordinates Xg, Yg previously obtained in step P8.
The third calculation target frame F that is three times as large as the above is set. here,
The calculation frame reduction / enlargement editor 24 reduces the calculation target frame F as indicated by the thick line in FIG.

Further, in step P9, the barycentric coordinates Xg, Yg are obtained for the inside of the third calculation object frame F, and thereafter,
The barycentric coordinate values Xg and Yg obtained in step P10 are set to the object 1
The center of gravity position of 0 is recognized as XG and YG, and various operations are performed. Since the other structures are similar to those of the first and second embodiments, the description thereof will be omitted.

As described above, according to the image measuring method of the robot controller for recognizing an object according to the third embodiment of the present invention, as shown in the processing flowchart of FIG. Deviation values sx and sy are calculated,
Then, in steps P5 and P8, a new calculation target frame F that is three times the standard deviation values sx and sy is set.

Therefore, when there are many objects 9 other than the objects having different shapes in the vicinity of the object 10 or the object 10
When the shape of is not determined, the optimum calculation target frame F can be set at high speed.

As a result, similarly to the first and second embodiments, the measurement information of the object 10 can be calculated accurately. (4) Description of Fourth Embodiment FIG. 9 shows a flowchart of image processing for recognizing an object according to a fourth embodiment of the present invention.

In FIG. 9, the difference from the first to third embodiments is that in the fourth embodiment, the calculation target frame F is set when a new calculation target frame F that is a constant multiple of the standard deviation values sx and sy is set. The convergence value between the barycentric coordinates of the target image inside F is set as a control variable i, for example, a control variable i = 5 in advance, and the calculation process is repeated until it becomes less than the target.

That is, as in the first to third embodiments,
The robot hand 27 is moved onto the area of the object 10 and an image acquisition process for the object 10 is performed. Further, in FIG. 9, in step P1, the camera input image is binarized by an appropriate level, and in step P2, a first calculation target frame F is set at a position where an object is most likely to exist. At this time, the control variable i = 1 is the calculation frame setting editor 2
When set to 1, the first calculation target frame F is set to the image acquisition target 16 as shown by the thin line in FIG.

Next, in step P3, the first calculation target frame F
The barycentric coordinates Xg, Yg are obtained for the inside of the. Up to this point, the procedure is the same as in the second embodiment. Furthermore, step P4
Then, the standard deviation is calculated for the inside of the first calculation target frame F. Here, similarly to the third embodiment, the calculation frame setting editor 21 sets the magnification k of the standard deviation values sx and sy of the target image inside the calculation target frame F to 3 times.

Then, in step P5, 2 is substituted for the control variable i. At this time, the control variable i = 2 is the data comparison unit 2
When set to 5, the calculation frame moving editor 22 moves the calculation target frame F set in the image acquisition target 16.

Next, in step P6, the i-th calculation target frame F is set with the center of gravity coordinates Xg, Yg previously obtained as the center and three times the standard deviation values sx, sy of the size of the calculation target frame F. .. Here, the calculation frame reduction / enlargement editor 24 reduces the calculation target frame F as shown by the dotted line in FIG. 7B.

Further, in step P7, the barycentric coordinates Xg, Yg are obtained for the inside of the i-th calculation target frame F. At this time, as in step P3, the barycentric coordinates Xg and Yg of the object 10 are obtained via the calculation unit 23.

Next, in step P8, the standard deviation is obtained for the inside of the i-th calculation target frame F. At this time, similarly to step P4, the calculation frame setting editor 21 sets the magnification k of the standard deviation values sx and sy of the target image inside the calculation target frame F to 3 times.

Then, in step P9, the distance between the i-th barycentric coordinate Xg and Yg calculated and the i-1th barycentric coordinate Xg and Yg position is obtained. Here, the calculation unit 23
Thus, the distance information between the centers of gravity is calculated based on the center of gravity coordinate values of the target image inside the calculation target frame F and the center of gravity coordinate values of the calculation target frame F.

Then, in step P10, 1 is added to the control variable i. Here, when i = 1 is added to the control variable i = 2 in the calculation frame setting editor 21, the calculation frame moving editor 22 moves the calculation target frame F set in the image acquisition target 16. Here, as shown by the thick line in FIG. 7A, the calculation target frame F for the third time is set as the image acquisition target 16.

Next, in step P11, it is determined whether or not the distance between the barycentric coordinates is below a certain amount. At this time, if it becomes less than a certain amount (YES), step P
Transition to 12. If it does not fall below the fixed amount (NO), the process returns to step P6.

Therefore, when it becomes less than a certain amount, the barycentric coordinates Xg, Yg obtained in step P12 are recognized as the barycentric positions XG, YG of the object 10 and various operations thereafter are performed. The rest of the configuration is the same as that of the first embodiment, so its explanation is omitted.

As described above, according to the image measuring method of the robot controller for recognizing an object according to the fourth embodiment of the present invention, as shown in the processing flowchart of FIG. 9, at step P4 (P8). When the standard deviation values sx and sy are calculated, and then the control variable i is set in step P5, a new calculation target frame F that is three times the standard deviation values sx and sy is set in step P6, and the center of gravity is set in step P10. Coordinate X
It is judged whether or not the distance between the g and Yg positions is below a certain amount.

Therefore, as in the first and second embodiments,
The barycentric coordinate value X of the object 10 related to the calculation target frame F
When g and Yg are repeatedly calculated, the target center position of the calculation target frame F can be asymptotically approached to the barycentric coordinate value of the object 10.

As a result, the dead time associated with the image measurement processing is suppressed as much as possible, and the measurement information of the object can be calculated at high speed. Although the embodiment in which the image processing apparatus of the present invention is applied to the eyes of the robot controller has been described, the same effect can be obtained when applied to pattern recognition processing and other image processing. Further, although the calculation target frame F has been described as having a quadrangular shape, it can be similarly applied to an arbitrary shape such as a circle or an ellipse.

Further, in each of the embodiments, the case where the image acquisition target 16 is the black target object 10 on the white background has been described.
The same effect can be obtained when the target object 10 is white on a black background. Although the standard deviation values sx and sy are shown as the method of determining the size of the calculation target frame F, the variance value may be applied.

Although the barycentric coordinates and the area have been described as the measured values in each embodiment, other measured values such as the inertial principal axis angle and the equivalent elliptic major axis / minor axis are used to calculate the inside of the calculation target frame F. Involved calculation processing may be performed.

Further, in each of the embodiments, the case of the measurement processing relating to the X-axis and Y-axis two-dimensional images has been described, but it is also applicable to the case of the measurement processing relating to a stereoscopic image such as a CT scanner.

[0134]

As described above, according to the image processing apparatus of the present invention, the range setting means, the range moving means, the calculating means and the range changing means are provided, and the target image inside the past calculation target frame is displayed. The position and size of the calculation target frame can be changed based on the barycentric coordinate value and the standard deviation value.

Therefore, even when the image of the object and the object other than the object are included in the initially set object frame, the object other than the object is sequentially inserted from the object frame. The position and size are automatically changed so as to exclude the image of. This makes it possible to measure the barycentric coordinate value from the area of the object with good reproducibility, and to apply the calculated barycentric coordinate value as the true barycentric position of the object.

Further, according to the image processing apparatus of the present invention, the calculation processing is repeated based on the preset control target value of the calculation range frame. Therefore, even when the first calculation target frame set first includes a part of the object that is not the target, by repeating the calculation process based on the control target value of the calculation range frame, The improper measurement is reduced. If the gravity center coordinate value cannot be measured, an alarm or the like is issued to prevent a control error or the like in advance.

Furthermore, according to the image processing apparatus of the present invention,
Since a new calculation target frame that is a constant multiple of the standard deviation value is set by the range setting means, when there are many objects other than the target object of different shapes near the target object or when the shape of the target object is undecided, It is possible to set the optimum calculation target frame at high speed.

Further, according to the image processing apparatus of the present invention, the calculating means is operated based on the barycentric coordinate value of the target image inside the past calculation target frame and the barycentric coordinate value of the target image inside the calculation target frame. Distance information between the centers of gravity is calculated, and the control target value and the distance information between the centers of gravity are compared by the comparison means.

Therefore, when the barycentric coordinate of the object is repeatedly calculated in relation to the calculation target frame, the center position of the target calculation target frame can be quickly brought close to the barycentric coordinate value of the target. Become.

As a result, the dead time related to the image measurement processing is suppressed as much as possible, and the measurement information of the object can be calculated at high speed. According to the robot controller of the present invention, the robot working unit, the image acquisition unit, the image measuring unit and the control unit are provided, and the image measuring unit is the image processing unit of the present invention.

Therefore, the position and size of the calculation target frame are fluidly and automatically changed by the image measuring means, so that the accurate barycentric coordinate value of the object can be calculated at high speed.
Therefore, as compared with the conventional example, useless control of the robot working unit is omitted, and the positioning process can be speeded up.

This greatly contributes to the provision of a highly reliable image measuring device and a robot controller having a high-performance object recognition function to which the image measuring device is applied.

[Brief description of drawings]

FIG. 1 is a principle diagram (1) of an image processing apparatus and a robot control apparatus according to the present invention.

FIG. 2 is a principle diagram (No. 2) of the image processing apparatus and the robot control apparatus according to the present invention.

FIG. 3 is a configuration diagram of a robot control device to which an image processing device according to an embodiment of the present invention is applied.

FIG. 4 is a flowchart of image processing for recognizing an object according to the first embodiment of the present invention.

FIG. 5 is a supplementary explanatory diagram of the flowchart according to the first embodiment of the present invention.

FIG. 6 is a flowchart of image processing for recognizing an object according to the second embodiment of the present invention.

FIG. 7 is a supplementary explanatory diagram of a flowchart according to another embodiment of the present invention.

FIG. 8 is a flowchart of image processing for recognizing an object according to the third embodiment of the present invention.

FIG. 9 is a flowchart of image processing for recognizing an object according to the fourth embodiment of the present invention.

FIG. 10 is an explanatory diagram of an object recognition method of a robot controller according to a conventional example.

[Fig. 11] Fig. 11 is an acquired image diagram of a target object for explaining the problems in the conventional example.

[Explanation of symbols]

 11 ... Range setting means, 12 ... Range moving means, 13 ... Computing means, 14 ... Range reducing / enlarging means, 15 ... Comparison means, 17 ... Robot working unit, 18 ... Image acquisition means, 19 ... Image measuring means, 20 ... Control means, F ... Calculation target frame, Xgp, Ygp ... Past barycentric coordinate value, Xgr, Ygr ... Present barycentric coordinate value.

Claims (6)

[Claims] 1. A range setting means (11) for setting a calculation target frame (F) to an image acquisition target (16) and a range for moving the calculation target frame (F) set to the image acquisition target (16). A moving means (12), a calculating means (13) for calculating the barycentric coordinates and standard deviation of the target image inside the calculation target frame (F), and a range changing means for reducing or enlarging the calculation target frame (F). (14) and changing the position or size of the calculation target frame (F) based on the barycentric coordinate value or standard deviation value of the target image inside the calculation target frame (F) in the past. Image processing device. 2. The image processing device according to claim 1, wherein
An image processing apparatus, wherein arithmetic processing is repeated based on a control target value of a preset calculation range frame (F). 3. The image processing apparatus according to claim 1, wherein
The image processing apparatus, wherein the range setting means (11) sets a new calculation target frame (F) that is a constant multiple of the standard deviation value. 4. The image processing apparatus according to claim 1, wherein
The calculation means (13) calculates distance information between the centers of gravity based on the barycentric coordinate values of the target image inside the calculation target frame (F) and the barycentric coordinate values of the target image inside the calculation target frame (F) in the past. An image processing device characterized by calculating. 5. The image processing apparatus according to claim 1,
An image processing apparatus comprising a comparison means (15) for comparing the control target value and distance information between the centers of gravity. 6. A robot working unit (17) for performing various teaching operations relating to an object (10), an image acquisition means (18) for acquiring an image of the object (10), and the object (10). Image measuring means (19) for measuring position information of
And the robot working unit (17), image acquisition means (1
8) and a control means (20) for controlling the input / output of the image measuring means (19), wherein the image measuring means (19) comprises the image processing device according to any one of claims 1 to 5. Robot controller.