JPH05272922A - Visual sensor apparatus - Google Patents

Abstract

PURPOSE:To make it possible to perform highly accurate measurement without enlarging the arranging area of markers even when a distance is changed by computing the position and the attitude with the images of the markers (markers neighboing other markers) facing each visual-sense input device in a region, where the distance to an object is small (large). CONSTITUTION:When a distance to an object 1 is small, a visual-sense input device 4A (4B) picks up the images of markers 2A and 2B (2D and 2E) on the object 1. A computer 5A (5B) computes the position and the attitude of the object based on the data of the markers 2A and 2B (2D and 2E) on the image of the picked-up image. On the contary, when the distance to the object 1 is large, the devices 4A (4B) picks up the images 2A-2E. The computer 5A (5B) computes the position and the attitude of the object 1 based on the data of the markers 2A-2E on the picked-up images. Namely, when the distance to the object 1 is large, the markers for devices 4A and 4B in the close proximity are shared, and the measurement can be performed without deteriorating the accuracy.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual sensor device used to detect a three-dimensional position and orientation of an object when the object is automatically operated or approached. is there.

[0002]

2. Description of the Related Art FIG. 19 is, for example, issued by the 2nd SAIRAS Executive Committee, 2nd Space Artificial Intelligence / Robot / Automation Symposium (November 17, 1988, 1
(8th) Lecture collection, P51 to P54 is a block diagram showing a conventional visual sensor device, in which 1 is an object, 2A is a marker placed on or near the object 1, and 3 is an object. A supporter that supports a marker 2B placed apart from the object 1, 4 is a visual input device such as a CCD camera that images the markers 2A and 2B, and 5 is the position of the object 1 based on the image output by the visual input device 4. A calculator (calculation device) for calculating the posture and the posture 6 indicates the visual field of the visual input device 4.

Next, the operation will be described. First, the visual input device 4 images the markers 2A and 2B. The three-dimensionally arranged markers 2A and 2B are projected and converted into an image by the visual input device 4, and the image obtained by the visual input device 4 is input to the computer 5. In the computer 5, each marker 2A,
Using the relative positional relationship of 2B (distance, angle, etc.), the position or orientation of the target object 1 is obtained from the position of the marker on the imaging surface.

FIG. 20 shows, for example, the Robotics and Mechatronics Lecture Meeting of Japan Society of Mechanical Engineers '90 (June 8, 1990).
(9th, 9th), Proceedings, P287 to P290, which are explanatory views showing a conventional method of detecting an object, in which 10 is an input image obtained by a visual input device such as a CCD camera, Reference numeral 8 indicates an object image, and 7 indicates a search area set for searching the object image 8.

Next, the operation will be described. First, a search area 7 for searching the object 1 is set in the image. This is a measure to reduce the number of data and speed up the processing by limiting the search area because it takes a lot of time to process the entire screen. At the start of measurement, the operator indicates the position of the search area 7.

From the second and subsequent measurements, the appearance position of the object image 8 is predicted from the previous measurement results, and the search area is set around it. If the position of the object image 8 on the screen is known, the position of the object 1 is calculated under the condition that the object 1 is on a certain known plane.

FIG. 21 is, for example, issued by the 2nd SAIRAS executive committee, 2nd Space artificial intelligence / robot / automation symposium (November 17, 1988, 1
FIG. 8 is a configuration diagram showing a conventional visual sensor device shown in Lectures, P51 to P54, in which 1 is an object, 2A and 2B are placed on the object 1 or in the vicinity of the object 1. It is a marker.

Reference numeral 3 is a marker 2 placed apart from the object 1.
A supporter that supports B, 4 is a visual input device such as a CCD camera that images the markers 2A and 2B, and 5 is a computer that calculates the position and orientation of the object 1 based on the image output by the visual input device 4 ( Reference numeral 6 denotes a visual field of the visual input device 4.

Next, the operation will be described. The visual input device 4 images the markers 2A and 2B. Each of the markers 2A and 2B arranged three-dimensionally is projected and converted into an image by the visual input device 4. The visual input device 4 outputs the data image to the computer 5. The computer 5 obtains the posture from the position of the marker image in the image using the relative positional relationship (distance, angle, etc.) between the markers 2A and 2B.

Also, conventionally, a device for irradiating light from a visual input device, detecting the light reflected by a corner cube reflector used as a marker with the visual input device, and detecting the position and orientation of an object from the obtained image. Has been considered.

FIG. 22 is a block diagram of the visual sensor device shown in, for example, the 31st Space Science and Technology Union Lecture Lecture P530-531, where 1 is an object and 30 is an object 1. Corner cube reflector as a fixed marker, 25 is a corner cube reflector 3
An illuminator that illuminates 0, 4 is a visual input device, 20 is a half mirror that transmits the light emitted from the illuminator 25, changes the direction of the light that is reflected by the corner cube reflector 30 and is directed toward the visual input device 4, 5 Is a calculator that calculates the position and orientation of the object 1 from the image output by the visual input device 4.

Next, the operation will be described. First, the light emitted from the illuminator 25 passes through the half mirror 20,
It is reflected by the corner cube reflector 30 mounted on the object 1. Due to the characteristics of the corner cube reflector 30, the reflected light is reflected in the direction parallel to the incident light and reaches the half mirror 20. This light is reflected on the half mirror 20 and reaches the visual input device 4. Since the corner cube reflector 30 is equivalent to placing a plane mirror perpendicular to the incident light, the visual input device 4 produces an image of the illuminator 25 at the four corner cube reflectors 30. The computer 5 processes this image to calculate the position and orientation of the object 1.

[0013]

Since the conventional visual sensor device is constructed as described above, in the device shown in FIG. 19, two or more devices may be redundantly constructed in order to enhance reliability. Was not considered. Therefore, in order to have a redundant configuration, it is necessary to mount two identical devices, which increases the area and weight occupied by the markers and the visual input device, which hinders the mounting of other devices. there were. In addition, only 2 markers are used with the conventional configuration.
If more than one device is shared and a redundant configuration is attempted, the visual input device is mounted at an angle with respect to the marker, and there is a problem that a sufficient visual field cannot be secured.

In the conventional example of FIG. 20, no particular consideration is given to the method of setting the search area, and when applied to the case of searching for a plurality of markers at the same time, in order to cope with a sudden change in the motion of the object, A large search area must be set. However, if a large number of markers are included in one search area and the process of identifying them is added, or if there is almost no movement, a large area must be processed. There was a problem such as not becoming.

In the conventional example shown in FIG. 21, when the approximate position and orientation of the object 1 are limited, the position and orientation of the object 1 can be detected if three markers are detected. Since it is redundant by using the five markers 2A and 2B, there is a problem that the marker arrangement area becomes large and the area for arranging other devices is limited.

The protruding marker 2B is located at the center of the four rear markers, and when the position or orientation of the object 1 changes, the protruding marker 2B is likely to hide the rear marker 2A. There was a problem such as.

Further, in the conventional example of FIG. 22, the case where the relative distance changes greatly is not taken into consideration, and the device designed to be used in the area where the relative distance is small is applied to the area where the relative distance is large. If this is attempted, the image formed by the light emitted from the illuminator 25 will become smaller and the amount of light will decrease, making it difficult to distinguish it from noise, and there is the problem that measurement cannot be performed. Conversely, if an apparatus designed to be used in a region where the relative distance is large is applied to a region where the relative distance is small, the image formed by the light emitted from the illuminator becomes large and it is difficult to detect it as one image. However, there is a problem that the processing time becomes long.

No particular treatment for ambient light is applied, and the corner cube reflector 30 is viewed from a certain direction.
The disturbance light that is incident on is normally reflected three times inside the corner cube reflector 30 and reflected in parallel with the incident light, but is reflected four times or more inside the corner cube reflector 30 and deviated from the incident direction. Light that is reflected in the direction is generated. When this light is input to the visual input device, an error occurs because the shape of the image formed by the light emitted from the illuminator 25 collapses, or when this light is strong light such as sunlight, smear or There is a problem that a phenomenon called blooming occurs and measurement cannot be performed at all. In order to avoid this problem, it is possible to put a hood around the corner cube reflector to prevent sunlight from hitting the corner cube reflector, but do not block the original signal light emitted from the illuminator. It is difficult to realize a hood that blocks only ambient light, and when the hood is attached, there is an adverse effect that a plurality of adjacent markers arranged on the object 1 are hidden behind the marker.

Further, in order to perform stable measurement, it is necessary that the brightness of the image formed by the light emitted from the illuminator and reflected by the corner cube reflector is higher than that of the surrounding area. The surrounding area of the reflector may be painted black, but in general, the light emitted from the illuminator is much weaker than the sunlight, and in the situation where the sunlight irradiates as a disturbance, the corner cube reflector part It is said that it is necessary to take measures such as not being able to obtain a sufficient contrast between the surrounding area and its surroundings, making the image processing operation unstable and limiting the usage conditions only when there is no strong ambient light. There was a problem.

The invention of claim 1 has been made to solve the above problems, and provides a visual sensor device which can be mounted in as small an area as possible even when a plurality of visual input devices are mounted. The purpose is to

It is an object of the present invention to provide a visual sensor device capable of stably measuring the position and orientation of an object without increasing the complexity of processing even when the motion of the object changes abruptly. And

It is an object of the present invention to provide a visual sensor device which has a structure in which the number of markers is minimized, and which can obtain as high measurement accuracy as possible and prevent hidden markers from hiding rearward markers. ..

It is an object of the present invention to provide a visual sensor device capable of measuring the position and orientation of an object in a wide range from a small relative distance area to a large relative distance area.

The inventions of claims 5, 6, and 7 are as follows:
Generally, when strong ambient light such as sunlight is shining on the corner cube reflector, ambient light may be reflected toward the visual input device, but even at that time, the ambient light reflected is attenuated to reduce its intensity. It is an object of the present invention to obtain a visual sensor device capable of stably extracting the image formed by the light emitted from the illuminator by reducing the value.

According to the inventions of claims 8 and 9, even when the object is exposed to strong ambient light such as sunlight, the contrast between the marker such as a corner cube reflector attached to the object and its surroundings is improved. It is an object of the present invention to obtain a visual sensor device that is sufficiently large and can stably extract a marker image.

[0026]

According to a first aspect of the present invention, there is provided a visual sensor device comprising a plurality of sets of markers which are arranged close to each other on an object and a set which is arranged at a position facing the markers. And a plurality of visual input devices for imaging the markers of
In the area where the distance to the object is small, the computer is caused to calculate the position and orientation only from the marker image of the marker facing each visual input device, and in the area where the distance to the object is large, each visual input device The position and orientation of the target object are calculated based on the marker images of the marker facing each other and other markers adjacent thereto.

The visual sensor device according to the invention of claim 2 is
The computer is provided with a plurality of markers arranged close to each other on the object and a visual input device arranged at a position facing the markers and imaging these markers, and A large search area is set around the marker image whose appearance position is predicted to perform the search, while the second marker image is obtained by the appearance position of the predicted marker image and the first search. Based on the information about the appearance position of the marker image, a search is performed with a small search area set, and the position and orientation of the target object are calculated.

The visual sensor device according to the invention of claim 3 is
Two markers arranged on the object and one marker arranged on a surface different from the arrangement surface of each of the markers described above at a position deviating from the straight line connecting the respective markers A visual input device that is placed at a position facing each other and that captures each of these markers is provided, and the computer is made to calculate the position and orientation of the target object based on the marker image output by the visual input device. It was done like this.

The visual sensor device according to the invention of claim 4 is
A marker with specular reflection is placed on the object, and an illuminator composed of a plurality of light sources is placed around the visual input device.For example, in a region where the relative distance between the object and the visual input device is small, the light of the visual input device is Only the light source near the axis is turned on, and as the relative distance increases, the light source farther from the optical axis of the visual input device is also turned on, and the size of the image of the illuminator reflected on the marker is changed. It is a thing.

The visual sensor device according to the invention of claim 5 is
A corner cube reflector is placed as a marker on the object, and the radius of the incident surface of the corner cube reflector or the radius of the circumscribing circle when the incident surface is processed into a polygon is r. The maximum angle of the normal between the incident surface of the reflector and the light emitted from the illuminator and incident on the corner cube reflector is θ, the refractive index of the corner cube reflector is n, and the direction perpendicular to the incident surface of the corner cube reflector. If the length is h,

It satisfies the following equation.

The visual sensor device according to the invention of claim 6 is
It suppresses the reflection of light at the portion near the outer circumference of the reflecting surface of the corner cube reflector.

A visual sensor device according to a seventh aspect of the invention is
The radius of the effective reflection area of the reflecting surface of the corner cube reflector is R, the length in the direction perpendicular to the entrance surface of the corner cube reflector is h, and the normal of the entrance surface of the corner cube reflector and the illuminator When the maximum value of the angle formed by the light radiated from and incident on the corner cube reflector is θ and the refractive index of the corner cube reflector is n, the reflection of light at the portion near the outer periphery of the reflection surface of the corner cube reflector is By suppressing and narrowing the effective reflection area,

This satisfies the following equation.

The visual sensor device according to the invention of claim 8 is
The angle formed by the normal to the surface of the object around the marker and the light emitted from the illuminator and incident on the marker is increased.

The visual sensor device according to the invention of claim 9 is
The surface of the object around the marker is processed into a triangular wave shape.

[0035]

When the distance to the object is small, the computer measures using a marker arranged in front of each visual input device, and when the distance to the object is large, each visual input is performed. By measuring by using the marker in front of the device and other markers in front of the visual input device that are close to each other, it is possible to increase the placement area of the marker even when the distance to the object changes. , It is possible to configure a redundant visual sensor device while maintaining high measurement accuracy.

According to the second aspect of the invention, in the marker search from the second marker, the information of the predicted marker appearance position and the marker appearance position obtained in the first search is used.
A small search area is set so that the position and orientation of an object can be calculated at high speed.

The three markers on the object according to the invention of claim 3 effectively use the rectangular field of view of the visual input device, and are arranged in the shape of “a” as seen from the visual input device, so as to jump out. The space between the left marker and the other marker behind is widened so that high measurement accuracy can be obtained, and the planar distance between the protruding marker and the other marker behind The possibility of the shadow of the protruding marker is reduced and the calculation result of the position and orientation by the computer is made highly reliable.

The illuminator in the invention of claim 4 is
In the area where the relative distance is small, only the light source close to the optical axis of the visual input device is turned on, and the area of the image created by the light emitted from the illuminator is kept small, simplifying the process of extracting this image and increasing the processing speed. shorten. Also, as the relative distance increases, the image becomes smaller and the amount of light decreases as it is, but by turning on the light source away from the optical axis of the visual input device, the area of the image created by the light emitted from the illuminator is increased. To maintain and increase the amount of light,
Make it easy to distinguish from noise.

Since the corner cube reflector according to the invention of claim 5 is made longer than the normal one in the direction perpendicular to the incident surface, the normal corner cube reflector causes the disturbance light incident from an angle to be hardly attenuated. Although it may be reflected toward the input device, such disturbance light is applied to the side surface having a reduced reflectance to be attenuated. As a result, of the disturbance light radiated to the corner cube reflector, the light reflected toward the visual input device is attenuated to have an intensity that does not pose a problem for measurement.

Since the corner cube reflectors according to the sixth and seventh aspects of the present invention suppress the reflection of light at the peripheral portion of the reflecting surface, the normal corner cube reflector almost completely attenuates the disturbance light incident at an angle. There was a case where the light was reflected toward the visual input device without incident, but such a disturbance light is applied to a surface whose reflection is suppressed to be attenuated. As a result, of the disturbance light radiated to the corner cube reflector, the light reflected toward the visual input device is attenuated to have an intensity that does not pose a problem for measurement.

In the object of the eighth aspect of the present invention, since the angle between the normal line of the surface of the object around the marker and the light that is emitted from the illuminator and is incident on the marker is increased, strong sunlight or the like is emitted. However, since the light around the marker is reflected in a direction other than the visual input device, the peripheral portion of the marker is observed to be dark when viewed from the visual input device. A large contrast can be obtained and the marker image can be stably extracted.

In the object according to the ninth aspect of the present invention, the surface around the marker is processed into a triangular wave shape, so that the light reflected by the surface processed into the triangular wave shape is visible even when irradiated with strong sunlight. Since the light is reflected or attenuated in a direction other than the input device, the peripheral area of the marker is observed dark when viewed from the visual input device. Therefore, a large contrast can be obtained between the marker image and its peripheral portion, and the marker image can be stably extracted.

[0043]

【Example】

Example 1. An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a visual sensor device according to an embodiment of the invention of claim 1, in which FIG.
A and 2D are markers placed on or near the object 1, and 3A and 3B are markers 2 placed apart from the object 1.
B, 2E supporters, 4A, 4B are markers 2A,
Visual input devices such as CCD cameras for imaging 2B, 2D and 2E, 5A and 5B are calculators (calculation devices) for calculating the position and orientation of the object 1 based on the images output by the visual input devices, 6A and 6B, 6C and 6D are visual input devices 4A and 4
The view of B is shown.

Further, by arranging only the marker set used when the distance to the object 1 is small in front of the visual input devices 4A and 4B and sharing the markers with each other when the distance to the object 1 is large. It eliminates the marker that was added to prevent deterioration of accuracy when the distance becomes large, reduces the number of markers, and reduces the layout area.

Next, the operation will be described. First, when the distance to the target object 1 is small, the visual input device 4A images the markers 2A and 2B on the target object 1, and the computer 5A uses the data of the markers 2A and 2B on the captured image to target the target object 1. Calculate the position and orientation of. Similarly, the visual input device 4B
Picks up images of the markers 2D and 2E on the object 1, and the computer 5
B calculates the position and orientation of the object 1 from the data of the markers 2D and 2E on the captured image.

On the other hand, when the distance to the object 1 is large, the visual input device 4A causes the marker 2 on the object 1 to move.
A, 2B, 2D and 2E are imaged, and the computer 5A calculates the position and orientation of the object 1 from the data of the markers 2A, 2B, 2D and 2E on the imaged image. Similarly, the visual input device 4B images the markers 2A, 2B, 2D, 2E on the object 1, and the computer 5B sets the markers 2A, 2 on the imaged image.
The position and orientation of the object 1 are calculated from the B, 2D, and 2E data. That is, in this embodiment, when the distance to the object 1 is large, no marker is added in order to suppress the deterioration of accuracy, but the adjacent visual input devices 4A, 4A,
By sharing the marker arranged for 4B, it is possible to measure the position and the like without degrading accuracy.

It should be noted that in the above-described embodiment, the redundant configuration has two
Although the case where a single visual sensor is mounted is shown, three or more visual sensors may be mounted, and the same effect as that of the above-described embodiment is obtained.

Further, in the above embodiment, when the distance to the object 1 is small, the visual input devices 4A and 4B that are close to each other do not share the marker, but when the distance to the object 1 is small, the adjacent visual input devices 4A and 4B do not share the marker. By sharing the marker among the sensors, it is possible to further reduce the number of markers and the layout area.

Example 2. FIG. 2 shows an embodiment of the invention of claim 2, in which 1 is an object 1, 2 is a plurality of markers placed on the object 1, and 4 is a CCD camera for picking up an image of the marker 2. The visual input device 5 is a calculator.
Further, in FIG. 3, 10 is an input image obtained by the visual input device 4, 11 is a marker image that is first tried to be found, 14 is another marker image, and 15 is a search area set to find the marker image 11. , 16 are search areas set for finding the marker image 14.

Next, the operation will be described. First, the visual input device 4 images the marker 2 on the target object 1, and the computer 5 processes the input image to calculate the position and orientation of the target object 1. Image processing is performed as follows.

First, the appearance position of the marker image 11 is predicted from the previous measurement result, and a large search area 15 is set around it. At the time of the first measurement, since there is no prior information and the appearance position cannot be predicted, the operator indicates the appearance position or searches for the marker image 11 by a special method.

The size of the search area is set so that it can follow the movement of the object 1 at the maximum speed. Therefore, the adjacent marker image 14 may be included in the set area. When two or more marker images 11 and 14 are detected when searching the set area and finding the marker image, the target marker image 11 is compared with the appearance order of the predicted marker images. decide.

When the marker image 11 is discovered, the difference between the predicted position of the marker image 11 and the actual position is obtained, and the difference is added to the appearance predicted position of the marker image 14 other than the marker image 11 to appear. Correct the predicted position. As a result, the prediction accuracy of the appearance position is improved, so that the marker image 14 can be found only by setting a small search area 16 around the corrected prediction position.

In this way, if all the marker images 11 and 14 are detected, the position and orientation of the object are calculated from the appearance positions in those images.

Example 3. FIG. 4 shows an embodiment of the invention of claim 3. In the figure, 2A is two markers arranged on two opposite vertices of a rectangle 17 which is similar to the visual field of the visual input device 4, and 2B is a marker which is offset from the center of the rectangle and which is projected upward, 17 Is a virtually assumed rectangle, 3 is a support for the marker 2B, 4 is a visual input device, 5 is a calculator, and 6 is a visual field of the visual input device.

Next, the operation will be described. First, as in the conventional case, the visual input device 4 uses the marker 2 of the object 1.
Images A and 2B are taken, and the computer 5 calculates the position and orientation of the object 1 from the marker image positions of the markers 2A and 2B in the taken images.

Here, a rectangle 17 which is similar to the visual field of the visual input device 4 at an appropriate position on the object 1 and which does not protrude from the visual field even if the object 1 takes an arbitrary position and posture in the measurement condition. Assume Then, two markers 2A are arranged on the opposite vertices of the rectangle 17. Next, on a straight line 19 passing through the center of the rectangle perpendicular to the diagonal line 18 as a straight line connecting the two markers 2A from the center of the rectangle, and at a position protruding upward from a position deviated from the center of the rectangle 17, One marker 2B is arranged and this marker 2B
Are supported by the support device 3.

By doing so, when viewed from the direction of the visual input device 4, the marker 2B is located at a position deviated from the straight line connecting the two markers 2A, so that the marker 2B is on the straight line connecting the respective markers 2A. Compared to the case, the space created by the markers 2A and 2B becomes wider. Further, the planar distance between the markers 2A and 2B also becomes large. Furthermore, it is possible to prevent the marker 2A from being hidden by the marker 2B.

In the above embodiment, the case where the total number of the markers 2A and 2B is three is shown, but the markers 2A and 2B are also used so as to be used for the measurement when the distance to the object 1 is large. Markers are arranged in addition to the surroundings of the marker, and when the distance to the object 1 is small, the marker 2A and the marker 2
Although only B is used for measurement, when the distance is large, the accuracy of measurement can be improved by additionally using the added marker for measurement.

In the above embodiment, the marker 2B is the straight line 1.
However, the position can be shifted according to the shape of the visual field of the visual input device 4.

Example 4. 5 is an overall configuration diagram of an embodiment of the invention of claim 4, and FIG. 6 is a front view of an illuminator in the embodiment. In the figure, 1 is an object, 2 is a specular marker attached to the object 1, 3 is a visual input device, 25 is an illuminator attached to the visual input device 4,
Reference numerals 25a, 25b and 25c are light sources forming the illuminator 25, and 25a is an innermost light source in a triple arrangement.
25b is a central light source in the triple arrangement, 25c is an outermost light source in the triple arrangement, and 24 is an illuminator 2.
A hole made in the center of 5 for passing light through the visual input device 4,
Reference numeral 5 denotes an object 1 obtained by processing an image output from the visual input device 4.
2 shows a computer that calculates the position and orientation of the.

FIG. 7 shows the light source 25a among the light sources 25a, 25b and 25c which are multiply arranged around the optical axis of the visual input device 4.
And 25a, 25b, 25c
The results of measuring the light energy focused on one light-receiving element of the visual input device 4 by changing the relative distance when all of are turned on are shown.

Next, the operation will be described. First, when the light sources 25a, 25b, 25c of the illuminator 25 are turned on, this light is reflected by the marker 2 attached to the object 1,
The visual input device 4 is reached through a hole 24 formed in the center of the illuminator 25. Since the marker 2 is specularly reflected, the shape of the light source of the illuminator 25 is directly captured by the visual input device 4. That is, a donut-shaped image in which the central portion corresponding to the hole 8 is dark is formed at a position where the marker 2 is located. If a plurality of markers 2 are arranged on the object 1, donut-shaped images are obtained by the number of the markers 2.
The position and orientation of the object 1 can be calculated from the positions of these images.

When the operation is started, all the light sources 25a, 25b and 25c arranged in triplicate are turned on to input an image, and the position and orientation of the object 1 are calculated from the obtained image. From the next time, if the relative distance is small, corresponding to the position obtained in the previous measurement, only the innermost light source 25a is turned on to input the image, thereby suppressing the size of the image to be small. The load of the process of extracting this image from is reduced, and high-speed processing is realized. When it is determined that the relative distance is medium, the light sources 25a, 25 in the innermost and middle portions
The b is turned on, an image is input, and measurement processing is performed. By doing so, an appropriate image size and light amount can be obtained. If it is determined that the relative distance is large, the triple light source 25
By inputting an image with all of a, 25b, and 25c turned on, the image is made as large as possible and the amount of light is increased, thereby making it possible to reliably identify the noise.

FIG. 7 shows the double light sources 25a, 2 from the inside.
5b is turned on, and the triple light sources 25a, 25b, 2
5 shows the results of measuring the amount of light (energy) condensed on one light receiving element of the visual input device 4 at each relative distance when all 5c are turned on. In the figure, what is indicated by □ is the light source 25.
In the case where all of a, 25b, and 25c are turned on, what is indicated by a circle is when only the light sources 25a and 25b are turned on.
As a tendency, it decreases rapidly up to a certain distance,
Beyond that distance, it becomes almost constant and then begins to decrease again. This is because in a region where the distance is short, only light emitted from one light source is focused on one light receiving element, so that the light decreases substantially in inverse proportion to the square of the distance. Since the light from the light source is focused on one light receiving element, the light coming from one light source decreases in inverse proportion to the square of the distance, but from the number of light sources proportional to the square of the distance. Since the emitted light is focused on one light receiving element, the total amount of light does not decrease. Since the number of light sources is finite,
Eventually the amount of light begins to decrease. That is, if the number of light sources to be turned on is increased in proportion to the square of the distance, a certain amount of light can be maintained up to a distant place.

Even in the example of FIG. 7, this tendency is almost recognized. The light amount sharply decreases up to a distance of about 4 m, but it hardly changes thereafter, and when the light sources 25a and 25b are turned on, the light amount starts again from about 10 m. When the light sources 25a, 25b, and 25c are turned on, the light emission starts to decrease again from about 12 m. Although there is a limit to increasing the output of each light source, it is possible to suppress the decrease in the light amount in the distance by increasing the number while maintaining the same output. However, if many light sources are turned on at positions close to each other, the size of the image of the marker becomes large and the processing time increases.
As in the present invention, it is necessary to increase or decrease the number of light sources that are turned on according to the distance.

Example 5. 8 is a diagram showing an embodiment of the invention of claim 5, FIG. 9 (a) is a perspective view showing a normal reflection characteristic of a corner cube reflector, and FIG. 9 (b) is an equivalent reflection characteristic of the corner cube reflector. 2A is a two-dimensional explanatory diagram, FIG. 10A is a diagram showing an example in which reflected light that is not parallel to the incident light is generated in the corner cube reflector, and FIG. 10B is an effect of this embodiment (parallel to the incident light). Principle of attenuating reflected light that is not parallel), FIG. 11 is an explanatory view used for deriving a conditional expression for attenuating reflected light that is not parallel to the incident light, and FIG. 12 is incidence of the corner cube reflector used in this embodiment. It is a table showing the length in the direction perpendicular to the plane.

In the figure, 30 is a corner cube reflector, 31, 32, and 33 are respective surfaces of the corner cube reflector, 31 is an incident surface, 32 is a reflecting surface, and 33 is a side surface which is processed to suppress reflection. In addition, the same components as those in FIG. 1 are designated by the same reference numerals, and duplicate description will be omitted.

The entire structure of the visual sensor device is almost the same as that of the first embodiment, but the point that the marker is limited to the corner cube reflector is different from the first embodiment. The light that has entered the corner cube reflector 30 is normally reflected by three different reflecting surfaces 32 as shown in FIG.
Go out from 1. At this time, the incident light and the reflected light are parallel. The reflection characteristic shown in FIG. 9A is equivalent to that shown in FIG.
It can be handled two-dimensionally as in (b).

FIG. 9 shows the reflection characteristics of a normal corner cube reflector, but as shown in FIG. 10A, light incident from an oblique direction may be reflected in a direction not parallel to the incident light. Light incident on the corner cube reflector 30 from an oblique direction is reflected by the reflecting surface 32 and then reaches the incident surface 31 without reaching other reflecting surfaces. Since this light has a large angle with the direction normal to the incident surface, a part or all of the light does not go out of the incident surface 31 and is reflected toward the inside of the corner cube reflector 30. After that, the reflection is repeated as shown in FIG. 10A, and the light exits from the incident surface 31 in due course. As a result, light whose incident direction and reflection direction are not parallel to each other is generated. That is, when the corner cube reflector 30 is irradiated with sunlight or the like, the light may be reflected by the corner cube reflector portion and input to the visual input device 4. Therefore, if the side surface 33 of the corner cube reflector 30 is lengthened as in the present embodiment shown in FIG.
Therefore, even if there is light reflected toward the inside of the corner cube reflector 30, this light strikes the side surface 33 and is attenuated. Therefore, even if this light goes out, the intensity is such that measurement is not hindered.

FIG. 11 shows a critical state in which reflected light that is not parallel to the incident light is generated, and the length h of the corner cube reflector 30 in the direction perpendicular to the incident surface 31 is as small as possible. In this case, strong reflected light that is not parallel to the incident light is generated. For the case of this figure, by solving the geometrical equation, as a condition that does not cause strong reflected light that is not parallel to the incident light,

The following is derived. Where θ is the normal of the entrance surface of the corner cube reflector 30 and the illuminator, and is the corner cube reflector 3 under the usage conditions of this device.
0 is the maximum angle formed by the light incident on 0, and r is the incident surface 31
Is the radius of the circle, the radius of the circumscribing circle when the incident surface 31 is processed into a polygon, and n is the refractive index of the corner cube reflector 30. Further, FIG. 12 shows the ratio of h and r when the invention of claim 5 is used. In addition, the minimum ratio of h and r for forming the corner cube reflector is also shown. When θ is determined from the usage conditions of the device, the condition regarding the ratio of h and r is obtained. It is necessary to increase h as the device is used until θ is large.

Example 6. Next, an embodiment of the inventions of claims 6 and 7 will be described with reference to the drawings. FIG. 13 is an explanatory view showing the principle that the reflected light which is not parallel to the incident light is attenuated as the effect of this embodiment, and FIG. 14 is the conditional expression for attenuating the reflected light which is not parallel to the incident light in the invention of claim 6. FIG. 15 is an explanatory view for explaining the derivation of FIG. 15, and FIG. 15 is an explanatory view for explaining that the reflection characteristic of the corner cube reflector 30 is not deteriorated by applying the present invention. In the figure,
34 is a region that suppresses reflection in the reflection surface 32 of the corner cube reflector 30, 36a, 36b, 36c are the thickness of the light flux that enters the corner cube reflector 30 and is reflected, 39 is the apex of the reflection surface 32, and others, The same components as those in the above-described embodiment are designated by the same reference numerals, and duplicated description will be omitted.

In the fifth embodiment, the ambient light is attenuated by lengthening the side surface, whereas in this embodiment, the reflection of the partial area 34 of the reflecting surface 32 of the corner cube reflector 30 is suppressed to achieve the same effect. I try to get the effect. The overall structure of the device is the same as that of the device of the fifth embodiment shown in FIG. Light incident on the corner cube reflector 30 from a certain direction is reflected by the reflecting surface 32, and then the incident surface 3
Even if there is light reflected toward the inside of the corner cube reflector 30 at 1, the light impinges on the region 34 of the reflection surface 32 where the reflection is suppressed, and is attenuated, so that the light does not interfere with the measurement even if it goes out. Becomes

FIG. 14 shows a critical state in which reflected light that is not parallel to the incident light is generated. If the area 34 that suppresses the reflection of the reflecting surface 32 is smaller than the state shown in this drawing, it is not parallel to the incident light. Reflected light is generated.
For the case of this figure, by solving the geometrical equation, as a condition that does not cause strong reflected light that is not parallel to the incident light,

The following is derived. Here, θ is the maximum value of the angle formed by the normal of the incident surface of the corner cube reflector 30 and the light emitted from the illuminator and incident on the corner cube reflector under the usage conditions of this device, and R is the reflection surface 32.
Is the radius of the effective reflection portion (area excluding the area 34), h is the thickness in the direction perpendicular to the incident surface 31, and n is the refractive index of the corner cube reflector 30.

In FIG. 15, 36a, 36b, 36c
Indicates the thickness of the light flux that is incident on and reflected by the corner cube reflector 30, and has the same thickness on both sides of the light flux that passes through the apex 39 of the reflecting surface 32. In order to perform stable measurement over the entire operating conditions of the device, it is necessary to always secure a luminous flux having a certain thickness or more. Figure 15
As shown in (a), when light enters from a direction perpendicular to the incident surface 31, the light beam 36a when using the present invention and the light beam 36b when not using the present invention have a smaller light beam 36a. However, there is no problem in particular because the thickness of the luminous flux has been secured with a margin. On the other hand, FIG.
As shown in (b), when the light is obliquely incident on the incident surface 31, the thickness of the light flux is reduced as compared with the case where the light is incident in the vertical direction, and thus the thickness of the light flux is further reduced. Although it is necessary to avoid this, in this case, as can be seen from the figure, the thickness of the light flux does not change even if the present invention is used or not used, and there is no problem.

Example 7. FIG. 16 shows an embodiment of the invention of claim 8. In the figure, reference numeral 37 denotes a region around the marker 2 which is inclined so that the angle formed by the normal direction of the surface and the direction of the light emitted by the illuminator 25 is increased, and other components similar to those in FIG. Are denoted by the same reference numerals and redundant description will be omitted.

Next, the operation will be described. First, the light source 2
5a. 25b and 25c are turned on, and the light reflected by the marker 2 on the object 1 and returned is imaged by the visual input device 4, and this image is processed by the computer 5 and the position and posture of the object 1 are processed. To measure. When the object 1 is irradiated with strong ambient light such as sunlight, the surface 37 is inclined so that the light hitting the surface 37 is not directly reflected toward the visual input device 4. Also, after several reflections, the visual input device 4
There is a case where there is light reflected in the direction of, but this light is attenuated each time it is reflected, and has an intensity that does not hinder measurement. As a result, when viewed from the visual input device 4, the marker 2 is brightly observed by the light emitted from the illuminator 25, but the peripheral portion thereof is darkly observed. Therefore,
The contrast between the image of the marker 2 and its peripheral portion is increased, and the marker image can be stably extracted. The surface portion of the object 1 which is not inclined like the surface 37 is observed to be very bright depending on the irradiation direction of the ambient light, but can be easily removed by image processing because it is far from the marker image. no problem.

In addition, in the above embodiment, the periphery of the marker is formed into a conical surface so as to have the inclination, but the same effect can be obtained even if the plane has a large inclination.

Example 8. FIG. 17 shows an embodiment of the invention of claim 9 and FIG. 18 is an explanatory view for explaining its operation. In the figure, reference numeral 38 denotes a region around the marker which is coated with a material for limiting the reflectance of the surface and has a triangular wave shape. Other than the above, the same components as those in FIG. A duplicate description will be omitted.

Next, the operation will be described. First, the light source 2
5a, 25b, 25c are turned on, and the light reflected by the marker 2 on the object 1 and returned is imaged by the visual input device 4, and this image is processed by the computer 5 to determine the position of the object 1. And posture. When the object 1 is irradiated with strong ambient light such as sunlight, when the triangular wave-shaped surface 38 is laterally irradiated as shown in FIG. Reflected in the direction largely deviated from
As shown in (b), when the triangular wave-shaped surface 38 is irradiated from a direction close to the vertical direction, the light is attenuated by being repeatedly reflected by the triangular wave-shaped surface 38. Due to these two effects, the light reflected in the direction of the visual input device 4 is suppressed to be very weak around the marker 2 so that the measurement is not hindered.

In the above-mentioned embodiment, the long mountain shape is arranged side by side, but the same effect can be obtained even if the pyramidal shape is spread all over the surface.

[0082]

As described above, according to the first aspect of the present invention, a plurality of sets of markers arranged close to each other on an object and a set of markers arranged in a position facing the markers are provided. With a plurality of visual input devices for imaging, in a region where the distance to the object is small, the computer is made to calculate the position and orientation only from the marker image of the marker facing each visual input device, and the object is In a region where the distance to is large, it is configured to calculate the position and the posture based on the marker images of the markers facing each visual input device and other markers adjacent to the marker. Even when there is a change, there is an effect that a redundant visual sensor device can be configured with high accuracy without increasing the arrangement area of the marker.

According to the second aspect of the present invention, it is provided with a plurality of markers arranged close to each other on the object, and a visual input device arranged at a position facing the markers and imaging these markers. In the search process of the first marker, the computer sets a large search area around the marker image whose appearance position is predicted, and causes the computer to search, while the second marker image is used to search for the predicted marker. Since the search is performed in a small search area setting based on the information of the appearance position of the image and the appearance position of the marker image obtained in the first search, the position and orientation of the object are calculated. Even if the motion of the object changes suddenly, there is an effect that the position and orientation of the object can be measured at high speed without increasing the complexity of processing.

According to the third aspect of the present invention, the two markers arranged on the object and the markers arranged at a position deviating from the straight line connecting the markers are arranged on a plane different from the plane on which the markers are arranged. A marker and a visual input device that is arranged at a position facing each of the markers and that captures each of the markers. Based on the marker image output by the visual input device, Since it is configured to calculate the position and orientation of the target object, while minimizing the number of markers to obtain the highest possible measurement accuracy, while preventing the marker behind from being hidden by the marker protruding forward, There is an effect that a highly reliable result of position and orientation measurement can be obtained.

According to the fourth aspect of the present invention, the marker which is specularly reflected is arranged on the object, and the illuminator composed of a plurality of light sources is arranged around the visual input device, according to the distance to the object. Since the size of the reflected image of the illuminator from the marker is changed by changing the number of light sources of the illuminator, it is possible to measure from near to distant with a simple configuration while keeping the measurement processing time short. There is an effect.

According to the fifth aspect of the present invention, the corner cube reflector is configured so as to satisfy the predetermined condition, whereby the ambient light reflected by the corner cube reflector and directed to the visual input device can be attenuated. There is an effect that stable measurement can be performed even under the condition that strong ambient light such as light is emitted.

According to the sixth aspect of the present invention, the means for suppressing the reflection near the outer periphery of the reflecting surface of the corner cube reflector is provided, so that the ambient light reflected by the corner cube reflector and traveling toward the visual input device can be attenuated. Therefore, there is an effect that stable measurement can be performed even in a situation where strong ambient light such as sunlight is emitted.

According to the invention of claim 7, by constructing the corner cube reflector so as to satisfy a predetermined condition, it is possible to attenuate the ambient light reflected by the corner cube reflector and directed to the visual input device. There is an effect that stable measurement can be performed even under the condition that strong ambient light such as light is emitted.

According to the invention of claim 8, since the light reflected from the peripheral portion of the marker toward the visual input device can be reduced by a simple structure in which the surface of the object around the marker is inclined, the sunlight or the like can be reduced. There is an effect that it is possible to stably measure even under the condition that the strong ambient light of is irradiated.

According to the invention of claim 9, the light reflected from the peripheral portion of the marker toward the visual input device can be reduced with a simple structure in which the surface of the object around the marker is made into a triangular wave shape, so that sunlight or the like can be reduced. There is an effect that it is possible to stably measure even under the condition that the strong ambient light of is irradiated.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing a visual sensor device according to an embodiment of the present invention.

FIG. 2 is a configuration diagram showing a visual sensor device according to an embodiment of the invention of claim 2;

FIG. 3 is an explanatory diagram illustrating a method of processing an input image input to the computer of FIG.

FIG. 4 is a configuration diagram showing a visual sensor device according to an embodiment of the invention of claim 3;

FIG. 5 is a configuration diagram showing a visual sensor device according to an embodiment of the invention of claim 4;

6 is a front view shown for explaining the operation of the illuminator in FIG. 5. FIG.

FIG. 7 is an explanatory diagram showing a result of an experiment for confirming the effect of the invention of claim 4;

FIG. 8 is a configuration diagram showing a visual sensor device according to an embodiment of the invention of claim 5;

9A and 9B are explanatory views showing reflection characteristics of a normal corner cube reflector, FIG. 9A is a perspective view, and FIG. 9B is an explanatory view in the case of two-dimensional equivalent conversion.

FIG. 10 is an explanatory view showing the action of the invention of claim 5;
(A) is a figure which shows the reflection characteristic of a normal corner cube reflector, (b) is a figure which shows the reflection characteristic of the corner cube reflector in the Example of this invention.

FIG. 11 is an explanatory diagram illustrating the derivation of a conditional expression regarding the length of the corner cube reflector in the direction perpendicular to the incident surface in the invention of claim 5;

FIG. 12 is an explanatory view showing the length of the corner cube reflector in the direction perpendicular to the incident surface when the invention of claim 5 is used.

FIG. 13 is an explanatory view showing the operation of the inventions of claims 6 and 7.

FIG. 14 is an explanatory diagram illustrating the derivation of a conditional expression regarding the radius of the effective reflection area of the reflection surface of the corner cube reflector in the invention of claim 6;

FIG. 15 is an explanatory diagram illustrating an operation of the inventions of claims 6 and 7;

FIG. 16 is a configuration diagram showing a visual sensor device according to an embodiment of the invention of claim 8;

FIG. 17 is a configuration diagram showing a visual sensor device according to an embodiment of the invention of claim 9;

FIG. 18 is an explanatory diagram illustrating an effect of the invention of claim 9;

19 is a configuration diagram showing a conventional visual sensor device corresponding to FIG.

FIG. 20 is an explanatory diagram showing a relationship between a conventional input image and an object image corresponding to FIG. 2.

21 is a configuration diagram showing a conventional visual sensor device corresponding to FIG.

22 is a configuration diagram showing a conventional visual sensor device corresponding to FIG.

[Explanation of symbols]

1 Object 2, 2A, 2B, 2D Marker 4, 4A, 4B Visual input device 5, 5A, 5B Calculator 11, 14 Marker image 15, 16 Search area 18 Diagonal line (straight line) 25 Illuminator 25a, 25b, 25c Light source 30 Corner Cubic reflector 31 Incident surface of the corner cube reflector 32 Reflective surface of the corner cube reflector 34 Area processed to suppress reflection (means for suppressing reflection) 37 Area with a large inclination (normal to the surface of the symmetrical object around the marker) And the incident angle of the light emitted from the light source of the illuminator and illuminating the marker is increased.) 38

Claims (9)

[Claims] 1. A plurality of sets of markers arranged close to each other on an object, and a plurality of visual input devices arranged at positions facing the markers to capture images of each set of markers, In a visual sensor device capable of calculating the position and orientation of the target object based on the marker image output by the input device, in a region where the distance to the target object is small, the marker facing each visual input device is The position and orientation are calculated from only the marker image, and in a region where the distance to the object is large, the position and the posture are calculated based on the marker images of the marker facing each visual input device and other markers adjacent to the marker. A visual sensor device comprising a calculator for calculating a posture. 2. A plurality of markers arranged close to each other on an object, and a visual input device which is arranged at a position facing the markers and picks up an image of these markers, the visual input device outputting In the visual sensor device capable of calculating the position and orientation of the target object based on the marker image, the large search area is set around the marker image whose appearance position is predicted in the first marker search process. The second marker image is searched for by setting a small search area based on the information on the predicted marker image appearance position and the marker image appearance position obtained in the first search. hand,
A visual sensor device comprising a calculator for calculating the position and orientation of an object. 3. Two markers arranged on the object,
At a position outside the straight line connecting these markers,
One marker arranged on a surface different from the arrangement surface of each marker, and arranged at a position facing each of the markers,
A visual sensor device comprising: a visual input device that images each of these markers; and a calculator that calculates the position and orientation of the object based on the marker image output by the visual input device. 4. A marker arranged on an object to specularly reflect the light emitted, a visual input device arranged at a position facing the marker, and a plurality of visual input devices arranged around the visual input device. With a illuminator consisting of a light source, in the visual sensor device that enables calculation of the position and orientation of the object from the image output by the visual input device, depending on the distance to the object, the illuminator A visual sensor device comprising a calculator for changing the size of a reflected image of an illuminator from a marker by changing the number of light sources turned on. 5. A marker using a corner cube reflector which is arranged on an object and reflects the irradiated light,
A visual input device arranged at a position facing the marker;
In the vision sensor device, the corner cube reflector is provided with an illuminator composed of a light source arranged around the visual input device, and a calculator for calculating the position and orientation of the object from an image output by the visual input device. If the incident surface is a circle, its radius, and if the incident surface is processed into a polygon, the radius of its circumscribing circle is r. Under normal conditions of use, the normal to the incident surface of the corner cube reflector and the above The maximum value of the angle formed by the light emitted from the illuminator and incident on the corner cube reflector is θ, the refractive index of the corner cube reflector is n, and the length in the direction perpendicular to the incident surface of the corner cube reflector is h.
If, then, A visual sensor device characterized in that 6. A marker using a corner cube reflector which is arranged on an object and reflects emitted light, a visual input device arranged at a position facing the marker, and the periphery of the visual input device. In a visual sensor device comprising an illuminator consisting of a light source for illuminating a marker arranged in, and a calculator for calculating the position and orientation of the object from the image output by the visual input device, in the reflecting surface of the corner cube reflector. A visual sensor device comprising means for suppressing reflection around the outer periphery. 7. A marker using a corner cube reflector that is arranged on an object and reflects the irradiated light,
A visual input device arranged at a position facing the marker;
In the visual sensor device including an illuminator that is arranged around the visual input device and includes a light source that illuminates a marker, and a calculator that calculates the position and orientation of the object from the image output by the visual input device, the corner The radius of the effective reflection area of the reflective surface of the cube reflector is R, the length in the direction perpendicular to the incident surface of the corner cube reflector is h, and from the normal of the incident surface of the corner cube reflector and the illuminator under use conditions. When the maximum value of the angle formed by the light that is irradiated and enters the corner cube reflector is θ and the refractive index of the corner cube reflector is n, it is effective to suppress reflection near the outer periphery of the reflection surface of the corner cube reflector. By narrowing the reflection area, A visual sensor device characterized in that 8. A marker arranged on an object, a visual input device arranged at a position facing the marker, an illuminator composed of a light source arranged around the visual input device and illuminating the marker, In a visual sensor device equipped with a calculator that calculates the position and orientation of the object from the image output by the visual input device, a marker is generated from the normal line of the surface of the object around the marker and the light source of the illuminator. A visual sensor device characterized in that an angle formed by an illuminating light and an incident direction is increased. 9. A marker arranged on an object, a visual input device arranged at a position facing the marker, an illuminator composed of a light source arranged around the visual input device and illuminating the marker, A visual sensor device comprising a calculator that calculates the position and orientation of the target from an image output by a visual input device, wherein the surface of the target around the marker has a triangular wave shape. ..