JP2020193945A - Measurement device, grasping system, method for controlling measurement device, and program - Google Patents

Abstract

To precisely measure the positional attitude of a target object containing much water.SOLUTION: The measurement device includes: projection means for projecting a random dot pattern of which the contrast ratio between a bright part and a dark part is 100 to 9 or smaller; imaging means for imaging an object to which the random dot pattern is projected; and measurement means for measuring the positional attitude of the object on the basis of the image taken by the imaging means.SELECTED DRAWING: Figure 2

Description

The present invention relates to a measuring device, a gripping system, a control method and a program of the measuring device.

For harvesting vegetables and fruits, there is known a device in which a sensor is mounted on a robot hand and a gripping operation is executed based on the detection result of the sensor to perform harvesting. For example, the position and orientation can be measured by projecting a pattern including a bright part and a dark part and photographing the pattern with a sensor.

Patent Document 1 discloses a work position / orientation measuring device that acquires a three-dimensional position / orientation of a work using a stereo camera.

Japanese Unexamined Patent Publication No. 2012-183616

However, in the technique described in Patent Document 1, in the case of a vegetable containing a large amount of water such as tomato, light is scattered inside, the contrast between the bright part and the dark part becomes ambiguous, and the measurement becomes unsuccessful. There is a problem.

The present invention has been made with the recognition of the above problems as an opportunity, and an object of the present invention is to provide a technique for accurately measuring the position and orientation of an object containing a large amount of water.

The measuring device according to one aspect of the present invention that achieves the above object is
A projection means for projecting a random dot pattern in which the contrast ratio between the bright part and the dark part is 100 for the bright part and 9 or less for the dark part.
An imaging means for photographing an object on which the random dot pattern is projected, and
A measuring means for measuring the position and orientation of the object based on a captured image taken by the photographing means,
It is characterized by having.

According to the present invention, it is possible to accurately measure the position and orientation of an object containing a large amount of water.

The accompanying drawings showing the embodiments of the present invention form a part of the specification and are used for explaining the present invention together with the description thereof.
It is a figure for demonstrating the configuration example of the gripping system which concerns on embodiment. It is a block diagram for demonstrating the configuration example of the measuring apparatus which concerns on embodiment. (A) It is a figure which shows an example of the random dot pattern which concerns on this Embodiment, (b) is the graph which shows the relationship between the pixel number of each pixel on the straight line drawn on the random dot pattern, and the luminance value. It is an image when the random dot pattern according to the embodiment is projected onto an object and photographed, and (b) an image of a color point cloud generated from a stereo image and a color image (an image showing a part where distance measurement has been performed). Is. This is an image taken by projecting a pattern with a low contrast ratio onto an object. (B) An image of a color point cloud generated from a stereo image and a color image (an image of a point cloud indicating a part where distance measurement was possible). ).

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more of the plurality of features described in the embodiments may be arbitrarily combined. In addition, the same or similar configuration will be given the same reference number, and duplicate description will be omitted.

<System configuration>
FIG. 1 is a diagram showing a configuration example of a gripping system according to the present embodiment. 1 is a gripping system. The gripping system 1 is a system for gripping and harvesting an object (vegetables such as tomatoes, fruits, etc.).

Reference numeral 10 denotes a gripping portion for gripping the object. The gripping portion 10 may be of a type that grips the object in a shape like a finger, or may be of a type that grips the object by sucking the object by a suction mechanism. Reference numeral 11 denotes a camera unit attached to the grip portion 10. Reference numeral 12 denotes an arm portion that supports the grip portion 10. Reference numeral 13 is a pillar portion. The arm portion 12 is configured to be movable in the vertical direction (direction of the arrow 14) along the pillar portion 13. 15 is a dolly. The object harvested by the grip portion 10 can be placed.

FIG. 2 is a configuration example of the measuring device according to the present embodiment. Reference numeral 100 is a measuring device. The measuring device includes a camera unit 11, a control unit 101, a storage unit 102, and a communication unit 103. The control unit 101, the storage unit 102, and the communication unit 103 may be configured integrally with the camera unit 11, or may be configured as a separate body from the camera unit 11, such as inside the pillar unit 13.

In the present embodiment, the camera unit 11 includes a projection unit 11a and a photographing unit 11b. The projection unit 11a includes a laser projector 110. The laser projector 110 can project a random dot pattern onto an object.

The photographing unit 11b includes a stereo camera 111 and a color camera 112. The photographing unit 11b photographs an object on which a random dot pattern is projected by the projection unit 11a.

The control unit 101 controls the operations of the projection unit 11a and the photographing unit 11b by reading and executing the program stored in the storage unit 102, and executes various processes. For example, the position / orientation measurement (distance measurement) of the object is performed based on the image captured by the photographing unit 11b.

Specifically, the control unit 101 controls the operations of the projection unit 11a and the imaging unit 11b, and photographs the object on which the random dot pattern is projected by the laser projector 110 with the stereo camera 111. Then, the control unit 101 acquires the captured stereo image. Then, after performing processing such as distortion correction processing and stereo parallelization processing, stereo matching is performed to acquire a parallax image. Then, the parallax image is converted into a point cloud. On the other hand, a color image of the same object is separately captured by the color camera 112 and subjected to distortion correction processing. Then, the point cloud whose distance can be measured is converted into the coordinate system of the color camera 112. Then, a point cloud is generated with the resolution of the color image, and the point cloud is colored. At this time, when a plurality of point clouds exist in one pixel, the closer point may be associated with the pixel. As a result, it is possible to acquire a color point cloud (a point cloud indicating a part where the distance can be measured) corresponding to the point cloud whose distance can be measured.

Further, the control unit 101 can also control the gripping unit 10 based on the measured position and orientation of the object to control the object to be gripped. The storage unit 102 stores a program in which the processing content executed by the control unit 101 is stored, an image captured by the photographing unit 11b, and a processing result executed by the control unit 101.

The communication unit 103 is used to communicate with the outside of the measuring device 100. For example, the processing result executed by the control unit 101, the image captured by the photographing unit 11b, and the like are transmitted to an external server (not shown).

<Random dot pattern>
Subsequently, the random dot pattern used in the present embodiment will be described with reference to FIGS. 3 (a) and 3 (b). FIG. 3A is a diagram showing an example of a random dot pattern according to the present embodiment. FIG. 3B is a graph showing the relationship between the pixel number of each pixel on the straight line drawn on the random dot pattern of FIG. 3A and the luminance value.

The random dot pattern according to the present embodiment is a fixed pattern formed so as to include 5000 or more dots at a projector projection angle of 45 deg × 45 deg (length 45 degrees and width 45 degrees). For example, a random dot pattern can be formed by passing the light emitted from the light source through a filter configured to form a predetermined pattern. The projection angle of the projector may be changed as long as the dot density is the same as the projection angle of the projector. This random dot pattern is a random dot pattern in which the contrast ratio between the bright part and the dark part is 100 for the bright part and 9 or less for the dark part. The greater the contrast between the bright and dark areas, the better the measurement accuracy. As shown in FIG. 3B, the contrast ratio indicates the ratio between the maximum luminance value 301 and the minimum luminance value 302.

Further, if the brightness values of the bright and dark areas are configured to include two or more pixels on the camera side, the texture becomes dense, so that the measurement accuracy can be further improved.

<Image example>
FIG. 4A shows an image taken by a stereo camera 111 by projecting a random dot pattern according to the present embodiment onto an object (fruit / vegetable containing water, for example, tomato). Further, FIG. 4B is an image of a color point cloud generated based on the stereo image corresponding to FIG. 4A (an image of a point cloud indicating a portion where distance measurement was possible).

It was confirmed that the photographed pattern can be clearly discriminated even in the fruit containing water. In addition, it was confirmed that the pattern could be irradiated densely on the pedicel 401, and that the measurement could be performed accurately even on a thin pedicel 401 having a diameter of about Φ4 mm. The pedicel 401 is a short pedicel that connects a single petal flower and a stem connected to an inflorescence in a flowering plant. Harvesting can be easily performed by rotating the object gripped by the gripping portion 10 with reference to the pedicel 401.

On the other hand, FIG. 5A is an image taken by the stereo camera 111 when a random dot pattern in which the contrast ratio between the bright part and the dark part is 100 for the bright part and more than 9 for the dark part is used. Is. Further, FIG. 5B is an image of a color point cloud generated from the stereo image corresponding to FIG. 5A (an image of a point cloud showing a portion where distance measurement was possible). If the contrast between the bright part and the dark part is small, the distance measurement accuracy is lowered and the distance measurement point cloud is also reduced.

<Bandpass filter>
Since sunlight has high intensity in a wide wavelength band, a bandpass filter that transmits only the vicinity of the wavelength (for example, 830 nm) of the laser projector 110 may be attached to the camera lens of the stereo camera 111. Since the intensity of light is inversely proportional to the square of the distance, the farther it is, the more it attenuates and is greatly affected by sunlight. However, the grip portion 10 of the robot used for harvesting tomatoes measures at a short distance of 100 mm to 500 mm. I hope I can. Therefore, the influence of sunlight can be sufficiently reduced by installing the bandpass filter. As a result, the measurement accuracy can be improved even outdoors.

<Other Embodiments>
Also, a program that implements one or more functions described in each embodiment is supplied to a system or device via a network or storage medium, and one or more processors in the computer of the system or device read this program. Can be executed. The present invention can also be realized by such an aspect.

<Summary of Embodiment>
Configuration 1. The measuring device (for example, 100) of the above embodiment is
A projection means (for example, 11a) that projects a random dot pattern in which the contrast ratio between the bright part and the dark part is 100 for the bright part and 9 or less for the dark part on the object.
An imaging means (for example, 11b) for photographing the object on which the random dot pattern is projected, and
A measuring means (for example, 101) that measures the position and orientation of the object based on a captured image captured by the photographing means, and
To be equipped.

This makes it possible to accurately measure the position and orientation of an object containing a large amount of water.

Configuration 2. In the measuring device (for example, 100) of the above embodiment,
The preprojection means projects the random dot pattern onto the object at a density including 5000 or more dots at a projector projection angle of 45 deg × 45 deg (45 degrees in length and 45 degrees in width) in the angle of view range.

In this way, by projecting a high-density random dot pattern in a predetermined angle of view range, the measurement accuracy can be improved.

Configuration 3. In the measuring device (for example, 100) of the above embodiment,
The imaging means includes a stereo camera (eg 111) and a color camera (eg 112).
The measuring means measures the position and orientation based on the stereo image taken by the stereo camera and the color image taken by the color camera.

This makes it possible to accurately measure the position and orientation of an object containing a large amount of water.

Configuration 4. In the measuring device (for example, 100) of the above embodiment,
The object is a tomato.

This makes it possible to accurately measure the position and posture of tomatoes.

Configuration 5. The gripping system (for example, 1) of the above embodiment is
The measuring device (for example, 100) according to any one of configurations 1 to 4 and
A gripping means (for example, 10) that grips the object based on the position and posture, and
To be equipped.

As a result, even an object containing a large amount of water can be appropriately gripped.

Configuration 6. The control method of the measuring device (for example, 100) of the above embodiment is
A projection step of projecting a random dot pattern in which the contrast ratio between the bright part and the dark part is 100 for the bright part and 9 or less for the dark part.
A shooting process for shooting an object on which the random dot pattern is projected, and
A calculation step of calculating the position and orientation of the object based on the captured image captured by the imaging step, and
Have.

This makes it possible to accurately measure the position and orientation of an object containing a large amount of water.

Configuration 7. The program of the above embodiment
Computer,
This is a program for causing the computer to function as the measuring device according to any one of the configurations 1 to 4.

As a result, the processing of the measuring device can be realized by the computer.

The invention is not limited to the above-described embodiment, and various modifications and changes can be made within the scope of the gist of the invention.

1: Grip system, 10: Grip, 11: Camera, 11a: Projection, 11b: Imaging, 101: Control, 111: Stereo camera, 112: Color camera

Claims (7)

A projection means for projecting a random dot pattern in which the contrast ratio between the bright part and the dark part is 100 for the bright part and 9 or less for the dark part, and
An imaging means for photographing the object on which the random dot pattern is projected, and
A measuring means for measuring the position and orientation of the object based on a captured image taken by the photographing means,
A measuring device characterized by being provided with. 1. The preprojection means is characterized in that the random dot pattern is projected onto the object at a projection angle of 45 deg × 45 deg (length 45 degrees and width 45 degrees) and a density including 5000 or more dots. The measuring device described in. The photographing means includes a stereo camera and a color camera.
The measurement according to claim 1 or 2, wherein the measuring means measures the position and orientation based on the stereo image taken by the stereo camera and the color image taken by the color camera. apparatus. The measuring device according to any one of claims 1 to 3, wherein the object is a tomato. The measuring device according to any one of claims 1 to 4,
A gripping means for gripping the object based on the position and posture, and
A gripping system characterized by comprising. It is a control method for measuring devices.
A projection step of projecting a random dot pattern in which the contrast ratio between the bright part and the dark part is 100 for the bright part and 9 or less for the dark part.
A shooting process for shooting an object on which the random dot pattern is projected, and
A calculation step of calculating the position and orientation of the object based on the captured image captured by the imaging step, and
A method for controlling a measuring device, which comprises. A program for causing a computer to function as the measuring device according to any one of claims 1 to 4.