WO2021044473A1

WO2021044473A1 - Multi-joint robot-arm control device and multi-joint robot arm device

Info

Publication number: WO2021044473A1
Application number: PCT/JP2019/034393
Authority: WO
Inventors: 航石井; 佳典原田; 雅史上野山; 和弘中谷
Original assignee: ヤマハ発動機株式会社
Priority date: 2019-09-02
Filing date: 2019-09-02
Publication date: 2021-03-11

Abstract

The present invention addresses the problem of obtaining a multi-joint robot-arm control device and a multi-joint robot arm device that can enhance versatility of the multi-joint robot arm by increasing types of work that the multi-joint robot arm can handle. The present invention comprises: a single monocular camera 24 provided in a multi-joint robot arm 11; an image processing unit 26 that detects an image of a target grape G(n) from images photographed by the monocular camera 24; a coordinate information processing unit 27 that calculates a position coordinate of the target grape G(n); and a driving control unit 28 that drives an actuator of the multi-joint robot arm 11. The monocular camera 24 captures an image of a work area W in a first image-capturing position P1 positioned at a predetermined distance from the work area W. The image processing unit 26 detects an image of the target grape G(n) from the image of the work area W. On the basis of the detected image of the target grape G(n), the coordinate information processing unit 27 calculates a position coordinate of the target grape G(n) in a coordinate system of the multi-joint robot arm 11.

Description

Articulated robot arm control device and articulated robot arm device

The present invention relates to an articulated robot arm control device and an articulated robot arm device.

An articulated robot arm control device that controls an articulated robot arm based on an image taken by a fixed camera and a hand camera provided at the tip of the arm is known. The articulated robot arm control device uses measurement data obtained from an image taken by the hand camera in order to improve the accuracy of the articulated robot arm, and obtains measurement data obtained from an image taken by the fixed camera. I am correcting.

For example, Patent Document 1 describes a robot arm, a first visual sensor that measures a measurement range including a movable range of the robot arm, a second visual sensor that is positioned at the tip of the robot arm, and the robot arm. A robot device having a control device for controlling the position and orientation of the robot device is disclosed. When the control device controls the position and orientation of the robot arm based on the measured value of the first visual sensor, the control device generates a command value given to the robot arm using the measured value of the second visual sensor. ..

JP-A-2018-2020608

In the control device of Patent Document 1, the appropriate arrangement of the fixed camera is determined in consideration of the usage environment of the articulated robot arm, the type of the object to be handled, the degree of freedom of the work area, and the like. Therefore, when the usage environment of the articulated robot arm, the type of the object to be handled, the degree of freedom of the work area, and the like change, the control based on the image taken by the fixed camera described in Patent Document 1 can be performed. There was a possibility that the articulated robot arm could not handle it.

It is required to increase the versatility of the articulated robot arm by increasing the types of work that can be handled by the articulated robot arm as described above. The type of work includes, for example, the usage environment of the articulated robot arm (outside, weather, vehicle mounting / fixing, etc.), the type of object (industrial parts, agricultural products, cooking utensils, etc.), the degree of freedom in the work area, and the like.

The present invention provides an articulated robot arm control device and an articulated robot arm device that can increase the versatility of the articulated robot arm by increasing the types of work that can be handled by the articulated robot arm.

The present inventors can increase the versatility of the articulated robot arm by increasing the types of work that can be handled by the articulated robot arm by using the conventionally proposed fixed camera and hand camera. An arm control device and an articulated robot arm device were examined. As a result of diligent studies, the present inventors have come up with the following configuration.

The articulated robot arm control device according to an embodiment of the present invention includes an imaging unit provided on the articulated robot arm, an image processing unit that detects an image of an object from an image captured by the imaging unit, and the object. It is provided with a coordinate information processing unit for calculating the position coordinates of the robot arm and a drive control unit for driving the actuator of the articulated robot arm. The imaging unit images the work area at a first imaging position away from the work area where the articulated robot arm works on the object, and the image processing unit captures the object from the image of the work area. The image of the object is detected, and the coordinate information processing unit calculates the position coordinates of the object in the coordinate system of the articulated robot arm based on the detected image of the object.

As described above, in the articulated robot arm control device, the imaging unit captures the work area at the first imaging position away from the work area where the articulated robot arm works on the object. Therefore, the object can be searched from the area including the work area. Further, the image processing unit detects an image of the object from the entire image of the captured work area, and the coordinate information processing unit in the coordinate system of the articulated robot arm based on the image of the object. The position coordinates of the object are calculated. Thereby, the articulated robot arm control device can identify the positions of a plurality of the objects in the work area. As described above, the imaging unit corresponding to the conventionally proposed hand camera captures the region including the working region at the first imaging position away from the working region, thereby fixing the conventionally proposed fixed region. It plays the role of a camera. Therefore, the articulated robot arm control device can omit the fixed camera in the conventional proposal using the fixed camera and the hand camera.

Moreover, as described above, the articulated robot arm control device calculates the position coordinates of a plurality of the objects in the work area, so that the optimum work path when the articulated robot arm performs the work. Can be calculated based on predetermined conditions. As a result, the articulated robot arm control device can efficiently cause the articulated robot arm to perform work on the object even if a plurality of the objects are randomly arranged in the work area. ..

Therefore, in the articulated robot arm control device, the degree of freedom in the usage environment of the articulated robot arm device can be increased by increasing the types of work that can be handled by the articulated robot arm. Therefore, the versatility of the articulated robot arm device can be increased.

From another point of view, the articulated robot arm control device of the present invention preferably includes the following configurations. The imaging unit further photographs the working area including the object at a second imaging position different from the first imaging position.

As described above, the articulated robot arm control device further images the work area where the object is located at the second imaging position different from the first imaging position, thereby imaging at the first imaging position. Of the images of the work area, a specific area or the specific object can be imaged at different angles or magnified. As a result, the articulated robot arm control device can acquire an image suitable for the state of the work area and the shape of the object.

Therefore, in the articulated robot arm control device that further images the work area where the object is located at the second imaging position different from the first imaging position, the types of work that can be handled by the articulated robot arm are increased. This makes it possible to increase the degree of freedom in the usage environment of the articulated robot arm device. Therefore, the versatility of the articulated robot arm device can be increased.

From another point of view, the articulated robot arm control device of the present invention preferably includes the following configurations. The second imaging position is a position closer to the working area than the first imaging position.

As described above, the articulated robot arm control device can acquire an entire image of the work area including the object from the image of the work area captured at the first imaging position. Further, the articulated robot arm control device can acquire a detailed image of a specific object among a plurality of objects imaged at the first imaging position by imaging at the second imaging position. As a result, the articulated robot arm control device searches the work area over a wide area using the image of the work area captured at the first imaging position by one of the imaging units, and images the image at the second imaging position. The work area can be locally searched by using the image of the work area.

Therefore, in the articulated robot arm control device that images the work area where the object is located at the second imaging position that is closer to the work area than the first imaging position, the work that can be handled by the articulated robot arm. By increasing the types of robot arm devices, the degree of freedom in the usage environment of the articulated robot arm device can be increased. Therefore, the versatility of the articulated robot arm device can be increased.

From another point of view, the articulated robot arm control device of the present invention preferably includes the following configurations. The imaging unit images the working area at an angle of view of 90 degrees or more in the imaging range.

As described above, since the imaging unit has an angle of view of 90 degrees or more in the range actually projected, the entire working area can be photographed in the vicinity of the working area. As a result, the articulated robot arm control device can secure an imaging position in which the working area is entirely included in the imaging range of the imaging unit within the movable range of the articulated arm robot.

Therefore, in the articulated robot arm control device having an imaging unit that images the work area with a diagonal angle of 90 degrees or more, the articulated robot arm device can be handled by increasing the types of work that can be handled by the articulated robot arm. It is possible to increase the degree of freedom in the usage environment of. Therefore, the versatility of the articulated robot arm device can be increased.

From another point of view, the articulated robot arm control device of the present invention preferably includes the following configurations. The imaging unit includes a monocular camera that can move in an arbitrary direction by driving the articulated robot arm and can acquire a parallax image of the work area.

As described above, the articulated robot arm control device can measure the distance to the work area or the object by moving the monocular camera with the articulated robot arm and taking an image. As a result, the articulated robot arm control device can image the work area or the object with the monocular camera and measure the distance to the work area or the object.

Therefore, in the articulated robot arm control device that images the work area with the monocular camera, the degree of freedom in the usage environment of the articulated robot arm device is increased by increasing the types of work that can be handled by the articulated robot arm. Can be done. Therefore, the versatility of the articulated robot arm device can be increased.

From another point of view, the articulated robot arm control device of the present invention preferably includes the following configurations. The amount of movement of the monocular camera in an arbitrary direction for capturing the parallax image differs between the first imaging position and the second imaging position.

As described above, the articulated robot arm control device has the monocular for capturing a parallax image at the first imaging position and the second imaging position closer to the working area than the first imaging position. By changing the amount of movement of the camera in an arbitrary direction, it is possible to adjust the measurement accuracy when measuring the distance to the work area or the object. As a result, the articulated robot arm control device measures the distance to the work area or the object with measurement accuracy based on the positional relationship between the first imaging position and the second imaging position and the work area. be able to.

Therefore, in the articulated robot arm control device in which the amount of movement in an arbitrary direction for capturing the disparity image of the monocular camera differs between the first imaging position and the second imaging position, the articulated robot arm corresponds to the movement amount. By increasing the types of work that can be done, the degree of freedom in the usage environment of the articulated robot arm device can be increased. Therefore, the versatility of the articulated robot arm device can be increased.

From another point of view, the articulated robot arm control device of the present invention preferably includes the following configurations. The imaging unit captures images from different imaging directions at the first imaging position and the second imaging position.

As described above, the articulated robot arm control device captures images from a different imaging direction from the first imaging position at the second imaging position, so that it is difficult to image an object due to backlight, obstacles, or the like. It can be avoided. As a result, the articulated robot arm control device can change the imaging position according to the surrounding conditions.

Therefore, in the articulated robot arm control device that captures images from different imaging directions at the first imaging position and the second imaging position, the articulated robot arm device can be operated by increasing the types of work that can be handled by the articulated robot arm. The degree of freedom in the usage environment can be increased. Therefore, the versatility of the articulated robot arm device can be increased.

From another point of view, the articulated robot arm control device of the present invention preferably includes the following configurations. The drive control unit corrects the control amount of the actuator of the articulated robot arm based on the calculated fluctuation of the position coordinates of the object.

As described above, the position of the articulated robot arm is changed by the drive control unit correcting the control amount of the actuator of the articulated robot arm based on the calculated fluctuation of the position coordinates of the object. Even so, the position of the imaging unit can be maintained by the articulated robot arm. As a result, the articulated robot arm control device can continue the work even if the surrounding state of the articulated robot arm changes.

Therefore, in the articulated robot arm control device that corrects the control amount of the actuator of the articulated robot arm based on the fluctuation of the position coordinates of the object, the types of work that can be handled by the articulated robot arm can be increased. It is possible to increase the degree of freedom in the usage environment of the articulated robot arm device. Therefore, the versatility of the articulated robot arm device can be increased.

From another point of view, the articulated robot arm device of the present invention preferably includes the following configurations. The articulated robot arm device includes the articulated robot arm, and the articulated robot arm is controlled by the articulated robot arm control device according to any one of the above.

As described above, the articulated robot arm is controlled by the articulated robot arm control device, so that the imaging unit provided on the articulated robot arm can be arbitrarily moved within the movable range of the articulated robot arm. It can be arranged at the position of, and the information of the work area or the object can be acquired. As a result, the articulated robot arm device can continue the work even if the state of the work area, the object, and the surroundings of the articulated robot arm changes.

Therefore, in the articulated robot arm device controlled by the articulated robot arm control device, the degree of freedom in the usage environment of the articulated robot arm device can be increased by increasing the types of work that can be handled by the articulated robot arm. .. Therefore, the versatility of the articulated robot arm device can be increased.

From another point of view, the articulated robot arm device of the present invention preferably includes the following configurations. The articulated robot arm has a tip rotating portion that can rotate around the axis of the most advanced link, and the tip rotating portion is provided with the imaging unit.

As described above, since the imaging unit is provided at the tip rotating unit that can rotate around the axis of the link, the imaging unit can be moved to a position where an object can be imaged. As a result, the articulated robot arm device can continue imaging even if the surrounding state of the articulated robot arm changes.

Therefore, in the articulated robot arm device in which the imaging unit is provided at the tip rotating portion that can rotate around the axis of the link, the articulated robot arm device can be handled by increasing the types of work that can be handled by the articulated robot arm. It is possible to increase the degree of freedom in the usage environment of. Therefore, the versatility of the articulated robot arm device can be increased.

The terminology used herein is used for the purpose of defining only specific examples, and there is no intention of limiting the invention by the terminology.

As used herein, "and / or" includes all combinations of one or more relatedly listed components.

As used herein, the use of "including, including," "comprising," or "having" and variations thereof is described in the features, processes, elements, components, and / or , Identifying the existence of their equivalents, but may include one or more of steps, actions, elements, components, and / or their groups.

In the present specification, "attached", "connected", "combined", and / or their equivalents are used in a broad sense and are "direct and indirect" attachments. Includes both connection and connection. Further, "connected" and "connected" are not limited to physical or mechanical connections or connections, but can include direct or indirect connections or connections.

Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

Terms defined in commonly used dictionaries should be construed to have meaning consistent with the relevant technology and meaning in the context of the present disclosure, unless expressly defined herein. , Is not interpreted in an ideal or overly formal sense.

It is understood that a number of techniques and processes are disclosed in the description of the present invention. Each of these has its own interests and can be used with one or more of the other disclosed techniques, or in some cases all.

Therefore, for the sake of clarity, the description of the present invention refrains from unnecessarily repeating all possible combinations of individual steps. However, the specification and claims should be read with the understanding that all such combinations are within the scope of the present invention.

This specification describes embodiments of the articulated robot arm control device and the articulated arm robot device according to the present invention.

In the following description, a number of specific examples will be given to provide a complete understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention can be practiced without these specific examples.

Therefore, the following disclosure should be considered as an example of the invention and is not intended to limit the invention to the particular embodiments set forth in the drawings or description below.

[Articulated robot arm]
In the present specification, the articulated robot arm means a robot arm having a plurality of joint portions connecting a plurality of links. The articulated robot arm includes a vertical articulated robot arm. Specifically, the vertical articulated robot arm is a robot arm of a serial link mechanism in which links are connected in series from the root to the tip by a rotary joint or a linear motion joint having one degree of freedom. The vertical articulated robot arm has a plurality of joints.

[Work area]
In the present specification, the work area is a process in which when an articulated robot arm works on an object, the articulated robot arm approaches the object to be worked on or the object to be worked on. Means the area through which the articulated robot arm may pass. The area through which the articulated robot arm passes when it leaves the position of one object is excluded.

[Angle of view]
In the present specification, the angle of view means an angle indicating a range actually captured by the camera. Specifically, the angle of view is a diagonal angle of view indicating an angle formed by the optical center of the lens forming two points diagonal to the light receiving surface of the image sensor. That is, the diagonal angle of view is determined by the effective focal length of the lens and the length of the diagonal line of the light receiving surface. When the size of the light receiving surface is the same, the diagonal angle of view becomes smaller as the effective focal length of the lens becomes longer, and becomes smaller as the diagonal line of the light receiving surface becomes shorter.

According to one embodiment of the present invention, there is provided an articulated robot arm control device and an articulated robot arm device capable of increasing the versatility of the articulated robot arm by increasing the types of work that can be handled by the articulated robot arm. it can.

It is a schematic diagram of the remote control vehicle provided with the articulated robot arm device which concerns on Embodiment 1 of this invention. It is a block diagram of the control composition of the remote control vehicle 1. It is a schematic diagram of the articulated robot arm device which concerns on Embodiment 1 of this invention. The plan view of the end effector which concerns on Embodiment 1 of this invention is shown. The side view of the end effector which concerns on Embodiment 1 of this invention is shown. It is a figure which shows typically the viewing angle in the monocular camera provided in the end effector. It is a block diagram of an articulated robot arm control device. It is a figure which shows the relationship between the movement amount of a monocular camera and the distance from the image formation position of a monocular camera to the crop to be harvested. It is a figure which shows an example of the processed image of the target grape and the harvesting order calculated by the articulated robot arm control device. It is a schematic diagram which shows an example of the movement of an end effector for generating a parallax image by a monocular camera. It is a control flow diagram of the articulated robot arm control device which concerns on Embodiment 1 of this invention. It is a block diagram of the articulated robot arm control device which concerns on other embodiment of this invention. It is a figure which shows the state which the imaging direction is different between the 1st imaging position and the 2nd imaging position in the articulated robot arm control device which concerns on other embodiment of this invention. It is a schematic diagram of the remote control vehicle provided with the articulated robot arm device which concerns on another embodiment of this invention.

Hereinafter, each embodiment will be described with reference to the drawings. In each figure, the same parts are designated by the same reference numerals, and the description of the same parts will not be repeated. The dimensions of the constituent members in each drawing do not faithfully represent the actual dimensions of the constituent members and the dimensional ratio of each constituent member.

<Overall configuration>
The remote-controlled vehicle 1 according to the first embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG. 1 is a schematic view of a remote-controlled vehicle 1 provided with an articulated robot arm device 10 according to a first embodiment of the present invention. FIG. 2 is a block diagram of the control configuration of the remote-controlled vehicle 1. In the present embodiment, the remote-controlled vehicle 1 is in front of the remote-controlled vehicle 1 in the direction from the end where the articulated robot arm is not arranged to the end where the articulated robot arm is arranged at both ends in the traveling direction. Defined as direction.

Arrows X, Y, and Z in the figure shown below indicate the direction of the coordinate axis which is the Cartesian coordinate system (hereinafter, simply referred to as "robot coordinate system") in the articulated robot arm device 10. In the robot coordinate system, the X-axis direction coincides with the front direction of the remote-controlled vehicle 1. Therefore, in the robot coordinate system, if the front direction of the remote control vehicle 1 is the X-axis direction and the vertically upward direction is the Z direction, the left direction when facing the front direction of the remote control vehicle 1 is the Y-axis direction. The origin of the robot coordinate system is the intersection of the axis of the S-axis motor unit 12 and the axis of the L-axis motor unit 14 in the articulated robot arm 11, which will be described later.

As shown in FIG. 1, the remote-controlled vehicle 1 is a four-wheeled vehicle that is remotely controlled by an external control signal. The remote-controlled vehicle 1 communicates with the vehicle body 2, the pair of wheels 3, the pair of wheels 4, the drive motor 5 for driving the

wheels

3 and 4, and the steering motor 6 for steering the

wheels

3 and 4. It includes a device 7, a battery 8, and a vehicle control device 9.

(Remote control vehicle)
In the remote-controlled vehicle 1, the pair of wheels 3 are located at the front of the vehicle body 2 and are located at the rear of the pair of wheels. Therefore, for example, one wheel 3 is a front wheel and a pair of wheels 4 are rear wheels. Each of the pair of

wheels

3 and 4 is steerable by a steering device (not shown) on the vehicle body 2. A harvest box H for storing the harvested material is mounted on the upper surface of the vehicle body 2.

The drive motor 5 is an actuator that applies a driving force to each of the pair of wheels 3. The drive motor 5 is provided on a pair of wheels 3. The drive motor 5 applies a driving force to the pair of wheels 3 via, for example, a speed reducer (not shown).

The steering motor 6 is an actuator that steers a pair of

wheels

3 and 4, respectively. The steering motor 6 is provided in a steering device (not shown). The steering motor 6 steers a pair of

wheels

3 and 4 by driving a steering device.

The communication device 7 transmits and receives control signals to and from the external operation terminal C. The communication device 7 is provided on the vehicle body 2. The communication device 7 receives a control signal transmitted from the external operation terminal C. Further, the communication device 7 transmits a control signal output from the vehicle control device 9 to the operation terminal C. The operation terminal C transmits a signal for remotely controlling the vehicle 1 to the communication device 7 as a control signal.

Battery 8 is a battery that can be charged and discharged. The battery 8 is provided on the vehicle body 2. The battery 8 is, for example, a lead storage battery, an alkaline storage battery, a lithium ion battery, or the like. The battery 8 supplies electricity to the drive motor 5, the steering motor 6, the communication device 7, the vehicle control device 9, and the articulated robot arm device 10 described later.

The vehicle control device 9 is a device that controls the remote-controlled vehicle 1. The vehicle control device 9 may substantially have a configuration in which a CPU, ROM, RAM, HDD, etc. are connected by a bus, or may have a configuration including a one-chip LSI or the like. The vehicle control device 9 stores various programs and data for controlling the operation of the drive motor 5, the steering motor 6, the communication device 7, and the like.

As shown in FIG. 2, the vehicle control device 9 is communicably connected to the drive circuit of the drive motor 5. As a result, the vehicle control device 9 can transmit a control signal to the drive circuit of the drive motor 5.

The vehicle control device 9 is communicably connected to the drive circuit of the steering motor 6. As a result, the vehicle control device 9 can transmit a control signal to the drive circuit of the steering motor 6.

The vehicle control device 9 is communicably connected to the communication device 7. As a result, the vehicle control device 9 can acquire the control signal transmitted from the external operation terminal C and received by the communication device 7. Further, the vehicle control device 9 can transmit a control signal to the external operation terminal C via the communication device 7.

The vehicle control device 9 is electrically connected to the battery 8. As a result, electric power is supplied to the vehicle control device 9 from the battery 8. The vehicle control device 9 is communicably connected to the battery 8. As a result, the vehicle control device 9 can acquire information about the state of the battery 8.

The remote-controlled vehicle 1 configured as described above is remotely controlled by a control signal transmitted from the external operation terminal C.

(Articulated robot arm device 10)
Next, the overall configuration of the articulated robot arm device 10 according to the first embodiment of the present invention will be described with reference to FIGS. 3 to 5. FIG. 3 is a schematic view of the articulated robot arm device 10 according to the first embodiment of the present invention. FIG. 4A is a plan view of the end effector according to the first embodiment of the present invention. FIG. 4B is a side view of the end effector according to the first embodiment of the present invention. FIG. 4C is a diagram schematically showing a viewing angle in a monocular camera provided on an end effector. FIG. 5 is a block diagram of the articulated robot arm control device 25. The articulated robot arm device 10 includes an articulated robot arm 11, an end effector 23, and an articulated robot arm control device 25.

(Articulated robot arm 11)
As shown in FIG. 3, the articulated robot arm 11 is a robot arm of a serial link mechanism in which the links are connected in series from the base end to the tip end by a rotary joint having one degree of freedom in the present embodiment. The articulated robot arm 11 is, for example, a 6-axis vertical articulated robot arm. The articulated robot arm 11 is provided on the front portion of the upper surface of the vehicle body 2 in the remote-controlled vehicle 1.

In the articulated robot arm 11, the S-axis motor unit 12, the L-axis motor unit 14, the U-axis motor unit 16, the B-axis motor unit 18, and the R-axis motor unit 20 are arranged in this order from the base end portion fixed to the remote-operated vehicle 1. And the T-axis motor unit 22 are connected in series by a link, respectively. The motor unit of each axis constitutes a rotary joint. The motor unit of each axis includes a motor, a speed reducer, an encoder and a drive circuit (not shown). The articulated robot arm 11 is controlled by the articulated robot arm control device 25. The articulated robot arm 11 acquires a control signal from the articulated robot arm control device 25 to the drive circuit of each axis. Further, the articulated robot arm 11 transmits information regarding the output of the motor of the motor unit of each axis and information from the encoder to the articulated robot arm control device 25.

The S-axis motor unit 12 is provided in the remote-controlled vehicle 1. The S-axis motor unit 12 is a rotary joint that rotates the entire articulated robot arm 11. The S-axis motor unit 12 is arranged so that the axis of the S-axis motor unit 12 extends in a direction perpendicular to the installation surface of the articulated robot arm 11. A base member 13 is fixed to the output shaft of the S-axis motor unit 12. The base member 13 is provided with an L-axis motor unit 14.

The L-axis motor unit 14 is a rotary joint that swings the lower bowl link 15. The L-axis motor unit 14 is arranged so that the axis of the L-axis motor unit 14 extends in a direction perpendicular to the axis of the S-axis motor unit 12. One side end of the lower bowl link 15 is fixed to the output shaft of the L-axis motor unit 14. A U-axis motor unit 16 is provided at the other end of the lower bowl link 15.

The U-axis motor unit 16 is a rotary joint that swings the upper arm link 17. The U-axis motor unit 16 is arranged so that the axis of the U-axis motor unit 16 extends in a direction parallel to the axis of the L-axis motor unit 14. One side end of the upper arm link 17 is fixed to the output shaft of the U-axis motor unit 16. A B-axis motor unit 18 is provided at the other end of the upper arm link 17.

The B-axis motor unit 18 is a rotary joint that swings the wrist vertical link 19. The B-axis motor unit 18 is arranged so that the axis of the B-axis motor unit 18 extends in a direction parallel to the axis of the U-axis motor unit 16. A wrist vertical link 19 is fixed to the output shaft of the B-axis motor unit 18. An R-axis motor unit 20 is provided on the wrist upper / lower link 19.

The R-axis motor unit 20 is a rotary joint that rotates the wrist rotation link 21. The R-axis motor unit 20 is arranged so that the axis of the R-axis motor unit 20 extends in a direction perpendicular to the axis of the B-axis motor unit 18. A wrist rotation link 21 is fixed to the output shaft of the R-axis motor unit 20. The wrist rotation link 21 is provided with a T-axis motor unit 22.

The T-axis motor unit 22 is a rotary joint that rotates the end effector 23. The T-axis motor unit 22 is arranged so that the axis of the T-axis motor unit 22 extends in a direction perpendicular to the axis of the R-axis motor unit 20. An end effector 23 is fixed to the output shaft of the T-axis motor unit 22.

The articulated robot arm 11 configured in this way has three degrees of freedom of translation in the X-axis, Y-axis, and Z-axis directions and three degrees of freedom of rotation around the X-axis, Y-axis, and Z-axis, depending on the motor unit of each axis. Has 6 degrees of freedom. Therefore, the articulated robot arm 11 can move the end effector 23 fixed to the output shaft of the T axis to an arbitrary position and take an arbitrary posture in the movable space of the articulated robot arm 11. Can be done.

(End effector)
As shown in FIGS. 3, 4A and 4B, the end effector 23 is a device that works on an object. The end effector 23 according to the first embodiment of the present invention is a harvesting device for harvesting an agricultural product as an object. The end effector 23 is fixed to the output shaft of the T-axis of the articulated robot arm 11. The end effector 23 includes a gripping device 23a for gripping the crop and a cutting device 23b for separating the crop from the branches and stems. The gripping device 23a and the cutting device 23b are driven by, for example, a motor. The end effector 23 harvests the object by cutting the stem on the trunk side of the gripping position with the cutting device 23b while the stem of the object to be harvested is gripped by the gripping device 23a.

(Articulated robot arm control device 25)
Next, the configuration of the articulated robot arm control device 25 will be described with reference to FIGS. 3 to 5. The articulated robot arm control device 25 is a device that controls the articulated robot arm 11, the end effector 23, and the monocular camera 24. The articulated robot arm control device 25 includes a monocular camera 24 which is an imaging unit.

(Monocular camera 24)
As shown in FIGS. 3, 4A, 4B and 4C, the monocular camera 24 is a camera that captures an object from a single viewpoint at a time. The monocular camera 24 is a digital camera using a CCD sensor or a COMPOS sensor. The monocular camera 24 has a diagonal angle of view of 90 degrees or more, which is formed by two points diagonal to the optical center of the lens and the light receiving surface of the CCD sensor or the COMPOS sensor.

The monocular camera 24 is provided on the end effector 23. The monocular camera 24 is arranged in the end effector 23 so that the gripping device 23a or the cutting device 23b of the end effector 23 is included in the angle of view of the monocular camera 24. The monocular camera 24 is rotatably provided around the axis of the T-axis by the T-axis motor unit 22 which is a tip rotating portion of the articulated robot arm 11. As a result, the monocular camera 24 is arranged on the circumference centered on the end effector 23.

As shown in FIG. 5, the articulated robot arm control device 25 may actually have a configuration in which a CPU, ROM, RAM, HDD, etc. are connected by a bus, or may consist of a one-chip LSI or the like. It may be a configuration. The articulated robot arm control device 25 stores various programs and data for controlling the operations of the articulated robot arm 11, the monocular camera 24, and the end effector 23.

The articulated robot arm control device 25 is connected to the battery 8 and can be supplied with electric power from the battery 8 and can acquire information about the state of the battery.

The articulated robot arm control device 25 includes an S-axis motor unit 12, an L-axis motor unit 14, a U-axis motor unit 16, a B-axis motor unit 18, an R-axis motor unit 20, and a T-axis motor unit of the articulated robot arm 11. It is connected to each of the drive circuits of the motor included in 22, and can transmit a control signal to the drive circuit of the motor of each axis. Further, the articulated robot arm control device 25 can acquire the rotation position information (encoder signal) of the motor from the motor unit of each axis.

The articulated robot arm control device 25 is connected to the monocular camera 24 and can acquire an image captured by the monocular camera 24.

The articulated robot arm control device 25 is communicably connected to the drive circuit of the end effector motor that drives the gripping device 23a and the cutting device 23b of the end effector 23, and drives the motor that drives the gripping device 23a and the cutting device 23b. A control signal can be sent to the circuit.

The articulated robot arm control device 25 is communicably connected to the vehicle control device 9 of the remote-controlled vehicle 1, acquires a control signal from the vehicle control device 9 of the remote-controlled vehicle 1, or is a vehicle control device of the remote-controlled vehicle 1. A control signal can be transmitted to 9.

The articulated robot arm control device 25 is communicably connected to the communication device 7 of the remote control vehicle 1 and can acquire a control signal from the external operation terminal C received by the communication device 7. Further, the articulated robot arm control device 25 continuously transmits the control signal generated by the articulated robot arm control device 25 or the image captured by the monocular camera 24 to the external operation terminal C via the communication device 7. be able to.

The articulated robot arm device 10 configured as described above can remotely control the articulated robot arm 11, the monocular camera 24, and the end effector 23 by a control signal from an external operation terminal C. That is, the articulated robot arm device 10 uses the articulated robot arm 11 as an actuator for moving the monocular camera 24 and the end effector 23 to an arbitrary position in an arbitrary posture. As described above, in the articulated robot arm device 10, since the monocular camera 24 captures images from the viewpoint from the articulated robot arm 11, remote control based on the image captured by the monocular camera 24 is easy. Further, the articulated robot arm device 10 can automatically perform a predetermined work based on a control signal from the operation terminal C.

(Control of articulated robot arm control device 25)
Next, the control of the articulated robot arm control device 25 will be described with reference to FIGS. 3, 5 to 8. FIG. 6 shows a relationship diagram between the movement amount T of the monocular camera and the distance L from the imaging position of the monocular camera to the crop to be harvested (target grape G (n)). FIG. 7 shows a processed image in which the target grape G (n) and the harvest order A are calculated. FIG. 8 shows a schematic diagram showing an example of the movement of the end effector for generating a parallax image by the monocular camera.

As shown in FIG. 3, the articulated robot arm control device 25 is articulated so as to image the work area W at the first imaging position P1 (see the black arrow) away from the work area W for harvesting crops. It controls the robot arm 11 and the monocular camera 24. The first imaging position P1 is a position within the movable range of the articulated robot arm 11 and including the entire working area W within the angle of view θ of the monocular camera 24. In other words, it is a position where the area for searching for harvestable crops can be imaged at once.

The articulated robot arm control device 25 controls the articulated robot arm 11 and the monocular camera 24 so as to image crops that can be harvested at a second imaging position P2 (see the black-painted arrow) different from the first imaging position P1. The second imaging position P2 is within the movable range of the articulated robot arm 11 and is closer to the working area W than the first imaging position P1. That is, it is a position where the crop to be harvested can be imaged in detail. The articulated robot arm control device 25 calculates the detailed position coordinates of the crop to be harvested at the second imaging position P2.

As a result, the articulated robot arm device 10 moves the monocular camera 24 to an arbitrary position and direction by using the articulated robot arm 11 as a moving actuator. It can be omitted. As a result, the articulated robot arm control device 25 can increase the degree of freedom in the usage environment of the articulated robot arm device 10. Therefore, the articulated robot arm control device 25 and the articulated robot arm device 10 can enhance the versatility of the articulated robot arm 11.

As shown in FIG. 5, the articulated robot arm control device 25 includes an image processing unit 26, a coordinate information processing unit 27, and a drive control unit 28.

The image processing unit 26 detects, for example, grape G (n) (hereinafter, simply referred to as “target grape G (n)”) as an agricultural product to be harvested from the image of the work area W captured by the monocular camera 24. It is a control device. The image processing unit 26 stores in advance the crop detection data D (see FIG. 7) acquired by learning about various images of the target grape G (n). Hereinafter, (n) of the target grape G (n) is a subscript (n is an integer) for distinguishing the grapes.

The image processing unit 26 causes the monocular camera 24 to image the work area W, and detects all the target grapes G (n) existing in the image captured based on the detection data D. Further, the image processing unit 26 continuously transmits the captured image to the external operation terminal C via the communication device 7 of the remote control vehicle 1. As a result, the articulated robot arm control device 25 can detect all the target grapes G (n) from the agricultural products existing in the work area W.

The coordinate information processing unit 27 is a control device that calculates the position coordinates G (n) (x, y, z) of the target grape G (n) detected by the image processing unit 26 in the robot coordinate system. The coordinate information processing unit 27 acquires an image of the work area W of the target grape G (n) from the image processing unit 26. Further, the coordinate information processing unit 27 acquires the posture information of the articulated robot arm 11 from the drive control unit 28.

The coordinate information processing unit 27 obtains the position coordinates of the target grape G (n) in the image of the work area W from the attachment position of the monocular camera 24 on the articulated robot arm 11 and the posture information of the articulated robot arm 11 to the robot coordinates. It is converted into the position coordinates G (n) (x, y, z) in the system. As a result, the articulated robot arm control device 25 can control the articulated robot arm 11 with reference to the image of the work area W. In the following description, the position coordinates G (n) (x, y, z) of the target grape G (n) are the position coordinates based on the first image Im1 from the first imaging position. It is described as (x1, y1, z1), and the position coordinates based on the second image Im2 from the second imaging position are described as the position coordinates G (n) (x2, y2, z2).

As shown in FIG. 6, the coordinate information processing unit 27 moves the first image Im1 captured at the first imaging position P1 away from the work area W by an arbitrary movement amount T in an arbitrary direction and images the image. A parallax image is generated from the auxiliary image Is1 of the first image Im1 and the distance L to the target grape G (n) is calculated. Specifically, the coordinate information processing unit 27 assists the first image Im1 and the first image Im1 which are two images captured by the monocular camera 24 at the first imaging position P1 by shifting the imaging position by the movement amount T. From the image Is1, the amount of movement T that the monocular camera 24 has moved to capture the two images, the focal distance f of the monocular camera 24, and the position coordinates Gm1 and Gs1 of the target grape G (n) in the two images. The distance L from the monocular camera 24 to the target grape G (n) is calculated. As a result, the articulated robot arm control device 25 can calculate the distance L to the target grape G (n) with the monocular camera 24. The lines Om and Os in FIG. 6 are the main optical axes of the monocular camera 24.

As shown in FIG. 7, when a plurality of target grapes G1, G2, and G3 exist in the first image Im1 of the work area W, the coordinate information processing unit 27 calculates the harvest order A for performing the harvesting work. .. The coordinate information processing unit 27 is based on the calculated position coordinates G1 (x1, y1, z1), G2 (x1, y1, z1), and G3 (x1, y1, z1) of the plurality of target grapes G1, G2, and G3. For example, the harvesting order A in which the harvesting work is performed in order from the position where the Z coordinate is large and the target grape G1 closest to the articulated robot arm 11 is calculated.

Further, the coordinate information processing unit 27 uses the communication device 7 of the remote control vehicle 1 to calculate the position coordinates G1 (x1, y1, z1) and G2 (x1, y1, z1) of the target grapes G1, G2, and G3. , G3 (x1, y1, z1) and the information of the harvest order A are transmitted to the external operation terminal C. As a result, the articulated robot arm control device 25 can efficiently perform the harvesting work in the work area W.

The drive control unit 28 is a control device that controls the motor of the end effector 23 and the motor unit of each axis of the articulated robot arm 11. The drive control unit 28 acquires the control signal of the articulated robot arm 11 from the external operation terminal C via the communication device 7 of the remote control vehicle 1, and provides the external operation terminal C with the posture information of the articulated robot arm 11. To send. Further, the drive control unit 28 acquires information on the position coordinates G (n) (x1, y1, z1) of the target grape G (n) and the harvest order A from the coordinate information processing unit 27. Further, the drive control unit 28 acquires the rotation position information of the motor from the motor unit of each axis.

The drive control unit 28 uses the control signal acquired from the external operation terminal C, the position coordinates G (n) (x1, y1, z1) of the acquired target grape G (n), information on the harvest order A, and each axis. A control signal is transmitted to the drive circuit of the motor of each axis based on the rotation position information of the motor unit of. As a result, the articulated robot arm control device 25 can arrange the end effector 23 at an arbitrary position and in an arbitrary posture by the articulated robot arm 11.

When the end effector 23 is moved to a predetermined position with respect to the target grape G (n), the drive control unit 28 transmits a control signal to the drive circuit of the motor of the gripping device 23a and the cutting device 23b in the end effector 23. .. As a result, the articulated robot arm control device 25 can perform the harvesting operation of the target grape G (n) by the end effector 23.

<Control>
Next, the harvesting operation of the target grape G (n) by the articulated robot arm device 10 will be described. In the present embodiment, the articulated robot arm device 10 harvests the target vine G (n), which is an agricultural product, from, for example, vines planted in a row. The remote-controlled vehicle 1 moves along a row of vines. It is assumed that the articulated robot arm device 10 is moved by the remote-controlled vehicle 1 to a position within the movable range of the articulated robot arm 11 to include a vine for harvesting work.

As shown in FIG. 3, the articulated robot arm control device 25 controls to start imaging from the external operation terminal C via the vehicle control device 9 of the remote control vehicle 1 or the communication device 7 of the remote control vehicle 1. When the signal is acquired, the imaging of the work area W is started.

The articulated robot arm control device 25 controls the motor units of each axis of the articulated robot arm 11 by the drive control unit 28, and is the first imaging position P1 separated from the work area W by a predetermined distance, and is the work area. The monocular camera 24 is moved to approximately the center of the robot coordinate system in W in the Z-axis direction and substantially in the center of the robot coordinate system in the work area W in the X-axis direction. Further, the drive control unit 28 controls the motor units of each axis of the articulated robot arm 11 to move the monocular camera 24 so that the imaging direction of the monocular camera 24 faces the work area W.

At this time, the monocular camera 24 is arranged so that the entire work area W is included in the angle of view of the monocular camera 24. Since the monocular camera 24 has a diagonal angle of view of 90 degrees or more, the entire work area W can be contained within the angle of view of the monocular camera 24 in the vicinity of the work area W.

The articulated robot arm control device 25 controls the monocular camera 24 by the image processing unit 26 to image the work area W from the first imaging position P1. As shown in FIG. 7, the image processing unit 26 detects all the target grapes G (n) to be harvested existing in the first image Im1 imaged based on the detection data D. In the present embodiment, the detected target grapes G (n) are grapes G1, grapes G2, and grapes G3.

As shown in FIG. 6, the articulated robot arm control device 25 controls the motor units of each axis of the articulated robot arm 11 by the drive control unit 28, and moves arbitrarily from the first imaging position P1 in an arbitrary direction. The monocular camera 24 is moved by the amount T.

The articulated robot arm control device 25 controls the monocular camera 24 by the image processing unit 26 to image the work area W from a position moved by an arbitrary movement amount T in an arbitrary direction, and assists the first image Im1. Image Is1 is acquired. The articulated robot arm control device 25 transmits the captured first image Im1 and the auxiliary image Is1 of the first image Im1 to the external operation terminal C via the communication device 7 of the remote control vehicle 1.

The articulated robot arm control device 25 uses the coordinate information processing unit 27 to obtain the attitude information of the articulated robot arm 11 at the time of capturing the first image Im1 and the mounting position of the monocular camera 24. The position coordinates G1 (x1, z1), G2 (x1, z1), and G3 (x1, z1) on the XZ plane in the robot coordinate system of the grape G2 and the grape G3 are calculated. Similarly, the coordinate information processing unit 27 calculates the position coordinates on the XX plane of the grape G1, the grape G2, and the grape G3 in the auxiliary image Is1 of the first image Im1 in the robot coordinate system.

The coordinate information processing unit 27 generates a disparity image from the first image Im1 and the auxiliary image Is1 of the first image Im1, and calculates the Y coordinates of the grape G1, the grape G2, and the grape G3 in the robot coordinate system. As a result, the coordinate information processing unit 27 has the position coordinates G1 (x1, y1, z1), G2 (x1, y1, z1), G3 (x1, y1, z1) of the grape G1, the grape G2, and the grape G3 in the robot coordinate system. ) Is calculated.

The articulated robot arm control device 25 normally calculates the position coordinates of at least one of the grapes G1, the grapes G2, and the grapes G3, for example, the position coordinates G2 (x1, y1, z1) of the grapes G2 by the coordinate information processing unit 27. If this is not possible, the drive control unit 28 transmits a control signal for moving the monocular camera 24 to the motor units of each axis of the articulated robot arm 11. The articulated robot arm control device 25 again uses the position coordinates G1 (x1, y1, z1), G2 (x1, y1, z1), G3 (x1, y1, z1) of the grapes G1, the grapes G2, and the grapes G3 in the robot coordinate system. ) Is calculated.

As shown in FIG. 7, the articulated robot arm control device 25 has the position coordinates G1 (x1, y1, z1), G2 (x1, y1, of the grape G1, the grape G2, and the grape G3) of the grape G1, the grape G2, and the grape G3 calculated by the coordinate information processing unit 27. Based on z1) and G3 (x1, y1, z1), the harvesting order A for performing the harvesting work is calculated. The coordinate information processing unit 27 calculates, for example, the harvesting order A in which the harvesting work is performed in the order of increasing Z coordinate and decreasing Y coordinate. In the present embodiment, in the articulated robot arm control device 25, for example, a harvesting order A in which the harvesting work is performed in the order of grape G1, grape G2, and grape G3 is set. The articulated robot arm control device 25 transmits the calculated position coordinates of the grape G1, the grape G2, and the grape G3 and the information of the harvest order A to the external operation terminal C via the communication device 7 of the remote control vehicle 1.

Next, the articulated robot arm control device 25 calculates detailed position coordinates in the order of grape G1, grape G2, and grape G3 based on the harvesting order A, and performs harvesting work.

As shown in FIG. 3, the articulated robot arm control device 25 controls the motor units of each axis of the articulated robot arm 11 by the drive control unit 28, and the second image is closer to the grape G1 than the first imaging position P1. The monocular camera 24 is moved to the imaging position P2. The articulated robot arm control device 25 controls the monocular camera 24 by the image processing unit 26 to image the grape G1 from the second imaging region and acquire the second image Im2 (see FIG. 6).

As shown in FIGS. 6 and 8, the articulated robot arm control device 25 controls the motor units of each axis of the articulated robot arm 11 by the drive control unit 28, and is in an arbitrary direction from the first imaging position P1. The monocular camera 24 is moved by an arbitrary movement amount T. The articulated robot arm control device 25 controls the monocular camera 24 by the image processing unit 26 to image the grape G1 from a position where the grape G1 is moved in an arbitrary direction by an arbitrary amount of movement T, and an auxiliary image of the second image Im2. Acquire Is2.

The articulated robot arm control device 25 transmits the captured second image Im2 and the auxiliary image Is2 of the second image Im2 to the external operation terminal C via the communication device 7 of the remote control vehicle 1.

The articulated robot arm control device 25 converts the position coordinates of the grape G1 in the second image Im2 into the position coordinates G1 (x2, z2) on the XX plane in the robot coordinate system by the coordinate information processing unit 27. Similarly, the coordinate information processing unit 27 converts the position coordinates of the grape G1 in the auxiliary image Is2 of the second image Im2 into the position coordinates on the XX plane in the robot coordinate system.

Further, the coordinate information processing unit 27 generates a parallax image from the second image Im2 and the auxiliary image Is2 of the second image Im2, and calculates the Y coordinate of the grape G1 in the robot coordinate system. As a result, the coordinate information processing unit 27 calculates the position coordinates G1 (x2, y2, z2) of the grape G1 in the robot coordinate system. Since the position coordinates G1 (x2, y2, z2) of the grape G1 are calculated from the second image Im2 captured at the second imaging position P2, which is closer to the grape G1 than the first imaging position P1, G1 (x1, y2, z2). It is more accurate than y1 and z1).

The articulated robot arm control device 25 controls the motor units of each axis of the articulated robot arm 11 by the drive control unit 28, and sets the end effector 23 up to the calculated position coordinates G1 (x2, y2, z2) of the grape G1. Move. The drive control unit 28 controls the motor of the end effector 23, grips the stalk of the vine G1 with the gripping device 23a, and separates the stalk of the vine G1 from the branch of the vine tree with the cutting device 23b. After that, the drive control unit 28 controls the motor units of each axis of the articulated robot arm 11 to store the harvested grape G1 in the harvest box H of the remote-controlled vehicle 1 (see FIG. 1).

Similarly, the articulated robot arm control device 25 calculates the detailed position coordinates G2 (x2, y2, z2) and G3 (x2, y2, z2) of the grape G2 and the grape G3 based on the harvest order A. Harvesting work is performed by the end effector 23.

(Operation of articulated robot arm control device 25)
Next, with reference to FIG. 9, the operation of the articulated robot arm control device 25 having the above configuration will be specifically described. FIG. 9 shows a control flow diagram of the articulated robot arm control device 25 according to the first embodiment of the present invention.

As shown in FIG. 9, when the flow of the harvesting work by the articulated robot arm control device 25 starts, in step S110, the articulated robot arm control device 25 is moved by the drive control unit 28 to each axis of the articulated robot arm 11. The motor unit is controlled to move the monocular camera 24 to the first imaging position P1 and shift the step to step S120.

In step S120, the articulated robot arm control device 25 controls the monocular camera 24 by the image processing unit 26 to capture the first image Im1 of the work area W from the first imaging position P1. Further, the articulated robot arm control device 25 detects the target grape G (n) from the first image Im1 based on the detection data D by the image processing unit 26, and shifts the step to step S130.

In step S130, the articulated robot arm control device 25 captures the auxiliary image Is1 of the first image Im1 from a position in which the monocular camera 24 is moved from the first imaging position P1 in an arbitrary direction by an arbitrary movement amount T. The step is shifted to step S140.

In step S140, the articulated robot arm control device 25 is subjected to the position coordinates G (n) of the target grape G (n) in the robot coordinate system from the auxiliary image Is1 of the first image Im1 and the first image Im1 by the coordinate information processing unit 27. (X1, y1, z1) is calculated, and the step is shifted to step S150.

In step S150, the articulated robot arm control device 25 determines whether or not all the position coordinates G (n) (x1, y1, z1) of the target grape G (n) can be calculated.
As a result, when all the position coordinates G (n) (x1, y1, z1) of the target grape G (n) can be calculated, the articulated robot arm control device 25 shifts the step to step S160.
On the other hand, when the target grape G (n) cannot be properly imaged and all the position coordinates G (n) (x1, y1, z1) of the target grape G (n) cannot be calculated, the articulated robot arm control device 25 The step is shifted to step S161.

In step S160, the articulated robot arm control device 25 performs the harvesting operation based on the position coordinates G (n) (x1, y1, z1) of all the target grapes G (n) calculated by the coordinate information processing unit 27. The harvest order A to be performed is calculated, and the step is shifted to step S170.

In step S170, the articulated robot arm control device 25 controls the motor units of each axis of the articulated robot arm 11 by the drive control unit 28, and the second of the target grapes G (n) selected based on the harvest order A. 2 The monocular camera 24 is moved to the imaging position P2, and the step is shifted to step S180.

In step S180, the articulated robot arm control device 25 controls the monocular camera 24 by the image processing unit 26 to image the second image Im2 of the target grape G (n) from the second imaging position P2. Further, the articulated robot arm control device 25 captures the auxiliary image Is2 of the second image Im2 from the position where the monocular camera 24 is moved from the second imaging position P2 in an arbitrary direction by an arbitrary movement amount T, and steps. To step S190.

In step S190, the articulated robot arm control device 25 is subjected to the position coordinates G (n) of the target grape G (n) in the robot coordinate system from the auxiliary image Is2 of the second image Im2 and the second image Im2 by the coordinate information processing unit 27. (X2, y2, z2) is calculated. Further, the articulated robot arm control device 25 controls the motor units of each axis of the articulated robot arm 11 by the drive control unit 28, and the target grapes at the calculated position coordinates G (n) (x2, y2, z2). G (n) is harvested and the step shifts to step S200.

In step S200, the articulated robot arm control device 25 determines whether or not all the target grapes G (n) have been harvested.

As a result, when all the target grapes G (n) are harvested, the articulated robot arm control device 25 ends the harvesting work and ends this flow.

On the other hand, if all the target grapes G (n) have not been harvested, the articulated robot arm control device 25 shifts the step to step S170.

In step S161, the articulated robot arm control device 25 controls the motor units of each axis of the articulated robot arm 11 by the drive control unit 28, and a new first imaging closer to the work area W than the first imaging position P1. The monocular camera 24 is moved to the position P1 and the step is shifted to step S162.

In step S162, the articulated robot arm control device 25 controls the monocular camera 24 by the image processing unit 26 to capture a new first image Im1 of the work area W from the new first imaging position Pn, and for detection. The target grape G (n) is detected from the new first image Im1 based on the data D, and the step is shifted to step S163.

In step S163, the articulated robot arm control device 25 captures the auxiliary image Is1 of the new first image Im1 from the position where the monocular camera 24 is moved from the new first imaging position P1 in an arbitrary direction by an arbitrary amount of movement. , Move the step to step S164.

In step S164, the articulated robot arm control device 25 receives the new position coordinates G of the target grape G (n) in the robot coordinate system from the auxiliary image Is1 of the new first image Im1 and the new first image Im1 by the coordinate information processing unit 27. (N) Calculate (x1, y1, z1) and shift the step to step S150.

From the above, in the present embodiment, the articulated robot arm control device 25 includes a monocular camera 24 provided on the articulated robot arm 11, an image processing unit 26, a coordinate information processing unit 27, and a drive control unit 28. I have. The monocular camera 24 images the work area W at the first imaging position P1 in which the articulated robot arm 11 is separated from the work area W and the work area W is completely included in the angle of view of the monocular camera 24. The image processing unit 26 detects the first image Im1 of the target grape G (n) from the image in the work area W. The coordinate information processing unit 27 of the target grape G (n) in the robot coordinate system of the articulated robot arm 11 based on the first image Im1 of the detected target grape G (n) and the auxiliary image Is1 of the first image Im1. The position coordinates G (n) (x1, y1, z1) are calculated.

As described above, the articulated robot arm control device 25 searches for the target grape G (n) from the area including the work area W by the monocular camera 24 taking an image of the work area W at the first imaging position P1. Can be done. Further, the coordinate information processing unit 27 can calculate the position coordinates of the target grape G (n) based on the first image Im1 and the auxiliary image Is1 of the first image Im1. As described above, the monocular camera 24 provided on the articulated robot arm 11 has a role of calculating detailed position coordinates for improving the work accuracy of the end effector 23 and the target grape G (n) included in the entire work area W. It plays a role in detecting. That is, the articulated robot arm control device 25 moves the monocular camera 24 by the articulated robot arm 11 to image the work area W and the target grape G (n), which are imaging targets having different imaging ranges. Can be done. As a result, the articulated robot arm control device 25 can omit the fixed camera in the conventional proposal using the fixed camera and the hand camera.

Moreover, as described above, the coordinate information processing unit 27 calculates the position coordinates G (n) (x1, y1, z1) of the plurality of target grapes G (n) in the work area W, so that the articulated robot The optimum harvesting order A when the arm 11 performs the harvesting operation can be calculated based on a predetermined condition. As a result, the articulated robot arm control device 25 can efficiently harvest the target grape G (n) existing at a random position.

Further, the monocular camera 24 further images the work area W including the target grape G (n) at the second imaging position P2 different from the first imaging position P1.

As described above, a specific region or a specific target grape G (n) is imaged from the second imaging position P2 at a different angle from the first image Im1 imaged from the first imaging position P1, or from the first imaging position P1. Can be magnified and imaged at a close position. As a result, the articulated robot arm control device 25 can search the work area W over a wide area using the first image Im1 imaged from the first imaging position P1, and the second image Im2 imaged from the second imaging position P2. The work area W can be searched locally using. That is, the articulated robot arm control device 25 can acquire an image suitable for the state of the work area W, the state around the target grape G (n), and the shape of the target grape G (n).

Further, since the monocular camera 24 takes an image with an angle of view θ of 90 degrees or more in the imaging range, the entire work area W can be imaged only by moving the monocular camera 24 within the movable range of the articulated robot arm 11. .. As a result, the articulated robot arm control device 25 can capture the entire working area W within the movable range of the articulated robot arm 11 with the monocular camera 24.

Further, the monocular camera 24 can generate a parallax image of the work area W by moving in an arbitrary direction by driving the articulated robot arm 11 and imaging the work area W at a plurality of places.

As described above, the articulated robot arm control device 25 can measure the distance L to the work area W or the target grape G (n) by moving the monocular camera 24 to take an image. Further, in the articulated robot arm control device 25, the distance between the two cameras is fixed by setting the movement amount T of the monocular camera 24 in an arbitrary direction according to the work environment and the work state. Compared with a stereo camera, the range of the distance at which the target grape G (n) can be recognized can be expanded, and the range that can be measured with a certain accuracy or higher can be expanded. As a result, the articulated robot arm control device 25 captures a plurality of target grapes G (n) in the work area W by the monocular camera 24, and is based on a parallax image according to the state of the target grapes G (n). The distance L to the target grape G (n) can be measured.

Further, the articulated robot arm 11 has a T-axis motor unit 22 that can rotate around the axis of the wrist rotation link 21 at the most advanced end, and has an end effector 23 on the output shaft of the T-axis motor unit 22 that is the tip rotating portion. A monocular camera 24 is provided. At this time, the monocular camera 24 is arranged so that at least one part of the gripping device 23a and the cutting device 23b of the end effector 23 is included in the angle of view of the monocular camera 24.

The articulated robot arm control device 25 can move the monocular camera 24 to a position where the target grape G (n) can be imaged by rotating the monocular camera 24 around the axis of the wrist rotation link 21. As a result, the articulated robot arm device 10 can continue imaging by rotating the monocular camera 24 to a position where imaging can be performed even if the surrounding state of the target grape G (n) changes. Further, in the articulated robot arm device 10, since the end effector 23 is projected within the angle of view of the monocular camera 24, remote control by the external operation terminal C can be easily performed.

In the articulated robot arm device 10 including the articulated robot arm control device 25 and the articulated robot arm control device 25 configured as described above, the articulated robot arm 11 is used as a moving actuator and the monocular camera 24 is positioned at an arbitrary position. Can be moved in the direction of. That is, the articulated robot arm device 25 and the articulated robot arm device 10 including the articulated robot arm control device 25 can image the work range from a position and a direction according to the work environment and the work state. Therefore, the articulated robot arm device 25 and the articulated robot arm device 10 including the articulated robot arm control device 25 can increase the types of work that can be handled by the articulated robot arm 11 and enhance versatility.

(Other embodiments)
Although the embodiments of the present invention have been described above, the above-described embodiments are merely examples for carrying out the present invention. Therefore, the embodiment is not limited to the above-described embodiment, and the above-described embodiment can be appropriately modified and implemented within a range that does not deviate from the purpose.

FIG. 10 shows a block diagram of the articulated robot arm control device 25 according to another embodiment of the present invention. In the above embodiment, the articulated robot arm device 10 has a first image Im1 captured by a monocular camera 24 provided on the articulated robot arm 11 and a first image obtained by moving an arbitrary amount of movement in an arbitrary direction. The distance to the target grape G (n) is calculated by using the auxiliary image Is1 of the image Im1. However, as shown in FIG. 10, the articulated robot arm device 10 further includes a distance measuring unit 29 including a laser ranging sensor and the like, and measures the distance to the target grape G (n) without relying on the monocular camera 24. You may. The articulated robot arm device 10 configured in this way captures the work area W with the monocular camera 24, and the coordinate information processing unit 27 acquires the value measured by the distance measuring unit 29 to obtain the target grape G (n). ) Position coordinates G (n) (x1, y1, z1) can be measured.

Further, the articulated robot arm control device 25 controls so that the amount of movement T of the monocular camera 24 for capturing a parallax image in an arbitrary direction differs between the first imaging position P1 and the second imaging position P2. May be good.

As described above, the articulated robot arm control device 25 changes the movement amount T of the monocular camera 24 for capturing the parallax image in an arbitrary direction, thereby moving the distance to the work area W or the target grape G (n). The measurement accuracy when measuring L can be adjusted. As a result, the articulated robot arm control device 25 can determine the distance in the work area W and the articulated robot arm even if the distance L between the first imaging position P1 and the second imaging position P2 to the work area W is different. The distance L to the work area W or the target grape G (n) can be measured with the measurement accuracy according to the work environment and the work state of 11.

Further, the articulated robot arm control device 25 may take an image from different imaging directions at the first imaging position P1 and the second imaging position P2.

FIG. 11 shows a layout diagram showing a state in which the first imaging position P1 and the second imaging position P2 and the imaging direction are different in the articulated robot arm control device 25 according to another embodiment of the present invention. As shown in FIG. 11, the articulated robot arm control device 25 moves the monocular camera 24 to the second imaging position P2 while maintaining the posture of the monocular camera 24 at the first imaging position P1. At this time, the positional relationship of the backlight in which the lens of the monocular camera 24 is directed in the direction of the sunlight SL (see the whitewashed arrow) and the target grape G (n) between the anti-monocular camera 24 and the elephant grape G (n). When the positional relationship is such that grapes, leaves, etc. other than) are present, the monocular camera 24 cannot capture the target grape G (n) from the second imaging position.

When the articulated robot arm control device 25 cannot acquire the position coordinates at the second imaging position due to backlight or an obstacle, the articulated robot arm control device 25 acquires images from different imaging directions at the first imaging position P1 and the second imaging position P2. Even when it is difficult to take an image of the target grape G (n) due to backlight, obstacles, or the like, it is possible to take an image in a state where the monocular camera 24 can take an image by changing the image taking direction and the image taking position.

As a result, the articulated robot arm control device 25 changes the position and posture of the monocular camera 24 according to changes in the work environment and the work state, so that the state of the work area W frequently changes. (N) can be imaged. That is, the articulated robot arm control device 25 can increase the degree of freedom in the usage environment of the articulated robot arm device 10 by increasing the recognition accuracy of the object by the monocular camera 24.

Further, the articulated robot arm control device 25 controls the motor units of each axis of the articulated robot arm 11 by the drive control unit 28 based on the calculated fluctuation amount of the position coordinates of the target grape G (n). The position coordinates G (n) (xc, yc, zc) of the target grape G (n) after the change may be corrected to the position coordinates G (n) (x, y, z) before the change.

FIG. 12 shows a schematic view of a remote-controlled vehicle provided with an articulated robot arm device according to another embodiment of the present invention. As shown in FIG. 12, the position of the remote-controlled vehicle 1 may change due to the influence of the muddy ground or the change in the weight balance of the remote-controlled vehicle 1 due to the change in the posture of the articulated robot arm 11. As a result, the origin of the robot coordinate system in the articulated robot arm device 10 is moved. That is, the position coordinates G (n) (x1, y1, z1) or the position coordinates G (n) (n) of the target grape G (n) calculated by the articulated robot arm control device 25 before the movement of the origin of the robot coordinate system. x2, y2, z2) are different from the position coordinates of the target grape G (n) in the robot coordinate system after the origin has moved.

The articulated robot arm control device 25 has the same first position coordinates G (n) (x1c, y1c, z1c) of the target grape G (n) based on the first image Im1 calculated by the coordinate information processing unit 27. If the position coordinates G (n) (x1, y1, z1) of the target grape G (n) based on the first image Im1 captured in the past from the imaging position are different, it is determined that the origin of the robot coordinate system is moving. ..

The articulated robot arm control device 25 has the position coordinates G (n) (x1c, y1c, z1c) of the target grape G (n) after the origin of the robot coordinate system has moved and before the origin of the robot coordinate system has moved. The motor unit of each axis of the articulated robot arm 11 is controlled so that the position coordinates G (n) (x1, y1, z1) coincide with each other. As a result, the monocular camera 24 is corrected so as to be in the position and posture before the origin of the robot coordinate system moves (see the black arrow). Therefore, the articulated robot arm control device 25 can continue the work under the control of the articulated robot arm 11 even if the surrounding state of the articulated robot arm 11 changes. That is, the articulated robot arm control device 25 can increase the degree of freedom in the usage environment of the articulated robot arm device 10.

In the above embodiment, the articulated robot arm control device 25 determines the harvesting order A of the target grape G (n) in the work area W by a predetermined program based on the position of the target grape G (n). However, this is just an example and is not limited to this. For example, the articulated robot arm control device 25 may determine the harvest order A by AI (artificial intelligence) or by applying a specific standard such as the size order of the target grape G (n).

Further, in the above-described embodiment, the articulated robot arm 11 is a 6-axis vertical articulated robot arm. The articulated robot arm 11 is, for example, an S-axis motor unit 12, an L-axis motor unit 14, and a U-axis motor unit 16. , B-axis motor unit 18, R-axis motor unit 20, and T-axis motor unit 22 are connected in series by links, respectively. The articulated robot arm 11 may have a structure in which the motor units of the respective axes are connected in the order of connection, the axial direction at the time of connection, and the like are established as an articulated robot arm.

Further, in the above-described embodiment, the articulated robot arm 11 is provided on the upper surface of the vehicle body 2 in the remote-controlled vehicle 1 and in front of the vehicle body 2, but this is an example and is not limited thereto. For example, the articulated robot arm 11 may be provided on the upper surface of the vehicle body 2 and on the rear side or the left or right side of the vehicle body 2. Further, the articulated robot arm 11 may be provided on the front-rear side surface or the left-right side surface of the remote-controlled vehicle 1.

Further, in the above-described embodiment, the articulated robot arm device 10 performs harvesting work of the target grape G (n), which is an agricultural product, but this is an example and is not limited thereto. For example, the articulated robot arm device 10 is a type of end effector 23 mounted on the articulated robot arm 11 regardless of whether it is outdoors or indoors, such as not only harvesting agricultural products outdoors but also handling industrial parts outdoors. Carry out the work according to.

Further, in the above-described embodiment, the end effector 23 is a device including a gripping device 23a and a cutting device 23b, and grips and cuts the target grape G (n), but is not limited thereto. The end effector 23 may be any device that performs a predetermined operation on the object.

Further, in the above embodiment, the articulated robot arm device 10 semi-automatically harvests the target grape G (n) by the control signal from the external communication terminal and the control signal from the articulated robot arm control device 25. However, this is just an example and is not limited to this. For example, the articulated robot arm device 10 may be configured to control the motor unit, the end effector 23, and the monocular camera 24 of each axis of the articulated robot arm 11 by a control signal from an external operation terminal C.

Further, in the above-described embodiment, the remote-controlled vehicle 1 is controlled by the vehicle control device 9 so as to move on a predetermined route when a control signal is acquired from the operation terminal C, but this is an example and is not limited thereto. Absent. For example, the remote-controlled vehicle 1 may be configured to move independently based on position information and map data from GNSS (Global Navigation Satellite System).

Further, in the above-described embodiment, the remote-controlled vehicle 1 is a four-wheeled vehicle including a pair of wheels 3 and a pair of wheels 4, but is not limited to this as an example. The remote-controlled vehicle may be a vehicle other than a four-wheeled vehicle, for example, a three-wheeled vehicle or a two-wheeled vehicle.

1 Remote control vehicle 2

Body

3, 4 Wheels 5 Drive motor 6 Steering motor 7 Communication device 8 Battery 9 Control device 10 Articulated robot arm device 11 Articulated robot arm 23 End effector 23a Gripping device 23b Cutting device 24 Monocular camera 25 Articulated robot arm control device 26 Image processing unit 27 Coordinate information processing unit 28 Drive control unit P1 First imaging position Im1 First image Is1 Auxiliary image of first image P2 Second imaging position Im2 Second image Is2 Auxiliary of second image image

Claims

An imaging unit provided on the articulated robot arm and
An image processing unit that detects an image of an object from an image captured by the imaging unit, and an image processing unit.
A coordinate information processing unit that calculates the position coordinates of the object,
A drive control unit for driving an actuator of the articulated robot arm is provided.
The imaging unit images the work area at a first imaging position located at a predetermined distance from the work area where the articulated robot arm works on the object.
The image processing unit detects an image of the object from the image of the work area and determines the image of the object.
The coordinate information processing unit is an articulated robot arm control device that calculates the position coordinates of the object in the coordinate system of the articulated robot arm based on the detected image of the object.
In the articulated robot arm control device according to claim 1,
The imaging unit is an articulated robot arm control device that further photographs the work area including the object from a second imaging position different from the first imaging position.
In the articulated robot arm control device according to claim 2.
The articulated robot arm control device whose second imaging position is closer to the working area than the first imaging position.
In the articulated robot arm control device according to any one of claims 1 to 3.
The imaging unit is an articulated robot arm control device that images the working area at an angle of view of 90 degrees or more in the imaging range.
In the articulated robot arm control device according to any one of claims 1 to 4.
The articulated robot arm control device includes a monocular camera that can move in an arbitrary direction by driving the articulated robot arm and can acquire a parallax image of the work area.
In the articulated robot arm control device according to claim 5.
An articulated robot arm control device in which the amount of movement in an arbitrary direction for capturing a parallax image of the monocular camera differs between the first imaging position and the second imaging position.
In the articulated robot arm control device according to any one of claims 2 to 6.
The imaging unit is an articulated robot arm control device that captures images from different imaging directions at the first imaging position and the second imaging position.
In the articulated robot arm control device according to any one of claims 1 to 7.
The drive control unit is an articulated robot arm control device that corrects the control amount of the actuator of the articulated robot arm based on the calculated fluctuation of the position coordinates of the object.
The articulated robot arm control device according to any one of claims 1 to 8.
With an articulated robot arm
With more
The articulated robot arm is an articulated robot arm device controlled by the articulated robot arm control device.
In the articulated robot arm device according to claim 9.
The articulated robot arm
With multiple joints
A plurality of links rotatably connected to the plurality of joints around the axis of each joint, and
It has a tip rotating portion that can rotate around the axis of the link located at the tip of the plurality of links.
The imaging unit is an articulated robot arm device provided in the tip rotating unit.