WO2020049765A1

WO2020049765A1 - Mobile manipulator, mobile robot, server, and mobile manipulator system

Info

Publication number: WO2020049765A1
Application number: PCT/JP2019/009017
Authority: WO
Inventors: 達也古賀; 聡庸金井
Original assignee: オムロン株式会社
Priority date: 2018-09-07
Filing date: 2019-03-07
Publication date: 2020-03-12
Also published as: JP7059872B2; JP2020040147A

Abstract

The present invention realizes a mobile manipulator with which it is possible to reduce workpiece conveying time. This mobile manipulator, mounted on an unmanned conveying vehicle (50), is provided with: a robot arm (11); an image capturing unit (14); and a control unit (21) which transmits and receives imaging result information based on an imaging result obtained by the image capturing unit (14) to and from an external device (60), and controls the operation of the robot arm (11) on the basis of the imaging result information.

Description

Mobile manipulator, mobile robot, server, and mobile manipulator system

The present disclosure relates to a mobile manipulator, a mobile robot, a server, and a mobile manipulator system for transporting a work.

Conventionally, manipulators for taking out and transporting a workpiece by a robot arm have been known. Further, in order to expand the working range of the manipulator, there is an increasing demand for a mobile manipulator that can load a manipulator on an automatic transport vehicle or the like and transport the workpiece while moving.

Patent Literature 1 includes a three-dimensional measurement unit that detects an arrangement state of a work arranged on a stocker, and the three-dimensional measurement unit performs a work removal operation for taking out the work by a robot arm. A manipulator for detecting an arrangement state of a work arranged on another stocker is disclosed.

As described above, the manipulator detects the arrangement state of the workpieces in parallel with the workpiece extracting operation, thereby shortening the time required for a series of workpiece extracting steps for sequentially extracting the workpieces from the plurality of stockers.

Japanese Unexamined Patent Publication "JP-A-2013-86184"

However, the above-described prior art cannot detect the arrangement state of the work arranged on the stocker located at a position distant from the manipulator. Therefore, when the manipulator moves, it is necessary to newly detect the arrangement state of the work. During that time, the work removal operation by the manipulator must be stopped. Therefore, as the number of movements of the manipulator increases, the time for detecting the arrangement state of the work increases.

An object of one embodiment of the present disclosure is to realize a mobile manipulator that can reduce a work transfer time by performing a work removal operation and a detection of a work arrangement state in parallel.

In order to solve the above problems, a mobile manipulator according to an aspect of the present disclosure is a mobile manipulator mounted on an automatic guided vehicle, and a robot arm that performs a gripping operation on an object, and the robot arm The imaging unit provided in the, imaging result information based on the imaging result by the imaging unit, and transmitting to an external device by communication, from the external device, receiving the imaging result information stored in the external device, A robot arm control unit that controls the operation of the robot arm based on the imaging result information.

According to one aspect of the present disclosure, a mobile manipulator that can reduce the work transfer time can be realized by performing the work removal operation and the detection of the arrangement state of the work in parallel.

FIG. 1 is a diagram schematically illustrating a mobile robot and a mobile manipulator system according to a first embodiment. FIG. 2 is a block diagram illustrating a main configuration of the mobile robot according to the first embodiment. FIG. 4 is a diagram schematically illustrating an example of an operation of the mobile robot according to the first embodiment. It is a figure which shows typically an example of operation | movement of the mobile robot which concerns on a reference example. 5 is a flowchart illustrating an example of a flow of a process of a control unit according to the first embodiment. FIG. 9 is a block diagram illustrating a main configuration of a mobile robot according to a second embodiment. 13 is a flowchart illustrating an example of a flow of a process of a control unit according to the second embodiment.

[Embodiment 1]
Hereinafter, an embodiment according to one aspect of the present invention (hereinafter, also referred to as “the present embodiment”) will be described with reference to the drawings.

§1 Application Example FIG. 1 schematically illustrates an example of a mobile robot 1 and a mobile manipulator system according to the present embodiment. FIG. 2 is a block diagram illustrating a main configuration of the mobile robot according to the present embodiment. 3 and 4 are diagrams schematically showing an example of the operation of the

mobile robots

1 and 1a according to the present embodiment or the mobile robots 101 and 101a according to the reference example of the present embodiment. First, an outline of an application example of the mobile robot 1 will be described with reference to FIGS.

As shown in FIGS. 1 and 2, the mobile robot 1 is a device for transporting a work (object) 2 and is configured to be movable. The mobile robot 1 includes a manipulator (mobile manipulator) 10 and an automatic guided vehicle 50. The manipulator 10 includes a robot arm 11 and a housing 15.

The robot arm 11 is a member that performs a gripping operation on the work 2. One end of the robot arm 11 includes a work holding unit 13 for holding the work 2 and a camera (imaging unit) 14. The housing unit 15 includes a control device 20 and a first communication unit 30 described below.

The mobile robot 1a also includes a manipulator 10a and an automatic guided vehicle, like the mobile robot 1. The manipulator 10a includes a workpiece gripper at one end of a robot arm and a camera 14a.

The control device 20 includes a control unit 21 and a first storage unit 40. The control unit 21 includes an image analysis unit 22, a robot arm control unit 23, an automatic guided vehicle control unit 24, and a camera control unit 25. The image analysis unit 22 performs image analysis on the captured image data (image capture result) captured by the camera 14. The robot arm control unit 23 controls the operation of the robot arm 11 based on the received imaging result information.

The first communication unit 30 transmits the captured image data captured by the camera 14 or the result of analyzing the captured image data by the image analysis unit 22 to the server (external device) 60 as captured result information. In addition, the first communication unit 30 receives the imaging result information stored in the server 60.

As described above, the mobile robot 1 mutually communicates with the other mobile robots 1 a and the like and the server 60 by the first communication unit 30. Thereby, a mobile manipulator system including the plurality of mobile robots 1 and the server 60 is formed.

An outline of the operation of the mobile robot 1 will be described with reference to FIGS. As shown in FIG. 3, at a timing (period) t1, the mobile robot 1 moves to a position in front of a stocker 3 on which a plurality of works (objects) 2 to be transported are placed.

Next, at timing t2, the mobile robot 1 captures an image of the workpiece 2 with the camera 14, and generates workpiece position information (position information of the target object) by the image analysis unit 22 from the obtained captured image data (see FIG. 3). "Image analysis"). The captured image data or the work position information is shared with another mobile robot 1a as imaging result information.

The mobile robots 101 and 101a according to the reference example shown in FIG. 4 differ from the mobile robots 1.1 and 1a according to the present embodiment in that the imaging result information cannot be shared with another mobile robot.

Next, at a timing t3, the mobile robot 1 recognizes the arrangement state of the work 2 based on the work position information and grips the work 2. Thereafter, the mobile robot 1 captures an image of the stocker 3. The mobile robot 1 generates new workpiece position information from captured image data while transporting the workpiece 2. The captured image data or the work position information is shared with another mobile robot 1a as imaging result information.

(4) From timing t4 to t6, the mobile robot 1 similarly carries the work 2 and analyzes the image. The work position information generated by the mobile robot 1 is shared with the mobile robot 1a as needed.

Here, by the beginning of the timing t7, the mobile robot 1a moves to the position before the stocker 3 where the mobile robot 1 is carrying. From timing t7, the mobile robot 1a transfers the work 2 instead of the mobile robot 1.

At timing t7, the mobile robot 1a receives the imaging result information generated by the mobile robot 1 at timing t6 before performing the gripping operation of the work 2. Thereby, the mobile robot 1a can recognize the arrangement state of the work 2 in the stocker 3. Therefore, the mobile robot 1a can immediately start transporting the work 2 without needing to image the stocker 3 and generate work position information.

(4) After timing t8, the mobile robot 1a conveys the work 2 a predetermined number of times, captures an image of the stocker 3, analyzes the image, and shares the imaging result information.

On the other hand, according to the mobile robots 101 and 101a according to the reference example, the work 2 is transported from the timing t11 to the time t16 in the same manner as the

mobile robots

1 and 1a.

However, the mobile robot 101 does not have a function of sharing the generated imaging result information with another mobile robot 101a. Therefore, the mobile robot 101a itself takes an image of the stocker 3 at timing t17 and performs image analysis. At timing t17, the mobile robot 101a cannot transfer the work 2 and must stop its operation.

On the other hand, in the

mobile robots

1 and 1a according to the present embodiment, the processing at the timing t16 and the processing at the timing t17 in FIG. 4 can be performed at the timing t6 in FIG. As described above, according to the

mobile robots

1 and 1a according to the present embodiment, it is possible to reduce the time required for the imaging and image analysis of the stocker 3 (for the period of t17) and the transfer work time in accordance with the movement of the work point.

§2 Configuration example (Robot arm 11)
The robot arm 11 includes a joint 12 formed to be rotatable and bendable in an arbitrary direction. The robot arm 11 may include one or more joints 12. One end of the robot arm 11 is formed so as to be rotatable and bendable in the housing 15.

(4) The work holding portion 13 includes a vacuum pad capable of vacuum-sucking the work 2. The work gripper 13 is not limited to this, and may include, for example, a pair of claws for sandwiching and gripping.

The camera 14 includes a solid-state imaging device and a lens that can three-dimensionally recognize an imaging target. As the solid-state imaging device, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor may be used. According to such a configuration, the spatial position of the workpiece 2 to be conveyed can be accurately grasped. Note that the camera 14 is not limited to such an example, and may include, for example, a solid-state imaging device and a lens that can only capture a two-dimensional image.

The camera 14 may be attached to any position of the mobile robot 1 instead of one end of the robot arm 11. For example, the camera 14 may be attached to the housing 15 or the automatic guided vehicle 50. According to such a configuration, while the robot arm 11 is moving, the camera 14 can capture an image of the workpiece 2 or the like with high accuracy in a stationary state.

(Control device 20)
The control device 20 includes a control unit 21 and a first storage unit 40. The control unit 21 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and controls each component. The first storage unit 40 is, for example, an auxiliary storage device such as a hard disk drive or a solid state drive, and stores various programs executed by the control unit 21, image data shared with other manipulators, imaging result information, and the like. .

(4) The image analysis unit 22 acquires captured image data captured by the camera 14 and analyzes the captured image data. The image analysis unit 22 first determines whether or not the workpiece 2 to be transported exists in the captured image data. If it is determined that the work 2 exists, the image analysis unit 22 generates work position information. According to such a configuration, the mobile robot 1 can generate work position information from captured image data captured by itself.

The imaging result information generated by the image analysis unit 22 may be output to the first communication unit 30 and the first storage unit 40. The imaging result information may be acquired by the robot arm control unit 23 and the automatic guided vehicle control unit 24.

The robot arm control unit 23 controls the operation of the robot arm 11. For example, based on the imaging result information or the like, the operation of the robot arm 11 is controlled to grip and transport the work 2, and the position and direction of the camera 14 provided in the robot arm 11 are adjusted.

The automatic guided vehicle control unit 24 controls the operation of the automatic guided vehicle 50 and moves the mobile robot 1 to a predetermined point. Further, the automatic guided vehicle control unit 24 may control the operation of the automatic guided vehicle 50 based on the imaging result information or the like, and finely adjust the position of the mobile robot 1. Further, the mobile robot 1 is moved to a position in front of the stocker 3 on which the work 2 to be transported is placed, based on the position information of the stocker 3 transmitted from the information management control unit 62 provided in the server 60 described later.

The camera control unit 25 controls the imaging operation of the camera 14.

(First communication unit 30)
The first communication unit 30 performs communication with the server 60 and the like. The first communication unit 30 may perform wired communication using a cable or the like, or may perform wireless communication. Further, the first communication unit 30 may communicate with the first communication unit of another mobile robot 1a.

(Automated guided vehicle 50)
The automatic guided vehicle 50 includes a wheel 51. The operation of the wheel 51 is controlled by the automatic guided vehicle control unit 24, so that the automatic guided vehicle 50 can move to an arbitrary point. Therefore, the manipulator 10 placed on the automatic guided vehicle 50 can transport the work 2 to a distant position that is not reachable by the robot arm 11. Further, the point where the manipulator 10 works can be changed as appropriate.

The manipulator 10 is not limited to the automatic guided vehicle 50 because the manipulator 10 is configured to be movable as the mobile robot 1. For example, it may be an unmanned aerial vehicle (drone) that can fly, or a bogie that is manually moved.

(Server 60)
The server 60 includes a second communication unit 61, an information management control unit 62, and a second storage unit 63. The information management control unit 62 controls writing to the second storage unit 63 according to a change in the state of the mobile robot 1 (such as updating a database according to movement of the mobile robot 1), and via the second communication unit 61. Communication control (such as information transmission according to the movement of the mobile robot 1).

The server 60 receives, for example, the imaging result information generated by the mobile robot 1 by the second communication unit 61 and stores the imaging result information in the second storage unit 63. The imaging result information is transmitted to another mobile robot 1a or the like as needed. Thereby, the imaging result information can be shared among the plurality of mobile robots 1.

As described above, by forming the mobile manipulator system including the plurality of mobile robots 1 and the server 60, the operation of the mobile robot 1 can be smoothly controlled.

§3 Operation Example FIG. 5 is a flowchart illustrating an example of a processing flow of the control unit 21 according to the present embodiment. An example of a processing flow of the control unit 21 will be described with reference to FIG.

First, the automatic guided vehicle controller 24 moves the mobile robot 1 to a position (designated point) in front of the stocker 3 on which the work 2 to be transferred is placed (S1).

The first communication unit 30 specifies the position and the like of the stocker 3 including the work 2 to be transported, and obtains imaging result information (second imaging result information) on the stocker 3 generated by another manipulator 10a from the server 60. I do. Next, the first communication unit 30 or the robot arm control unit 23 determines whether the current imaging result information of the stocker 3 can be received from the server 60 (S2). In the present operation example, the imaging result information transmitted and received between the first communication unit 30 and the server 60 is work position information generated from the captured image data by the image analysis unit 22.

If the work position information has not been received or if the received work position information is unavailable (NO in S2), the robot arm control unit 23 images the robot arm 11 with the stocker 3 including the work 2. (S3).

できない As a case where the work position information cannot be received, it is conceivable that there is no imaging result information including the work 2 to be transported.

As a case where the received work position information is information that cannot be used, for example, a case where the target work 2 does not exist at the position indicated by the received work position information can be considered. For example, the arrangement of the work 2 on the stocker 3 may be changed after the work position information of the stocker 3 generated most recently. When the robot arm 11 cannot grasp the work 2 indicated by the work position information, the robot arm control unit 23 determines that the received work position information is information that cannot be used.

Then, the camera control unit 25 causes the camera 14 to image the stocker 3 (S4). The image analysis unit 22 performs an image analysis process on the captured image data (S5), and generates work position information in the stocker 3 (S6).

As described above, the manipulator 10 can generate the work position information by itself even when appropriate work position information for detecting the position of the work 2 cannot be received.

Next, based on the work position information received from the server 60 (YES in S2) or based on the work position information generated by the image analysis unit 22 in step S6, the robot arm control unit 23 operates the robot arm 11 To perform the gripping operation of the work 2 (S7).

Here, after the work 2 is gripped, the robot arm control unit 23 moves the robot arm 11 to a position for imaging the stocker 3 including the work 2 (S8). Then, the camera control unit 25 causes the camera 14 to image the stocker 3 (S9).

According to such a configuration, the arrangement state of another work 2 after the work 2 is gripped by the robot arm 11 can be imaged. Therefore, it is possible to generate optimum work position information for the next gripping operation.

(4) Next, the robot arm control unit 23 starts the transfer of the work 2. At substantially the same time as the start of the conveyance, the image analysis unit 22 starts an image analysis process of the captured image data (S10), and generates work position information (first imaging result information) in the stocker 3 (S11). . The work position information is output to the first communication unit 30 and transmitted from the first communication unit 30 to the server 60 (S12). At this time, the work position information may also be output to the first storage unit 40 and stored in the first storage unit 40.

After the transmission of the work position information is completed, the transfer of the work 2 is completed (S13). The transfer of the work 2 may be completed substantially simultaneously with the completion of the transmission or before the completion of the transmission of the work position information. Next, the control unit 21 determines whether or not the mobile robot 1 moves to another point based on the transfer plan information of the work 2 transmitted from the information management control unit 62 provided in the server 60 (S14). If the mobile robot 1 does not move and continues the transfer operation at the same position (NO in S14), the process returns to step S2, and a series of processing is performed again.

At this time, the work position information received in step S2 may be obtained from the first storage unit 40 by the work position information generated by itself in step S11, or the work position information transmitted by itself in step S12 may be renewed by the server 60. May be received. When the plurality of mobile robots 1 transport the work 2 on the same stocker 3, the latest work position information in the stocker 3 may be received from the server 60.

On the other hand, when the mobile robot 1 moves to another location (YES in S14), the automatic guided vehicle control unit 24 moves the mobile robot 1 to a position before the stocker 3 on which the workpiece 2 to be transported next is placed. (Point), and a series of processing is performed again.

As described above, when the mobile robot 1 moves to the work point and there is another mobile robot 1a working at the work point after the movement, the mobile robot 1 moves to the work position generated by the mobile robot 1a. Receive and use information. Therefore, after moving, the mobile robot 1 can recognize the position of the work 2 at the work point after the movement based on the received work position information. Therefore, immediately after moving to the new work point, the transfer of the work 2 can be started immediately without performing the imaging of the work 2 and generating the work position information by itself.

{Accordingly, the transport work time can be reduced by the time required for imaging the work 2 and generating the work position information. The effect of shortening the transfer operation time increases as the movement of the mobile robot 1 increases.

[Embodiment 2]
Another embodiment of the present disclosure will be described below. For convenience of description, members having the same functions as those described in the above embodiment are denoted by the same reference numerals, and description thereof will not be repeated.

§1 Configuration Example FIG. 6 is a block diagram illustrating a main configuration of the mobile robot 1 according to the present embodiment. A configuration example of the mobile robot 1 according to the present embodiment will be described with reference to FIG.

移動 The mobile robot 1 according to the present embodiment is different from the mobile robot 1 according to the first embodiment in that the control unit 21 does not include the image analysis unit 22 but includes an imaging result information processing unit 22a instead.

The imaging result information processing unit 22a outputs the image data captured by the camera 14 to the first communication unit 30, and transmits the work position information transmitted from the server 60 to the robot arm control unit 23 and the automatic guided vehicle control unit 24. Output. However, unlike the image analysis unit 22, the analysis of the captured image data is not performed. Therefore, the captured image data captured by the camera 14 is not subjected to image analysis, and the captured image data itself is transmitted to the server 60.

In this case, the image analysis unit 64 included in the server 60 analyzes the received captured image data and generates work position information including the position information of the work 2. The work position information generated by the server 60 is transmitted to the mobile robot 1 from the second communication unit 61 provided in the server 60. Note that the captured image data received by the server 60 and the work position information generated by the server 60 are stored in the second storage unit 63.

That is, in the present embodiment, the imaging result information transmitted from the mobile robot 1 to the server 60 is the captured image data itself, and the imaging result information transmitted from the server 60 to the mobile robot 1 is generated from the captured image data. Work position information.

According to the configuration in which the control unit 21 includes the image analysis unit 22, the mobile robot 1 can analyze the captured image data by itself and generate the imaging result information including the position information of the work 2. On the other hand, according to the configuration in which the control unit 21 does not include the image analysis unit 22, the load on the control unit 21 can be reduced.

§2 Operation Example FIG. 7 is a flowchart illustrating an example of the flow of processing of the control unit 21 according to the present embodiment. An example of a processing flow of the control unit 21 will be described with reference to FIG.

(4) The automatic guided vehicle control unit 24 moves the mobile robot 1 to a position (designated point) in front of the stocker 3 on which the work 2 to be transferred is placed (S21). The first communication unit 30 of the manipulator 10 specifies the position of the stocker 3 including the work 2 to be transported, and requests the server 60 for work position information (S22).

The information management control unit 62 of the server 60 is generated based on work position information (first imaging result information) on the stocker 3 generated based on the captured image data of the manipulator 10 or captured image data of another manipulator 10a. Work position information (second imaging result information) on the stocker 3 obtained is acquired from the second storage unit 63 and output to the second communication unit 61. The second communication unit 61 transmits the work position information to the first communication unit 30 of the manipulator 10 (S41).

The first communication unit 30 of the manipulator 10 receives the work position information (S23). Next, the first communication unit 30 or the robot arm control unit 23 determines whether the current work position information of the stocker 3 has been received from the server 60 (S24).

If the work position information could not be received, or if the received work position information is information that cannot be used (NO in S24), the robot arm control unit 23 images the robot arm 11 with the stocker 3 including the work 2. (S25).

Next, the camera control unit 25 causes the camera 14 to image the stocker 3 (S26). The captured image data is transmitted to the server 60 by the first communication unit 30 (S27).

Next, the server 60 receives the captured image data (S42), and performs image analysis processing by the image analysis unit 64 (S43). Thereby, the work position information based on the captured image data received by the server 60 is generated (S44).

According to such a configuration, it is possible to cause the server 60 to perform the process of calculating the work position information from the captured image data. Therefore, the control unit 21 does not need to include the image analysis unit 22, and the load on the control unit 21 can be reduced.

As described above, when the manipulator 10 cannot receive appropriate work position information to detect the position of the work 2, the manipulator 10 takes an image of the stocker 3 on which the work 2 is placed, and transmits the image to the server 60, thereby obtaining the work position. Information can be obtained.

Next, when the work position information is received from the server 60 (YES in S24), the robot arm controller 23 controls the operation of the robot arm 11 based on the received work position information, and the work 2 is gripped. (S28).

Here, after the work 2 is gripped, the robot arm control unit 23 moves the robot arm 11 to a position for imaging the stocker 3 including the work 2 (S29). Then, the camera control unit 25 causes the stocker 3 to take an image of the camera 14 (S30).

(4) Next, the robot arm control unit 23 starts the transfer of the work 2. Further, during the transport, the first communication unit 30 transmits the captured image data to the server 60 (S31). At this time, the captured image data may also be output to the first storage unit 40 and stored in the first storage unit 40.

(4) After the transmission of the captured image data is completed or almost simultaneously with the completion of the transmission, the transport of the work 2 is completed (S32). Next, the control unit 21 determines whether or not the mobile robot 1 moves to another point based on the transfer plan information of the work 2 transmitted by the information management control unit 62 provided in the server 60 (S33). When the mobile robot 1 does not move and continues the transport operation at the same point (NO in S33), the process returns to step S22, and a series of processes is performed again.

On the other hand, when the mobile robot 1 moves to another location (YES in S33), the automatic guided vehicle control unit 24 moves the mobile robot 1 in front of the stocker 3 on which the workpiece 2 to be transported next is placed (another location). (S21), and a series of processing is performed again.

[Example of software implementation]
The control block of the control unit 21 (in particular, the image analysis unit 22) may be realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, or may be realized by software.

In the latter case, the control unit 21 includes a computer that executes instructions of a program that is software for realizing each function. The computer includes, for example, one or more processors and a computer-readable recording medium storing the program. Then, in the computer, the object of the present disclosure is achieved by the processor reading the program from the recording medium and executing the program. As the processor, for example, a CPU (Central Processing Unit) can be used. Examples of the recording medium include "temporary tangible media" such as ROM (Read Only Memory), tapes, disks, cards, semiconductor memories, and programmable logic circuits. Further, a RAM (Random Access Memory) for expanding the above program may be further provided. Further, the program may be supplied to the computer via an arbitrary transmission medium (a communication network, a broadcast wave, or the like) capable of transmitting the program. Note that one embodiment of the present disclosure can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[Summary]
A mobile manipulator according to an aspect of the present disclosure is a mobile manipulator mounted on an automatic guided vehicle, a robot arm that performs a gripping operation on an object, an imaging unit provided on the robot arm, The imaging result information based on the imaging result by the imaging unit is transmitted to an external device by communication, the imaging result information stored in the external device is received from the external device, and the robot is configured based on the imaging result information. A robot arm control unit for controlling the operation of the arm.

According to the above configuration, the imaging result information can be shared among a plurality of mobile manipulators. Therefore, when the mobile manipulator moves to the work point and there is another mobile manipulator working at the work point after the movement, the imaging result information generated by the another mobile manipulator is received and used. it can. Therefore, the mobile manipulator does not need to image the target object and generate the imaging result information by itself in order to detect the position of the target object.

Note that the imaging result information may be position information of a target which is an analysis result of the imaging image data (imaging result) imaged by the imaging unit, or the imaging image data itself. Further, the external device may be a server described below or another mobile manipulator. A specific mobile manipulator may have a server function, and the imaging result information may be mutually transmitted and received between a plurality of mobile manipulators.

The control unit may include an image analysis unit that analyzes image data captured by the imaging unit and calculates position information of the object as the imaging result information. According to the above configuration, the mobile manipulator can calculate the position information of the target object from the captured image data and control the gripping operation by the robot arm based on the position information.

The control unit may transmit image data obtained by the imaging unit to the external device as the imaging result information. According to the above configuration, it is possible to cause a server or the like to perform a process of calculating position information of a target object from captured image data. Therefore, the control unit does not need to include the image analysis unit, and the load on the control unit can be reduced.

The imaging unit may perform imaging after the gripping operation by the robot arm is completed. According to the configuration, it is possible to image the arrangement state of another target object after the target object is gripped by the robot arm. Therefore, optimal imaging result information for the next gripping operation can be generated and shared with another mobile manipulator.

In other words, the mobile manipulator that has received the imaging result information does not need to perform a process of excluding information on the previously transported object from the imaging result information. Therefore, processing for controlling the operation of the robot arm from the imaging result information can be simplified.

The control unit acquires the imaging result information corresponding to a point at which the gripping operation is performed from the external device before the robot arm performs the gripping operation, and acquires the imaging result information of the target object based on the imaging result information. The gripping operation by the robot arm may be controlled by recognizing a position.

According to the above configuration, after the movable manipulator has moved, the position of the target at the point after the movement can be recognized from the received imaging result information. Therefore, it is possible to skip the step of capturing the target object and generating the imaging result information by itself, and immediately start transporting the target object. Therefore, it is possible to reduce the transport work time by the time required for imaging the target object and generating the imaging result information.

The control unit, before performing the gripping operation of the robot arm, if the imaging result information could not be obtained from the external device, and causes the imaging unit to perform imaging, and the captured image data by the imaging unit And acquiring the position information of the object based on the position information and controlling the gripping operation by the robot arm based on the position information.

The control unit causes the imaging unit to perform imaging when the imaging result information acquired from the external device before the robot arm performs a gripping operation is information that cannot be used, and performs imaging by the imaging unit. Position information of the object based on an image may be acquired, and the gripping operation by the robot arm may be controlled based on the position information.

According to the above configuration, if there is no appropriate imaging result information for detecting the position of the target, the mobile manipulator acquires the position information of the target by itself and controls the gripping operation by the robot arm. it can. In addition, in order to acquire the position information of the target object based on the captured image data, the mobile manipulator may perform image analysis by itself, or transmit the captured image data to a server or the like, and analyze the analysis result from the server or the like. You may receive it.

移動 Further, a mobile robot according to an aspect of the present disclosure includes any of the above-described mobile manipulators and the automatic guided vehicle. According to the above configuration, the mobile manipulator can be moved by the automatic guided vehicle.

サーバ Further, the server according to an aspect of the present disclosure transmits and receives the imaging result information to and from any of the mobile manipulators described above.

In addition, a mobile manipulator system according to an aspect of the present disclosure includes any one of the mobile manipulators described above and the server. According to the above configuration, sharing of imaging result information among a plurality of mobile manipulators can be performed via the server.

The present disclosure is not limited to the embodiments described above, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present disclosure.

1, 1a, 101, 101a Mobile robot 2 Work 10 Manipulator (mobile manipulator)
11 Robot arm 13 Work gripper 14 Camera (imaging unit)
22, 64 Image analysis unit 23 Robot arm control unit 24 Automatic guided vehicle control unit 30, 30a First communication unit 40 First storage unit 50 Automatic guided vehicle 60 Server (external device)
61 second communication unit 63 second storage unit

Claims

A mobile manipulator mounted on an automatic guided vehicle,
A robot arm that performs a gripping operation on an object,
An imaging unit provided on the robot arm;
A communication unit configured to transmit first imaging result information based on an imaging result obtained by the imaging unit to an external device by communication, and receive second imaging result information based on an imaging result obtained by another mobile manipulator from the external device; ,
A control unit that controls the operation of the robot arm based on the first imaging result information or the second imaging result information;
A mobile manipulator comprising:
The mobile manipulator according to claim 1, further comprising: an image analysis unit that analyzes image data captured by the imaging unit and obtains position information of the object as the first imaging result information.
The mobile manipulator according to claim 1, wherein the communication unit transmits image data captured by the imaging unit to the external device as the first imaging result information.
The mobile manipulator according to any one of claims 1 to 3, wherein the imaging unit performs imaging after the gripping operation by the robot arm is completed.
Before the robot arm performs the gripping operation, the communication unit acquires the second imaging result information corresponding to a point at which the gripping operation is performed, from the external device,
The mobile manipulator according to any one of claims 1 to 4, wherein the control unit recognizes a position of the object based on the second imaging result information and controls the gripping operation by the robot arm. .
The control unit, before the robot arm performs a gripping operation, when the second imaging result information cannot be acquired from the external device, causes the imaging unit to perform imaging, and an image captured by the imaging unit. The mobile manipulator according to any one of claims 1 to 4, wherein the gripping operation by the robot arm is controlled based on data.
The controller, when the second imaging result information acquired from the external device before the robot arm performs a gripping operation is information that cannot be used, causes the imaging unit to perform imaging, and the imaging unit The mobile manipulator according to any one of claims 1 to 4, wherein the gripping operation by the robot arm is controlled based on image data captured by the robot arm.
A mobile robot comprising the mobile manipulator according to any one of claims 1 to 7 and the automatic guided vehicle.
A server that transmits and receives the first imaging result information and the second imaging result information to and from the mobile manipulator according to any one of claims 1 to 7.
A mobile manipulator according to any one of claims 1 to 7,
A mobile manipulator system comprising: a server that transmits and receives the first imaging result information and the second imaging result information to and from the mobile manipulator.