JP4578438B2 - Robot device - Google Patents

Description

  The present invention relates to a robot apparatus, and more particularly to a robot apparatus configured to perform a gripping operation of an object.

  Recently, the development of robots (robot devices) has been rapidly progressing, and applications in a wide range of fields are expected. In particular, for robots that are symbiotic with humans and classified under the name “humanoid”, development related to collaborative work with people, such as supporting human behavior, has been energetically promoted.

  A typical example of cooperative work with people is a work such as carrying luggage. In the case of such work, the robot performs an operation of moving the hand unit provided at the tip of the arm unit to the position of the target object (luggage) by the movement of the arm unit and gripping the target object. That is, the robot needs to perform a series of recognition processes and operations such as recognition of the object position, positioning of the hand unit at the object position, and gripping of the object by the hand unit. Therefore, the robot needs to have the ability to recognize the position of the object to be grasped and the ability to position the hand part at a position where the object can be grasped by the arm part.

  A technique called stereoscopic sensing (visual sensing) is known for position recognition of a grasped object or the like. Stereoscopic sensing is an image measurement technique using an image captured by a camera, and a robot such as a gripping object from a plurality of target images captured so as to obtain an appropriate parallax (parallax angle) for the gripping object. Measure and recognize the position relative to. Alternatively, a technique called visual servo (or visual feedback) is known for positioning the hand unit on an object. In visual servoing, the position of the object obtained by stereoscopic sensing is the same as the position data of the arm and hand obtained by stereoscopic sensing, and feedback control based on the position data of the arm and hand is used to move the hand with the arm. And position it at a position where it can be gripped. For example, Patent Documents 1 to 3 disclose such robot technology.

JP 2005-88175 A JP 2003-31670 A JP 2003-117867 A

  It is desired that the object grasping operation in the robot, particularly the object grasping operation as a cooperative operation with a person, can be performed with smoothness similar to a natural operation of a person. From this point of view, stereoscopic sensing and visual servoing based on it have responded appropriately to the smoothness requirement, but are not necessarily sufficient, and there is room for further improvement.

  Further, there are still many problems with the gripping operation of the object by the hand unit that is necessary after positioning with respect to the object. In grasping an object by the hand unit, it is necessary to find and grasp the object with a stable grasp point so that the object can be grasped stably. In order to quickly identify the stable gripping point, depth information (size information in the depth direction) of the object is required. However, with the conventional technology that only uses stereoscopic sensing, sufficient depth information cannot be acquired when the object is approached at the shortest distance by a linear motion with respect to the object. Therefore, conventionally, the gripping operation of the hand unit is performed by stress sensing. In stress sensing, a stress sensor is embedded in the hand part. The stress state applied to the hand part at the start of the gripping operation is detected by the stress sensor, and a stable gripping point is found by trial and error based on the detection of the stress state. To grip. In such a stress sensing method, it takes time to search for a stable gripping point. As a result, the gripping operation becomes awkward and the smoothness is inferior.

The present invention has been made against the background of the above circumstances, a robot having an object gripping function, a challenge to make it perform a gripping operation by the hand of the object more smoothly Yes.

In the present invention, in order to solve the above-described problem , an arm portion, a hand portion that is provided at the tip of the arm portion and performs an operation of gripping an object, a visual sensor that images the object, and an image captured by the visual sensor A visual sensing processing unit that acquires position information of the target object based on a plurality of target object images obtained by performing the movement of the arm unit based on the positional information of the target object obtained by the visual sensing processing unit. In a robot apparatus including an arm motion control unit that controls and positions the hand unit at a position for gripping the object, a hand unit sensor that images the object on the side of the hand unit that grips the object And a hand sensing processing unit that acquires position information of the object based on a plurality of object images obtained by photographing with the hand unit sensor, the arm motion control unit In the process of controlling the movement of the arm part and positioning the position of the hand part to a position for gripping the object, the hand part and the hand part based on the position information of the object obtained by the visual sensing processing part When the distance to the object is equal to or greater than a threshold value, the movement of the arm unit is controlled using the position information of the object obtained by the visual sensing processing unit, and the hand unit and the object When the distance is less than a threshold value, the movement control of the arm unit is performed using the position information of the object obtained by the hand sensing processing unit by the hand unit sensor .

In such a robot apparatus, a stable gripping point of an object can be obtained as side information of the object by a hand sensing processing unit using a hand part sensor . Therefore, there is no need to search for a stable gripping point by trial and error as in the conventional stress sensing described above, the stable gripping point can be determined quickly, and the gripping operation of the object can be performed more smoothly.

In the robot apparatus as described above, it is preferable to provide a hand unit sensor on the side of the hand unit that holds the object . By doing in this way, a hand part sensor can be given various functions other than hand sensing , and functionality can be raised more.

Further, in the robot apparatus of the present invention, the object position information obtained by the visual sensing processing unit by the visual sensor and the object position information obtained by the hand sensing processing unit by the hand part sensor are properly used according to the distance between the hand part and the object. Thus, the movement control of the arm part in positioning the hand part with respect to the object can be performed with higher accuracy, and therefore the approaching operation to the object until the gripping operation can be performed more smoothly.

In the present invention, the arm motion control unit uses the position information of the object obtained by the visual sensing processing unit, and the object is a dynamic object that moves or a static object that is stationary. When the object is a dynamic object, the object image used in the visual sensing processing unit has a low resolution. When the object image is a static object, the object image is a high object image. It is possible to switch to the resolution .

  According to such a robot apparatus, for a dynamic object, by applying a low resolution process in which the resolution of the object image, that is, the speed of the position information acquisition process is more important than the accuracy of the position information, A gripping operation on an object, particularly an approach operation to an object in the gripping operation can be performed more smoothly. On the other hand, for static objects, the approach operation can be performed more smoothly by applying high image resolution processing that places greater importance on the accuracy of position information than the speed of position information acquisition processing.

Further, in the present invention, the arm motion control unit, before Symbol brightness of an image obtained by photographing the object at hand section sensor in the condition that falls below the threshold value, the hand portion is the It can be determined that the positioning has been completed with respect to the object .

  Positioning completion status determination in such a robotic device is based on the fact that when the hand unit is positioned with respect to the target, the hand unit sensor is shielded by the target and the image of the hand unit sensor becomes dark. Made. According to such a robot apparatus, it becomes possible to more easily determine the positioning completion state. Therefore, it becomes possible to cause the hand unit to take a positioning completion state with higher accuracy. The positioning operation of the hand part on the object in the gripping work can be performed more smoothly.

With this embodiment, the hand portion and the arm portion, and configured to include left and right, the arm motion control unit, based on an image obtained by photographing the object in front Symbol right and left hand portions sensor estimating the size of the object, if the estimated size is equal to or greater than the threshold value, and controls the gripping use both the right and left hand portions, if the estimated size is less than said threshold value The gripping using one of the left and right hand portions can be controlled .

  According to such a robot apparatus, it is possible to perform an efficient, that is, wasteful gripping operation according to the size of the object, and smoother the approaching operation to the target object in the gripping work, particularly the gripping work. You can do it.

  For the robot apparatus as described above, in the case of gripping using either one of the left and right hand parts, based on the positional relationship of the object with respect to the left and right hand parts, any of the left and right hand parts is selected as the target. It is more preferable to be able to decide whether to use for gripping an object.

Furthermore, the present invention can be configured to include a work history database that can store arm motion control data related to motion control of an arm mechanism unit including the arm portion and the hand portion regarding a gripping work performed in the past . In this case, when performing a new gripping work , the arm motion control unit searches the work history database, and if arm motion control data applicable to the gripping work to be newly performed can be extracted, extracted control of a series of movements of the arm mechanism unit based on the arm motion control data can be configured intends row.

  Thus, by making it possible to utilize the motion control data in the gripping work performed in the past, the processing time in the motion control can be shortened, and the object up to the gripping operation of the target by the hand unit can be reduced. The approach operation can be performed more smoothly.

  For the robot apparatus as described above, it is preferable that the applicability of the arm motion control data can be determined based on the shape pattern of the object in terms of more effective determination.

  According to the present invention as described above, it is possible to more smoothly perform a gripping operation by a hand portion of an object with respect to a robot having an object gripping function. Further, according to the present invention as described above, the approaching operation to the object up to the gripping operation by the hand unit can be performed with higher smoothness.

  Hereinafter, modes for carrying out the present invention will be described. FIG. 1 shows an overall appearance structure of the robot apparatus 1 according to the first embodiment with respect to a gripping operation state for the object 2. The robot apparatus 1 includes a body unit 12 having a head mechanism unit 11 attached to the upper end, left and right arm mechanism units 13L and 13R attached to shoulders of the body unit 12, and further to the lower end of the body unit 12. An attached traveling unit 14 is provided. In response to a voice command from a person, for example, the user can approach the object 2 and hold it to carry it. The left and right elements such as the arm mechanism units 13L and 13R have the same configuration on the left and right. Therefore, in the following description, L and R may be appropriately omitted as in “arm mechanism unit 13”.

  The head mechanism unit 11 can perform a predetermined operation such as swinging and incorporates left and right visual sensors 21L and 21R. The visual sensors 21L and 21R are used for visual sensing (stereoscopic sensing). That is, it is used for recognizing the position of the object 2 or the like under conditions that enable stereoscopic viewing. As described above, stereoscopic sensing is an image measurement technique that uses an image captured by a camera. For example, the robot apparatus 1 of the object 2 from a plurality of images captured so as to obtain an appropriate parallax for the object 2. Measure and recognize the position relative to. Therefore, digital imaging means is used for the visual sensor 21. Further, the two visual sensors 21L and 21R are provided apart by a distance so that appropriate parallax can be obtained.

  The body unit 12 incorporates a control system, which will be described later, for controlling various operations of the robot apparatus 1.

  The arm mechanism unit 13 includes an arm part 22 (22L, 22R) and a hand part 23 (23L, 23R) connected to the tip thereof. For example, in the case of gripping work of the object 2, the movement of the arm part 22 is performed. Thus, the hand 23 is positioned on the object 2 and the hand 2 is then held by the hand 23 so that the work can be performed.

  The hand unit 23 is configured to have a structure suitable for carrying the object 2 or performing a button pressing operation such as pressing an operation button of an elevator. A hand sensor 24 (24L, 24R) is incorporated in the hand unit 23 as a hand unit sensor. Since the hand sensors 24 are provided in left and right pairs, they function as a proximity visual sensor for proximity visual sensing (proximity stereoscopic sensing) that captures the object 2 at a proximity position in the case of an operation such as gripping. The hand sensor 24 functions as a side-view sensor for side-view sensing that captures the object 2 from its side. Such a hand sensor 24 is configured by using digital photographing means. Here, the side surface of the object 2 is a surface that intersects the front surface with the direction of the object 2 captured by visual sensing as the front surface. Side-viewing sensing refers to side-view information, depth information, or stable gripping point of the object 2 based on the side-view image of the object 2 captured by the side-view sensor, specifically the hand sensor 24. This is a technique for acquiring side information such as information. In the case of this embodiment, the hand sensor 24 is used as a side-view sensor, and therefore it can also be called hand sensing.

  The traveling unit 14 includes a traveling drive unit 26 and left and right traveling wheels 27 (27L, 27R) connected to the traveling drive unit 26, and is a unit that performs traveling for moving motion of the robot apparatus 1. Then, the traveling wheel 27 is rotationally driven to cause the robot apparatus 1 to travel.

  FIG. 2 shows the configuration of the control system of the robot apparatus 1. The control system 30 in this embodiment includes a system control unit 31, a motion integrated control unit 32, a travel control unit 33, an arm motion control unit 34, a head motion control unit 35, a visual sensing information processing unit 36, and a hand sensing information processing unit. 37.

  The system control unit 31 is an overall control unit that performs overall control related to work performed by the robot apparatus 1 and associated motion. Control targets include dialogue interface control in voice recognition for voice commands from humans, formulation of work plans based on commands from humans, battery management, control of mechanical systems and electrical systems, and the like.

  The motion integrated control unit 32 creates an operation plan for each of the head mechanism unit 11, the arm mechanism unit 13, and the traveling unit 14 according to the contents of the work plan formulated by the system control unit 31. In addition, the motion integration control unit 32 uses the data such as the position information as described later from the visual sensing information processing unit 36 and the hand sensing information processing unit 37 to obtain necessary control data based on the created motion plan. It produces | generates in a data format and provides it to each control part 33-35 of driving | running | working, arm movement, and head movement. Here, the predetermined data format is a data format so that the control units 33 to 35 for running, arm motion, and head motion can execute necessary control processing.

  The travel, arm motion, and head motion control units 33 to 35 correspond to functional units (any one of the head mechanism unit 11, the arm mechanism unit 13, and the travel unit 14) using the control data received from the motion integrated control unit 32. Drive). As a result, the robot apparatus 1 performs the work planned by the system control unit 31.

  The visual sensing information processing unit 36 and the hand sensing information processing unit 37 perform sensing information processing according to the content of the work planned by the system control unit 31. When the work is gripping the object 2, the visual sensing information processing unit 36 recognizes the positional relationship between the robot apparatus 1 and the object 2 and the positional relationship between the object 2 and the hand unit 23 by visual sensing using the visual sensor 21. The position information obtained thereby is provided to the motion integrated control unit 32. Similarly, when the work is gripping the object 2, the hand sensing information processing unit 37 performs processing for recognizing the positional relationship between the object 2 and the hand unit 23 by proximity position visual sensing, and from the side surface side of the object 2. Processing for recognizing the side shape and depth of the object 2 or the stable gripping point of the object 2 by side-viewing sensing based on the image is performed, and the position information and depth information obtained thereby are used as side information to the motion integration control unit 32. provide. Then, the motion integration control unit 32 provides the side information provided from the visual sensing information processing unit 36 and the hand sensing information processing unit 37 to the arm motion control unit 34, and the arm motion control unit 34 receives the information from the arm motion control unit 34. The positioning of the hand part 23 with respect to the object 2 by movement and the gripping operation of the object 2 by the hand part 23 are controlled.

FIG. 3 shows an image of the processing performed by the visual sensing information processing unit 36. Two of the visual sensor 21 (21L, 21R) by a distance _{L 0} is isolated by, both have the same focal length f. Under this condition, the visual sensing information processing unit 36 uses the three-dimensional coordinates (x, y) of the point P on the object 2 from an image (object front image) obtained by photographing the object 2 from the front direction with the visual sensor 21. , Z). Since the focal distance f of the visual sensor 21 corresponds to the coordinate in the Z-axis direction of the object front image, the coordinates of the point P in the object front image by the visual sensor 21L are (x _L , y _L , f), and the visual The coordinates of the point P in the front image of the object by the sensor 21R are (x _R , y _R , f). At this time, the point P is given by the following equation (1).

  The visual sensing information processing unit 36 includes an image processing unit 41, an image recognition processing unit 42, and a position information output unit 43. The image processing unit 41 takes in the output signal of the visual sensor 21 and generates two-dimensional image data (target object front image). The image recognition processing unit 42 extracts a feature point of the object 2 in the front image of the object as a point P using a computer program such as a differential calculation process. The position information output unit 43 obtains the three-dimensional coordinates (x, y, z) of the feature point P from the front image of the object by each of the two visual sensors 21L and 21R according to the expression (1), and uses it to determine the position of the object 2 Information is sent to the motion integrated control unit 32.

  FIG. 4 shows an image of processing performed by the hand sensing information processing unit 37. The hand sensing information processing unit 37 generates side information such as depth information of the object 2 from the image of the side surface 2s of the object 2 (object side image) taken by the hand sensor 24. For this purpose, the hand sensing information processing unit 37 includes an image processing unit 45, an image recognition processing unit 46, and an output data generation unit 47. The image processing unit 45 takes in the output signal of the hand sensor 24 and generates two-dimensional image data (object side image). The image recognition processing unit 46 extracts the side surface 2s of the target object 2 from the target side image or extracts a stable gripping point for the extracted side surface 2s. The output data generation unit 47 measures the depth of the side surface 2s of the extracted object 2 or calculates the three-dimensional coordinates of the extracted stable gripping point based on the three-dimensional coordinates of the feature point P. They are sent to the motion integrated control unit 32.

  FIG. 5 schematically illustrates an example of a specific configuration of the arm mechanism unit 13 in association with a specific configuration example of the arm motion control unit 34, and also schematically illustrates a configuration example of a drive unit in the arm mechanism unit 13. Shown in In the arm mechanism unit 13, an arm portion 22 and a hand portion 23 are integrally configured with respect to the drive structure. That is, the arm mechanism unit 13 includes seven rotary type drive units 51 (51a to 51g) in which the arm unit 22 and the hand unit 23 are integrated in a drive structure, and the rotational movements of these drive units 51 are independent. Thus, complicated movements and operations of the arm unit 22 and the hand unit 23 can be realized. 5, the drive units 51a, 51c, and 51e rotate in a direction perpendicular to the paper surface, and the drive units 51b, 51d, 51f, and 51g are in a direction parallel to the paper surface. It means to rotate.

  Each drive unit 51 has basically the same configuration. Specifically, as shown in FIG. 5B, the rotation of the motor 52 is transmitted to the gear 54 through the drive shaft 53, and the drive unit 51 is rotated by receiving the rotation of the gear 54. ing. Further, the rotation amount of the drive unit 51 can be detected by the encoder 57 from the gear 55 meshing with the gear 54 via the gear shaft 56 as the rotation amount of the gear 54. Detection data of the encoder 57 is provided to the arm motion control unit 34 via wiring (not shown) mounted on the arm mechanism unit 13 and a connector 58.

  The arm motion control unit 34 includes one arm motion control circuit 34a, and also includes seven motor drive control circuits 62 (62a to 621g) provided corresponding to the seven drive units 51a to 51g, respectively. . Data from the encoder 57 is input to the motor drive control circuit 62. The motor drive control circuit 62 sends a drive signal to the motor 52 while monitoring the data from the encoder 57 until the rotation amount of the drive unit 51 given in advance from the arm motion control circuit 34a is executed. The arm mechanism unit 13 performs its operation by such a procedure.

  The movement of the arm mechanism unit 13 as described above, for example, the movement of positioning the hand unit 23 on the object 2 for grasping the object 2 is performed by using the visual sensing information processing unit 36 and the hand sensing information processing unit 37 together. Control is based on the obtained hand position information, that is, the positional relationship information between the hand unit 23 and the object 2. FIG. 6 shows a block diagram for the motion control of such an arm mechanism unit 13.

  When a command value with the position of the object 2 as the movement destination position of the hand unit 23 is input, the command value is converted into a rotation angle in the drive unit 51 of the arm mechanism unit 13, that is, through a joint angle conversion process. An arm motion control loop is executed. As a result of the movement of the arm part 22 by the arm movement control loop, the position of the hand part 23 changes. The position of the hand unit 23 is fed back by visual sensing processing by the visual sensing information processing unit 36 and hand sensing processing by the hand sensing information processing unit 37. In such hand position information processing, position information obtained by hand sensing processing and position information obtained by visual sensing processing are used in combination in a procedure described later. Then, while calculating the difference between the position of the object 2 (movement destination position) and the position of the hand unit 23 based on such position information, the above-described processing is repeated until the difference becomes 0, whereby the hand unit 23 is moved to the object 2. Can be accurately moved to the position.

  FIG. 7 is a block diagram showing a specific configuration example of the arm motion control loop. The arm motion control unit 34 uses an angular velocity and an angular acceleration in addition to the rotation angle for causing each driving unit 51 in the arm mechanism unit 13 to perform a necessary operation. In this case, it is necessary to prevent the commanded angular velocity or angular acceleration from exceeding the allowable value of the rotational operation of the motor, and therefore limiter processing is performed. FIG. 7B shows the constraint condition of the angular velocity with respect to the upper limit value and the lower limit value of the joint angle. The angular velocity ω at the joint angle θ in the vicinity of the upper limit value and the lower limit value of the joint angle is given by the following equations (2) and (3).

ω = ωmax × sqrt {α × (θmax−θ)} ………… (2)
ω = ωmin × sqrt {α × (θ−θmin)} (3)
Here, ωmax and ωmin are respectively an upper limit value and a lower limit value of the angular velocity, θmax and θmin are respectively an upper limit value and a lower limit value of the rotation angle of the drive unit 51, and α is an attenuation coefficient as a constant.

  FIG. 8 shows the flow of processing in the arm motion control loop as described above. The command value θRef of the rotation angle (joint angle) of the drive unit 51 is read, the difference Δθ between the current rotation angle θAct of the drive unit 51 and the command value θRef is obtained, and the angular velocity command value ωRef is obtained from the difference Δθ. When the angular velocity command value ωRef deviates from the allowable range, the angular velocity upper limit value ωmax and the lower limit value ωmin are limited, that is, limiter processing is performed. For this purpose, the constraint relationship between the rotation angle and the angular velocity of the drive unit 51 shown in FIG. 7B is applied to the angular velocity command value ωRef. Next, a difference Δω between the angular velocity command value ωRef and the current angular velocity ωAct is obtained, and an angular acceleration command value βRef is obtained from the difference Δω. When the angular acceleration command value βRef is out of the allowable range, limiter processing is performed with the upper limit value βmax and the lower limit value βmin of the angular acceleration. Next, the angular acceleration command value βRef subjected to the limiter process is output, and the angular velocity command value ωRef is obtained again by integrating the execution period of the control. Next, the angular velocity command value ωRef is output, and the command value θRef of the rotation angle of the drive unit 51 is calculated. Finally, the command value θRef is output as a new rotation angle of the drive unit 51. Here, in the processing as described above, when the smoothing process is performed on the angular velocity and the angular acceleration, the rotation angle command value θRef of the drive unit 51 output in the final stage becomes more gradually changed than the original angle and becomes an absolute value. May shift. Therefore, in order to prevent an error in the rotation angle of the drive unit 51, correction processing such as performing offset processing on the angular acceleration command value βRef is added to the algorithm shown in FIG. By doing so, high positional accuracy during operation of the arm mechanism unit 13 can be maintained.

  The above-described motion control of the arm mechanism unit 13 is performed according to the procedure shown in FIG. 9 when the hand 2 is positioned with respect to the target 2 and then the target 2 is gripped. First, in step 201, the position information of the target object 2 is acquired from the visual sensing information processing unit 36 (target position acquisition process). In step 202, a command value with the position of the object 2 as the movement destination position of the hand unit 23 is generated (movement destination command value generation process). In step 203, the position (hand position) of the hand unit 23 is acquired from the visual sensing information processing unit 36 (hand unit position acquisition process). In step 204, a distance D between the object position and the hand portion position is obtained (distance calculation processing). In step 205, it is determined whether the distance D is 0 (first distance determination processing). If the result of the first distance determination is negative, step 206 is performed.

  In step 206, it is determined whether the distance D is equal to or smaller than the distance threshold value (second distance determination process). When the result of the second distance determination is negative, in step 207, the joint rotation angle of the arm mechanism unit 13 for setting the distance D to 0 is calculated based on the position information by the visual sensing information processing unit 36. In step 208, the movement of the arm mechanism unit 13 is controlled based on the calculated joint rotation angle (arm movement control process). On the other hand, if the result of the second distance determination is affirmative, the process of step 209 is performed. In step 209, the acquisition of position information of the object 2 by the hand sensing information processing unit 37 is started (position information acquisition start processing by hand sensing). As described above, the hand sensing information processing unit 37 also performs proximity visual sensing that captures the object 2 at the proximity position. The proximity visual sensing enables position information with higher accuracy than visual sensing by the visual sensor 21. In step 210, the position of the object 2 is corrected based on position information obtained by high-precision proximity visual sensing. That is, the destination command value is corrected (destination command value correction process). When the destination command value is corrected, the processing of step 207 and step 208 is performed based on the corrected destination command value. By repeating the processing of step 203 to step 208 until the determination result in step 205 becomes affirmative, the hand unit 23 is positioned with respect to the object 2.

  If the determination result in step 205 is affirmative, positioning is completed, and the object 2 is gripped by the hand unit 23 in steps 211 and 212. In step 211, side information of the object 2 (information on the stable holding point in the case of gripping) is acquired by hand sensing (side-viewing sensing) (object side information acquisition processing). In step 212, the object 2 is gripped based on the side information acquired in step 211, that is, the stable holding point (gripping process).

  As described above, in the gripping motion control using hand sensing, the position information by visual sensing and the position information by hand sensing are selectively used according to the distance of the hand unit 23 with respect to the object 2. The movement control of the arm mechanism unit 13 in the approach movement up to the positioning with respect to the object 2 can be performed with higher accuracy, thereby making the approach movement smoother. In addition, according to the gripping motion control using the hand sensing as described above, the gripping operation of the object 2 can be performed more smoothly. That is, since the stable gripping point of the object 2 can be obtained as side information of the object 2 by hand sensing using the hand sensor 24, it is necessary to search for the stable gripping point by trial and error as in the conventional stress sensing described above. The stable gripping point can be determined quickly, and the gripping operation of the object 2 can be performed more smoothly.

  Note that a process for performing fine adjustment on the stable gripping point obtained by hand sensing may be added. That is, the gripping stabilization is finely adjusted by searching for the gripping point that enables more stable gripping in the state where the object 2 is gripped by the stable gripping point obtained by hand sensing, by the above-described stress sensing. It is a process to make.

  Here, the distance threshold value used in step 206 serves as a reference for determining the use start timing of the position information by hand sensing. As an example, such a distance threshold is set in correlation with a visual sensing error or an arm mechanism error. The visual sensing error is a position measurement error in visual sensing using the visual sensor 21. In visual sensing using the visual sensor 21, a certain amount of measurement error cannot be avoided due to a large distance between the visual sensor 21 and the object 2, and high-precision control is performed at a distance smaller than the measurement error. Becomes difficult. Therefore, when the distance D is equal to or smaller than the measurement error in visual sensing, a distance threshold is set in correlation with the visual sensing error in order to use position information by hand sensing. The arm mechanism error is an operation error of the arm mechanism unit 13. The drive unit 51 of the arm mechanism unit 13 is formed by combining the motor 52, the gear 54, and the like as described above, and the rotation thereof is detected by the encoder 57. In such an arm mechanism unit 13, a certain degree of arm mechanism error is unavoidable due to mechanical backlash between the motor 52 and the gear 54, or detection errors and delays of the encoder 57. In view of this, such an arm mechanism error is also taken into consideration in setting the distance threshold value, so that more accurate control can be performed.

  The above is a case where the start of use of position information by hand sensing is determined by the distance threshold, but in addition to this, a form in which visibility is also used as a reference for starting use of hand sense position information is possible. When the target object 2 is small, the target object 2 may be hidden from the visual sensor 21 by the approaching hand unit 23. In such a case, visual sensing using the visual sensor 21 cannot be used, and visual recognition by the visual sensor 21 cannot be performed. Therefore, the visibility criterion is to switch to hand sensing when such a state occurs.

  Next, a second embodiment will be described. This embodiment is the same as the first embodiment except that an image resolution switching process is added to the motion control of the arm mechanism unit 13. Therefore, only the image resolution switching process will be described below. In the image resolution switching process, when controlling the movement of the arm mechanism unit 13 for positioning the hand unit 23 on the object 2, it is determined whether the work object, for example, the object 2 is moving or stationary, This is a process of switching the resolution of an image generated by the image processing unit 41 of the visual sensing information processing unit 36 according to the determination.

FIG. 10 shows a process flow in the image resolution switching process. In the image resolution switching process, the CPU obtains the initial position P ₀ of the object 2 as step 301 (operation target initial position acquisition processing). Then, in step 302, the position P of the object 2 after a predetermined time is acquired (work target position acquisition process after a predetermined time). Then, in step 303, it determines (position sameness judging) whether _{P 0} and P are the same. If the determination in step 303 is affirmative, it is determined that the object 2 has not moved and the image resolution is set to a high resolution. Specifically, the data from the visual sensor 21 is transferred to the image processing unit 41 without thinning out (step 304: high resolution processing). On the other hand, if the determination in step 303 is negative, the object 2 is moving and the image resolution is set to a low resolution. Specifically, the data from the visual sensor 21 is thinned out at a predetermined ratio and transferred to the image processing unit 41 (step 305: low image resolution processing).

  FIG. 11 shows a timing chart in the motion control loop of the arm mechanism unit 13. (A) in the figure is a case of low resolution processing, and (b) is a case of high image resolution processing. In the motion control of the arm mechanism unit 13, as described above, the position information of the object 2 generated by the image processing in the visual sensing information processing unit 36 is output to the motion integrated control unit 32, and the motion is based on the position information. Based on the control data (trajectory data) generated by the integrated control unit 32, the arm motion control unit 34 controls each drive unit 51 of the arm mechanism unit 13, and in the meantime, a hand that changes due to the motion of the arm mechanism unit 13 is controlled. It is repeated that the position of the unit 23 is captured by visual sensing or hand sensing, and the position information is fed back to the motion control of the arm mechanism unit 13.

  In such a motion control loop, the low-resolution processing (a) requires a short processing time in the visual sensing information processing unit 36. For example, when the work target moves such as when the robot apparatus 1 receives an article from a human hand or when the article gripped by the robot apparatus 1 is handed to a person, the movement of the dynamic work target is followed. However, it is necessary to acquire the position information. In such a case, by applying low resolution processing in which the processing speed of the visual sensing information processing unit 36 is more important than the resolution of the image, that is, the accuracy of the position information, work such as gripping on a dynamic work target is performed. The approach movement to the work object in can be performed more smoothly.

  On the other hand, in the high image resolution processing of (b), the processing time in the visual sensing information processing unit 36 becomes long. When the work target of the robot apparatus 1 is stationary, the processing by the visual sensing information processing unit 36 does not necessarily require high speed as in the case of a dynamic work target. In such a case, by applying high-image resolution processing that places greater importance on the accuracy of position information than the processing speed in the visual sensing information processing unit 36, the work target in a work such as gripping a static work target The approach movement to can be performed more smoothly. Here, in the high image resolution processing, as shown in the example in the figure, the processing time in the visual sensing information processing unit 36 becomes long, thereby causing a shift between the period of the visual sensing process and the period of the arm motion control. There may be a situation in which arm motion control is performed in a state where new position information cannot be obtained from the information processing unit 36. In such a case, arm motion control is performed using the previous position information while new position information cannot be obtained.

  Next, a third embodiment will be described. The present embodiment is characterized in a method for detecting a state in which positioning for gripping the object 2 in the gripping operation of the object 2 and the start of the gripping operation can be started. This is the same as the first embodiment. Therefore, only the positioning completion state detection method will be described below.

  Positioning the hand unit 23 in a state in which a gripping operation can be started with respect to the object 2 means that the hand unit 23 is in a state along the side surface of the object 2. In this state, the hand sensor 24 incorporated in the palm portion of the hand portion 23 is shielded by the object 2. That is, the image photographed by the hand sensor 24 becomes dark overall. Accordingly, by determining the image brightness (image brightness) by the hand sensor 24, it is possible to detect the shielding state of the hand sensor 24, that is, the positioning completion state. The positioning completion state detection method in this embodiment is based on such a principle.

  FIG. 12 shows a processing flow in the positioning completion state detection method based on the image brightness index. In the positioning completion state detection, an image of a work target is acquired as a step 401 (work target image acquisition process), and brightness data L of the work target image is acquired as a step 402 (lightness data acquisition process). When the brightness data L is acquired, in step 403, the brightness data L is compared with a brightness threshold value (lightness determination process). In the example shown in the figure, “0” is used as the brightness threshold value. If the determination result of step 403 is negative, the process returns to step 401 and is repeated. On the other hand, if the determination result in step 403 is affirmative, the positioning is complete, and therefore, in step 404, the arm mechanism unit 13 is commanded to stop operating (operation stop command processing).

  According to the positioning completion state detection method as described above, it is possible to cause the hand unit 23 to take a higher-accuracy positioning completion state, and thus the operation by the hand unit 23 can be performed more smoothly. Further, according to the positioning completion state detection method as described above, the hand sensor 24 can be used also for the positioning completion state detection, so that the sensor system in the robot apparatus 1 can be simplified.

  Next, a fourth embodiment will be described. The present embodiment is characterized in that the use control of the hand unit 23 can be performed when performing an operation such as gripping the target object 2 and is the same as the first embodiment except that. Therefore, only the hand portion use control will be described below.

  There are various sizes of the object 2 to be grasped, and there are cases where it is necessary to make “two-hand grasp” using both the left and right hand parts 23L and 23R for grasping, and the left and right hand parts There are cases where “one-handed gripping” with either one of 23L and 23R is sufficient. When “single-handed grip” is sufficient, “single-handed grip” can be used to make the operation smoother. In the case of “single-handed grip”, the left and right hand portions 23L and 23R can be used properly for the object 2 according to the situation, so that the work can be performed more smoothly.

  From this point of view, in the present embodiment, the hand unit separate control function is incorporated in the robot apparatus 1. FIG. 13 shows a flow of processing in the hand portion use control. In the hand portion use control, as step 501, an image of a work target is acquired by the visual sensor 21 (work target image acquisition process). In step 502, the size of the work target is estimated based on the acquired work target image (work target size estimation process). In the illustrated example, the volume V is used as the size of the work target. The process for estimating the volume V or the like of the work target is a general image processing such as extracting a work target area from the work target image by edge detection or the like and estimating the volume V or the like based on the extracted area. It can be done with a technique.

  When the estimated volume V is obtained, it is determined in step 503 whether the estimated volume V is equal to or greater than a volume product threshold value (size threshold value) (size determination process). If the determination result in step 503 is affirmative, it means “two-handed gripping”, and therefore the process proceeds to step 504 to instruct “two-handed gripping” (two-handed gripping instruction processing). Specifically, an operation start signal for gripping work is transmitted to both arm mechanism units 13L and 13R. On the other hand, if the determination result in step 503 is negative, “one-handed grip” is indicated, and the process proceeds to step 505. In step 505, the left / right usage determination, that is, which of the left / right hand portions 23L and 23R is used is determined (right / left usage determination processing). In the example of the figure, the right / left use determination is performed based on the positional relationship of the work object with respect to the robot apparatus 1, and thus the positional relationship determination processing is performed. The determination of the positional relationship can be performed, for example, by determining the position of the center of gravity of the work object from the work object image and determining whether the position of the center of gravity is on the left or right of the center of the work object image. If it is determined in step 505 that the left side is to be used, a gripping work start signal is transmitted to the left arm mechanism unit 13L in step 506 (left one-hand gripping instruction process). On the other hand, if it is determined in step 505 that the right side is to be used, in step 507, a gripping work start signal is transmitted to the right arm mechanism unit 13R (right one-handed gripping instruction process).

  Next, a fifth embodiment will be described. The present embodiment is characterized in that the control system is configured so that past work history can be used when performing work such as gripping the object 2, and the same as the first embodiment except for that. It is. Accordingly, only the control system for using the past work history will be described below.

  FIG. 14 shows a configuration of the control system 60 according to the present embodiment. The control system 60 of the present embodiment is similar to the system control unit 31, the motion integrated control unit 32, the travel control unit 33, the arm motion control unit 34, the head motion control unit 35, and visual sensing information similar to those in the first embodiment. A processing unit 36 and a hand sensing information processing unit 37 are provided, and a work history database 61 is further provided. The control system 60 can be used by being mounted on the robot apparatus 1 of FIG.

  In the work history database 61, data on work performed in the past is accumulated. For example, if the work history data is a gripping operation of the target object 2, the hand unit 23 is positioned with respect to the target object by the movement of the arm mechanism unit 13, and then the object 2 is gripped as one work unit. It is configured by combining data on the object 2 in the unit of work and actual data on motion control of the arm mechanism unit 13.

  FIG. 15 shows a specific configuration example of the work history database 61. The work history database 61 in this example includes an object shape pattern storage area 62, an arm motion data storage area 63, a correlation data storage area 64, an interface control unit 65, a pattern recognition processing unit 66, a search processing unit 67, and an address management unit. 68, and the shape pattern of the object 2 is used as data about the object 2.

  In the object shape pattern storage area 62, the shape pattern of the object handled in the past work is stored as shape pattern data 62D. In the arm motion data storage area 63, past performance data regarding motion control of the arm mechanism unit 13 is stored as arm motion control data 63D. The arm motion control data 63D is generated in correspondence with the shape pattern data 62D stored in the object shape pattern storage area 62. In the correlation data storage area 64, data relating to the association between the shape pattern data 62D and the arm motion control data 63D is stored as correlation data 64D. The correlation data 64D is configured to associate both data with the addresses of the shape pattern data 62D and the arm motion control data 63D. The interface control unit 65 functions as an interface when the exercise integration control unit 32 refers to data stored in the work history database 61. The pattern recognition processing unit 66 recognizes the shape pattern (new work object shape pattern) of the object in the work to be performed. The search processing unit 67 searches the object shape pattern storage area 62 for the new work object shape pattern recognized by the pattern recognition processing unit 66. The address management unit 68 performs address management of the storage areas 62, 63, and 64 for the object shape pattern, arm motion data, and correlation data.

  Hereinafter, the main points of the motion control of the arm mechanism unit 13 using the work history database 61 will be described. For example, when a work plan for gripping the object 2 is given from the system control unit 31 to the motion integrated control unit 32, the motion integrated control unit 32 sends the object 2 together with the position information of the target 2 from the visual sensing information processing unit 36. Get shape data. The motion integrated control unit 32 that has acquired the shape data of the object 2 accesses the work history database 61. Access to the work history database 61 is performed using the shape data of the object 2. The work history database 61 that has received the shape data of the object 2 via the interface control unit 65 recognizes a new work object shape pattern from the shape data by the pattern recognition processing unit 66. The new work object shape pattern recognized by the pattern recognition processing unit 66 is transferred to the search processing unit 67. In response to this, the search processing unit 67 accesses the object shape pattern storage area 62 and stores the shape stored therein. The pattern data 62D is searched for the same (or almost the same) as the new work object shape pattern. If the shape pattern data 62D is not identical to the new work object shape pattern, the result is transmitted to the motion integrated control unit 32. In this case, the motion integrated control unit 32 newly generates motion control data and sends it to the arm motion control unit 34. The newly generated motion control data is sent to the work history database 61 and registered as work history data.

  On the other hand, when the search processing unit 67 extracts the shape pattern data 62D as being the same as the new work object shape pattern, the search processing unit 67 sets the address of the arm motion control data 63D corresponding to the extracted shape pattern data 62D. The correlation data storage area 64 is searched, and the corresponding arm motion control data 63D is extracted from the address through the address management unit 68. The arm motion control data 63 </ b> D thus taken out is transferred to the motion integrated control unit 32 via the interface control unit 65. The motion integrated control unit 32 that has received the arm motion control data 63D sends the arm motion control data 63D to the arm motion control unit 34 as motion control data in the gripping operation of the object 2. The processing for retrieving the arm motion control data 63D by referring to the work history database 61 requires a shorter processing time than the processing in which the motion integrated control unit 32 newly generates motion control data. Therefore, according to the present embodiment in which the past work history can be used, the approach operation to the object 2 up to the grasping operation of the object 2 by the hand unit 23 can be performed more smoothly.

  Although some of the embodiments for carrying out the present invention have been described above, the present invention is not limited to these embodiments, and can be implemented in various forms without departing from the spirit of the present invention. it can. For example, each of the embodiments described above is for a robot device that travels and autonomously moves, but can also be applied to a walking robot device, and can also be applied to a robot device that does not have a moving function.

It is a figure which shows the whole external appearance structure of the robot apparatus by 1st Embodiment. It is a figure which shows the structure of the control system of the robot apparatus of FIG. FIG. 6 is a diagram illustrating an image of a process performed by a visual sensing information processing unit. It is a figure which makes the image of the process of a hand sensing information processing part into an image, and shows it. It is a figure which shows typically the structural example of an arm mechanism unit, and the structural example of a drive part. It is a block diagram about the motion control of an arm mechanism unit. It is a block diagram of the structural example of an arm motion control loop. It is a figure which shows the flow in an arm motion control loop. It is a figure which shows the flow of the motion control process of an arm mechanism unit. It is a figure which shows the flow of a process in an image resolution switching process. It is a figure which shows the timing chart in the motion control loop of an arm mechanism unit. It is a figure which shows the flow of a process in the positioning completion state detection method by an image brightness parameter | index. It is a figure which shows the flow of the process in hand part separate control. It is a figure which shows the structure of the control system in the robot apparatus by 5th Embodiment. It is a figure which shows the structural example of a work history database.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Robot apparatus 2 Object 13 Arm mechanism unit 21 Visual sensor 22 Arm part 23 Hand part 24 Hand sensor (side view sensor, hand part sensor)
36 Visual Sensing Information Processing Unit (Stereoscopic Sensing Information Processing Unit)
37 Hand Sensing Information Processing Unit 61 Work History Database 63D Arm Movement Control Data

Claims (7)

Based on an arm unit, a hand unit that is provided at the tip of the arm unit and performs an operation of gripping an object, a visual sensor that images the object, and a plurality of object images obtained by imaging with the visual sensor A visual sensing processing unit for acquiring position information of the object, and controlling the movement of the arm unit based on the position information of the object obtained by the visual sensing processing unit to In a robot apparatus including an arm motion control unit positioned at a gripping position ,
The position information of the object is acquired based on a hand part sensor that photographs the object on the side of the hand part that grips the object and a plurality of object images obtained by photographing with the hand part sensor. A hand sensing processing unit,
The arm movement control unit controls the movement of the arm unit to position the position of the hand unit at a position where the object is gripped, in the position information of the object obtained by the visual sensing processing unit. When the distance between the hand unit and the object is greater than or equal to a threshold value, the movement of the arm unit is controlled using the position information of the object obtained by the visual sensing processing unit, and the hand When the distance between the part and the object is less than a threshold value, the movement of the arm part is controlled using the position information of the object obtained by the hand sensing processing part by the hand part sensor. A robotic device characterized by In the robot apparatus according to claim 1,
The arm motion control unit uses the position information of the object obtained by the visual sensing processing unit to determine whether the object is a moving dynamic object or a static object that is stationary. When the object is a dynamic object, the object image used in the visual sensing processing unit is switched to a low resolution , and when the object is a static object, the object image is switched to a high resolution. features and be Carlo bot device. The robot apparatus according to claim 1 or 2,
The arm motion control unit, the brightness of the image obtained by photographing the object in the previous SL hand section sensor in the condition that falls below the threshold value, positioning the hand unit with respect to the object features and to Carlo bot device to determine that it is now completed state. The robot apparatus according to any one of claims 1 to 3,
The arm part and the hand part are respectively provided on the left and right,
The arm motion controller estimates a size of the object based at previous SL right and left hand portions sensor image obtained by photographing the object, if the estimated size is equal to or greater than the threshold value controls as a gripping use both the right and left hand portions, wherein when the estimated size is less than said threshold value, characterized a control child with a grip to use either one of the left and right hand portions and to Carlo bot apparatus. The robot apparatus according to claim 4, wherein
In the case of gripping using either one of the left and right hand parts , the arm motion control unit determines which of the left and right hand parts is the object based on the positional relationship of the object with respect to the left and right hand parts. robot apparatus according to claim to decide whether to use the grip. The robot apparatus according to any one of claims 1 to 5,
Furthermore, it includes a work history database capable of storing arm motion control data relating to exercise control of the arm mechanism unit including the hand portion and the arm portion for gripping the work performed in the past,
The arm motion control unit searches the work history database when performing a new gripping work, and when arm motion control data applicable to the gripping work to be newly performed can be extracted, the extracted arm set of motion features and to Carlo bot device rows Ukoto the control of the arm mechanism unit based on the motion control data. The robot apparatus according to claim 6, wherein
The arm motion control unit, the arm motion control characteristics and to Carlo bot device and Turkey be determined on the basis of the shape pattern of the object on the applicability of the information of the data.

Images