JP2017136677A - Information processing apparatus, information processing method, robot control apparatus, and robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately recognize a position and a posture of a target object even when the target object cannot be accurately recognized even by surface position information.SOLUTION: The information processing apparatus includes acquisition means for acquiring surface position information about an object, determination means for determining a measurement position for measuring the object based on the surface position information acquired by the acquisition means, measurement means for measuring contact position information when it contacts the object at the measurement position determined by the determination means, generation means for generating map information representing spatial position information about the object based on the contact position information measured by the measurement means, and recognition means for recognizing a position and a posture of the object based on the map information generated by the generation means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus, an information processing method, a robot control apparatus, and a robot system.

  2. Description of the Related Art Conventionally, a robot apparatus that uses a visual sensor to grasp the position and orientation of a target object is known as a technique for a robot to grip target objects that are randomly stacked. In Patent Document 1, a teaching model is created by imaging an object having the same shape as the target object from a plurality of directions, and matching is performed between the teaching model and image data obtained by imaging the stacked target objects. Recognizing the position and orientation of a target object is disclosed.

Japanese Patent No. 3300682

In measurement using a visual sensor, only information on the surface position of an imaged object can be acquired. Therefore, in the technique described in Patent Document 1, when the target object is wrapped with a plastic bag or a cushioning material, the target object in the image data does not match the teaching model. Therefore, in this case, the position and posture of the target object that is the contents wrapped in the plastic bag or the cushioning material cannot be properly recognized, and the robot cannot grip the target object.
Accordingly, an object of the present invention is to appropriately recognize the position and orientation of a target object even when the target object cannot be accurately recognized only by surface position information.

  In order to solve the above-described problem, an aspect of the information processing apparatus according to the present invention is an information processing apparatus that recognizes an object including a target object and a peripheral object, and obtains a surface position information of the object Based on the surface position information acquired by the acquisition unit, a determination unit that determines a measurement position for actually measuring the object, and contact when the object is contacted at the measurement position determined by the determination unit A measurement unit that measures position information; a creation unit that creates map information indicating spatial position information of the object based on the contact position information measured by the measurement unit; and a creation unit created by the creation unit Recognition means for recognizing the position and orientation of the object based on the map information.

  According to the present invention, even when the target object cannot be accurately recognized only by the surface position information, the position and orientation of the target object can be appropriately recognized.

It is a figure which shows the structural example of the robot system of 1st embodiment. It is a figure which shows the hardware structural example of information processing apparatus. It is a functional block diagram of the information processor in a first embodiment. It is a flowchart for demonstrating the operation | movement in 1st embodiment. It is a figure for demonstrating the operation | movement in 1st embodiment. It is a figure which shows the structural example of the robot system of 2nd embodiment. It is a functional block diagram of the information processor in a second embodiment. It is a flowchart for demonstrating the operation | movement in 2nd embodiment. It is a figure which shows the structural example of the robot system of 3rd embodiment.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.
The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed depending on the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment.
(First embodiment)
In the first embodiment, three-dimensional position information of an object measured based on image information of an object (target object and surrounding peripheral objects) obtained by imaging with an imaging device, and measurement information by a force sensor To create object map information. Then, the position and orientation of the target object are recognized using the created map information. In this embodiment, the target object is an object to be operated by the robot, and includes, for example, parts such as a plastic bag or a toner cartridge wrapped in a buffer material. Moreover, the measurement by a force sensor means measuring the contact position by a robot by a position detection sensor when it is detected that the robot is contacting an object by the force sensor. Further, the map information is spatial position information of the target object and surrounding objects.

FIG. 1 is a diagram illustrating a configuration example of a robot system 100 including an information processing apparatus 20 according to the present embodiment.
The robot system 100 includes a robot 10 and an information processing device 20. The robot 10 is an articulated robot, for example, and includes a manipulator such as a robot arm and an end effector 11 such as a robot hand. Further, the robot 10 includes a position / orientation changing mechanism that can change the position / orientation of the end effector 11 by changing the angle of each joint of the manipulator. The position / orientation changing mechanism may be driven by an electric motor, or may be driven by an actuator that operates with fluid pressure such as hydraulic pressure or air pressure. The position / orientation changing mechanism is driven in accordance with operation instruction information output from the information processing apparatus 20.

  The end effector 11 is a tool for realizing an operation according to the type of the target object of the robot 10. The end effector 11 has a chuck mechanism that can be driven by a motor and can hold the target object. A hand using a suction pad to be sucked can be used. The end effector 11 is detachably attached to the robot arm, and can be exchanged according to the type of workpiece. The robot 10 is not limited to an articulated robot, and may be a movable machine capable of numerical control (NC).

The robot 10 performs an action determined by the information processing apparatus 20 and performs an operation of transporting or grasping the target object 41. In the present embodiment, the action is an operation of the robot 10 for recognizing or operating the target object 41.
The target object 41 is a part such as a toner cartridge that is supposed to be gripped and transported by the robot 10 and may be wrapped in a plastic bag or a buffer material. That is, a peripheral object 42 such as a plastic bag or a cushioning material surrounding the target object 41 exists around the target object 41. The peripheral object 42 is all objects other than the target object 41 located around the target object 41. For example, the peripheral object 42 is placed in a container containing the stacked target objects 41 or around the container. Includes objects.

The robot system 100 further includes an imaging device 31, a light source 32, a force sensor 33, and a position detection sensor 34.
The imaging device 31 is a visual sensor including a camera, a sensor that detects light, a photodiode, and the like, and acquires image information of the target object 41 and the peripheral object 42. The imaging device 31 outputs the acquired image information to the information processing device 20. The light source 32 is configured by, for example, a projector, and projects uniform illumination light and pattern light onto the target object 41 and the peripheral object 42 by emitting visible light or emitting infrared light from a laser light source. . The imaging device 31 captures images of the target object 41 and the peripheral object 42 onto which light is projected by the light source 32. Note that the imaging device 31 can also capture images of the target object 41 and the peripheral object 42 in a state where light is not projected by the light source 32. As illustrated in FIG. 1, the imaging device 31 and the light source 32 may be fixedly disposed above the imaging target space, may be mounted on the robot 10, or may be mounted on another work machine. A plurality of imaging devices 31 may be arranged.

  The force sensor 33 is attached to the base of the end effector 11 of the robot 10. The force sensor 33 is configured by a strain gauge or a piezoelectric element, and measures the force (reaction force) applied to the robot 10 when the robot 10 is in contact with the target object 41 or the peripheral object 42 and is a measurement result. The haptic information is output to the information processing apparatus 20. The force sensor 33 may be a 6-axis force sensor (Force / Torque sensor) or a 3-axis force sensor. Further, the force sensor 33 may be a one-dimensional pressure sensor or a contact sensor that determines the presence or absence of contact.

  The position detection sensor 34 detects the position of the robot 10 and outputs position detection information as a detection result to the information processing apparatus 20. The position detection sensor 34 is configured to include an encoder attached to the robot 10, and can detect the position of the robot 10 by using forward kinematics based on angle information acquired from the encoder. The position detection sensor 34 can be used together with the force sensor 33 to detect the position of the robot 10 when the robot 10 is in contact with the target object 41 or the peripheral object 42. The method for detecting the position of the robot 10 is not limited to the method using the position detection sensor 34 described above.

The information processing apparatus 20 is a robot control apparatus that controls the operation of the robot 10. The information processing apparatus 20 is configured by, for example, a personal computer (PC). FIG. 2 is an example of a hardware configuration of the information processing apparatus 20. The information processing apparatus 20 includes a CPU 21, a ROM 22, a RAM 23, an external memory 24, an input unit 25, a display unit 26, a communication I / F 27, and a system bus 28.
The CPU 21 controls the operation of the information processing apparatus 20 in an integrated manner, and controls each component (22 to 27) via the system bus 28. The ROM 22 is a non-volatile memory that stores a program necessary for the CPU 21 to execute processing. The program may be stored in the external memory 24 or a removable storage medium (not shown). The RAM 23 functions as a main memory and work area for the CPU 21. That is, the CPU 21 loads various programs from the ROM 22 to the RAM 23 when executing the processes, and implements various functional operations by executing the programs.

The external memory 24 stores various data and various information necessary for the CPU 21 to perform processing using a program, for example. The external memory 24 stores various data and various information obtained by the CPU 21 performing processing using a program, for example. The input unit 25 is configured by, for example, a keyboard or a mouse pointing device, and an operator can give an instruction to the information processing apparatus 20 via the input unit 25. The display unit 26 includes a monitor such as a liquid crystal display (LCD). The communication I / F 27 is an interface for communicating with an external device. The system bus 28 connects the CPU 21, ROM 22, RAM 23, external memory 24, input unit 25, display unit 26, and communication I / F 27 so that they can communicate with each other.
In this way, the information processing apparatus 20 is connected to the robot 10, which is an external device, the imaging device 31, the light source 32, and the force sensor 33 via the communication I / F 27 so as to be able to communicate with each other. To control the operation.

FIG. 3 is a functional block diagram of the information processing apparatus 20.
The information processing apparatus 20 includes an image acquisition unit 201, a surface position measurement unit 202, a map information creation unit 203, a measurement position determination unit 204, and an action plan unit 205. Furthermore, the information processing apparatus 20 includes a force sense information acquisition unit 206, a force measurement unit 207, a position information acquisition unit 208, a position measurement unit 209, and a gripping position determination unit 210. The surface position measurement unit 202, the map information creation unit 203, the measurement position determination unit 204, the force measurement unit 207, and the position measurement unit 209 constitute a recognition unit 220 that performs processing for recognizing the position and orientation of the target object 41. . The recognition unit 220 can operate as an object recognition device.
The image acquisition unit 201 acquires image information output from the imaging device 31. The image acquisition unit 201 converts the acquired image information into image data and outputs the image data to the surface position measurement unit 202. The image acquisition unit 201 is configured by, for example, a capture board or a memory (RAM). The surface position measurement unit 202 measures information (surface position information) on the surface positions of the target object 41 and the peripheral object 42 based on the image data acquired from the image acquisition unit 201. The surface position measurement unit 202 outputs the measured surface position information to the map information creation unit 203.

  The map information creation unit 203 is based on surface position information measured by the surface position measurement unit 202, force information measured by a force measurement unit 207 described later, and position information measured by a position measurement unit 209 described later. To create map information. The map information creating unit 203 acquires force information measured by the force measuring unit 207 via the position measuring unit 209. The map information is spatial position information of the target object 41 and the peripheral object 42. In the present embodiment, the map information is position information of the target object 41 and the peripheral object 42 in a three-dimensional space. The map information creation unit 203 first creates map information based on the surface position information, and then updates the map information created based on the surface position information based on the force information and the position information. The correct map information.

  The surface positions of the target object 41 and the peripheral object 42 can be obtained from the surface position information measured based on the image data. However, as described above, when the target object 41 is a toner cartridge wrapped in a plastic bag, the surface position obtained from the surface position information is the surface position of the plastic bag, not the surface position of the toner cartridge. As described above, there are cases where the exact position and orientation of the target object 41 cannot be obtained from the surface position information measured based on the image data.

  Therefore, in this embodiment, in order to recognize the target object 41 more accurately, the target object 41 is directly touched by the robot 10 and the target object 41 is measured. Specifically, position information (contact position information) of the robot 10 when the robot 10 contacts the target object 41 is measured using the force sensor 33 and the position detection sensor 34. Then, the map information is updated using the measured position information of the robot 10. That is, the map information creating unit 203 actually creates the target object 41 using the force sensor 33 and the position detection sensor 34 with respect to the surface position of the target object 41 measured using the imaging device 31 when creating the map information. The result of the detailed position of the target object 41 measured by touching is added. In this way, the position and orientation of the target object 41 are measured in more detail. The map information creation unit 203 outputs the created map information to the measurement position determination unit 204 and the grip position determination unit 210.

  The measurement position determination unit 204 determines a measurement position for actually measuring the detailed position and orientation of the target object 41 based on the map information input from the map information creation unit 203. The map information input to the measurement position determination unit 204 is map information created by the map information creation unit 203 based on the surface position information. The measurement position determination unit 204 refers to the input map information and determines the position determined as having a high existence probability of the target object 41 as the measurement position. The measurement position determination unit 204 outputs the determined measurement position to the action planning unit 205.

  The behavior planning unit 205 plans the behavior of the robot 10 based on the measurement position input from the measurement position determination unit 204. Specifically, the action planning unit 205 creates a trajectory of the robot 10 for moving the end effector 11 to the measurement position and a trajectory of the robot 10 for realizing the contact operation to the target object 41 at the measurement position. The operation instruction information is output to the robot 10. The action planning unit 205 can also input the grip position of the target object 41 from the grip position determination unit 210 described later. When the action plan unit 205 inputs a grip position, the action plan unit 205 plans the action of the robot 10 based on the input grip position. Specifically, the action planning unit 205 creates a trajectory of the robot 10 for moving the end effector 11 to the gripping position and a trajectory of the robot 10 for realizing the gripping operation of the target object 41 at the gripping position. The operation instruction information is output to the robot 10.

  The force information acquisition unit 206 acquires the force information output from the force sensor 33 and outputs the acquired force information to the force measurement unit 207. The haptic information acquisition unit 206 is configured by, for example, a memory (RAM). Based on the haptic information input from the haptic information acquisition unit 206, the force measuring unit 207 applies a force (reverse force) applied to the robot 10 when the end effector 11 of the robot 10 is in contact with the target object 41 or the peripheral object 42. Force). The force measuring unit 207 outputs the measured force information to the position measuring unit 209.

  The position information acquisition unit 208 acquires the position detection information output from the position detection sensor 34 and outputs the acquired position detection information to the position measurement unit 209. The position information acquisition unit 208 is configured by a memory (RAM), for example. Based on the force information input from the force measurement unit 207 and the position detection information input from the position information acquisition unit 208, the position measurement unit 209 makes contact with the target object 41 or the peripheral object 42 by the end effector 11 of the robot 10. Measure the position of the contact point when The position measurement unit 209 outputs the measured position information (contact position information) to the map information creation unit 203.

The gripping position determination unit 210 determines the gripping position of the target object 41 based on the map information output from the map information creation unit 203. The map information input to the grip position determination unit 210 is detailed map information created by the map information creation unit 203 based on the force information and the position information. The gripping position determination unit 210 refers to the input map information and determines a position where the robot 10 can grip the target object 41 as a gripping position. The gripping position determination unit 210 outputs the determined gripping position to the action planning unit 205.
3 can be realized by the CPU 21 shown in FIG. 2 executing a program stored in the ROM 22 or the external memory 24.

Hereinafter, the operation of the robot system 100 will be described.
FIG. 4 is a flowchart illustrating a robot control processing procedure executed by the information processing apparatus 20. The process shown in FIG. 4 is realized by the CPU 21 of the information processing apparatus 20 reading and executing a program stored in the ROM 22 or the external memory 24. However, part or all of the processing in FIG. 4 may be realized by dedicated hardware. The process in FIG. 4 is started when the operator activates the robot system 100, for example. However, the start timing is not limited to when the robot system 100 is activated.

First, in S1, the CPU 21 performs a system initialization process. That is, the CPU 21 loads a program stored in the ROM 22 or the external memory 24, develops it on the RAM 23, and makes it executable. In addition, the parameters of each device connected to the information processing apparatus 20 are read and returned to the initial position to make it usable.
Next, in S2, the image processing unit 201 acquires image information of the target object 41 and the peripheral object 42 output by the imaging device 31, and the process proceeds to S3. In S3, the surface position measurement unit 202 measures the surface positions of the target object 41 and the peripheral object 42 based on the image information acquired by the image processing unit 201 in S2. Since the image information obtained from the imaging device 31 is information of a three-dimensional point group, the surface position measurement unit 202 recognizes the surface irregularities of the target object 41 and the peripheral object 42 based on the acquired image information. Can do. Note that the surface positions of the target object 41 and the peripheral object 42 may be determined with reference to a parts database (parts DB) stored in advance in the external memory 24 or the like. The component DB holds information on the component type and shape of the target object 41 that is the recognition target. The shape information of the target object 41 stored in the component DB is three-dimensional model data such as CAD data or CG data (polygon data). Note that the information on the shape of the target object stored in the component DB may be configured from a set of two-dimensional images obtained by observing the target object 41 and the peripheral object 42 from multiple directions.

  In S4, the map information creation unit 203 creates map information based on the surface position information measured by the surface position measurement unit 202 in S3. Hereinafter, a method for creating map information will be specifically described. In order to create map information, it is necessary to drop information obtained from a plurality of different sensors into one piece of spatial map information. However, the imaging device coordinate system of the imaging device 31, the force sensor coordinate system of the force sensor 33, and the robot coordinate system of the robot 10 are all different. Therefore, it is necessary to unify these coordinate systems.

First, a world coordinate system Σw is set as a reference coordinate system in the work space. The displacement from the world coordinate system Σw to the robot coordinate system Σr is defined as (RX, RY, RZ). Further, a 3 × 3 rotation matrix representing the posture of the robot 10 is defined as RM.
The displacement from the robot coordinate system Σr to the tip coordinate system Σf of the robot 10 is (FX, FY, FZ). Further, the 3 × 3 rotation matrix representing the posture of the tip of the robot 10 is FM. Further, the displacement from the tip coordinate system Σf to the coordinate system Σe at the tip of the end effector 11 of the robot 10 is (EX, EY, EZ). Further, the 3 × 3 rotation matrix representing the attitude of the tip of the end effector 11 is EM. The tip of the end effector 11 is a portion that contacts the target object 41 or the peripheral object 42.

The displacement from the tip coordinate system Σf of the robot 10 to the force sensor coordinate system Σt is (TX, TY, TZ). A 3 × 3 rotation matrix representing the posture of the force sensor 33 is TM. Further, the displacement from the robot coordinate system Σr to the imaging device coordinate system Σc is (CX, CY, CZ). A 3 × 3 rotation matrix representing the attitude of the imaging device 31 is CM. Further, the displacement from the imaging device coordinate system Σc to the target object coordinate system Σv is (VX, VY, VZ). A 3 × 3 rotation matrix representing the posture of the target object 41 is VM. Here, the displacement of the target object 41 viewed from the world coordinate system Σw is (WX, WY, WZ), and the 3 × 3 rotation matrix representing the attitude is WM.
When the robot 10 is in contact with the target object 41, the position of the tip of the end effector 11 attached to the tip of the robot 10 matches the position of the target object 41 photographed by the imaging device 31. Therefore, the following formulas (1) and (2) hold.

なお、ワールド座標系Σｗから見た、接触動作時にロボット１０にかかる力を４×４の行列ＷＴとおくと、行列ＷＴは以下の式で求まる。この（３）式を用いて、力の方向をワールド座標系Σｗで考慮することができる。

If the force applied to the robot 10 during the contact operation as seen from the world coordinate system Σw is set as a 4 × 4 matrix WT, the matrix WT is obtained by the following equation. Using this equation (3), the direction of force can be considered in the world coordinate system Σw.

The map information creation unit 203 creates map information based on the above definition. That is, the map information creation unit 203 uses the above equations (1) to (3) to unify the coordinate systems of different sensors, and when the coordinate systems are unified, use information obtained from each sensor. To create map information.
In S4, the map information creation unit 203 first arranges voxels in the work space of the world coordinate system Σw. Here, the region where the voxel exists is a region where the target object 41 or the peripheral object 42 may exist. Further, the size of the voxel is appropriately set according to the accuracy required for the gripping operation of the target object 41.
Next, the map information creation unit 203 performs a process of cutting all voxels existing outside the surface positions of the target object 41 and the peripheral object 42 using the surface position information. The surface position information is three-dimensional point cloud data, and this surface position information is obtained as information of the imaging device coordinate system Σc. Therefore, the surface position of the target object 41 is expressed in the world coordinate system Σw using the above equation (2). Can be converted. When a point group exists in a voxel arranged in the world coordinate system Σw, the voxel is a target to be cut. The area where the voxel is cut is processed as a part where no object exists. As described above, map information indicating a space where the target object 41 and the peripheral object 42 are approximately present is created.

  In S5, the measurement position determination unit 204 determines the measurement position of the target object 41 based on the map information created in S4. Specifically, the measurement position determination unit 204 refers to the map information, and selects a portion having a convex surface based on information on the surface irregularities of the target object 41 and the peripheral object 42. Then, the selected part is determined as a measurement position for measuring the target object 41 in detail, assuming that the target object 41 has a high existence probability. When the position of the target object 41 can be estimated using the component DB, the measurement position may be determined based on the estimated position. As described above, when the rough position and orientation of the target object 41 can be estimated from the component DB, the measurement position for directly checking whether the estimated position and orientation are correct is determined.

  In S6, the action plan unit 205 determines an action plan of the robot 10 for realizing measurement based on the measurement position based on the measurement position determined by the measurement position determination unit 204 in S5. Next, in S7, the action plan unit 205 outputs the operation instruction information to the robot 10 based on the action plan determined in S6. Thereby, the robot 10 moves the end effector 11 to the measurement position, and performs a contact operation on the target object 41 at the measurement position. The operation of the robot 10 at this time is not only the movement of the end effector 11 to the surface position measured by the surface position measurement unit 202 but also the acquisition of force information while moving deeper than the surface position. As shown in FIG. 5A, when the target object 41 is wrapped in a peripheral object 42 such as a plastic bag or a cushioning material, the end effector 11 crushes the peripheral object 42 as shown in FIG. Moving.

  In S <b> 8, the force information acquisition unit 206 acquires force information output from the force sensor 33. The position information acquisition unit 208 acquires the position detection information output from the position detection sensor 34 in synchronization with the acquisition of the force information by the force information acquisition unit 206. In S9, the force measurement unit 207 measures force information indicating the force applied to the robot 10 based on the force sense information acquired by the force sense information acquisition unit 206 in S8. Further, the position measurement unit 209 measures position information indicating the position of the robot 10 based on the position detection information acquired by the position information acquisition unit 208 in S8.

  The force information and the position information may be repeatedly measured while the robot 10 performs a contact operation on the target object 41. In this case, in the flowchart shown in FIG. 4, S7 to S9 are written as a one-way flow, but the processes of S7 to S8, S8 to S9, S9 to S7, and S7 to S9 are repeated at high speed. That is, the robot 10 acquires force sense information / position information while moving, and performs force measurement / position measurement. Furthermore, not only S7-S9 but the map information of S10 may also be updated. In that case, instead of returning from S9 to S7, the process flow from S7 to S8, S8 to S9, S9 to S10, and S10 to S7 is repeated at high speed. Here, the contact operation is performed in a range in which the magnitude of the force applied to the robot 10 does not exceed a preset upper limit value. Whether or not the magnitude of the force applied to the robot 10 exceeds the upper limit value is determined based on the force information measured in S9. During the contact operation, the force applied to the robot 10 increases according to the contact force to the target object 41. Therefore, the upper limit value of the force is set to a force that does not cause a problem such as damage to the target object 41 by the contact operation. For example, when the target object 41 is a toner cartridge wrapped in a plastic bag, the force that does not crush the plastic bag and break the toner cartridge is set as the upper limit value.

When the robot 10 crushes the plastic bag and contacts the surface of the toner cartridge, and the force applied to the robot 10 reaches the upper limit value, the information processing apparatus 20 stops the contact operation of the robot 10. The position of the robot 10 at the time when the contact operation is stopped can be recognized as the actual surface position of the target object 41.
In S10, the map information creation unit 203 updates the map information created in S4 based on the force information and position information measured in S9, and creates new map information. In S10, the map information creation unit 203 uses the position information acquired in S9 as a result of the actual operation of the robot 10 at the measurement position. Then, the map information created in S4 is subjected to a process of cutting a voxel corresponding to the space in which the robot 10 has moved.

  At that time, the map information creation unit 203 uses the 3D model information of the robot 10 held in advance, and the position that the robot 10 occupies at each time based on the forward kinematics from the joint angle data (dedicated position). Calculate Similarly, for the end effector 11 part, the exclusive position is calculated from the position and posture of the tip of the robot 10 using the previously stored 3D model information of the end effect 11. As described above, the exclusive position can be calculated based on the data of each joint angle of the robot 10 which is position detection information obtained from the position detection sensor 34 installed in the robot 10. At that time, as in the coordinate system unification process described above, a translational vector of displacement is obtained from the length between links, a 3 × 3 rotation matrix is obtained from the joint angle, and forward kinematics is used. As a result, the three-dimensional position and orientation of the robot 10, that is, the displacement (FX, FY, FZ) from the robot coordinate system Σr to the tip coordinate system Σf of the robot 10 and the rotation matrix FM can be obtained.

  The exclusive position obtained as described above is calculated as position information in the world coordinate system Σw by unifying the coordinate system described above. Therefore, the voxel corresponding to the calculated exclusive position can be cut. Even when the robot 10 is in contact with the target object 41 or the surrounding object 42, the robot 10 continues to move when the force measured by the force measuring unit 207 is smaller than the upper limit value. Therefore, in this case, the map information creation unit 203 determines that the robot 10 is not yet completely in contact with the target object 41, and at the same time the robot 10 continues to move, performs a spatial sweep and scrapes the voxels. . When the target object 41 is wrapped with something like a plastic bag, the robot 10 continues to operate when the end effector 11 crushes the plastic bag in a space where only the plastic bag exists. Therefore, the area in the voxel corresponding to the space in which only the plastic bag exists is scraped.

When the robot 10 comes into contact with the target object 41 and the force applied to the robot 10 reaches the upper limit value, the robot 10 stops its operation at that time. At this time, the map information creation unit 203 stops the process of scraping the voxels, determines that there is no object in the space swept up to the contact location, and that there is an object at the contact position, and ends the creation of map information.
In creating map information, a method that does not arrange voxels may be used. For example, surface position information that is three-dimensional point cloud data may be converted into surface format data, and the surface positions of the target object 41 and the peripheral object 42 may be expressed using polygons. In this case, map information may be created by generating the surface based on the three-dimensional position information with respect to the space swept by the robot 10 using the surface position information created by the polygon as an initial value. In addition, when creating map information in two dimensions, map information may be created using a technique similar to the above on a two-dimensional plane using pixels instead of voxels.

In S11, the gripping position determination unit 210 determines whether or not the target object 41 can be gripped by the end effector 11 of the robot 10 based on the map information creation result in S10. If gripping is possible, it is determined that the measurement is to be terminated, and if gripping is impossible, it is determined that the measurement is to be continued.
Specifically, the position and orientation of the target object 41 to be grasped are estimated based on the map information, and the grasping is performed based on the estimated position and orientation of the target object 41 and the shape of the end effector 11 of the robot 10. Judgment is made. When the end effector 11 grips the side surface of the target object 41 from above, the end effector 11 needs to enter the side surface of the target object 41. In this case, first, based on the map information, it is determined whether or not there is a space on the target object 41 where the robot 10 can move. The presence / absence of space is determined based on whether or not voxels are removed from the map information.

  If there is a space, it is determined that the robot 10 can move over the target object 41. Then, whether or not there is a space for the end effector 11 to enter the side surface of the target object 41 is determined by an operation involving contact. Then, when there is a space for the end effector 11 on the side surface of the target object 41, the end effector 11 can grip the target object 41, so that the measurement is finished and the process proceeds to S12. On the other hand, when it is determined that the end effector 11 cannot hold the target object 41, the process returns to S5 and measurement is continued. As described above, the gripping position determination unit 210 determines a position having a space in which the end effector 11 as a gripping part of the robot 10 can be inserted as a position where the robot 10 can grip the target object 41.

  If it is determined in S11 that gripping is not possible, the process of determining the measurement position in S5 is performed again. At this time, the result of the process so far may be used. For example, in determining whether or not the gripping is possible in S11, the robot is operating. When determining the measurement position, the next measurement position is obtained from another measurement position. Specifically, it is assumed that there are a plurality of candidate positions in determining the current measurement position. At this time, as described in the description of S5, a portion where the existence probability of the target object 41 is high is determined as a measurement position using an uneven state of the surface. When this existence probability is used as an evaluation function, the position where this evaluation function shows the maximum value is determined as the measurement position in this measurement, but in the next measurement, the evaluation function is set to the next largest value after the maximum value. The indicated position is determined as the measurement position. These results may be included in the created map information. In this way, not only simply repeating the same process but also updating the result of the current process in the map information as needed to use it in the next process. By doing so, the robot 10 does not move many times to a position where it cannot be gripped. Further, the robot 10 can move in the order of the high possibility of gripping. Therefore, even if the target object 41 fails to be gripped in the first process, the probability that the target object 41 can be gripped in the second and subsequent processes increases.

  In S11, FIG. 4 describes “Can be gripped?”, But this is not only a determination of whether or not the robot can be gripped, but also a determination of whether or not the robot 10 can move to a position where a gripping operation can be performed. Including. When the robot 10 moves, there may be a case where the robot 10 cannot move to a position where a gripping operation is scheduled to be performed due to an unexpected failure or the like. In that case, it is determined that the grip is not possible, and the process of S5 is performed.

In S12, the gripping position determination unit 210 determines the gripping position of the target object 41 based on the map information created in S10. The grip position determination unit 210 determines the grip position of the target object 41 from the positions determined to be grippable in S11.
In S13, the action planning unit 205 determines an action plan of the robot 10 for realizing the gripping operation of the target object 41 by the gripping position determined in S12. When determining an action plan, the action plan unit 205 refers to an action database (action DB) stored in advance in the external memory 24 or the like based on the gripping position. The operation DB holds information on component types and work procedures. The robot action plan can be determined using a general route search algorithm. As the route search algorithm, for example, a sampling-based search method such as RRT (Rapidly-exploring Random Tree) or PRM (Probabilistic RoadMap) can be used. At this time, using the map information, three-dimensional position information such as where the target object 41 is and where there is an obstacle is input. By using such a method, the action planning unit 205 determines an action plan of the robot 10 that can hold the target object 41 while avoiding an obstacle. However, another method may be used as long as a similar result can be obtained.

Next, in S <b> 14, the action plan unit 205 outputs operation instruction information to the robot 10 based on the action plan determined in S <b> 13. As a result, the robot 10 moves the end effector 11 to the gripping position and performs a gripping operation on the target object 41 at the gripping position. The target object 41 held by the robot 10 is transported to a predetermined location in accordance with the work procedure accumulated in the operation DB.
In S <b> 15, the CPU 21 determines whether there is a next target object 41 to be recognized based on the image information acquired by the image acquisition unit 201. Then, the CPU 21 determines that the recognition process is terminated when the target object 41 does not exist, and terminates the process illustrated in FIG. 4. If the target object 41 exists, the CPU 21 determines that the recognition process is to be continued and performs S2. Return to.

  As described above, the information processing apparatus 20 determines a measurement position for measuring the position and orientation of the target object 41 in more detail based on the surface position information measured based on the image information obtained from the imaging apparatus 31. Then, the robot 10 is moved to the determined measurement position. Then, the robot 10 is actually brought into contact with the target object 41 at the measurement position, and the target object 41 is based on the force information and the position detection information measured using the force sensor 33 and the position detection sensor 34 during the contact operation. Measure the detailed position of. The position and orientation of the target object 41 are recognized by creating map information. As a result, even for the target object 41 wrapped in a plastic bag or cushioning material, the position and orientation of which cannot be accurately recognized only by the imaging device 31 can be appropriately recognized for the gripping operation. Therefore, the number of cases where the target object 41 can be gripped increases, and work efficiency can be improved.

In the present embodiment, the contact operation with respect to the target object 41 is performed once. However, when the target object 41 cannot be recognized by a single contact operation, the map information is created by repeatedly touching and measuring. Also good. Thereby, the recognition accuracy can be improved.
As described above, in the present embodiment, the surface position measurement unit 202 measures (acquires) the surface position information of the objects (the target object 41 and the peripheral object 42), and the measurement position determination unit 204 uses the surface position information. The measurement position for actually measuring the target object 41 is determined. The force measuring unit 207 and the position measuring unit 209 measure contact position information when the target object 41 is touched at the measurement position. The map information creation unit 203 recognizes the position and orientation of the target object 41 based on the surface position information and the contact position information.

  Thus, since the contact position information is measured by actually touching the target object 41, the detailed position of the target object 41 can be confirmed. Thereby, the position and attitude | position of the target object 41 which is the content wrapped in the plastic bag or the buffer material can be recognized appropriately. That is, even if the actual position and orientation of the target object 41 cannot be recognized only from the surface position information, the actual position and orientation of the target object 41 can be recognized. Therefore, the number of cases in which the target object 41 can be gripped by the robot 10 is increased, and work efficiency can be improved.

Further, the map information creating unit 203 creates map information indicating the spatial position information of the target object 41 based on the surface position information and the contact position information, and based on the created map information, the map information of the target object 41 is created. Recognize position and posture. As described above, the map information is created in consideration of the position information actually confirmed by touching the target object 41, so that the position and orientation of the target object 41 can be recognized appropriately.
Further, when creating the map information, the map information creating unit 203 first creates map information based on the surface position information, and then based on the surface position information based on the contact position information measured at the measurement position. Update the created map information. As described above, the map information is created by integrating the surface position measurement of the target object 41 by the imaging device 31 and the position measurement results by the force sensor 33 and the position detection sensor 34. Thereby, final map information can be created by appropriately supplementing information that is insufficient in the rough map information created based on the surface position information. Therefore, even if the position and orientation of the target object 41 cannot be obtained only by information acquired from the imaging device 31, the robot 10 can be operated by appropriately recognizing the shape of the target object 41.

Moreover, the measurement position determination part 204 detects the position where the surface is convex shape based on surface position information, and determines the detected position as a measurement position. Therefore, the target object 41 and the peripheral object 42 can be mixed, and a part having a high existence probability of the target object 41 can be selected and actually measured.
Further, the surface position measurement unit 202 acquires surface position information based on image information obtained by imaging the target object 41 and the peripheral object 42. Therefore, the surface position information can be easily acquired.

  The force measurement unit 207 measures (acquires) a reaction force when it comes into contact with the target object 41, and the position measurement unit 209 measures (acquires) the position of the contact point with the target object 41. Then, the position measurement unit 209 measures the contact position information by measuring the position of the contact point when the reaction force measured by the force measurement unit 207 reaches a predetermined upper limit value. Therefore, when the target object 41 is wrapped with a soft member such as a plastic bag, the member surrounding the target object 41 can be crushed and directly contacted with the target object 41 to measure contact position information. . In that case, if the force which does not destroy the target object 41 which is a crushing plastic bag or a buffer material is set as the upper limit value, the position and posture of the target object 41 can be accurately recognized.

  The gripping position determination unit 210 determines a gripping position at which the robot 10 can grip the target object 41 based on the position and orientation of the target object 41 recognized from the map information created by the map information creation unit 203. As described above, the gripping position determination unit 210 determines the gripping position based on the map information created in consideration of the position information actually confirmed by touching the target object 41. Therefore, a position where gripping can be performed with high accuracy can be determined, and the gripping operation by the robot 10 can be performed with high probability. At this time, the gripping position determination unit 210 determines a position having a space in which the gripping part of the robot 10 can be inserted as a gripping position, so that the gripping position can be appropriately determined.

(Second embodiment)
Next, a second embodiment of the present invention will be described.
In the second embodiment, a gripping force measurement sensor that measures gripping force at the time of gripping operation is further added to the first embodiment described above, and using the gripping force information measured by the gripping force measurement sensor, It is configured to create map information in more detail. Specifically, first, as in the first embodiment described above, the surface position information of the target object 41 and the peripheral object 42 is measured using the imaging device 31. Next, as in the first embodiment described above, the measurement position of the target object 41 is determined based on the surface position information, and the contact operation with respect to the target object 41 is performed at the measurement position. Then, during the contact operation, the position information when the robot 10 is in contact with the target object 41 is measured using the force sensor 33 and the position detection sensor 34, and the target object 41 is gripped based on the measured position information. Determine the position. In the present embodiment, gripping width information of the target object 41 is measured using the gripping force measurement sensor and the position detection sensor 34 during the gripping operation at the gripping position. The shape for gripping the target object 41 is recognized by creating map information by integrating the measurement results of the surface position information, the position information, and the grip width information.

  In the first embodiment, the target object 41 is recognized from information before the robot 10 grips the target object 41. Therefore, when the target target object 41 is mixed among a plurality of types of objects having different sizes, the target target object 41 cannot always be gripped. Therefore, in the second embodiment, the gripping width measured during the gripping operation is collated with the component DB, and the size (width) of the target target object 41 is referred to, whereby the object gripped by the robot 10 by the gripping operation is determined. It is determined whether or not the target object 41 is a target. Also, map information is created including during the gripping operation.

FIG. 6 is a configuration example of the robot system 100 according to the second embodiment. In FIG. 6, parts having the same configurations as those of the first embodiment described above are denoted by the same reference numerals as those in FIG. 1, and hereinafter, the parts having different configurations will be mainly described.
The robot system 100 according to the present embodiment is different from FIG. 1 in that a gripping force measurement sensor 35 is attached to the tip of the end effector 11 of the robot 10. The gripping force measurement sensor 35 is configured by a strain gauge or a piezoelectric element, measures the gripping force when the robot 10 is gripping the target object 41, and transmits gripping force information as a measurement result to the information processing apparatus 20 ′. Output. The gripping force measurement sensor 35 may be a 6-axis force sensor or a 3-axis force sensor. Further, the gripping force measurement sensor 35 may be a one-dimensional pressure sensor or a contact sensor that determines the presence or absence of contact.

FIG. 7 is a functional block diagram of the information processing apparatus 20 ′ in the present embodiment. The information processing apparatus 20 ′ illustrated in FIG. 7 is different from the information processing apparatus 20 illustrated in FIG. 2 in that a gripping force information acquisition unit 211 and a gripping force measurement unit 212 are added.
The grip force sense information acquisition unit 211 acquires grip force sense information output from the grip force measurement sensor 35 and outputs the acquired grip force sense information to the grip force measurement unit 212. The grasping force sense information acquisition unit 211 is configured by, for example, a memory (RAM). The gripping force measurement unit 212 measures the force applied to the robot 10 when the object is gripped by the end effector 11 of the robot 10 based on the gripping force sense information input from the gripping force sense information acquisition unit 211. The gripping force measuring unit 212 outputs the measured gripping force information to the position information measuring unit 209. A recognition unit that performs processing for recognizing the position and orientation of the target object 41 by the surface position measurement unit 202, the map information creation unit 203, the measurement position determination unit 204, the force measurement unit 207, the position measurement unit 209, and the gripping force measurement unit 212. 220 '.

FIG. 8 is a flowchart showing a robot control processing procedure executed by the information processing apparatus 20 ′. The process shown in FIG. 8 is the same as that shown in FIG. 4 except that the processes of S21 to S26 are added between the processes of S14 and S15 of FIG. Therefore, the following description will focus on the different parts of the process.
In S <b> 21, the grip force sense information acquisition unit 211 acquires grip force sense information output by the grip force measurement sensor 35. Further, the position information acquisition unit 208 acquires the position detection information output by the position detection sensor 34 in synchronization with the acquisition of the grip force sense information by the grip force sense information acquisition unit 211. Next, in S22, the gripping force measurement unit 212 measures gripping force information when the robot 10 grips the target object 41 based on the gripping force sense information acquired by the gripping force sense information acquisition unit 211 in S21. Further, the position measurement unit 209 measures the position of the tip of the end effector 11 when the robot 10 grips the target object 41 based on the position detection information acquired by the position information acquisition unit 208 in S21. That is, the position measuring unit 209 measures the grip width information of the target object 41 gripped by the robot 10.

  The gripping force information and the gripping width information are repeatedly measured while the robot 10 performs a gripping operation on the target object 41. The gripping operation is performed in a range where the magnitude of the force applied to the robot 10 when the target object 41 is gripped does not exceed a preset upper limit value. Whether or not the magnitude of the force applied to the robot 10 exceeds the upper limit value is determined based on the gripping force information measured in S22. During the gripping operation, the force applied to the robot 10 increases according to the gripping force on the target object 41. Therefore, the upper limit value of the force is set to a force that does not cause a problem such as damage to the target object 41 by the gripping operation. For example, when the target object 41 is a toner cartridge wrapped in a plastic bag, the force that does not crush the plastic bag and break the toner cartridge is set as the upper limit value.

When the robot 10 crushes the plastic bag and comes into contact with the surface of the toner cartridge, and the force applied to the robot 10 reaches the upper limit value, the information processing apparatus 20 ′ stops the gripping operation of the robot 10. The grip width by the end effector 11 of the robot 10 when the gripping operation is stopped can be recognized as the actual size of the target object 41.
In S23, the map information creation unit 203 updates the map information created in S10 based on the gripping force information and gripping width information measured in S22, and creates new map information. In S23, the map information creation unit 203 uses the grip width measured in S22 as information indicating the size of the target object 41. As described above, the map information creation unit 203 creates map information in consideration of grip width information obtained by actually gripping the target object 41, and recognizes the target object 41.

  The map information creation unit 203 first performs unification processing for each coordinate system. The unification of the coordinate system of the robot 10, the imaging device 31, and the force sensor 33 is as described in the first embodiment. Therefore, the coordinate system of the gripping force measurement sensor 35 will be described below. In the present embodiment, the displacement from the tip coordinate system Σf of the robot 10 to the gripping force measurement sensor coordinate system Σg is defined as (GX, GY, GZ). Further, a 3 × 3 rotation matrix representing the posture of the gripping force measurement sensor 35 is defined as GM. Here, assuming that the gripping force applied to the robot 10 during the gripping operation as seen from the world coordinate system Σw is a 4 × 4 matrix GT, the matrix GT is obtained by the following equation (4). Using this equation (4), the direction of the gripping force can be considered in the world coordinate system Σw.

When creating the map information, the map information creating unit 203 unifies the coordinate systems of different sensors using the above equation (4) and the above-described equations (1) to (3). Then, when the coordinate system is unified, the map information creation unit 203 creates map information using information obtained from each sensor. In S23, the map information creation unit 203 performs processing for adding the result of actually grasping the target object 41 to the map information created in S10.
In S24, the CPU 21 uses the map information created in S23 and the shape information of the target object 41 obtained by referring to the component DB, so that the object gripped by the robot 10 is the target target object 41. It is determined whether or not there is. In the determination, the possible width of the target object 41 is estimated in consideration of the deformation of the peripheral object 42 surrounding the target object 41. If the grip width of the target object 41 is within the estimated width, the CPU 21 determines that the object gripped by the robot 10 is the target object 41. If it is determined that the gripped object is the target object 41, it is determined that the operation is to be continued, and the process proceeds to S25. On the other hand, if it is determined that the gripped object is not the target object 41, the process returns to S12, the grip position is determined again, and the creation of map information is repeated.

In S <b> 25, the action plan unit 205 determines an action plan of the robot 10 for realizing an operation on the grasped target object 41. When determining an action plan, the action plan unit 205 refers to information on the part type and work procedure stored in the action DB based on the gripping position.
Next, in S26, the action plan unit 205 outputs the operation instruction information to the robot 10 based on the action plan determined in S25. Thereby, the robot 10 conveys the target object 41 gripped by the end effector 11 to a predetermined place in accordance with the work procedure accumulated in the operation DB.
Thus, in this embodiment, when the target object 41 is recognized and gripped by the same method as in the first embodiment described above, the gripping width of the object gripped using the gripping force measurement sensor 35 is determined. Measure. Based on the measured grip width, it is determined whether or not the gripped object is the target object 41, and map information is created in more detail. Thereby, even when there are a plurality of types of target objects 41 having different sizes, each of them can be recognized with high accuracy. Therefore, the number of cases in which different types of target objects 41 can be accurately grasped increases, and the possibility of continuing the work can be expanded.

  Specifically, the grip force measurement unit 212 measures grip width information when the target object 41 is gripped at the grip position, and the map information creation unit 203 further uses the grip width information to determine the position of the target object 41 and Recognize posture. Therefore, the shape for gripping the target object 41 can be recognized in more detail. For example, even when there are a plurality of types of target objects 41 having different sizes, the possibility of recognizing the position and orientation of the target object 41 with higher accuracy and continuing the operation by the robot 10 can be expanded. In addition, since the grip width when the target object 41 is actually gripped is measured, the size of the target object 41 can be measured with high accuracy. Therefore, it is possible to appropriately determine whether or not the gripped object is the target target object 41.

(Third embodiment)
Next, a third embodiment will be described.
In the third embodiment, in contrast to the first and second embodiments described above, when the end effector 11 is inserted in an occluded area, the created map information is used to interfere with the surrounding object 42. An action plan for the robot 10 can be made. Here, the processing procedure of the robot control is the same as the flowchart shown in FIG. 4, but the processing in S13 is different.
When the end effector 11 is inserted into a region where occlusion occurs, for example, as shown in FIG. This is the case when it is shielded. In this case, as shown in FIG. 9B, a region A1 in which the target object 41 can be recognized and a region A2 in which the target object 41 cannot be recognized even when the imaging device 31 tries to capture the target object 41 are generated. In this state, even if the target object 41 can be gripped, when the target object 41 is transported and taken out, it interferes with the peripheral object 42 that cannot be imaged by the imaging device 31, thereby holding the target object being gripped. 41 may be dropped.

Therefore, in the present embodiment, the map information created and updated in S4 and S10 is used in the process of S13. The map information created until the target object 41 is gripped includes not only the target object 41 but also information on the peripheral object 42. For this reason, in the action plan when the robot 10 grips the target object 41 and then transports and takes it out, the space in which the robot 10 can move is limited in consideration of spatial information in which the peripheral object 42 exists.
When acquiring the position information of the peripheral object 42, a proximity sensor that detects the distance to the approaching object may be used instead of the force sensor 33. In this case, when the measurement object is in contact with the target object 41, the distance information to the target object 41 is directly acquired by the proximity sensor, whereby the contact position information can be measured and the map information can be created. When the proximity sensor is installed at a position close to the force sensor 33 shown in FIGS. 1 and 6, the proximity sensor is a distance from the base of the end effector 11 to the position of the contact point with respect to the target object 41 of the robot 10. Information can be acquired. Furthermore, if proximity sensors are arranged at a plurality of locations, map information can be created by measuring distance information at a plurality of locations at once. Therefore, when there is occlusion in the measurement by the imaging device 31, in the take-out operation after grasping the target object 41, the operation of the robot 10 that does not interfere with the peripheral object 42 around the target object 41 is planned in a shorter time. Can do.

Moreover, you may measure using both the force sensor 33 and a proximity sensor. By using both the force sensor 33 and the proximity sensor, map information is created by measuring a plurality of locations at once with the proximity sensor while performing position measurement with high accuracy by measurement by the force sensor 33. Can do. Thereby, the target object 41 and the peripheral object 42 can be recognized in a shorter time, and the action of the robot 10 can be appropriately planned. The proximity sensor may be an infrared sensor, or may be a magnetic type or ultrasonic type sensor.
By doing so, when the target object 41 held by the robot 10 is transported, the possibility that the target object 41 interferes with the surrounding object 42 during transport is reduced, and therefore the target object 41 may fall during transport. The working efficiency can be improved.

(Modification)
In each of the embodiments described above, when the robot 10 fails in the gripping operation, the gripping position determination unit 210 excludes the gripping position that has failed in the gripping operation from the gripping position candidates that are to be gripped next, and grips it from another location. The position may be determined. Thus, by creating the map information, it is possible to perform processing for preventing the same failure in the robot operation. Thereby, the working time by the robot 10 can be shortened.

  In each of the above embodiments, the material of the target object 41 is recognized from the force information when the robot 10 contacts the target object 41, and the target object 41 is gripped using the recognized information. Good. For example, when the target object 41 is wrapped in a plastic bag as in the above-described embodiments, a slight force is applied to the robot 10 when the robot 10 contacts the plastic bag. In that case, since the space where the plastic bag exists can be known from the force applied to the robot 10, the target object 41 may be operated (conveyed) by holding the plastic bag. As a result, even when the target object 41 itself cannot be gripped, the target object 41 can be operated and the possibility that the work can be continued can be improved.

  Further, in each of the above embodiments, after the detailed position measurement of the target object 41 using the force sensor 33, the surface position measurement of the target object 41 by the imaging device 31 may be performed again. When the force sensor 33 performs a plurality of measurements, the deviation may be accumulated due to sensor drift. Therefore, by appropriately measuring the surface position by the imaging device 31, the above-described deviation can be corrected and map information can be created. Thereby, since the influence by the shift | offset | difference of sensor drift can be suppressed and map information can be produced more correctly, possibility that an operation | work can be performed correctly can be improved.

Further, in each of the above embodiments, the measurement by the imaging device 31 may be initially performed a plurality of times, and the surface position information may be measured in more detail. Thereby, the frequency | count of measurement by the subsequent force sensor 33 or a proximity sensor can be decreased. As a result, the target object 41 can be recognized in a shorter time and the action of the robot 10 can be planned.
Further, in each of the above embodiments, the case has been described in which the target object 41 whose position and orientation are recognized is gripped and transported by the robot. However, if the purpose is to recognize the position and orientation of the target object 41, an operation by a robot such as gripping or transporting may not be performed. The effect of the present invention is that the position and orientation of the target object 41 can be properly recognized even when the target object 41 cannot be accurately recognized only by the surface position information. Therefore, information regarding the position and orientation of the recognized target object 41 is presented (displayed) to the operator, or the target object 41 is inspected based on the recognized position and orientation of the target object 41. Also good. Here, the inspection of the target object 41 is to confirm the shape of the target object 41 or to confirm whether it is in a desired position.

Further, based on the recognized position and orientation of the target object 41, the robot may perform an operation without gripping the target object 41. Here, the operation that does not involve gripping is an operation that touches the target object 41 without gripping and changes the position of the target object 41. In this way, when the robot performs an operation on the target object 41, the operation may be an operation with gripping or an operation without gripping.
Furthermore, the robot for acquiring surface position information and contact position information may be one or plural. Further, the robot for acquiring the surface position information and the contact position information may be the same as or different from the robot that performs an operation such as gripping or transporting the target object 41.

In each of the above embodiments, the case where the robot 10 is an assembly-type robot has been described. However, the robot 10 is not limited to an assembly-type robot, and may be a mobile robot.
Further, in each of the above embodiments, the case where the map information is three-dimensional information has been described, but the map information may be two-dimensional information. Further, the map information may be created based on only two pieces of information, that is, the force information measured by the force measuring unit 207 and the position information measured by the position measuring unit 209.
In each of the above embodiments, the imaging device 31 and the light source 32 may be a laser range finder device. Furthermore, stereo measurement may be performed using two imaging devices 31 without using the light source 32, or only one imaging device 31 may be used without using the light source 32. In other words, any configuration that can measure surface position information for recognizing the surface positions of the target object 41 and the peripheral object 42 may be used.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  DESCRIPTION OF SYMBOLS 10 ... Robot, 11 ... End effector, 20, 20 '... Information processing apparatus, 31 ... Imaging device, 32 ... Light source, 33 ... Force sensor, 34 ... Position detection sensor, 35 ... Gripping force measurement sensor, 41 ... Target object 42 ... peripheral objects, 100 ... robot system

Claims (17)

An information processing apparatus for recognizing an object composed of a target object and a surrounding object,
Obtaining means for obtaining surface position information of the object;
Determination means for determining a measurement position for actually measuring the object based on the surface position information acquired by the acquisition means;
Measuring means for measuring contact position information when contacting the object at the measurement position determined by the determining means;
Creating means for creating map information indicating spatial position information of the object based on the contact position information measured by the measuring means;
An information processing apparatus comprising: recognition means for recognizing the position and orientation of the object based on the map information created by the creation means. The creating means includes
First creating means for creating the map information based on the surface position information obtained by the obtaining means;
2. The information according to claim 1, further comprising: a second creation unit that updates the map information created by the first creation unit based on the contact position information measured by the measurement unit. Processing equipment. The determining means includes
The position according to claim 1 or 2, wherein a position where the surface of the object has a convex shape is detected based on the surface position information acquired by the acquisition means, and the detected position is determined as the measurement position. The information processing apparatus described. The acquisition means includes
The information processing apparatus according to claim 1, wherein the surface position information is acquired based on image information obtained by imaging the object. The measuring means includes
First acquisition means for acquiring a reaction force when contacting the object;
Second acquisition means for acquiring a position of a contact point with the object,
The position of the contact point acquired by the second acquisition unit when the reaction force acquired by the first acquisition unit reaches a predetermined upper limit value is measured as the contact position information. The information processing apparatus according to any one of claims 1 to 4. The measuring means includes
Comprising a third acquisition means for acquiring a distance to the object approached;
The information according to any one of claims 1 to 5, wherein the contact position information is measured based on a distance acquired by the third acquisition unit when the object is in contact with the object. Processing equipment.   7. The apparatus according to claim 1, further comprising second determination means for determining a gripping position at which the robot can grip the object based on the position and orientation of the object recognized by the recognition means. The information processing apparatus according to claim 1. The second determining means includes
The information processing apparatus according to claim 7, wherein a position having a space around which the grip portion of the robot can be inserted is determined as the grip position. Further comprising second measuring means for measuring grip width information when the object is gripped at the gripping position determined by the second determining means;
The recognition means is
9. The information processing apparatus according to claim 7, wherein the position and orientation of the object are recognized by further using the grip width information measured by the second measuring unit.   10. The robot control apparatus according to claim 1, further comprising a control unit that controls an operation of a robot that performs work on the object.   The robot control apparatus according to claim 10, comprising at least one of a sensor and the robot.   The robot system according to claim 11, wherein the sensor includes a visual sensor that images the object and acquires image information.   The robot system according to claim 11, wherein the sensor includes a force sensor that acquires a reaction force when the sensor comes into contact with the object.   The robot system according to claim 11, wherein the sensor includes a position detection sensor that acquires a position and a posture of the robot.   The robot system according to any one of claims 11 to 14, wherein the sensor includes a force sensor that acquires a reaction force when the object is gripped. Obtaining surface position information of an object composed of a target object and surrounding objects;
Determining a measurement position for actually measuring the object based on the acquired surface position information;
Measuring contact position information when contacting the object at the determined measurement position;
Creating map information indicating spatial position information of the object based on the measured contact position information;
Recognizing the position and orientation of the object based on the created map information.   A program for causing a computer to function as each unit of the information processing apparatus according to any one of claims 1 to 9.

Images