JP2003256025A - Robot motion teaching method and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To remotely teach complicated motions of a robot by using only a two-dimensional human interface. <P>SOLUTION: By precedently defining geometrical elements such as a trajectory in a format including a spacial constraint between the geometrical elements determined from requirements for objective work by a parametric modeling method and providing a library and menu of them, a variety of constraint conditions can be set in a few steps in an actual teaching phase, a necessity of an operator to constantly consider detailed spatial information such as depth during operation of the robot is eliminated, and fast teaching with little labor is provided. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot motion teaching method and device, and more particularly to a motion teaching method and device for remotely teaching a robot tool trajectory through a camera image.

[0002]

2. Description of the Related Art As a method of giving a motion trajectory to a robot arm for assembling or processing an object, (i) recording a robot's passing position by using a teaching box or the like, and directly moving the robot in that order. There are a teaching method and (ii) a method of inputting and programming position / orientation data in the work space by numerical values or mathematical expressions.

The direct teaching has the following problems. (1) It is difficult to teach highly accurate position / orientation using the teaching box. (2) In the case of remote work, it is difficult to remotely teach the robot's spatial position / orientation because depth information is missing only from the camera image. (3) Since each recorded position / orientation is given independently, it is difficult to satisfy a certain spatial relationship (for example, the constraint that the tool is always on the same plane inclined).

The teaching by the program has the following problems. (4) The positions and shapes of objects and obstacles in the work space coordinate system must be known in advance. (5) The operator cannot intuitively understand the teaching using numerical values or mathematical expressions.

In order to solve the above-mentioned problems, the present inventor filed an application in advance (see Japanese Patent Laid-Open No. 2001-60108), and image information of a work space in which a robot teaching operator is imaged by a camera. From this, by creating a simple 3D geometric element that directly or indirectly corresponds to the real space, a method for teaching the trajectory of the tool for the target work safely and easily away from the robot Proposed.

This is because the operator creates a simple geometric element based on the three-dimensional measurement information with the input device while looking at the display device on which the camera image obtained from the three-dimensional image measurement device is displayed. The robot system is taught by defining the coordinate system necessary for defining the work trajectory of the work robot and determining the trajectory using them.
Further, by using a measuring device driving device such as a robot arm for changing the position / orientation of the three-dimensional image measuring device, it becomes possible to give a highly accurate teaching trajectory over the entire working space.

[0007]

The method previously proposed by the present applicant is based on the premise that the orbits and the geometrical elements forming them are all defined online. Some geometrical elements should be indirectly defined due to their complicated relationship with many other geometrical elements, but such defining work had to be performed for each teaching of each work. for that reason,
Even if you want to teach the same work, you have to create each geometric element from the beginning. In addition, the more complicated the work trajectory, the higher the geometric knowledge and the deeper understanding of the desired work structure, which limits the number of workers who can teach.

Therefore, the present invention solves such a problem and uses a general device having a normal two-dimensional interface function as an interface device, but it is complicated and changes the trajectory depending on the situation. It is an object of the present invention to provide a means that can be taught quickly and easily.

[0009]

[Means for solving the problem] A trajectory is defined in advance by a parametric method in a format including spatial constraints between geometric elements determined by work requirements, and at the actual teaching stage, a trajectory is selected from a trajectory menu. A trajectory can be created by simply teaching the minimum necessary information that defines the trajectory, and the operator can operate it without considering the detailed depth at the time of execution, enabling high-speed and less fatigue teaching. To do.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION Regarding Motion Teaching System First, a system configuration example using the motion teaching apparatus of the present invention will be described with reference to FIG. The motion teaching device includes a data processing unit and a 2D interface unit, and teaches a motion of the manipulation mechanism unit according to an instruction from a worker to execute the work.

<About Manipulation Mechanism Section> As the manipulation mechanism section, a robot hand for holding an object as a work tool, a robot arm to which the hand is attached, and a visual device such as a stereo camera are mounted on a moving mechanism. Consider one with a different configuration.

<Regarding Data Processing Unit> The data processing unit includes a three-dimensional image measuring device, a geometric element generating device, a geometric element database, and a constraint condition solving device. The three-dimensional image measuring device measures the spatial position designated by the operator on the real space image obtained from the visual device. The relationship between the measurement space coordinate system by the three-dimensional image measuring device and the tool coordinate system set in the robot hand is known by calibration or the like, or can be successively estimated by camera or other sensor feedback information. To do. In addition to three-dimensional image measurement using a stereo camera as a visual device, measurement can be performed by moving the camera using a single camera to obtain an image of another viewpoint, measuring distance with focus, laser It is also possible to perform three-dimensional image measurement by utilizing the principle of a range finder using a light source such as.

The geometric element generation device instantiates the definition of various geometric elements including trajectories and the definition in the geometric element database according to a user's instruction from the 2D interface. The constraint condition solving device adjusts the parameters of the geometric element so that various constraints given by the user are satisfied. For example, when constraining the tool coordinate system on a specified trajectory or constraining it to move parallel to a line element or area element, rotate it around a line element or point element. There are cases such as restraint. Although the data processing unit is represented separately in FIG. 1, it may be configured to move together with the manipulation mechanism unit by being mounted on the movement mechanism of the manipulation mechanism unit.

<About 2D Interface> 2D
The interface unit is an interactive interface including a display device and an input device. The display device is an ordinary two-dimensional display, and the input device is an ordinary two-dimensional pointing device such as a mouse or a touch panel. The display device displays the processing content and the like in the data processing unit. Figure 2
Is a configuration example of the display screen. In the image presentation area, a real space image obtained from a visual device, which also serves as an input to the three-dimensional image measuring device, is displayed, and geometrical elements such as teaching trajectories in the model space can be displayed while superimposing the real space image. In the definition tree display area, the definition relationship of geometric elements is displayed in a tree format as described later, and the details of the selected geometric element are displayed in the detailed display area below. In addition to these, the display device displays a pointer, various menus such as a trajectory creation menu, and numerical data such as sensor information that the operator wants to know.

The input device enables movement of the pointer displayed on the display device and selection operation at the pointer position (depression of the mouse button, touch operation of the touch panel, etc.), thereby specifying the position on the image and selecting the menu. ,drag&
It is possible to perform drop operations, draw straight lines, curves, etc. In the present invention, since it is premised that the operator operates the robot via the 2D interface unit, it is not necessary to be able to directly see the actual working environment or the manipulation mechanism unit. A normal two-dimensional device can be used as the interface unit. Therefore, mobile phones and PDAs
A general mobile device having a two-dimensional interface such as (a mobile information terminal) and a wireless communication device can also be used, and it is possible to widely and easily teach the operation of the robot remotely. The details of the operation of the 2D interface unit will be described below.

(Library creation and menu of trajectories according to work) In order to parametrically define geometrical elements such as trajectories according to work, the target work is typified, and concretely what kind of constraint condition is used. Since it is necessary to consider what should be described, it must be done by a person who has sufficient geometric knowledge and can understand the essence of the intended work. However, at the stage of instantiating geometrical elements such as once defined trajectories, it is sufficient to have a limited knowledge about the outline of work and conditions necessary for instantiation, and it is necessary to understand the entire parametric definition of trajectories. There is no.

Therefore, according to the present invention, a person with sufficient knowledge defines geometrical elements parametrically according to the work, saves them as a library, and prepares them so that they can be displayed as a menu. It is possible to divide the teaching work into two stages, that is, an execution stage for executing a work satisfying various constraint conditions while selecting a geometric element from the library of the created parametric geometric elements.

In order to instantiate the parametrically defined geometric elements in the preparation stage in the execution stage, create as many local models of the environment and objects as necessary and select the desired definition from the menu. The assigned geometric element is assigned. At this time, since detailed modeling of the environment or the entire object is not required, it is possible to quickly and easily teach an unknown object or environment.

FIG. 3 shows an example of the configuration of the dropdown menu display. From the file menu, you can read the definition of geometric elements saved in the storage medium or write the defined geometric elements in reverse. Geometric elements are created by selecting the type of geometric element from the element creation menu and then selecting the definition method from the submenu. Select the target work from the work selection menu and instantiate the parametrically defined trajectory. Since the menu display becomes larger as the number of types of work increases, the selection becomes easier by classifying the library of geometric elements by the category of work or the type of element and displaying the menu hierarchically.
The data display method on the display device is set from the setting menu.

(Display of Definition Structure of Geometric Element) For details of the definition of the geometric element, the relationship between the geometric elements in the definition structure is graphically displayed in a tree format in the definition tree display area of FIG. 2 as shown in FIG. . Also, by selecting the geometric element icon on the tree display with the pointer, a specific geometric element in the definition structure is selected, and detailed data of the selected geometric element is displayed in the detailed display area of each geometric element definition in FIG. Display as in the example of No. 5. In each example of FIG. 5, underlined parts represent parameters that can be changed, and can be changed by an operator as appropriate.

The geometric element is a model space image in which the state in the three-dimensional model space is projected on a two-dimensional screen,
An overlay is displayed on the real image so as to correspond to the real image from the camera. FIG. 6 is an example of a camera image and an image in which a model space image is overlay-displayed on the camera image. Such a display makes it easier for the operator to understand the relationship between the real space and the model space. Even in the image presentation area, each geometric element can be selected by the selection operation on the model image by the pointer. The detailed data of the selected geometric element can be displayed and changed in the detailed display area as in the case of the definition tree display.

(Instancing by Drag and Drop of Parametrically Defined Geometric Element) A geometric element on the definition history of a certain geometric element is selected by pointing on the definition tree display or the model image display to select it. And drag it onto the neighborhood of another geometric element
By dropping, it is possible to newly generate a geometric element instance based on the geometric element of the drop destination or update an already existing instance. This makes it possible to easily generate a geometric element that is determined by a complicated constraint condition only by a very simple pointer operation such as selection from a menu and drag and drop.

[About Parametric Definition of Geometric Elements]
A parametric definition of a geometric element, which is an important concept in the present invention, will be described. In the field of CAD, a method called parametric design is adopted to facilitate the modeling of products. JIS B3401-19
In the 93 CAD terminology, parametric design is defined as "a design method for easily generating an internal model of a computer by classifying a shape of a product or a part thereof and giving it with parameters such as dimensions".

An example of a commercially available three-dimensional CAD using a parametric design method is Parametric Technol.
There is "Pro / ENGINEER" from ogy. Such a three-dimensional C
In AD, the shape model is defined as a set of constraint conditions, rather than simply trying to handle the shape. Here, the constraint is a condition that defines the size and positional relationship of the shape model, and includes a dimensional constraint that defines the size and dimensions and a geometric constraint that defines the positional relationship.

The dimensional constraint is to constrain the shape to the size of the dimensional value designated by the operator. If the operator changes the dimension value, the size of the shape data changes according to the instruction. In geometric constraints, line segments are parallel to each other, line segments and arcs are in contact (tangency is satisfied), endpoints are shared (continuity is satisfied).
Is a constraint condition that determines the positional relationship between the shape elements.

In the present invention, from the same idea as the parametric modeling of the product shape in CAD,
"A modeling method to easily generate a geometric model inside a computer by categorizing the definition of geometric elements, giving parameters such as position and distance, and describing the geometric relationships with other geometric elements. Will be called a parametric modeling method for geometric elements. In other words, instead of categorizing the model of the product, parametrically categorizing the definition of the robot motion trajectory and the geometrical elements used to define the trajectory makes it possible to easily command the desired robot motion. To Here, a constraint that defines a position, a distance, and the like with a numerical value is referred to as a numerical constraint, and a constraint that defines a relationship between geometric elements is referred to as a geometric constraint.

The characteristic of parametric modeling is that the definition history is held in the form of numerical constraints or geometric constraints. On the other hand, in normal geometric modeling, only numerical data such as coordinate values are held in each geometric element. For example, suppose that a line segment s is defined from the points a and b, and then a point c is defined as the midpoint of the line segment s. In normal geometric modeling, the point c is the coordinate value (x , Y,
It only holds the numerical data of z), not how it was calculated. Therefore, the data of the points a and b and the points c become irrelevant after the definition calculation, and even if the points a and b are changed, the points c are not affected. On the other hand, in the parametric modeling, it is possible to reflect the definition history of the geometric element and reflect the change of the point a or the point b to the point c automatically or as needed. In addition, a new point g can be easily created as a midpoint between the points e and f by simply assigning the points a and b to different points e and f using this data structure as it is.

As described above, since the parametric modeling method retains the definition history, it is very easy to correct geometric elements created through a complicated definition process and to create new geometric elements of the same type. Therefore, if you perform parametric modeling in advance of geometrical elements such as trajectories that include various numerical constraints and geometric constraints that are determined according to the intended work of the robot, you can minimize the complicated definitions online. The trajectory can be taught only by creating necessary geometric elements, and the trajectory can be easily corrected.

[Restriction] The operation teaching of the robot is the position of the coordinate system (tool coordinate system) set in the work tool,
This can be done by changing the posture. The coordinate system has 6 degrees of freedom, but by giving appropriate constraint conditions according to the work,
The degree of freedom can be reduced, and by updating the definition parameters of the coordinate system while the constraint condition solving device sequentially satisfies the constraint conditions, it becomes possible to easily control the motion of the robot using only the two-dimensional interface. .

An example of the constraint condition is translational or rotational constraint with respect to a reference geometric element. Constraints that move the coordinate system parallel to line and surface elements, and rotations around point and line elements. Generally, an arbitrary state under such a constraint can be designated by an operation input with two degrees of freedom.

As another constraint condition, a trajectory is given,
There is a method of constraining the coordinate system on the orbit. Since the coordinate system on the orbit can be expressed by a scalar variable by a method described later, an arbitrary state on the orbit can be designated by an operation input with one degree of freedom. As another example of the constraint condition, there is a method of giving a limit value to the values that the position and orientation can take and the amount of change over time. This makes it possible to prevent the position and orientation from deviating from within a certain area. For example, by specifying free space as that area, the operator can freely avoid the collision with the object while automatically avoiding collision with the object. You will be able to operate the system. Alternatively, by giving a limit value to the amount of change over time, it becomes possible to safely operate the robot while avoiding sudden movements.

[Regarding the Orbit] In the present invention, the orbit means
It refers to the path of a coordinate system that changes according to the value of one continuous scalar variable, and when the scalar value is given, the coordinate system is determined in a one-to-one correspondence. As a concrete orbital expression, a plurality of coordinate systems and their order are given, and their position vector and rotation vector are separately interpolated continuously by a linear function or a spline function, etc. A possible method is to find a path in a continuous coordinate system that is smoothly connected.

The operator freely modifies the value of this scalar variable online, changes it with time, and controls it with a feedback controller as a control variable to obtain the coordinate system, and the target of the tool coordinate system of the robot. It can be controlled as a value. The simplest example of the trajectory definition is a method of internally dividing two coordinate systems of a start point and an end point with a linear function. Specifically, when the start point coordinate system and the end point coordinate system are internally divided by the scalar parameter u that changes from 0 to 1, first, the position vector of each coordinate system and the rotation vector from the start point coordinate system of the end point coordinate system are calculated. A position vector obtained by internally dividing these position vectors by u and a rotation vector obtained by multiplying the rotation vector by the change rate of u (here, u
Change rate of (u- “u value in the reference frame of rotation”) /
(“U value in coordinate system after rotation” − “u value in reference coordinate system of rotation”), where (u-0) / (1-0) =
Since u is u, u itself is calculated, and the coordinate system determined by these position vector and rotation vector is used as the interpolation coordinate system. This interpolation coordinate system draws a trajectory that continuously interpolates between two coordinate systems according to changes in u. Similarly, when three or more coordinate systems are given, the position vector of each coordinate system is obtained in the same manner, and the rotation vector is the rotation vector from the immediately previous coordinate system. And then interpolate them to form a more complicated trajectory. Here, curve interpolation such as spline interpolation can also be applied to the position vector.

In this definition method, the trajectory is already expressed by the geometric constraint in that the trajectory is constructed from the relations with a plurality of given coordinate systems, and the trajectory is parametrically defined by this alone. However, by defining those coordinate systems by the parametric modeling method from the relationship with other geometric elements, it becomes possible to add various constraints to the trajectory according to the purpose. It eliminates the need for the person to operate while actually being aware of the motion constraint necessary for work in the actual operation process.

[About Geometric Element and Definition Method] Next, an example of a specific method for defining a general geometric element including a trajectory will be described. The types of geometric elements are points, straight lines, line segments,
Consider a curve, curved section, plane, plane area, curved surface, curved surface area, rectangular parallelepiped, cylinder, vector, coordinate system, orbit. Among these, a straight line, a line segment, a curve, and a curved section are collectively referred to as a line element, a plane, a plane region, a curved surface, and a curved surface region are collectively referred to as a surface element, a rectangular parallelepiped, and a cylinder are collectively referred to as a primitive element. However, the geometrical element is not limited to this, and the essence of the present invention does not change even if another element convenient for the intended work teaching is introduced. There are a direct definition method, an indirect definition method, and an associated definition method as a method of creating a geometric element. Each definition method will be described below.

<Direct definition method> The operator directly inputs the numerical constraint parameters of each geometric element (this is called a general direct definition method), or while using a three-dimensional image measuring apparatus as in the following example. The method of giving an object by directly measuring it is the direct definition method.

(A) From the point designated by the operator in the two-dimensional camera image displayed on the point definition display device, the three-dimensional position in the working space corresponding to the position of that point is measured by three-dimensional image measurement. Find it and define it as a point element.

(B) From the linear image edge traced by the operator in the two-dimensional camera image displayed on the straight line definition display device, the three-dimensional positions of a plurality of points on the traced image edge are three-dimensionally determined. Lines in space are obtained by image measurement, and they are approximated by least squares, and defined as line elements.

(C) From the area specified by the operator in the two-dimensional camera image displayed on the plane definition display device, three-dimensional positions of a plurality of points included in the area are obtained by three-dimensional image measurement, A plane in space is obtained by approximating them by least squares, and defined as a plane element.

(D) Definition of curved surface Similar to the definition of the plane, an approximate curved surface passing through the three-dimensional positions of a plurality of points in the area designated by the operator is obtained and defined as a curved surface element.

<Indirect Definition Method> A method of defining a new geometric element using a geometric element already defined is called an indirect definition method. A specific example is shown below. However, other than the following, various definition methods using geometric operations are possible. Further, it is also a method of a general indirect definition method that a geometrical element is translated, rotated, scaled or otherwise converted to create another geometrical element.

(A) Definition of points a. From the already defined surface element and line element specified by the operator, a point is defined as an intersection of them. I. A point is defined as an intersection of two straight lines on the same plane defined by the operator. (If the two straight lines are not completely on the same plane, the intersection point shall be approximated by the midpoint of the line segment connecting the shortest points of the two straight lines.) C. Corresponds to the position of a point on the two-dimensional camera image specified by the operator and the surface element already defined by the operator, and on the surface element and on the camera image. Define a point in space. (Because specifying a point on a two-dimensional camera image specifies a set of points that can be projected at that point in space, that is, a straight line, this is a special case of finding the intersection of a surface element and a straight line. This corresponds to the case.) D. The operator defines a line element that has already been defined and a point on the line in the two-dimensional display image of the line element, thereby defining the point on the actual line element that projected the point.

(B) Definition of straight line a. A straight line passing through the two points defined by the operator is defined. I. From the 2 planes already defined by the operator,
A straight line is defined as their intersection line. C. When a stereo camera device is used, a worker draws one straight line on each of two different two-dimensional camera images shown on the display device so that they are projected straight lines on each image. Define the straight line inside.
(Drawing a straight line on a two-dimensional camera image imposes a constraint on the plane in space, so this corresponds to the special case of finding the line of intersection of two planes.)

(C) Definition of line segment a. From the already defined two points designated by the operator, a line segment with those points as endpoints is defined. I. A line segment is defined as a section on the straight line by designating an already defined straight line designated by the operator and two points on the two-dimensional display image of the straight line.

(D) Definition of curve a. The curve is defined by applying a curve interpolation method such as a spline curve using a plurality of already defined points designated by the operator. I. A curve in space is defined by projecting a two-dimensional curve arbitrarily drawn by the worker on a plane or a plane region defined by the worker from the normal line direction of the plane. C. A curve in the space is defined by projecting a two-dimensional curve arbitrarily drawn by the worker from a vector direction specified by the worker, on a curved surface or a curved surface area specified by the worker.

(E) Definition of curve section a. The operator specifies a curve that has already been defined and two points on the two-dimensional display image of the curve, and the curve section is a section of the curve sandwiched by points in the space corresponding to the two points. Is defined.

(F) Definition of plane a. From the already defined three points designated by the operator, a plane including them is defined. I. A plane including them is defined from two intersecting straight lines which are defined by the operator and which are already defined. C. From the already defined lines and points specified by the operator, define the plane containing them.

(G) Definition of plane area a. An operator defines a plane which has already been defined and a closed area on the two-dimensional display image of the plane, thereby defining the plane area as a portion where the area is projected onto the plane. I. The operator designates two previously defined line segments that share one point as an end point, thereby defining a parallelogram region in a space having the two line segments as two sides as a plane region.

(H) Definition of curved surface a. From an already defined plurality of three or more points designated by the operator, an approximate curved surface passing through them is defined by applying an interpolation method. I. A curved surface is defined as a locus when one of the two curves defined by the operator is translated along the other curve.

(I) Definition of curved surface area a. An operator defines a curved surface area that has already been defined and a closed area on the two-dimensional display image of the curved surface, thereby defining the curved surface area as a portion projected on the curved surface.

(J) Definition of rectangular parallelepiped a. An already defined line segment designated by the operator is taken as one side, and a rectangular parallelepiped having a given three-sided ratio is superposed on the actual image at a given rotation angle around the designated line segment. The operator defines the rectangular parallelepiped by appropriately changing the lengths of the two sides given in advance and the rotation angle around the designated line segment while looking at the two-dimensional display image displayed as. I. While the operator defines the plane rectangular area defined by the operator as one surface and sees a two-dimensional display image in which a rectangular parallelepiped having a given height is displayed in an overlapping manner with the actual image, A rectangular parallelepiped is defined by changing the given height appropriately. C. A rectangular parallelepiped having three side lengths given in advance with the origin of the previously defined coordinate system specified by the operator as one vertex and three axial directions as the sides of the rectangular parallelepiped is displayed superimposed on the actual image. While watching the two-dimensional display image
The operator defines a rectangular parallelepiped by appropriately changing the three side lengths given in advance.

(K) Definition of a cylinder a. A two-dimensional display image in which a cylinder with a given diameter whose center point is an already defined line segment specified by the operator and whose both end points are the centers of the end faces of the cylinder is displayed superimposed on the actual image. While watching, the operator defines the cylinder by appropriately changing its given diameter. I. From a plane already defined by the operator and a point on the plane, a real image of a cylinder having a given diameter and height such that the plane is the end face and the point is the center of the end face While looking at the two-dimensional display image that is superimposed and displayed, the operator defines the cylinder by appropriately changing the diameter and height given in advance. C. When the stereo camera device is used, two two-dimensional straight lines corresponding to the edges on both sides of the cylinder that the operator wants to define are drawn on the two camera images, respectively, so that they become projected edges on the side surface. The cylindrical surface is defined by an approximate calculation from the geometrical conditions, and the operator sets the center of both end surfaces of the cylinder on the two-dimensional display image of the central axis of the cylindrical surface.
Define a cylinder by specifying the position of the points.

(L) Definition of vector a. A vector that connects the two points defined by the operator and their order is defined. I. From a previously defined straight line or line segment specified by the operator, a vector of length and orientation specified by the worker along the straight line or line segment is defined. C. From the already defined plane or plane area designated by the operator, a vector of the length and orientation designated by the operator in the normal direction is defined. D. From the already defined curved surface (or curved surface area, the same applies below) specified by the operator and the point on that curved surface, the length and direction specified by the operator in the normal direction at that point position on that curved surface Define a vector. E. A new vector is defined by an operator-specified operation such as a constant multiple, an inner product, an outer product, or the like, from one or more vectors that have already been specified by the worker.

(M) Definition of coordinate system a. From a previously defined point specified by the operator and two vectors that are orthogonal to each other, using that point as the origin, these two vector directions, and the vector direction obtained by the cross product of these two vectors as the main axis direction. Defines the coordinate system for the direction specified by the user. I. From an already defined point and plane (or plane area, the same applies below) specified by the operator and a straight line included in that plane, use that point as the origin, and the direction of that straight line is orthogonal to that straight line within that plane. A coordinate system having an orientation designated by the operator is defined such that the direction and the normal direction of the plane are the three principal axis directions. C. An already defined trajectory designated by the operator and an arbitrary scalar value between 0 and 1 define a coordinate system on the trajectory corresponding to the scalar value.

(N) Definition of orbit a. The trajectory is calculated by linearly interpolating or spline interpolating these coordinate systems from the multiple coordinate systems already defined by the operator and the scalar value that increases monotonically from 0 to 1 accompanying each coordinate system. Define. I. From the previously defined trajectory designated by the operator and the coordinate system and an arbitrary scalar value from 0 to 1, an trajectory added to the trajectory so as to pass through the coordinate system at the scalar value is defined.

<Attached Definition Method> When defining a geometric element by the direct definition method and the indirect definition method, it may be convenient later to define a geometric element that is closely related to the geometric element. . For example, a straight line is associated with a vector corresponding to its direction vector, a line segment is associated with two point elements corresponding to both end points thereof and a vector corresponding to a direction vector, and a plane is equivalent to a normal vector. Vectors etc. are attached, and eight points corresponding to each apex of the rectangular parallelepiped, twelve line segments corresponding to each side, six plane areas corresponding to each surface, and all the axial directions with the center of gravity as the origin. A coordinate system that is parallel to the sides of the cylinder is attached, and the cylinder has a line segment corresponding to its central axis, two plane areas corresponding to the top and bottom surfaces, and coordinates having an axis parallel to the central axis with the center of gravity as the origin. In some cases, a system or the like is attached, and a point corresponding to the origin of the coordinate system or a vector corresponding to each axial direction is attached.

[0057]

EXAMPLE Next, a specific example of teaching an object operation trajectory will be described. <Step 1: Parametric definition of trajectory, saving in library, menu creation> As a concrete example of work by a robot, consider picking and placing an object. Below is the trajectory that the tool coordinate system set for the hand should move from picking up the object to placing it (pick this
Call the place trajectory). Pick-and-place is a work in which an object is gripped in the initial coordinate system, lifted a little, moved in the air, brought closer to the target coordinate system, and then separated in the target coordinate system.

As an example of the configuration of such a trajectory, a coordinate system F2 slightly distant from a coordinate system (starting point coordinate system) F1 when picking up an object is passed, and a coordinate slightly above a target position. Consider a trajectory T1 that goes through a system F3 and then reaches a final target coordinate system (orbit end coordinate system) F4. F1 and F4 are coordinate systems specified by the operator, which are determined depending on the current state of the object and the target state. If F1 and F4 are decided, the coordinate systems F2 and F3 are selected from the framework of the pick and place work described above.
The typical case of and the trajectories passing through F1 to F4 can be automatically obtained by the following parametric modeling.

An example of parametric modeling of the trajectory T1 will be described step by step. (1) Let F1 be the starting point coordinate system. (It does not need to be instantiated in the model space, but it may be defined in any way in practice, but as an initial value, for example, the position of (100,300,350) in the reference coordinate system (world coordinate system, etc.) F0 is set. Set the origin as a coordinate system with no rotation, etc. (Use the general direct definition method.) (2) Similarly, set the end coordinate system as F4. (3) Appropriate length in the Z-axis direction of the reference coordinate system F0 (eg 200)
The vector of is defined as the vector V12. (Use of the general direct definition method.) (4) Define the vector V43 as in (3). (5) Convert the coordinate system F1 with the vector V12 to F2.
And (General use of translation.) (6) Coordinate system F4 translated by vector V43
And (General use of translation.) (7) Define the trajectory that passes through the coordinate system F1, F2, F3, F4 corresponding to the scalar value 0, 0.2, 0.8, 1 as T1. (Use of indirect definition method (n).)

The definition tree of the trajectory T1 is as shown in FIG.
Further, the detailed data of the geometric element can be displayed as shown in FIG. In this display, underlined parts are parameters that can be changed. The trajectory T1 thus defined is saved in the library by selecting Save Definition from the file menu of FIG. At the same time, T1 should be displayed in the work selection menu. The display name on the menu is input from a software keyboard or the like displayed on the display device.

<Step 2: Creation of coordinate system for pick and place> The coordinate system at the time of picking up an object and the coordinate system at the time of place are given by the operator when the work is executed. If those coordinate systems have already been modeled, the trajectory can be instantiated easily by the drag and drop operation described above. Create their coordinate systems first if they are not already modeled. Note that this coordinate system creating step may be performed after the temporary instantiation of the trajectory at the beginning of step 3.

The purpose of the work here is to move the pen stand on the table to the file case on the table in the situation as shown in the camera image of FIG. A method of defining the pickup coordinate system G1 and the place coordinate system G4 in the situation like the camera image of FIG. 6 will be described step by step. (1) A point corresponding to one corner of the table surface is designated on the camera image to define a point P1. (Use the direct definition method (a).) (2) Similar to (1), point elements corresponding to two other corners, P
Define 2, P3. (3) A plane S1 passing through the points P1, P2, P3 is defined. (Indirect definition method
(f) Use of a. (4) By designating the plane S1 and pointing the center position of the bottom surface of the pen stand on the camera image, the center of the bottom surface of the pen stand is defined as the point P4. (Use of the indirect definition method (a) c.) (5) Specify the plane S1 and the point P4 to define the cylinder C1 whose diameter and height are adapted to the pen stand. (Use of the indirect definition method (k) b.) (6) Translate and rotate as necessary to make it easier to grip the pen stand with the hand, with respect to the coordinate system of the center of gravity position defined along with the cylinder C1. The coordinate system to which is added is G1. (Use of the accompanying definition method and general translation and rotation.) (7) Trace one side of the bottom of the file case on the screen to define a straight line, and specify the end points of the side on the straight line to specify the line. Define the minute L1. (Direct definition method (b) and indirect definition method
(c) Use of b. ) (8) Specify the line segment L1 and give the rotation angle around the line segment and the side length so that it matches the shape of the file case
Is defined. (Use of indirect definition method (j) A.) (9) Define a cylinder C2 that is a movement of the cylinder C1 so that the bottom surface of the cylinder C1 and the top surface of the rectangular parallelepiped B1 coincide with each other. (General use of translation and rotation.) (10) G4 is the coordinate system in which the same translation and rotation as in (6) is added to the coordinate system of the position of the center of gravity defined with the cylinder C2. (Use of ancillary definitions and general translation and rotation.)

<Step 3: Generation of Trajectory Instance> From the work selection menu of FIG. 3, the parametric definition of the target work created in step 1 is selected.
At this point, the trajectory is provisionally instantiated, and the trajectory is displayed in the definition tree and the model space.
Then, the provisionally instantiated trajectory is reassigned to match the desired work trajectory. First, F1 is selected in the definition tree display or model space image display, and F1 is assigned to the actual pickup coordinate system by dragging and dropping it to the pickup coordinate system G1 defined in step 2. Similarly, F4 is assigned to the place coordinate system G4 by dragging and dropping. The other coordinate systems F2 and F3 that make up the trajectory are described by the geometric constraints with other geometric elements, so if F1 and F4 are determined, the trajectory T1 is automatically determined and the entire trajectory T1 is selected. It has been instantiated correctly for place work.

During the drag & drop operation, when the pointer reaches the vicinity of a geometric element that can be assigned by dragging, the geometric element is highlighted so that the geometric element can be dropped. If it is shown to the worker, the operation becomes easier. As described above, it is possible to easily instantiate the entire pick-and-place work trajectory by simply applying the coordinate system of the start point and the end point by the drag and drop operation. It is possible to actually move the tool coordinate system along the trajectory by, for example, operating the slider by a pointer operation to change the value of the scalar variable.

Although the movement from the pick coordinate system is considered here, before the pick and place trajectory is executed, a movement trajectory for approaching the hand from the current hand coordinate system to the pick coordinate system is required. After the place, a motion trajectory to move from the place coordinate system to another retracted coordinate system is required, but these can also be generated by the same procedure as the above pick & place trajectory.

<Step 4. Orbit change> Step 1,
A few steps are the basic work flow, but depending on the situation,
Sometimes we want the trajectory to be different from the one obtained from the pre-specified constraints. For example, if you want F2 to be diagonally above rather than just above F1 to avoid obstacles, or if you want to increase the distance further, or F2 and F3
If you want to add another coordinate system between, and so on.

For example, when it is desired to make F2 obliquely upward or change the distance, it is possible to select V12 from the definition tree display, change the value of the numerical constraint parameter, or replace it with another vector. To add another transit coordinate system, define a coordinate system G5 defined at a position that avoids obstacles, for example, a trajectory T2 that is added to the trajectory T1 as passing through a position of 0.5 on the trajectory (indirect definition). Law (n)
B) is used, and it becomes a pick and place trajectory with avoidance motion. After instantiating the parametric definition in this way, you can change the parameters of each geometric element involved in the definition, replace the geometric element itself with another, or use the instantiated geometric element to create a new one. The trajectory can be easily changed by creating geometric elements.

<Step 5. Creating a menu of changed trajectories> However, if the same change is needed each time, it is more efficient to teach it by saving it as a new parametric definition trajectory in the library so that you can select it from the menu. Go up. This operation can be performed by selecting the trajectory T2 changed in step 4 with the pointer, saving the definition from the file menu of FIG. 3, and displaying the menu. If you make a menu like this, by using T2 in the same situation, T1
There is no need to change after instantiating.

[0069]

EFFECTS OF THE INVENTION It is possible to easily remotely teach the movement and position of a robot in a six-dimensional space only by a normal two-dimensional interface consisting of a display and a pointer input means.

[Brief description of drawings]

FIG. 1 is a diagram showing a system configuration example using a motion teaching apparatus of the present invention.

FIG. 2 is a diagram showing a configuration example of a display display.

FIG. 3 is a diagram showing a display example of a drop-down menu.

FIG. 4 is a diagram graphically displaying a relationship between geometric elements in a definition structure in a tree format.

FIG. 5 is a diagram showing a display example of detailed data of a selected geometric element.

FIG. 6 is a diagram showing a display example of an image presentation area.

Continued front page    F-term (reference) 3C007 AS01 JS02 JS07 JU02 JU03                       JU12 JU14 KS03 KS04 KS17                       KT03 KT05 KT18 LS05 LS10                       MT01                 5H269 AB33 BB09 EE19 JJ20 KK03                       QB02 QC01 QC03 QC10 QD03                       QE02 QE07 QE10 QE22 QE26                       RB01 RB08 SA08 SA10 SA11

Claims (10)

[Claims] 1. A three-dimensional measuring device capable of measuring spatial coordinates corresponding to a point specified on a camera image, a real space image taken by a camera, and a model space image displaying a geometric model corresponding to the real space image. With a display that can be overlapped,
In a robot motion teaching method for defining a work trajectory by creating a simple geometric element corresponding to an actual space image in a model space by using a pointing device having at least two degrees of freedom, a parametric modeling method is used. By defining geometrical elements such as trajectories in advance, it is possible to create geometrical elements such as trajectories suitable for individual situations by only assigning some geometrical elements related to the definition and changing parameters. Method. 2. A database of geometric elements such as trajectories defined by the parametric modeling method,
The robot operation teaching method according to claim 1, wherein a geometric element such as a trajectory can be instantiated by making it selectable from a menu. 3. A geometric element such as a trajectory defined by the parametric modeling method, wherein any geometric element related to the definition of the geometric element can be replaced with another geometric element by a drag-and-drop operation. 3. The robot operation teaching method according to 1 or 2. 4. A parametric modeling of a geometrical element such as a trajectory defined by the parametric modeling method, wherein the geometrical element related to the definition of the geometrical element is translated, rotated, or aligned based on an arbitrary geometrical element. The robot motion teaching method according to claim 1, wherein the original geometric element defined by the method can be redefined. 5. A portable information terminal having a display capable of displaying two-dimensional information and a pointing device capable of two-dimensional input as a teaching interface and capable of exchanging information with another device by wireless communication is used. The robot operation teaching method according to claim 1. 6. A three-dimensional measuring device capable of measuring spatial coordinates corresponding to a point specified on a camera image, a real space image captured by a camera, and a model space image displaying a geometric model corresponding to the real space image. With a display that can be overlapped,
A robot motion teaching device that consists of a pointing device with at least two degrees of freedom and defines a work trajectory by creating a simple geometric element corresponding to a real space image in a model space. A robot motion teaching device in which geometrical elements such as trajectories suitable for individual situations can be created only by allocating some geometrical elements related to the definition and changing parameters by predefining geometrical elements such as . 7. A database of geometric elements such as trajectories defined by the parametric modeling method,
The robot motion teaching apparatus according to claim 6, wherein geometrical elements such as trajectories can be instantiated by making selections from a menu. 8. A geometric element such as a trajectory defined by the parametric modeling method, wherein any geometric element related to the definition of the geometric element can be replaced with another geometric element by a drag-and-drop operation. 6. The robot operation teaching device according to 6 or 7. 9. A parametric modeling of a geometrical element such as a trajectory defined by the parametric modeling method, wherein the geometrical element related to the definition of the geometrical element is translated, rotated, or aligned based on an arbitrary geometrical element. 9. The robot motion teaching apparatus according to claim 6, wherein the original geometric element defined by the method can be redefined. 10. A portable information terminal having a display capable of displaying two-dimensional information and a pointing device capable of two-dimensional input as a teaching interface and capable of exchanging information with another device by wireless communication is used.
The robot operation teaching device according to claim 6.