JP3823799B2 - Position and orientation control method by visual servo - Google Patents

Description

【００３７】
いま、例えば、上述の図５に示したＣＣＤカメラ１５の移動を角度を変えて８回分の画像データを取り込んだ場合について考える。一方の姿勢角ＲｘあるいはＲｙを固定して他方の姿勢角ＲｙあるいはＲｘについて変化させたときに、ワークＷのベクトルＶ１〜Ｖ５のそれぞれの比の値Ｌλ（λ＝１〜５）と、角度φ１〜φ５の比の値θλ（λ＝１〜５）との値を、式（１）に従って関数値ｆ（Ｒｘ，Ｒｙ）として求める。８回分の画像データに対応して８個の関数値ｆ１〜ｆ８が得られる。
【００３８】
ここで、得られた関数値ｆ１〜ｆ８をそれぞれ検出されている姿勢角ＲｘあるいはＲｙに対してプロットすると、図６に示すように、誤差値を示す関数値ｆ１〜ｆ８を結んだ曲線として得られる。この関数曲線は、参照画像を撮影した位置つまり基準位置を通るときにゼロになる筈である。ただし、固定している姿勢角がずれている場合や、測定の誤差を考慮すると、図中破線で示すようにゼロを取らない場合があるので、判定基準は、誤差値の関数値をゼロで判断するのではなく、許容誤差値以下であることを条件とする。
【００３９】
さて、ビジュアルサーボによる位置姿勢制御では、関数値ｆ４〜ｆ６に対応するデータはすべて収束した状態としてこれらの間の違いをＣＣＤカメラ１５の画像の違いとして認識することができないとすると、この範囲では姿勢角の検出精度が低いことになる。ここで、ビジュアルサーボによる位置姿勢制御で、例えば、ｆ５に対応した姿勢角を検出位置Ｒｄとして得られている場合を考えると、関数値ｆがゼロになる基準位置Ｒｏに対してΔＲだけずれていることになる。そして、このずれた量ΔＲを補正値として取り込んでメモリに記憶する。
【００４０】
このような補正値を姿勢角Ｒｘ，Ｒｙのそれぞれについて求めることで、得られた補正値の分だけ位置姿勢を修正することで、ビジュアルサーボによる位置姿勢の制御では得られなかった精度で、基準位置に略一致する位置姿勢とすることができるようになる。
【００４１】
図７及び図８は、ベクトルＶ１〜Ｖ５を定めて位置姿勢の検出を行なうことで、精度の向上を図ることができる根拠を示したものである。いま、ＣＣＤカメラ１５の位置Ｐから、距離ｄだけ離れた位置にある長さ２Ｌの線分ＡＢを撮影した場合について考える。ＣＣＤカメラ１５の光軸ｚに対して線分ＡＢが中点Ｍで直交する状態である場合には、ＣＣＤカメラ１５のスクリーン（距離ｆ）に右半分の長さＡＭ及び左半分ＢＭの長さは等しくＩとして得ることができる（下記の式（２）参照）。この場合、中点Ｍは前述の基準点Ｐ０に相当し、両端部Ａ，Ｂは特徴点に相当している。
【００４２】
また、この線分ＡＢが角度θだけ傾いた状態つまり相対的にＣＣＤカメラ１５の姿勢角がθだけ傾いた状態を想定すると、そのときＣＣＤカメラ１５で撮影した線分Ａ′Ｂ′のスクリーン上での画素数Ｉ′(pix) 、Ｉ″(pix) は下記の式 （３）、（４）で示される。
【００４３】
【数２】

【００４４】
上式中、ｓの値はＣＣＤカメラ１５の画像面上での実寸を画素に換算する計数であり、例えば、図８の計算をするに際して用いた値では、ＣＣＤカメラ１５の１画素サイズが０．００７４ｍｍ（７．４μｍ）であるから、画像面上で０．０７４ｍｍの長さがある場合には１０画素分の長さとして観測される。したがって、Ｉ(pix) をｍｍから画素に換算する計数ｓの値は１３５（１／０．００７４の近似値）となる。
【００４５】
上述の場合、線分Ａ′Ｂ′が遠ざかった右半分Ａ′Ｍの画素数Ｉ′(pix) は、少なく見えるようになり、近付いた左半分Ｂ′Ｍの画素数Ｉ″(pix) は一旦増加して見えるようになり角度θが大きくなるに従って減少するように変化していく。これらＩ′(pix) およびＩ″(pix) の値について、それぞれＩ(pix) の値との差を演算した結果ΔＩ′、ΔＩ″を、姿勢角θを横軸にとってプロットすると、図８に示すような曲線として得られる。また、図８には、線分の全体の画素数Ｉ′(pix) ＋Ｉ″(pix) について２Ｉ(pix) との差を演算した結果を、同様にしてプロットした結果も示している。
【００４６】
この図８のプロットから、線分を中点Ｍで分割したそれぞれについて画素数を比較すると、姿勢角θに対する変化量が全体の画素数について比較した値よりも大きい変化を示していることがわかる。つまり、図２に示したように、基準点を中心として特徴点との間を結んでできるベクトルＶ１〜Ｖ５のそれぞれについて比較をすることで検出能力を高めることができるのである。このことは、特徴点間を結んで得られるベクトルを定義して、これらについて同様の演算を行なうことでも同じ効果を得ることができる。
【００４７】
このような本実施形態によれば、ＣＣＤカメラ１５の位置姿勢を示す縦揺れ角および偏揺れ角に対応した姿勢データを、ビジュアルサーボによる位置姿勢制御が収束した状態で、ＣＣＤカメラ１５を所定角度以上傾けて得られる画像データから補正量を推定するようにしたので、センサなどを別途に設けることなくビジュアルサーボによる位置姿勢制御で精度を上げることが難しい姿勢角についての制御を精度良く行なうことができるようになる。
【００４８】
また、本実施形態によれば、ワークＷの参照画像として、基準点と複数の特徴点との間を結んで得られるベクトルＶ１〜Ｖ５およびそれらの間の角度φ１〜φ５を定義して、これらを補正量の算出に際して誤差値の関数として設定するようにしたので、分解能を高めて精度の良い補正量算出を行なうことができるようになる。
【００４９】
（第２の実施形態）
図９および図１０は、本発明の第２の実施形態を示すもので、以下、第１の実施形態と異なる部分について説明する。
この実施形態においては、ＣＣＤカメラ１５による図３のステップＳ３で行なわれると同様のＲｘ，Ｒｙ検出処理のための画像データの取り込みを、ビジュアルサーボによる位置姿勢制御が収束した後に行なうのではなく、図１０に示すように、ビジュアルサーボによる位置姿勢制御が収束するまでの間、つまりワークＷへのアプローチを行なっている期間中（ステップＰ１）に行なうようにしたところである。
【００５０】
図示のように、ビジュアルサーボによる位置姿勢制御を行なっている期間中（ステップＰ１）では、ＣＣＤカメラ１５をビジュアルサーボによる位置姿勢制御のために、角度αだけ大きく変化させながらアプローチさせる（ステップＰ２）。このようにしてアプローチしながらそのときＣＣＤカメラ１５により撮影した画像データを取り込んで記憶させ（ステップＰ３）、誤差が許容値以下になったら（ステップＰ４で「ＹＥＳ」と判断）、ビジュアルサーボを終了する。ここで角度αは、Ｒｘ，Ｒｙの検出処理に必要となる程度の縦揺れ角および偏揺れ角に相当する角度を示している。
【００５１】
次に、それまでに得られた画像データを元にして、Ｒｘ，Ｒｙの検出処理を実施する（ステップＰ５）。この検出処理の内容は、第１の実施形態で述べたと同様である。これによって補正量が算出されると、補正量による位置姿勢の制御を行なって目的位置に移動させる（ステップＰ６）。
【００５２】
このような第２の実施形態によっても、第１の実施形態と同様の効果を得ることができると共に、ビジュアルサーボによる位置姿勢制御のアプローチの間にＲｘ，Ｒｙ検出処理のための画像データを取得するので、アプローチ期間中の動作が若干遅くなるものの、収束位置に達した状態では、計算を行なうのみで迅速に位置姿勢の補正を行なうことができるようになる。
【００５３】
（第３の実施形態）
図１１は、本発明の第３の実施形態を示すもので、第２の実施形態と異なるところは、ＣＣＤカメラ１５の傾き角度をチェックして本来のビジュアルサーボによるアプローチで傾き角度がαよりも大きい場合には（ステップＰ７で「ＹＥＳ」と判断）、上述したステップＰ２をジャンプするようにしたところである。
【００５４】
これにより、必要以上にＣＣＤカメラ１５を傾けたりすることなく、迅速且つ正確に補正量の演算を行なうことができ、このような第３の実施形態によっても同様の作用効果を得ることができるようになる。
【００５５】
（第４の実施形態）
図１２および図１３は本発明の第４の実施形態を示すもので、上記各実施形態と異なるところは、図２に示したワークＷに対してこれとは形状の異なるワークＷａを対象とする場合の位置姿勢の補正を行なう場合を取り扱うようにしたところである。ここで、ワークＷａは、図１２に示すように、前述したワークＷよりもＸ軸方向とＹ軸方向との長さの比が大きく、上述した方法でＲｘ，Ｒｙを検出しようとすると、短い側の検出精度を上げることが難しくなるような条件のものである。
【００５６】
すなわち、図１２に示すワークＷａは上述した条件を示す模式的な形状を示しており、４個の特徴点ｐ１〜ｐ４が菱形の頂点に対応した位置にある。この場合、Ｙ軸方向の長さを示す特徴点ｐ１とｐ３との間の距離Ｌ１とＸ軸方向の長さを示す特徴点ｐ２とｐ４との間の距離Ｌ２との関係は、Ｌ１がＬ２よりも大きく、その比が所定以上となる形状である。
【００５７】
前述のように、距離Ｌ１はＸ軸回りの回転角Ｒｘの検出精度に関係し、距離Ｌ２はＹ軸回りの回転角Ｒｙの検出精度に関係するので、上記したようにＬ１，Ｌ２の長さの関係からすると、式（２）で示した関係を考慮すると距離Ｌ２の回転角Ｒｙの変位に対する変化量は小さく、距離Ｌ１の場合に比べて検出精度が低いことがわかる。極端な場合、距離Ｌ２がゼロに近いワークの場合には、実効的にはＹ軸回りの回転角Ｒｙの検出は不可能となる。
【００５８】
そこで、このような場合に対処するために、第１に、検出精度を向上させるための方法を提案すると共に、第２に、検出精度に限界があるような条件を認識するように設定して検出精度を高めることができない場合に処理時間を短縮することで、実用上の効果を得るようにした。
【００５９】
まず、上記した第１の方法について説明する。この場合には、図１２に示しているように、特徴点ｐ２とｐ４とを結ぶ線分の長さが短いので、そのままでは精度を上げることができない。そこで、このような場合には、あらかじめ設定した条件により、対象となるワークＷａの特徴点間を結ぶ線分から得られるベクトルが所定寸法以下（カメラ１５の撮像画像の範囲内における寸法で決める）のときに、検出精度の向上を図るために、カメラ１５の位置をビジュアルサーボで得られる基準位置よりもワークＷａに近付けて検出処理を行なう。
【００６０】
これにより、図１３に示すように、短い方の線分Ｌ２がカメラ１５の撮影画面内で長くとることができるようになり、検出精度の向上を図ることができるようになる。なお、この場合に、カメラ１５をワークＷａに近付けると、長い方の線分Ｌ１は撮影画面からはみ出してしまうので、先に前述同様の方法でＸ軸回りの回転角Ｒｘについて検出処理を行ない、この後、カメラ１５を近接させて図１３に示すような撮影画像を得て、Ｙ軸回りの回転角Ｒｙの検出処理を行なうことでＲｘ，Ｒｙの両者について検出を行なう。
【００６１】
この結果、Ｒｘ，Ｒｙの双方の値について検出精度を確保しつつ、適切な検出処理時間で行なうことができ、得られた結果に基づいて精度良く補正量を演算して求めることができる。
【００６２】
次に、第２の方法について説明する。これは、検出精度に限界がある場合に、第１の方法のようにして検出精度を上げることが難しい場合や、その回転角の姿勢データの精度が要求されないような場合に適用するものである。まず、このような条件が成立するか否かをあらかじめ設定した条件に基づいて判定し、検出精度を上げなくとも良いと判断された場合には、その回転軸に関する補正量の検出を行なわないようにする。
【００６３】
このことは、検出精度の向上が望めない状況で検出精度の向上のための検出処理を継続することが、いたずらに時間を要する結果とならぬように、あらかじめ判定した結果に基づいて検出処理を停止することで、処理速度の向上を図るようにしたものである。これにより、必要とされる精度のみの向上を図ることで、短時間で効率の良い検出動作を行なうことができるようになり、実用上で大きなメリットを得ることができるようになる。
【００６４】
（他の実施形態）
本発明は、上記実施形態にのみ限定されるものではなく、次のように変形また拡張できる。
第１の実施形態と第２の実施形態を複合的に行なうようにすることもできる。例えば、第２の実施形態のようにワークＷにアプローチする過程で角度を振って画像データを記憶しながらビジュアルサーボを行ない、収束した時点で補正量を求める場合の精度を向上させるために、さらに第１の実施形態のように画像データを取り込むようにすることができる。これにより、両者の利点を柔軟に取り入れて、迅速且つ正確な位置姿勢制御を行なうことができるようになる。
【００６５】
対象物の参照画像の登録において、ベクトルを設定する方法は、必要に応じて選択的に実施することができる。ベクトルを設定しない方法を採用しても、従来技術のビジュアルサーボによる位置姿勢の制御に比べて、位置姿勢の精度を向上させることは可能である。
【００６６】
上記各実施形態においては、Ｒｘ，Ｒｙの検出を行なうための誤差値を求める関係式として式（１）を用いているが、これに限らず、基準位置における位置姿勢との誤差をＲｘ，Ｒｙの関係において評価することができて、その誤差値を元にしてＲｘ，Ｒｙの検出を行なうことができる関係式であれば他の式を用いることもできる。
【００６７】
上記各実施形態において説明したロボット本体１１は、対象物であるワークＷとの相対的に移動していない場合を想定しているが、これに限らず、対象物が移動している場合や、ロボット本体が移動可能な場合あるいは、双方が移動している状態でも適用することができ、これらの場合のように相対的に移動する物体に対してその速度ベクトルを補間する処理を実施することで同様にして位置姿勢制御を精度良く行なうことができるものである。
【００６８】
さらに、制御対象装置としては、ロボットのみならず、位置姿勢を制御すべき車両や、対象物の形状や動作などを正確に認識する認識装置などについても適用することができる。
【図面の簡単な説明】
【図１】本発明の第１の実施形態を示す全体の概略構成図
【図２】ワークの基準点、特徴点の設定とベクトルの設定状態の説明図
【図３】ビジュアルサーボと補正量の算出の処理過程を示すフローチャート
【図４】ビジュアルサーボによる位置姿勢制御の概略説明図
【図５】ＣＣＤカメラのアプローチと角度を振る場合の作用説明図
【図６】カメラの姿勢角に対する誤差値の関数の値を示した図
【図７】基準点と特徴点との間のベクトルの傾きに対する変化を説明するための図
【図８】傾き角度と画素の関係を示す図
【図９】本発明の第２の実施形態を示す図５相当図
【図１０】図３相当図
【図１１】本発明の第３の実施形態を示す図３相当図
【図１２】本発明の第４の実施形態を示す基準位置からワークＷａを撮影している状態を示す作用説明図
【図１３】基準位置よりも近接した位置における図１２相当図
【図１４】従来例を示す対象物の説明図
【図１５】対象物の傾きと誤差値の説明図
【図１６】傾き角度に対する線分幅の変化を説明する図
【図１７】画素データの説明図
【符号の説明】
１１はロボット本体（制御対象装置）、１２は制御装置、１３はロボットコントローラ、１４は視覚装置、１５はＣＣＤカメラ（カメラ）、Ｗはワーク（対象物）である。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a position / orientation control method using visual servoing that controls position and orientation based on image data captured by a camera.
[0002]
[Problems to be solved by the invention]
There is a visual servo technique in which a workpiece is photographed by a camera attached to the hand and the position and orientation of the hand are controlled when the workpiece is gripped by a robot or the like. This is a method in which the position and orientation of the hand is controlled quickly and accurately by registering a reference image in advance and controlling the position and orientation so that the workpiece can be seen in the same manner as the reference image. In this case, the reference image is used when the robot hand is placed in a position where position / posture control is desired and the workpiece is gripped and taken with a camera attached to the hand.
[0003]
By adopting such visual servo technology, it is possible to accurately control the position and orientation within the range of resolution of the image that the robot's hand can be captured by the camera, but there is no technical problem at the research level. Although it seemed like this, inventors' research has revealed that the following problems occur in practical use.
[0004]
For example, this occurs when positioning accuracy is required in operations such as gripping an object with a robot hand or attaching parts. Due to the nature of the image taken by a CCD camera or the like, the number of pixels There is a limit to the improvement in accuracy due to the resolution. That is, assuming that the optical axis direction of the camera is the z-axis and the captured screen is the xy plane, in the visual servo, the position (x, y) in the xy plane obtained from the two-dimensional image data captured by the camera and around the z axis Although high-precision control can be performed with respect to the rotation (Rz), there is a limit to the accuracy with respect to rotation (Rx, Ry) around the x-axis or y-axis and correction in the depth direction (z). .
[0005]
For example, as shown in FIGS. 14 (a) and 14 (b), when photographing the objects P1 and P2 by the camera 1, with respect to a minute change in the pitch angle that is a rotation angle around the x-axis Rx, The change in the number of pixels of the CCD in the photographed image of the camera 1a (position tilted by a minute angle in the Rx direction) has a quantization error when it is changed from the position of FIG. 15A to (b) or (c). Therefore, there is a limit to the range that can be captured as a change on the image. In this case, the resolution is currently about ± 2 °, for example.
[0006]
In the objects P1 and P2, the height dimensions h1 and h2 are h1> h2, and the dimension in the depth direction (dimension in the z-axis direction) is larger in the object P1. On the other hand, the change amount Error1 is larger than Error2, and the change angle can be easily grasped. However, in the case of the object P2 having a small dimension in the depth direction, the correction accuracy of the change angle of the pitch angle is low (see FIG. 15 (d)).
[0007]
In the object P1 as shown in FIG. 16A, when the width dimension is D, the width dimension d on the photographed image with respect to the change in Rx is as shown in FIG. 16B. expressed. That is, this relationship is
d / D = cos (Rx)
Therefore, in the region where Rx is small, the amount of change with respect to the angle is small, and as shown in FIG. 17, the range that can be counted as the difference in the number of pixels pix is widened, and the resolution is reduced accordingly. . That is, in the visual servo, the accuracy of the pitch angle (pitch) and the yaw angle (yaw), which are elements of the posture angle, greatly depends on the quantization accuracy of the camera 1.
[0008]
In order to avoid the above-described problems and increase the resolution, for example, a sensor is provided separately from the camera 1 and the position and orientation control based on the detection signal of the sensor converges in a state where the control of the position and orientation by the visual servo has converged. It is necessary to improve control accuracy by performing correction.
[0009]
The present invention has been made in view of the above circumstances, and its purpose is to provide a position and orientation closer to the reference position of the reference image with the position and orientation control converged even when only the position and orientation control by visual servo is used. An object of the present invention is to provide a position and orientation control method using visual servoing that can be controlled with high accuracy.
[0010]
[Means for Solving the Problems]
According to the first aspect of the present invention, at least the pitching estimated from the data of the plurality of images stored in the second process is stored in the first process. Estimate the position and orientation corresponding to the reference position where the error data is below the allowable error amount from the correlation between the orientation data indicating the angle and yaw angle and the error data indicating the error between the image data and the reference image data In the third process, the correction amount to the position and orientation corresponding to the estimated reference position is calculated, and in the fourth process, the third position is obtained with respect to the position and orientation in the state where the position and orientation control by the visual servo has converged. Since the position and orientation of the control gymnastic apparatus are corrected based on the correction amount calculated in the process of, from the image captured by the camera, a region that is a dead zone of the pitch angle and yaw angle indicating the position and orientation, that is, In the position and orientation control of dual servo, even if it is in a converged state and is included in the quantization error and in a region where no difference appears as image data, the position and orientation can be further corrected to control the position and orientation more accurately. become able to. Thereby, the control accuracy of the position and orientation can be improved without providing a separate sensor or the like.
[0011]
Then, in the first process, with the position and orientation control by visual servo converged, the position and orientation of the camera are The posture data is moved so as to change more than a predetermined amount so that it appears as a change in the number of pixels. Since it is executed by storing the image when changing, the position and orientation control by visual servo is converged, that is, the position closest to the reference position closest to the target object From the first step, the position and orientation can be corrected by detecting the correction amount of the position and orientation accurately and quickly.
[0012]
Claim 2 According to the invention The second Since the process 1 is executed by storing the camera image during the period until the position and orientation control by the visual servo is converged, the position and orientation control by the visual servo is executed. The image captured by the camera in the process of approaching can be captured as the first process, the correction amount can be calculated in parallel with the visual servo control and processing, and the position and orientation can be controlled quickly. Will be able to.
[0013]
And When performing the first process, when controlling the position and orientation of the camera during the period until the position and orientation control by the visual servo converges If the change does not appear as a change in the number of pixels, the posture data changes by moving the camera so that the change in the position and orientation of the camera changes by a predetermined amount or more so as to appear as a change in the number of pixels. As described above, in the above case, if the camera does not actually shake much when approaching the object, that is, if the change amount of the posture data does not shake more than the determination angle for obtaining the correction amount, In order to perform the correction accurately, the control is performed so that the posture for capturing the image is taken. Therefore, the correction amount value is surely calculated even when approaching while performing the first process. Will be able to.
[0014]
Claim 3 According to the invention, in each of the above inventions, the image data of the target object stored as the reference image includes a plurality of feature points with respect to a reference point and a plurality of vectors connecting the reference points. When the second process is performed, the position / orientation is estimated based on error data calculated based on at least the length of vectors or the ratio of lengths between vectors. When there is a shift in the pitch angle or yaw angle of the camera posture data between the image captured by the camera and the reference image, the feature point on one side with respect to the reference point of the object is Unlike when judging based on the total length of the object, when the vector that can be connected becomes longer and the vector that is formed by connecting the feature point on the other side becomes shorter, the difference between the two is detected. ,Appearance To enhance the detection capability for data, it is possible to perform a high correction amount calculation accuracy.
[0015]
Claim 4 According to the invention, in each of the above inventions, it is assumed that the image data of the object stored as the reference image includes a plurality of feature points with respect to the reference point and a plurality of vectors connecting the feature points. When the second process is set, the position / orientation is estimated based on error data calculated based on at least the length of vectors or the ratio of lengths between vectors. As a result, the detection capability for posture data can be improved and the correction amount can be calculated with high accuracy.
[0016]
Claim 5 According to the invention of claim 3 and 4 In the invention, the image data of the object to be stored as the reference image is obtained by estimating the position and orientation of the plurality of set vectors based on the error data calculated based on the angle between the vectors. Therefore, the correction amount calculation process with high accuracy can be performed by more strictly comparing the reference image data with respect to the reference image data.
[0017]
Claim 6 According to the invention of claim 3 Or 5 In this invention, the error data is calculated as the sum of squares of the difference between the image taken for the length or angle of the vector and the reference image, and the position and orientation are estimated by the error data calculated by the above method. Since the posture data is used when obtaining the minimum value, the minimum value is obtained by using the value corresponding to the sum of the absolute values of the differences from the reference image as the function value for the posture data value as a variable. The minimum value of the function, that is, the posture data of the position and posture closest to the reference image can be obtained. In this case, the function described above may be plotted corresponding to the variable, and the calculation may be performed so as to obtain the minimum value thereof, or the variable in a range in which the function value is simply equal to or less than the allowable error value. It can also be obtained by adopting an intermediate value.
[0018]
In this case, the claim 7 As described above, the position / orientation estimation is performed by obtaining an error data function using the attitude data as a variable from the error data value and the pitch angle and yaw angle value of the attitude data. It is better to do this by calculating the value.
[0019]
Claim 8 According to the invention, in each of the above inventions, the first to fourth processes are performed by moving the camera position from the convergence position to a position close to the convergence position in the state where the position and orientation control by the visual servo has converged. Even if accuracy is not obtained in the range of the captured image at the convergence position, it is possible to obtain an enlarged captured image by capturing at a close position, thereby obtaining image data with improved accuracy with respect to posture data. As a result, the position and orientation can be corrected with higher accuracy. Further, this makes it possible to improve the position / orientation correction accuracy even in the case where there is posture data whose accuracy is difficult to increase due to the shape of the object.
[0020]
Claim 9 According to the invention of claim 8 In the present invention, in the state where the position / orientation control by the visual servo has converged, the attitude data that needs to be improved in detection accuracy in the attitude data is moved from the convergence position to a position close to the position, so that it is necessary. Only with respect to the posture data, by obtaining a captured image of the object for correcting the position and orientation while moving to a close position, accurate correction based on the enlarged image can be performed.
[0021]
Claim 10 According to the invention, in each of the above-described inventions, the determination operation for excluding the posture data whose shape of the target object cannot improve the detection accuracy from the posture data as the reference image data is excluded from the target to be corrected for the position and posture. When the position / orientation control by visual servo has converged, calculation processing for position / orientation correction is performed for those that are not excluded from the attitude data, so attitude data for which improvement in detection accuracy cannot be expected Thus, the position and orientation control process can be quickly performed without spending time for the useless detection process.
[0022]
DETAILED DESCRIPTION OF THE INVENTION
(First embodiment)
Hereinafter, a first embodiment in which the present invention is applied to an articulated robot for assembly will be described with reference to FIGS. FIG. 1 schematically shows an assembly robot system. This system includes a robot body 11 as a control target device and a control device 12 that controls the robot body 11. The control device 12 includes a robot controller 13, a visual device 14, and a CCD camera 15. Has been.
[0023]
The robot body 11 is configured as a 6-axis small vertical articulated robot for assembly. In the robot body 11, a shoulder portion 17 is provided on a base portion 16 so as to be rotatable (turnable) about a vertical axis, and a pair of lower arms 18 extending upward at both left and right end portions of the shoulder portion 16 are provided. It is connected so as to be rotatable about a horizontal axis. A front end portion of the intermediate arm portion 19 is coupled to be rotatable about a horizontal axis so as to be sandwiched from both sides by the upper end portion of the lower arm portion 18.
[0024]
Further, the rear end portion of the upper arm portion 20 extending forward is connected to the front end portion of the intermediate arm portion 19 so as to be coaxially rotatable, and the front end side of the upper arm portion 20 is bifurcated to the left and right. The wrist part 21 is connected so as to be rotatable about a horizontal axis. Further, a circular flange portion 22 is connected to the front end portion of the wrist portion 10 so as to be coaxially rotatable, and the flange portion 22 is provided in a state in which a hand for gripping the workpiece W as an object can be attached. ing. Although not shown, a servomotor for driving each axis is incorporated in the robot body 11.
[0025]
The CCD camera 15 described above is for photographing the workpiece W and is attached to the upper arm unit 20. A signal line for transmitting the image signal of the CCD camera 15 is provided integrally with a cable 23 for controlling the robot body 11. The robot controller 13 includes a ROM and a RAM that store a control program mainly using a CPU, and includes a servo control unit that outputs a control command to the robot body 11 as a drive signal.
[0026]
The visual device 14 is mainly composed of a CPU for performing visual servoing, and includes a ROM and a RAM for storing a position and orientation control program, an interface circuit for inputting an image signal from the CCD camera 15, an image processing unit, an image The configuration includes various circuits such as a memory.
[0027]
Then, as will be described later, the control device 12 captures the image signal of the CCD camera 15 by the visual device 14, and outputs a control signal in a state in which the position and orientation are in agreement with the reference image registered in advance. When output to the robot controller 13, the robot controller 13 controls to drive the position and orientation of the robot body 11 to be moved based on this.
[0028]
FIG. 2 shows registration data when a workpiece W as an object is registered as a reference image taken by the CCD camera 15 from the standard position. Here, as an example of setting a reference image in which the features of the workpiece W are introduced, a member as illustrated will be described as an example. First, for this workpiece W, an arbitrary reference point P0 is set as a predetermined part in the central part, for example, and characteristic points in the peripheral part are set as characteristic points P1 to P5. Data of vectors V1 to V5 formed by connecting the reference point P0 and the feature points P1 to P5 is set. Further, data of adjacent angles φ1 to φ5 of the vectors V1 to V5 are set and registered as reference image data.
[0029]
Next, the operation of the present embodiment will be described with reference to FIGS. The control device 12 controls the position and orientation of the robot body 11 according to the flowchart shown in FIG. When the control operation is started, the control device 12 shifts to visual servo control, which will be described later, through various settings such as initial settings (step S1). When the control cycle is converged by proceeding with visual servo control, the error becomes equal to or less than an allowable value (determined as “YES” in step S2), and the process proceeds to the next Rx, Ry detection process (step S3). ).
[0030]
Here, the correction amount calculation processing referred to in the present invention is performed, and when the error value is equal to or less than the allowable value (determined as “YES” in step S4), the correction value obtained at that time is stored, and the robot The position and orientation of the main body 11 are moved so as to be in a state according to the target position, that is, the correction value (step S5). As a result, accurate position and orientation control that cannot be obtained by conventional position and orientation control using only visual servo can be performed.
[0031]
Next, the visual servo position and orientation control executed in step S1 will be briefly described. FIG. 4A shows registration data for registering an image when the workpiece W is photographed by the CCD camera 15 at the standard position as a reference image. The reference position here indicates the position and orientation state finally obtained by controlling the robot body 11. At the stage of visual servo position / orientation control, since the control using the image data shown in FIG. 2 is not performed, the shape of the workpiece W is simply shown as a rectangular parallelepiped.
[0032]
Now, when the robot body 11 is in an arbitrary position and orientation, the visual device 14 refers to it based on the images W1 and W2 of the workpiece W taken by the CCD camera 15 (see FIGS. 4B and 4C). Data indicating the position and orientation to be controlled by comparing with the image is output to the robot controller 13. The robot controller 13 controls the position and orientation of the robot body 11 so that the image captured by the CCD camera 15 looks the same as the reference image.
[0033]
In this way, the control of the position and orientation of the robot main body 11 is advanced, and when the position and orientation control of the visual servo by the CCD camera 15 converges, that is, when the same captured image as the reference image is obtained, the position and orientation is changed. It will be confirmed. However, in this state, since the accuracy of the pitch angle and the yaw angle of the CCD camera 15 is low as described above, these attitude data include errors that cannot be detected with an upper limit of about ± 2 °. There may be.
[0034]
Next, the Rx and Ry detection processing performed in step S3 of the flowchart of FIG. 3 will be described. In this detection process, first, in the state where the position and orientation control by the visual servo has converged as described above, the orientation of the CCD camera 15 changes, for example, the rotation angle Rx around the x axis by a predetermined value or more as shown in FIG. (In the figure, the converged state is the 15 position, and the positions 15a and 15b). In this case, the amount of change in Rx is set so large that it appears as a change in the number of pixels when viewed in the image taken by the CCD camera 15.
[0035]
Thereby, based on the change in the number of pixels, the rotation angle around the x-axis Rx or the y-axis Ry, that is, the value of the pitch angle or yaw angle as posture data is obtained. Then, an error value function f (Rx, Ry) for the values of Rx, Ry changed at this time is defined as follows. In this case, the error value function f (Rx, Ry) is used to compare the reference image data with the vectors V1 to V5 and φ1 to φ5 described in FIG.
[0036]
[Expression 1]

[0037]
Now, for example, consider the case where the movement of the CCD camera 15 shown in FIG. When one posture angle Rx or Ry is fixed and the other posture angle Ry or Rx is changed, the ratio value Lλ (λ = 1 to 5) of each vector V1 to V5 of the workpiece W and the angle φ1 A value with a ratio value θλ (λ = 1 to 5) of ˜φ5 is obtained as a function value f (Rx, Ry) according to the equation (1). Eight function values f1 to f8 are obtained corresponding to the image data for eight times.
[0038]
Here, when the obtained function values f1 to f8 are plotted with respect to the detected posture angles Rx or Ry, respectively, as shown in FIG. 6, a curve obtained by connecting the function values f1 to f8 indicating error values is obtained. It is done. This function curve should be zero when passing through the position where the reference image is taken, that is, the reference position. However, if the fixed posture angle is deviated or if measurement error is taken into consideration, there may be cases where zero is not taken as shown by the broken line in the figure. The condition is not to judge but to be equal to or smaller than an allowable error value.
[0039]
In the position / orientation control by visual servo, if all the data corresponding to the function values f4 to f6 are in a converged state and the difference between them cannot be recognized as the difference in the image of the CCD camera 15, the range is within this range. The detection accuracy of the posture angle is low. Here, in the position / orientation control by visual servo, for example, when the attitude angle corresponding to f5 is obtained as the detected position Rd, the function value f is shifted by ΔR with respect to the reference position Ro at which the function value f becomes zero. Will be. Then, the shifted amount ΔR is taken in as a correction value and stored in the memory.
[0040]
By obtaining such correction values for each of the posture angles Rx and Ry, by correcting the position and posture by the amount of the obtained correction value, it is possible to obtain the reference with accuracy that cannot be obtained by the position and posture control by the visual servo. A position and orientation that substantially matches the position can be obtained.
[0041]
FIG. 7 and FIG. 8 show the grounds on which accuracy can be improved by detecting the position and orientation by determining the vectors V1 to V5. Consider a case where a line segment AB having a length of 2L at a position separated by a distance d from the position P of the CCD camera 15 is photographed. When the line segment AB is orthogonal to the optical axis z of the CCD camera 15 at the midpoint M, the right half length AM and the left half BM length on the screen (distance f) of the CCD camera 15. Can be equally obtained as I (see equation (2) below). In this case, the middle point M corresponds to the aforementioned reference point P0, and both end portions A and B correspond to feature points.
[0042]
Further, assuming that the line segment AB is inclined by an angle θ, that is, a state where the attitude angle of the CCD camera 15 is relatively inclined by θ, the line segment A′B ′ photographed by the CCD camera 15 at that time is on the screen. The pixel numbers I ′ (pix) and I ″ (pix) are expressed by the following equations (3) and (4).
[0043]
[Expression 2]

[0044]
In the above equation, the value of s is a count for converting the actual size on the image plane of the CCD camera 15 into pixels. For example, in the value used in the calculation of FIG. Since it is .0074 mm (7.4 μm), when there is a length of 0.074 mm on the image plane, it is observed as a length of 10 pixels. Therefore, the value of the count s for converting I (pix) from mm to pixels is 135 (an approximate value of 1 / 0.0074).
[0045]
In the above case, the pixel number I ′ (pix) of the right half A′M where the line segment A′B ′ has moved away appears to be small, and the pixel number I ″ (pix) of the approaching left half B′M is Once it appears to increase, it changes so as to decrease as the angle θ increases. For these values of I ′ (pix) and I ″ (pix), the difference from the value of I (pix), respectively. When the calculated results ΔI ′ and ΔI ″ are plotted with the posture angle θ as the horizontal axis, a curve as shown in FIG. 8 is obtained. In FIG. 8, the total number of pixels I ′ (pix) in the line segment is obtained. The result of calculating the difference between + I ″ (pix) and 2I (pix) is also plotted.
[0046]
From the plot of FIG. 8, when the number of pixels is compared for each of the line segments divided by the midpoint M, it can be seen that the amount of change with respect to the posture angle θ shows a larger change than the value compared for the total number of pixels. . That is, as shown in FIG. 2, the detection capability can be enhanced by comparing each of the vectors V1 to V5 formed by connecting the feature point with the reference point as the center. The same effect can be obtained by defining vectors obtained by connecting feature points and performing similar operations on these vectors.
[0047]
According to the present embodiment, the attitude data corresponding to the pitch angle and the yaw angle indicating the position and orientation of the CCD camera 15 is set to a predetermined angle while the position and orientation control by the visual servo is converged. Since the correction amount is estimated from the image data obtained by tilting as described above, it is possible to accurately control the posture angle, which is difficult to increase the accuracy of the position and posture control by visual servo without providing a separate sensor. become able to.
[0048]
Further, according to the present embodiment, as the reference image of the workpiece W, the vectors V1 to V5 obtained by connecting the reference point and the plurality of feature points and the angles φ1 to φ5 between them are defined, and these Is set as a function of the error value when calculating the correction amount, it is possible to increase the resolution and calculate the correction amount with high accuracy.
[0049]
(Second Embodiment)
FIG. 9 and FIG. 10 show a second embodiment of the present invention. Hereinafter, parts different from the first embodiment will be described.
In this embodiment, image data for Rx and Ry detection processing similar to that performed in step S3 of FIG. 3 by the CCD camera 15 is not performed after the position and orientation control by visual servo has converged. As shown in FIG. 10, it is performed until the position and orientation control by the visual servo converges, that is, during the period when the approach to the workpiece W is being performed (step P1).
[0050]
As shown in the figure, during the position / orientation control by visual servo (step P1), the CCD camera 15 is approached while being largely changed by the angle α for the position / orientation control by visual servo (step P2). . While approaching in this way, the image data photographed by the CCD camera 15 is captured and stored (step P3), and when the error is less than the allowable value (determined as “YES” in step P4), the visual servo is terminated. To do. Here, the angle α indicates an angle corresponding to the pitch angle and the yaw angle required to detect Rx and Ry.
[0051]
Next, Rx, Ry detection processing is performed based on the image data obtained so far (step P5). The contents of this detection process are the same as described in the first embodiment. When the correction amount is calculated in this way, the position and orientation are controlled by the correction amount and moved to the target position (step P6).
[0052]
According to the second embodiment, the same effect as that of the first embodiment can be obtained, and image data for Rx and Ry detection processing is acquired during the position / orientation control approach by visual servo. Therefore, although the operation during the approach period is slightly delayed, the position and orientation can be corrected quickly only by calculating in the state where the convergence position has been reached.
[0053]
(Third embodiment)
FIG. 11 shows a third embodiment of the present invention. The difference from the second embodiment is that the inclination angle of the CCD camera 15 is checked and the inclination angle is larger than α by the original visual servo approach. If it is larger (determined as “YES” in Step P7), the above-described Step P2 is just jumped.
[0054]
As a result, the correction amount can be calculated quickly and accurately without tilting the CCD camera 15 more than necessary, and similar effects can be obtained by the third embodiment. become.
[0055]
(Fourth embodiment)
12 and 13 show a fourth embodiment of the present invention. The difference from each of the above embodiments is a workpiece Wa having a shape different from that of the workpiece W shown in FIG. In this case, the case of correcting the position and orientation is handled. Here, as shown in FIG. 12, the workpiece Wa has a larger length ratio between the X-axis direction and the Y-axis direction than the workpiece W described above, and is short when it is attempted to detect Rx and Ry by the above-described method. This is a condition that makes it difficult to increase the detection accuracy on the side.
[0056]
That is, the workpiece Wa shown in FIG. 12 has a schematic shape showing the above-described conditions, and the four feature points p1 to p4 are at positions corresponding to the vertices of the rhombus. In this case, the relationship between the distance L1 between the feature points p1 and p3 indicating the length in the Y-axis direction and the distance L2 between the feature points p2 and p4 indicating the length in the X-axis direction is such that L1 is L2. It is a shape that is larger than the predetermined ratio.
[0057]
As described above, since the distance L1 is related to the detection accuracy of the rotation angle Rx around the X axis, and the distance L2 is related to the detection accuracy of the rotation angle Ry around the Y axis, the lengths of L1 and L2 as described above. From this relationship, it can be understood that the change amount with respect to the displacement of the rotation angle Ry of the distance L2 is small and the detection accuracy is lower than that in the case of the distance L1 in consideration of the relationship expressed by the equation (2). In an extreme case, in the case of a workpiece whose distance L2 is close to zero, it is impossible to effectively detect the rotation angle Ry around the Y axis.
[0058]
Therefore, in order to deal with such a case, firstly, a method for improving the detection accuracy is proposed, and secondly, a condition that the detection accuracy is limited is set to be recognized. When the detection accuracy could not be increased, the processing time was shortened to obtain a practical effect.
[0059]
First, the first method described above will be described. In this case, as shown in FIG. 12, since the length of the line segment connecting the feature points p2 and p4 is short, the accuracy cannot be improved as it is. Therefore, in such a case, the vector obtained from the line segment connecting the feature points of the target workpiece Wa is not larger than a predetermined dimension (determined by the dimension within the range of the captured image of the camera 15) according to preset conditions. Sometimes, in order to improve the detection accuracy, the detection process is performed with the position of the camera 15 closer to the workpiece Wa than the reference position obtained by visual servoing.
[0060]
As a result, as shown in FIG. 13, the shorter line segment L2 can be taken longer in the shooting screen of the camera 15, and the detection accuracy can be improved. In this case, when the camera 15 is moved closer to the work Wa, the longer line segment L1 protrudes from the shooting screen, so the detection process is first performed on the rotation angle Rx around the X axis in the same manner as described above. Thereafter, the camera 15 is brought close to the camera to obtain a photographed image as shown in FIG. 13, and detection of both the Rx and Ry is performed by detecting the rotation angle Ry around the Y axis.
[0061]
As a result, both the values of Rx and Ry can be performed in an appropriate detection processing time while ensuring the detection accuracy, and the correction amount can be accurately calculated based on the obtained result.
[0062]
Next, the second method will be described. This is applied when it is difficult to increase the detection accuracy as in the first method when the detection accuracy is limited, or when the accuracy of the posture data of the rotation angle is not required. . First, whether or not such a condition is satisfied is determined based on a preset condition, and if it is determined that the detection accuracy does not need to be increased, the correction amount related to the rotation axis is not detected. To.
[0063]
This means that the detection process based on the result of the determination in advance is not necessary so that continuing the detection process for improving the detection accuracy in a situation where the improvement of the detection accuracy cannot be expected does not result in an unnecessary time. By stopping, the processing speed is improved. Thereby, by improving only the required accuracy, an efficient detection operation can be performed in a short time, and a great merit can be obtained in practice.
[0064]
(Other embodiments)
The present invention is not limited to the above embodiment, and can be modified or expanded as follows.
The first embodiment and the second embodiment can be performed in combination. For example, in order to improve the accuracy when the visual servo is performed while storing the image data while changing the angle in the process of approaching the workpiece W as in the second embodiment and the correction amount is obtained at the time of convergence, Image data can be captured as in the first embodiment. Thereby, both advantages can be taken in flexibly, and quick and accurate position and orientation control can be performed.
[0065]
In registering a reference image of an object, a method of setting a vector can be selectively performed as necessary. Even if a method in which a vector is not set is adopted, it is possible to improve the position and orientation accuracy as compared with the position and orientation control by the conventional visual servo.
[0066]
In each of the above embodiments, Expression (1) is used as a relational expression for obtaining an error value for detecting Rx and Ry. However, the present invention is not limited to this, and the error from the position and orientation at the reference position is represented by Rx and Ry. Other expressions can be used as long as the relational expression can be evaluated based on the error value and Rx and Ry can be detected based on the error value.
[0067]
The robot main body 11 described in each of the above embodiments assumes a case in which the robot body 11 is not moving relative to the workpiece W, which is an object, but is not limited to this. It can also be applied when the robot body is movable or when both are moving, and by performing the process of interpolating the velocity vector for the relatively moving object as in these cases Similarly, the position and orientation control can be performed with high accuracy.
[0068]
Furthermore, the control target apparatus can be applied not only to a robot but also to a vehicle whose position and orientation are to be controlled, a recognition apparatus that accurately recognizes the shape and operation of an object, and the like.
[Brief description of the drawings]
FIG. 1 is an overall schematic configuration diagram showing a first embodiment of the present invention.
FIG. 2 is an explanatory diagram of setting of a reference point and a feature point of a workpiece and a vector setting state
FIG. 3 is a flowchart showing a process of visual servo and correction amount calculation.
FIG. 4 is a schematic explanatory diagram of position and orientation control by visual servoing.
FIG. 5 is an explanatory diagram of the action when the CCD camera approach and angle are shaken.
FIG. 6 is a diagram showing a function value of an error value with respect to the posture angle of the camera.
FIG. 7 is a diagram for explaining a change with respect to a vector gradient between a reference point and a feature point;
FIG. 8 is a diagram illustrating a relationship between an inclination angle and a pixel.
FIG. 9 is a view corresponding to FIG. 5 showing a second embodiment of the present invention.
FIG. 10 is a view corresponding to FIG.
FIG. 11 is a view corresponding to FIG. 3, showing a third embodiment of the present invention.
FIG. 12 is an operation explanatory diagram illustrating a state in which a workpiece Wa is imaged from a reference position according to the fourth embodiment of the present invention.
13 is a view corresponding to FIG. 12 at a position closer to the reference position.
FIG. 14 is an explanatory diagram of an object showing a conventional example.
FIG. 15 is an explanatory diagram of an object inclination and an error value;
FIG. 16 is a diagram for explaining a change in line width with respect to an inclination angle;
FIG. 17 is an explanatory diagram of pixel data.
[Explanation of symbols]
11 is a robot body (control target device), 12 is a control device, 13 is a robot controller, 14 is a visual device, 15 is a CCD camera (camera), and W is a work (object).

Claims (10)

The control unit is configured so that an object is previously captured as a reference image from a reference position by a camera attached to the control target device and is registered, and the control is performed so that the image of the object captured by the camera is in the same state as the reference image. In the position / orientation control method by visual servo that controls the position / orientation of the target device,
A first step of storing images taken of the object by the camera in different postures;
From the correlation between the posture data indicating at least the pitch angle and the yaw angle estimated from the data of the plurality of stored images and the error data indicating the error between the image data and the reference image data, error data is obtained. A second process of estimating a position and orientation corresponding to the reference position where is equal to or less than an allowable error amount;
A third step of calculating a correction amount to a position and orientation corresponding to the estimated reference position;
And a fourth step of correcting the position and orientation of the control gymnastic apparatus based on the correction amount calculated in the third step with respect to the position and orientation in a state where the position and orientation control by the visual servo has converged. And
In the first process, in a state where the position / orientation control by the visual servo has converged, the position / orientation of the camera is moved so as to change more than a predetermined degree so as to appear as a change in the number of pixels, so that the attitude data changes. A position / orientation control method using visual servoing, which stores an image when the image is selected . The control unit is configured so that an object is previously captured as a reference image from a reference position by a camera attached to the control target device and is registered, and the control is performed so that the image of the object captured by the camera is in the same state as the reference image. In the position / orientation control method by visual servo that controls the position / orientation of the target device,
A first step of storing images taken of the object by the camera in different postures;
From the correlation between the posture data indicating at least the pitch angle and the yaw angle estimated from the data of the plurality of stored images and the error data indicating the error between the image data and the reference image data, error data is obtained. A second process of estimating a position and orientation corresponding to the reference position where is equal to or less than an allowable error amount;
A third step of calculating a correction amount to a position and orientation corresponding to the estimated reference position;
And a fourth step of correcting the position and orientation of the control gymnastic apparatus based on the correction amount calculated in the third step with respect to the position and orientation in a state where the position and orientation control by the visual servo has converged. And
In the first process, an image of the camera during a period until the position and orientation control by the visual servo is converged is stored, and during a period until the position and orientation control by the visual servo is converged If the position and orientation of the camera does not appear as a change in the number of pixels when the position and orientation of the camera is controlled, the camera is moved so that the change in the position and orientation of the camera appears as a change in the number of pixels. A position and orientation control method using visual servo , wherein control is performed so that the angle changes . In the position and orientation control method by visual servo according to claim 1 or 2 ,
The image data of the object to be stored as the reference image is set to include a plurality of feature points with respect to a reference point and a plurality of vectors connecting to the reference point, and the second A position / orientation control method using visual servo , wherein when performing the process, the position / orientation is estimated based on error data calculated based on at least the length of the vector or the ratio of lengths between vectors . In the position and orientation control method by the visual servo according to any one of claims 1 to 3,
The image data of the object to be stored as the reference image is set to include a plurality of feature points with respect to a reference point, and includes a plurality of vectors connecting the feature points. A position / posture control method using visual servo, wherein the position / posture is estimated based on error data calculated based on at least the length of the vector or the length ratio between vectors . In the position and orientation control method by visual servo according to claim 3 or 4,
The image data of the object to be stored as the reference image is a visual servo characterized in that the position / orientation is estimated based on error data calculated based on an angle between vectors for the set vectors. Position and orientation control method. In the position and orientation control method by visual servo according to any one of claims 3 to 5,
The calculation of the error data is obtained as the sum of squares of the difference between the image taken for the length or angle of the vector and the reference image,
The position / orientation control method using visual servo, wherein the position / orientation estimation is used as attitude data for obtaining a minimum value of error data calculated by the above method. The position and orientation control method using visual servo according to claim 6 ,
The position and orientation is estimated by calculating an error data function using the posture data as a variable from the error data value and the pitch angle and yaw angle value of the posture data, and calculating the minimum value of the error data function. A position and orientation control method using visual servo, characterized in that it is performed by obtaining the above. In the position and orientation control method by visual servo according to any one of claims 1 to 7,
The position and orientation by visual servo, wherein the first to fourth processes are performed by moving the position of the camera from the convergence position to a position close to the convergence position in a state where the position and orientation control by the visual servo has converged Control method. The position and orientation control method by visual servo according to claim 8,
In a state in which the position / orientation control by the visual servo has converged, the attitude data of the attitude data that needs to be improved in detection accuracy is moved to a position close to the convergence position and executed. Position and orientation control method by servo. In the position and orientation control method by visual servo according to any one of claims 1 to 9,
For posture data that cannot be improved in detection accuracy of the posture data as the reference image data as the shape of the target object, a determination operation is performed to exclude the target from the position and posture correction target,
Position / orientation control by visual servo, in which the position / orientation control by visual servo converges , arithmetic processing for correcting the position / orientation is executed for those not excluded from the attitude data Method.

Images