JP5049246B2 - Object shape evaluation device - Google Patents

Object shape evaluation device Download PDF

Info

Publication number
JP5049246B2
JP5049246B2 JP2008278401A JP2008278401A JP5049246B2 JP 5049246 B2 JP5049246 B2 JP 5049246B2 JP 2008278401 A JP2008278401 A JP 2008278401A JP 2008278401 A JP2008278401 A JP 2008278401A JP 5049246 B2 JP5049246 B2 JP 5049246B2
Authority
JP
Japan
Prior art keywords
measurement
distance
points
point
adjacent
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP2008278401A
Other languages
Japanese (ja)
Other versions
JP2010107300A (en
Inventor
昌孝 戸田
聡彦 吉川
正志 神谷
俊一 金子
秀則 高氏
皓之 栢場
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hokkaido University NUC
Aisin Corp
Original Assignee
Aisin Seiki Co Ltd
Hokkaido University NUC
Aisin Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aisin Seiki Co Ltd, Hokkaido University NUC, Aisin Corp filed Critical Aisin Seiki Co Ltd
Priority to JP2008278401A priority Critical patent/JP5049246B2/en
Publication of JP2010107300A publication Critical patent/JP2010107300A/en
Application granted granted Critical
Publication of JP5049246B2 publication Critical patent/JP5049246B2/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Measurement Of Optical Distance (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Length Measuring Devices With Unspecified Measuring Means (AREA)

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for evaluating the shape of an object in which the congruence transformation between a measurement point group and a reference point group is properly performed, even when the measurement error of a specific area is large, due to the conditions on measurement techniques of a measurement device, and as a result, the object shape evaluation of the measurement object is performed properly. <P>SOLUTION: The device of evaluating the shape of the object aligns the measurement points and the reference points, on the basis of sequential convergence processing for sequential convergence of the distance between many measurement points corresponding to the shape of the measurement object and many reference points corresponding to the reference shape of the measurement object, and evaluates the shape of the measurement object, on the basis of the measurement point data and the reference point data after the alignment. In the alignment processing, an inter-adjacent-point distance weight coefficient is determined, on the basis of the inter-adjacent-point distance between adjacent measurement points, or the inter-adjacent-point distance between the adjacent reference points, and the inter-adjacent-point distance weight coefficient is used, when a sequential convergence evaluation value in the sequential convergence processing is to be obtained. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

本発明は、測定対象物の形状に対応する多数の測定点とこの測定点に対応する多数の基準点とを位置合わせすることで測定対象物の形状を評価する物体形状評価装置に関する。   The present invention relates to an object shape evaluation apparatus that evaluates the shape of a measurement object by aligning a large number of measurement points corresponding to the shape of the measurement object and a large number of reference points corresponding to the measurement points.

測定点群と基準点群とを位置合わせ(マッチング)することで測定対象物に対する測定結果を評価する従来の技術として、ステレオ視とICP(Iterative Closest Point)アルゴリズムを組み合わせて、被計測物体の位置・姿勢を精度良く計測する画像計測方法が知られている(例えば、特許文献1参照)。このステレオ画像計測方法では、計測対象の3次元形状を複数台のカメラ画像をステレオ処理することにより計測点群が求められ、計測対象の既知の形状情報とステレオカメラの相対位置・姿勢とから可視部が予測され、予測された可視部の形状情報のみを用い前記計測点群の空間密度に合わせたモデル点群が決定され、前記計測点群とモデル点群の間でICPアルゴリズムを用いてマッチングさせ、最も評価関数が小さなモデル群に対応する位置・姿勢を計測結果として採用される。   As a conventional technique for evaluating the measurement result for a measurement object by aligning (matching) the measurement point group and the reference point group, the position of the object to be measured is combined with stereo vision and an ICP (Iterative Closest Point) algorithm. An image measurement method that accurately measures the posture is known (for example, see Patent Document 1). In this stereo image measurement method, a measurement point cloud is obtained by stereo-processing a plurality of camera images of the three-dimensional shape of the measurement target, and is visible from the known shape information of the measurement target and the relative position and orientation of the stereo camera. The model point cloud is determined according to the spatial density of the measurement point group using only the predicted shape information of the visible part, and matching is performed between the measurement point group and the model point group using an ICP algorithm. The position / orientation corresponding to the model group having the smallest evaluation function is adopted as the measurement result.

しかしながら、ICPアルゴリズムは、原則として例外値を含まない位置データ同士の位置決めまたは融合(両データの最も一致する合同変換を求める問題)を行う手法である。従って、位置決め対象となる距離データ集合内に重複しない不一致部分、またはデータの追加部分などが含まれていると、それらが例外値として働き正しい状態への収束を阻害するという問題が生じる。   However, the ICP algorithm is a technique for positioning or merging position data that does not include an exceptional value in principle (problem for obtaining the most congruent congruent transformation of both data). Therefore, if there is a non-overlapping part or an additional part of data included in the distance data set to be positioned, there arises a problem that these function as exception values and hinder convergence to the correct state.

このようなICPアルゴリズムの問題点を解消するために、M推定を導入したICP位置決め法が提案されている(例えば、非特許文献1参照)。このM推定ICP位置決め法では、2つの点群(例えば、測定点群と基準点群)の対応関係を更新しながら合同変換を逐次収束させる際、残差量(例えば、測定点と基準点との距離)に応じて重みを設定する。この重みによって対応点毎に位置決めにおける評価価値を変化させることにより、例外値による悪影響を抑制することが可能となる。しかしながら、測定対象物の三次元形状を測定するような場合、測定装置の測定技術上の条件から、測定対象物の傾斜した面などに対しては測定誤差が大きくなり、測定点と基準点との距離に基づく重み付けだけでは、測定点群と基準点群との合同変換を適正に逐次収束させることが困難となる。   In order to solve such problems of the ICP algorithm, an ICP positioning method in which M estimation is introduced has been proposed (see, for example, Non-Patent Document 1). In this M-estimated ICP positioning method, when the congruent transformation is successively converged while updating the correspondence between two point groups (for example, a measurement point group and a reference point group), a residual amount (for example, a measurement point and a reference point) The weight is set according to the distance. By changing the evaluation value in positioning for each corresponding point by this weight, it is possible to suppress the adverse effect due to the exceptional value. However, when measuring the three-dimensional shape of the measurement object, the measurement error increases for the inclined surface of the measurement object due to the measurement technical conditions of the measurement device, and the measurement point and the reference point Only by weighting based on the distance, it is difficult to properly and sequentially converge the joint transformation between the measurement point group and the reference point group.

特開2008−14691号公報(段落番号0006、図1)Japanese Patent Laying-Open No. 2008-14691 (paragraph number 0006, FIG. 1) 金子俊一・他著「M推定を導入したロバストICP位置決め法」精密工学会誌V0l.67,No.8,2001、1276-1280Shunichi Kaneko et al. “Robust ICP positioning method with M estimation”, Journal of Precision Engineering V0l.67, No.8, 2001, 1276-1280

上記実状に鑑み、本発明の目的は、測定装置の測定技術上の条件から、特定の測定領域でその測定誤差が大きくなるような場合においても測定点群と基準点群との合同変換が適正に行われ、その結果、測定対象物の物体形状評価が適正に行われる物体形状評価装置を提供することである。   In view of the above situation, the object of the present invention is that the joint conversion between the measurement point group and the reference point group is appropriate even when the measurement error becomes large in a specific measurement region due to the measurement technical conditions of the measurement apparatus. As a result, an object shape evaluation apparatus is provided in which the object shape evaluation of the measurement object is appropriately performed.

上記目的を達成するため、本発明に係る物体形状評価装置の特徴構成は、測定対象物の形状に対応する多数の測定点の位置情報を含む測定点データを入力する測定データ入力部と、前記測定対象物の基準形状に対応する多数の基準点の位置情報を含む基準点データを格納する基準データ格納部と、対応する測定点と基準点との間の距離を逐次収束させる逐次収束処理に基づいて前記測定点と前記基準点とを位置合わせする位置合わせ処理手段と、前記位置合わせ処理手段による位置合わせ処理後の前記測定点データと前記基準点データとに基づいて前記測定対象物の形状を評価する形状評価手段とを含み、前記位置合わせ処理手段は隣接する前記測定点の間の隣接点間距離又は隣接する前記基準点の間の隣接点間距離に基づいて隣接点間距離重み係数を決定する重み係数決定手段を有し、当該隣接点間距離重み係数を前記逐次収束処理における逐次収束評価値を求める際に用いることである。   In order to achieve the above object, a feature configuration of the object shape evaluation apparatus according to the present invention includes a measurement data input unit that inputs measurement point data including position information of a large number of measurement points corresponding to the shape of the measurement object, A reference data storage unit that stores reference point data including position information of a large number of reference points corresponding to the reference shape of the measurement object, and a sequential convergence process that sequentially converges the distance between the corresponding measurement point and the reference point An alignment processing means for aligning the measurement point and the reference point based on the shape, and a shape of the measurement object based on the measurement point data and the reference point data after the alignment processing by the alignment processing means Shape evaluation means for evaluating the distance between adjacent points based on a distance between adjacent points between adjacent measurement points or a distance between adjacent points between adjacent reference points. It has a weighting coefficient determination means for determining a look coefficients is to use the distance weight factor between the adjacent points when determining the sequential convergence evaluation value in the sequential convergence process.

この特徴構成によれば、測定点群と基準点群のいずれかの点群においてその点群に属する各点(測定点又は基準点)に対して隣接する点との距離である隣接点間距離に基づいて重み係数が割り当てられる。割り当てられた重み係数は、対応する測定点と基準点との間の距離を逐次収束させる逐次収束処理における逐次収束評価値を求める際に用いられ、隣接点間距離によって逐次収束評価値に及ぼす影響度が調整される。例えば、隣接点間距離が他に比べて大きくなっている測定点、あるいは、隣接点間距離が他に比べて大きくなっている基準点に対応している測定点ではわずかな測定点のずれが測定点と基準点との間の位置座標値の大きな差となってしまう可能性がある。これは、レーザビームなどによる測定面が測定基準面に対して直角に近い角度で湾曲していたり傾斜していたりしている場合、測定走査ピッチの僅かなずれが測定点の大きなずれを導き、基準点と測定点との間の位置対応が悪くなるからである。このような測定状況においては、隣接点間距離が大きな点に対してより小さな重み係数を割り当てて、その点が逐次収束評価値に及ぼす影響を抑制することで、測定点群と基準点群との間の適正な合同変換が実現できる。その結果、測定対象物の物体形状評価が適正に行われることになる。   According to this feature configuration, the distance between adjacent points that is the distance between each point belonging to the point group (measurement point or reference point) in either the measurement point group or the reference point group. A weighting factor is assigned based on The assigned weighting factor is used to determine the sequential convergence evaluation value in the sequential convergence process that sequentially converges the distance between the corresponding measurement point and the reference point, and the effect of the distance between adjacent points on the sequential convergence evaluation value The degree is adjusted. For example, there is a slight measurement point deviation at a measurement point where the distance between adjacent points is larger than the other points, or at a measurement point corresponding to a reference point where the distance between adjacent points is larger than the others. There may be a large difference in the position coordinate value between the measurement point and the reference point. This is because, when the measurement surface by a laser beam or the like is curved or inclined at an angle close to a right angle with respect to the measurement reference surface, a slight deviation of the measurement scanning pitch leads to a large deviation of the measurement point, This is because the positional correspondence between the reference point and the measurement point is deteriorated. In such a measurement situation, a smaller weighting factor is assigned to a point having a large distance between adjacent points, and the influence of the point on the convergence evaluation value is sequentially suppressed, so that the measurement point group and the reference point group Appropriate congruent conversion between can be realized. As a result, the object shape of the measurement object is properly evaluated.

また、本発明に係る物体形状評価装置におけるさらなる特徴構成として、前記重み係数決定手段は前記対応する測定点と基準点との間の対応点間距離に基づいて対応点間距離重み係数を決定し、当該対応点間距離重み係数を前記位置合わせ処理手段が前記逐次収束処理における逐次収束評価値を求める際に用いることも好適である。この特徴は、いわゆるM推定法として知られているロバストICP(Iterative Closest Point)位置決め法であり、非特許文献1で詳しく説明されているのでここでの詳しい説明は省略するが、対応する測定点と基準点との間の対応点間距離に基づいて決定される対応点間距離重み係数が前記逐次収束処理における逐次収束評価値を求める際に用いられ、大きな対応点間距離をもつ測定点(いわゆる外れ点)によって逐次収束評価値に及ぼす影響度が調整される。これにより、この好適な物体形状評価装置では、対応する測定点と基準点との間の対応点間距離に基づいて決定される対応点間距離重み係数と、同一点群における隣接点間距離に基づいて決定される隣接点間距離重み係数とで、誤差を導くと推定される測定点データを排除またはその影響力を小さくすることができる。その結果、測定対象物の物体形状評価がさらに適正に行われることになる。   Further, as a further characteristic configuration in the object shape evaluation apparatus according to the present invention, the weighting factor determination means determines a distance weighting factor between corresponding points based on a distance between corresponding points between the corresponding measurement point and a reference point. It is also preferable to use the distance weight coefficient between corresponding points when the alignment processing means obtains the sequential convergence evaluation value in the sequential convergence processing. This feature is a robust ICP (Iterative Closest Point) positioning method known as a so-called M estimation method, which has been described in detail in Non-Patent Document 1 and will not be described in detail here. A distance weight factor between corresponding points determined based on the distance between corresponding points between the reference point and the reference point is used when obtaining a sequential convergence evaluation value in the sequential convergence process, and a measurement point having a large distance between corresponding points ( The degree of influence on the successive convergence evaluation value is adjusted by so-called outliers. Thereby, in this preferable object shape evaluation apparatus, the distance weight between corresponding points determined based on the distance between corresponding points between the corresponding measurement point and the reference point, and the distance between adjacent points in the same point group. With the distance weight coefficient between adjacent points determined based on the measurement point data estimated to introduce an error, the influence of the measurement point data can be eliminated or reduced. As a result, the object shape evaluation of the measurement object is performed more appropriately.

重み係数決定手段による隣接点間距離に基づく隣接点間距離重み係数の決定の際、測定点群における隣接点間距離と基準点群における隣接点間距離のうちのいずれを採用してもよいが、基準点群における隣接点間距離を採用する場合、測定の前に予め隣接点間距離重み係数を求めておくことができる。同形状の測定対象物が複数ある場合、共通の基準点データ(基準点群)から求められた隣接点間距離重み係数をそれぞれの物体形状評価において利用できるので効果的である。その際、予め求められた隣接点間距離重み係数が予め算定され、装置内でテーブル化されているとさらに好適である。   In determining the distance weight between adjacent points based on the distance between adjacent points by the weight coefficient determining means, either the distance between adjacent points in the measurement point group or the distance between adjacent points in the reference point group may be adopted. When the distance between adjacent points in the reference point group is adopted, the distance weight coefficient between adjacent points can be obtained in advance before measurement. When there are a plurality of measurement objects of the same shape, it is effective because the distance weight coefficient between adjacent points obtained from common reference point data (reference point group) can be used in each object shape evaluation. At that time, it is more preferable that the distance weight coefficient between adjacent points obtained in advance is calculated in advance and tabulated in the apparatus.

隣接点間距離重み係数は広すぎる隣接点間距離を有する点(測定点又は基準点)が逐次収束評価値に及ぼす影響を抑制することなので、前記隣接点間距離重み係数が隣接点間距離をパラメータとする閾値関数によって求められるようにするだけで効果的であり、またそれにより隣接点間距離重み係数を求める演算が簡単となる。その際、1つの閾値を境にして、「0ないしは0に近い数」と「1」を値としてもつ閾値関数が好適である。   Since the distance weighting factor between adjacent points suppresses the influence of the point (measurement point or reference point) having a distance between adjacent points that is too wide on the convergence evaluation value, the distance weighting factor between adjacent points determines the distance between adjacent points. It is effective only to be obtained by using a threshold function as a parameter, and the calculation for obtaining the distance weight coefficient between adjacent points is thereby simplified. In this case, a threshold function having “0 or a number close to 0” and “1” as values with one threshold as a boundary is preferable.

本発明による物体形状評価装置は、隣接点間距離が大きな点に対してより小さな重み係数を割り当てることでその点が逐次収束評価値に及ぼす影響を抑制する。従って、測定走査ピッチの僅かなずれが測定点の大きなずれを導くことで基準点と測定点との間の位置対応が悪くなるような測定系の物体形状評価に特に適している。このことから、本発明による物体形状評価装置の測定点データがスリット光によって照射された前記測定対象物の撮影画像を処理することによって得られた距離画像から取り出された三次元位置データである場合に、適している。   The object shape evaluation apparatus according to the present invention assigns a smaller weight coefficient to a point having a large distance between adjacent points, thereby suppressing the influence of the point on the successive convergence evaluation value. Therefore, a slight shift in the measurement scanning pitch leads to a large shift in the measurement point, which is particularly suitable for the object shape evaluation of the measurement system in which the position correspondence between the reference point and the measurement point is deteriorated. Therefore, when the measurement point data of the object shape evaluation apparatus according to the present invention is three-dimensional position data extracted from a distance image obtained by processing a captured image of the measurement object irradiated with slit light. Suitable for.

まず、本発明による物体形状評価装置で採用されている、基準点群と測定点群との位置合わせを行う逐次収束処理における逐次収束評価に用いられる重み係数の算出原理を説明する。ここでは、逐次収束処理としてICPアルゴリズムに基づいた方法が採用されている。このICPアルゴリズムでは、図1で模式的に示されているように、基準点群(基準点データ)の各点について最も近い測定点群(測定点データ)の点を対応点とし、各対応点距離の2乗和を最小とする合同変換パラメータを推定して、逐次収束させていく。このようなICPアルゴリズムによる位置合わせでは、図1で示しているような等測定走査ピッチで測定点を決定していくような形状測定の場合、測定面の姿勢によっては僅かな走査ピッチのずれが大きな測定点のずれを導くことになる(図1では、測定点S4、S5、S6がこれに当てはまる)。従って、想定している位置座標と実際の位置座標との誤差が大きいことが予想される測定点とそれに対応する基準点とをそのまま逐次収束処理における逐次収束評価に用いることは好ましくない。このような問題を回避するため、本発明で適用されている改善されたICPアルゴリズムでは、以下に説明するような重み係数を取り入れている。   First, the calculation principle of the weighting coefficient used for the successive convergence evaluation in the successive convergence processing employed in the object shape evaluation apparatus according to the present invention for aligning the reference point group and the measurement point group will be described. Here, a method based on the ICP algorithm is adopted as the sequential convergence process. In this ICP algorithm, as schematically shown in FIG. 1, each point of the reference point group (reference point data) is a point corresponding to the closest measurement point group (measurement point data). The congruent transformation parameter that minimizes the sum of squares of the distance is estimated and converged sequentially. In the alignment by such ICP algorithm, in the case of shape measurement in which measurement points are determined at an equal measurement scan pitch as shown in FIG. 1, a slight scan pitch shift may occur depending on the orientation of the measurement surface. This leads to a large displacement of the measurement points (in FIG. 1, measurement points S4, S5, S6 apply to this). Therefore, it is not preferable to use a measurement point at which an error between an assumed position coordinate and an actual position coordinate is expected to be large and a reference point corresponding thereto as it is for the successive convergence evaluation in the successive convergence process. In order to avoid such problems, the improved ICP algorithm applied in the present invention incorporates weighting factors as described below.

図1では、基準点群(基準点データ):Mに含まれる基準点はmで示されており、測定順序に対応させて添え字(自然数)が付与されている。測定点群(測定点データ):Sに含まれる測定点はsで示されており、基準点と同様に測定順序に対応させて添え字(自然数)が付与されている。測定点群を基準点群に重ねる位置決めのための従来のICPアルゴリズムを用いた逐次収束処理では、測定点群中の各測定点について基準点群の中で最も近い基準点を対応点とし、各対応点間の距離の2乗和が最小となる合同変換パラメータ(R、t)が求められる。ここで、Rは回転行列で、tは並進移動ベクトルである。その際、対応する基準点と測定点の組み合わせの中で、上述した大きい誤差が予想される測定点との組み合わせたものを他の組み合わせと同様に扱うと無視できない誤差の影響を受ける可能性がある。従って、測定点又は前記基準点の隣接点間距離に基づいて隣接点間距離重み係数を割り当て、その悪影響を抑制する。ここでは、基準点の間の隣接点間距離に基づいてその対応点間の距離に対する重み係数を算定することにするが、もちろん測定点の間の隣接点間距離に基づいてその対応点間の距離に対する重み係数を算定してもよい。   In FIG. 1, a reference point group (reference point data): a reference point included in M is indicated by m, and a subscript (natural number) is given corresponding to the measurement order. Measurement point group (measurement point data): The measurement point included in S is indicated by s, and a subscript (natural number) is given corresponding to the measurement order in the same manner as the reference point. In the sequential convergence process using the conventional ICP algorithm for positioning the measurement point group over the reference point group, the closest reference point in the reference point group is set as the corresponding point for each measurement point in the measurement point group. A joint transformation parameter (R, t) that minimizes the sum of squares of the distances between corresponding points is obtained. Here, R is a rotation matrix and t is a translation vector. At that time, among the corresponding combinations of reference points and measurement points, if the combination of the above-mentioned measurement points where a large error is expected is handled in the same way as other combinations, there is a possibility that it will be affected by errors that cannot be ignored. is there. Therefore, the distance weight coefficient between adjacent points is assigned based on the distance between adjacent points of the measurement point or the reference point, and the adverse effect is suppressed. Here, the weighting coefficient for the distance between the corresponding points is calculated based on the distance between the adjacent points between the reference points. Of course, the distance between the corresponding points is determined based on the distance between the adjacent points between the measurement points. A weighting factor for the distance may be calculated.

三次元座標(xi,yi,zi)を有する基準点:miの隣接点間距離:diは、三平方の定理で求められ、その二乗は、
i 2=xi 2+yi 2+zi 2となる。
隣接点間距離重み係数:γiは、重み関数をΓとすると、
γi=Γ(di 2)で求められる。
例えば、重み関数:Γを次のような閾値関数とすることと好都合である。
i 2が閾値:dth以上の時、γi=0.01
i 2が閾値:dth未満の時、γi=1
閾値:dthは基準点群や測定点群の特性によって適切に決めることにより、想定している位置座標と実際の位置座標との誤差が大きいことが予想される測定点とそれに対応する基準点とが逐次収束評価に及ぼす悪影響を抑制することができる。
Three-dimensional coordinates (x i, y i, z i) the reference point having: m i between adjacent point distance: d i is calculated by the Pythagorean theorem, the square is
d i 2 = x i 2 + y i 2 + z i 2
The distance weight coefficient between adjacent points: γ i is the weight function Γ,
It is obtained by γ i = Γ (d i 2 ).
For example, it is convenient to set the weight function: Γ to the following threshold function.
When d i 2 is a threshold value: d th or more, γ i = 0.01
When d i 2 is less than the threshold: d th γ i = 1
The threshold value: d th is determined appropriately according to the characteristics of the reference point group and the measurement point group, so that the measurement point expected to have a large error between the assumed position coordinate and the actual position coordinate and the corresponding reference point Can suppress adverse effects on the successive convergence evaluation.

なお、このようなICPアルゴリズムに基づく位置合わせにおいて、上述した非特許文献に開示されているようなM推定を用いることが有用である。図1を用いて、M推定を導入したICPアルゴリズムを簡単に説明する。このM推定の導入は、対応する測定点と基準点との間の対応点間距離に基づいて決定された対応点間距離重み係数を逐次収束処理における逐次収束評価に用いることである。例えば、対応点間距離をeiとすると、対応点間距離重み係数:ρiは、重み関数をΡとすると、
ρi=Ρ(ei) で求められる。ここでも、重み関数:Ρを次のような閾値関数とすることができる;
|ei|が設定幅:Bi以下の時、
ρi=(Bi 2/2 )(1−(1−(ei/Bi2
|ei|が設定幅:Biを越える時、
ρi=(Bi 2/2 )。
It should be noted that it is useful to use M estimation as disclosed in the above-mentioned non-patent document in the alignment based on such an ICP algorithm. The ICP algorithm in which M estimation is introduced will be briefly described with reference to FIG. The introduction of M estimation is to use the distance weight factor between corresponding points determined based on the distance between corresponding points between the corresponding measurement point and the reference point for the successive convergence evaluation in the successive convergence process. For example, assuming that the distance between corresponding points is e i , the distance weight coefficient between corresponding points: ρ i is
ρ i = Ρ (e i ) Again, the weight function: Ρ can be a threshold function as follows:
When | e i | is less than the set width B i ,
ρ i = (B i 2/ 2) (1- (1- (e i / B i) 2)
When | e i | exceeds the set width B i
ρ i = (B i 2/ 2).

隣接点間距離重み係数:γiに加えて対応点間距離重み係数:ρiも逐次収束評価に用いる場合には、トータル重み係数:wは、隣接点間距離重み係数:γiと対応点間距離重み係数:ρiとをパラメータとする関数から導くことができる。従って、逐次収束処理における逐次収束評価値:Jは、対応点の数をNとすれば、対応点間距離:eiと対応点間距離重み係数:ρiと隣接点間距離重み係数:γiとをパラメータとする評価関数:Hを用いて導出することができる。
J=(1/N)ΣH(ei,ρi,γi
演算を簡単化するために、トータル重み係数:wiを各重み係数の乗算とすれば、
J=(1/N)ΣH(ei,wi)、wi=ρi×γi
となる。
When the distance weight coefficient between adjacent points: γ i and the distance weight coefficient between corresponding points: ρ i are also used for successive convergence evaluation, the total weight coefficient: w is the distance weight coefficient between adjacent points: γ i and the corresponding points. It can be derived from a function having a distance weight coefficient: ρ i as a parameter. Thus, sequential convergence process sequential convergence evaluation value in: J, if the number of corresponding points is N, between the corresponding point distance: e i and between the corresponding point distance weighting factor: [rho i with the neighboring point distance weight factor: gamma An evaluation function with i as a parameter can be derived using H.
J = (1 / N) ΣH (e i , ρ i , γ i )
In order to simplify the calculation, if the total weight coefficient: w i is multiplied by each weight coefficient,
J = (1 / N) ΣH (e i , w i ), w i = ρ i × γ i
It becomes.

次に、上述したアルゴリズムを用いて位置合わせされた基準点群(基準点データ)と測定点群(測定点データ)とから、効率的にめくり上がりなどの特定の表面欠陥を判定するアルゴリズムを図2の模式図を用いて説明する。
まず、対応点間距離:eiが予め設定されている対応点間距離閾値(第1閾値):TA以上となる連続した測定点群を誤対応領域として抽出する。図2では、測定点S4、S5、S6、S7が抽出されている。さらに、誤対応領域に属する測定点の分布密度が算定される。簡単に分布密度を算定するため、例えば、それらの測定点の隣接点間距離:diを用いることができる。つまり、隣接点間距離:diが予め設定されている隣接点間距離閾値(第2閾値):TL未満となる測定点を近傍測定点とみなす。これにより、所定以上の分布密度を有する対応点(近傍測定点)が抽出されたことになる。さらに、この近傍測定点の数が予め設定されている近傍点数閾値(第3閾値):TC以上となるかどうかチェックされる。近傍測定点の数が近傍点数閾値より大きい場合この近傍測定点群によって規定される領域、つまりこの近傍測定点群を含む表面領域が表面欠陥として判定される。この表面欠陥判定アルゴリズムは、めくり上がり欠陥など表面層が剥がれるような表面欠陥領域に対するレーザビームなどを用いた表面形状測定の結果、「その測定点群の対応点距離が正常領域に比べて大きくなる」、及び「その測定点群が互いに密集した点群として存在する」という本願発明者の知見に基づいている。
Next, an algorithm for efficiently determining a specific surface defect such as turning up from the reference point group (reference point data) and the measurement point group (measurement point data) aligned using the above-described algorithm is illustrated. This will be described with reference to FIG.
First, a corresponding point distance threshold (first threshold): TA corresponding to a distance between corresponding points: ei set in advance is extracted as a miscorresponding region. In FIG. 2, measurement points S4, S5, S6, and S7 are extracted. Further, the distribution density of the measurement points belonging to the erroneous correspondence area is calculated. In order to easily calculate the distribution density, for example, the distance between adjacent points of the measurement points: di can be used. In other words, adjacent point distance: di is between adjacent points is preset distance threshold (second threshold value) is regarded as the near measurement point the measurement point is less than TL. As a result, corresponding points (neighboring measurement points) having a distribution density greater than or equal to a predetermined value are extracted. Further, it is checked whether or not the number of neighboring measurement points is equal to or greater than a preset neighboring point threshold (third threshold): TC. When the number of neighboring measurement points is larger than the neighboring point count threshold, a region defined by this neighboring measurement point group, that is, a surface region including this neighboring measurement point group is determined as a surface defect. As a result of surface shape measurement using a laser beam or the like for a surface defect area where the surface layer is peeled off, such as a rolled-up defect, this surface defect determination algorithm is “the corresponding point distance of the measurement point group becomes larger than the normal area ”And“ the measurement point group exists as a dense point group ”.

上述したような、基準点群と測定点群との位置合わせアルゴリズム及び表面欠陥判定アルゴリズムを採用した、表面欠陥評価装置の一例を説明する。この表面欠陥評価装置は、表面に多数の直線上の深溝が整列形成されている測定対象物の溝断面を検査する表面検査装置として構成されている。図3は、そのような表面検査装置の構成を模式的に示す斜視図である。   An example of a surface defect evaluation apparatus that employs the alignment algorithm between the reference point group and the measurement point group and the surface defect determination algorithm as described above will be described. This surface defect evaluation apparatus is configured as a surface inspection apparatus that inspects a groove section of a measurement object in which a large number of straight deep grooves are aligned on the surface. FIG. 3 is a perspective view schematically showing the configuration of such a surface inspection apparatus.

この表面検査装置は、測定装置部1と、この測定装置部1に対する制御及びその測定結果に対する評価を行うコントローラ100を備えている。測定装置部1は、測定系の主な構成要素として、スリット光を発生させるレーザタイプのスリット光源ユニット2と、測定対象物のスリット光が照射されている領域を撮像する撮像ユニット3とを備えている。このスリット光源ユニット2と撮像ユニット3とは測定ヘッドMHとして一体的に組み付けられている。また測定装置部1は、機構系の主な構成要素として、基台10と、基台10に立設された門形フレーム11、門形フレーム11の中央部分で測定ヘッドMHを昇降可能に支持している昇降機構12を備えている。さらに、測定対象物のポジショニング機構として、測定対象物を載置させるとともに回転する回転テーブル13、回転テーブル13をX−Y平面(スリット光軸に直交する平面)上で移動させるためのX−Y移動機構を構成するX方向移動可能なXステージ14及びY方向移動可能なYステージ15を備えている。   The surface inspection apparatus includes a measurement apparatus unit 1 and a controller 100 that performs control on the measurement apparatus unit 1 and evaluates the measurement result. The measurement apparatus unit 1 includes, as main components of the measurement system, a laser-type slit light source unit 2 that generates slit light, and an imaging unit 3 that captures an area of the measurement object irradiated with the slit light. ing. The slit light source unit 2 and the imaging unit 3 are integrally assembled as a measurement head MH. In addition, the measuring apparatus unit 1 supports the base 10, the portal frame 11 erected on the base 10, and the central portion of the portal frame 11 so that the measuring head MH can be moved up and down as main components of the mechanical system. A lifting mechanism 12 is provided. Furthermore, as a positioning mechanism for the measurement object, the rotation table 13 that rotates while placing the measurement object, and the XY for moving the rotation table 13 on the XY plane (a plane orthogonal to the slit optical axis). An X stage 14 that can move in the X direction and a Y stage 15 that can move in the Y direction are provided.

コントローラ100は、実質的にはコンピュータユニットとして形成されており、光源制御部80、画像メモリ81、画像処理部82、三次元測定データ演算部83、本発明に特に関係する評価モジュール90、昇降機構制御部84、回転テーブル制御部85、Xステージ制御部86、Yステージ制御部87を備えている。回転テーブル制御部85、Xステージ制御部86、Yステージ制御部87はそれぞれ、回転テーブル13、Xステージ14、Yステージ15の動作を制御して、測定対象物を測定平面(X−Y平面)内の適正な測定位置に設定する。昇降機構制御部84は、昇降機構12の動作を制御して、測定ヘッドMHの測定対象物までの高さを測定可能高さに設定する。   The controller 100 is substantially formed as a computer unit, and includes a light source control unit 80, an image memory 81, an image processing unit 82, a three-dimensional measurement data calculation unit 83, an evaluation module 90 particularly related to the present invention, and a lifting mechanism. A control unit 84, a rotary table control unit 85, an X stage control unit 86, and a Y stage control unit 87 are provided. The rotary table control unit 85, the X stage control unit 86, and the Y stage control unit 87 control the operations of the rotary table 13, the X stage 14, and the Y stage 15, respectively, so that the measurement object is measured on the measurement plane (XY plane). Set to the appropriate measurement position. The elevating mechanism control unit 84 controls the operation of the elevating mechanism 12 and sets the height of the measuring head MH to the measurement object to a measurable height.

スリット光源ユニット2は、図4に示すように、スリット光源としてのレーザスリット投光器20と、レーザスリット投光器20から出たスリット光をその光軸に平行な平行光とするシリンドリカルレンズ21とを備えている。シリンドリカルレンズ21により、レーザスリット投光器20から出た扇状に拡がっていくスリット光はスリット光軸に平行な平行光に変換され、測定対象物を照射する。   As shown in FIG. 4, the slit light source unit 2 includes a laser slit projector 20 as a slit light source, and a cylindrical lens 21 that converts the slit light emitted from the laser slit projector 20 into parallel light parallel to the optical axis. Yes. By the cylindrical lens 21, the slit light that spreads out in a fan shape from the laser slit projector 20 is converted into parallel light parallel to the slit optical axis, and irradiates the measurement object.

撮像ユニット3は、テレセントリックレンズユニット30と、面状に配置された多数の受光素子(CCDやCMOS)からなる撮像部31と、テレセントリックレンズユニット30の被写体側に配置されたP偏光板32とを備えている。スリット光光源ユニット2からのスリット光が測定対象物の表面に照射され、そこで反射した反射光が、撮像ユニット3の撮像光軸に沿って、P偏光板32とテレセントリックレンズユニット30とを通過して撮像部31に達する様子が図5に示されている。スリット光光源ユニット2のスリット光軸と撮像ユニット3の撮像光軸とが交差する交差角、つまり撮像角αは、この実施の形態では約11度という極めて狭い角度を採用している。従って、図5において点線で示すように、撮像部31の撮像面31aが撮像光軸に直角となる姿勢であると、スリット光軸方向に沿った測定深さの範囲がテレセントリックレンズユニット30の被写界深度を超えていると測定深さの範囲においてピントの合わない領域が生じる。これを回避するため、撮像部31の撮像面31aを撮像光軸に対してあおり角βを作り出すように傾け、あおり撮影の原理で被写界深度を稼いでいる。これにより、テレセントリックレンズユニット30のもつ被写界深度以上の測定範囲においてもピンボケのない撮影画像が取得できる。   The imaging unit 3 includes a telecentric lens unit 30, an imaging unit 31 made up of a large number of light receiving elements (CCD and CMOS) arranged in a plane, and a P polarizing plate 32 arranged on the subject side of the telecentric lens unit 30. I have. The slit light from the slit light source unit 2 is irradiated on the surface of the measurement object, and the reflected light reflected there passes through the P polarizing plate 32 and the telecentric lens unit 30 along the imaging optical axis of the imaging unit 3. FIG. 5 shows how the image pickup unit 31 is reached. In this embodiment, an extremely narrow angle of about 11 degrees is adopted as the crossing angle at which the slit optical axis of the slit light source unit 2 and the imaging optical axis of the imaging unit 3 intersect, that is, the imaging angle α. Therefore, as shown by a dotted line in FIG. 5, when the imaging surface 31 a of the imaging unit 31 is in a posture perpendicular to the imaging optical axis, the range of the measurement depth along the slit optical axis direction is the object of the telecentric lens unit 30. If the depth of field is exceeded, an out-of-focus area occurs in the measurement depth range. In order to avoid this, the imaging surface 31a of the imaging unit 31 is tilted with respect to the imaging optical axis so as to create the tilt angle β, and the depth of field is gained by the principle of tilt shooting. As a result, a photographed image without blur can be acquired even in a measurement range that is greater than the depth of field of the telecentric lens unit 30.

撮像ユニット3からコントローラ100に送られてきた撮像画像(画像データ)は、画像メモリ81に展開される。さらに、必要に応じて、画像処理部82によって座標変換やレベル補正、エッジ検出などの画像処理を施され、スリット光による光切断線Sが検出される。三次元測定データ演算部83は、スリット光の照射点や照射角度、スリット光軸と撮像光軸とのなす角度が既知なので、画像処理部82で検出された光切断線Sの座標値から三角測量法に基づいて演算することで、光切断線Sつまり複数の直線状深溝を形成している測定対象物の3次元断面形状に対応する多数の測定点データ(距離画像)を得ることができる。ここでいう距離画像とは、測定点としての画素にその三次元位置座標値を割り当てた測定データである。なお、三角測量法に基づく演算に代えて、その演算結果を格納したテーブルを用いる方法を採用してもよい。   The captured image (image data) sent from the imaging unit 3 to the controller 100 is developed in the image memory 81. Furthermore, if necessary, the image processing unit 82 performs image processing such as coordinate conversion, level correction, and edge detection, and the light cutting line S by the slit light is detected. Since the three-dimensional measurement data calculation unit 83 knows the irradiation point and irradiation angle of the slit light, and the angle formed between the slit optical axis and the imaging optical axis, the three-dimensional measurement data calculation unit 83 determines the triangle from the coordinate value of the light cutting line S detected by the image processing unit 82. By calculating based on the surveying method, it is possible to obtain a large number of measurement point data (distance images) corresponding to the optical cutting line S, that is, the three-dimensional cross-sectional shape of the measurement object forming a plurality of linear deep grooves. . The distance image here is measurement data in which a three-dimensional position coordinate value is assigned to a pixel as a measurement point. Instead of the calculation based on the triangulation method, a method using a table storing the calculation result may be adopted.

三次元測定データ演算部83によって生成された測定点データは評価モジュール90に転送される。評価モジュール90は、図6に示すように、表面評価モジュール90Aと欠陥評価モジュール90Bからなる。表面評価モジュール90Aは、転送されてきた測定点データに上述した位置合わせアルゴリズムを適用して測定点群の基準点群への位置合わせを行い測定対象物の表面形状を評価する。欠陥評価モジュール90Bは、表面評価モジュール90Aから出力された測定点群の基準点群への位置合わせ結果に基づいて測定対象物の表面欠陥を評価する。表面評価モジュール90A及び欠陥評価モジュール90Bは、それぞれの評価を測定対象物において予め区分けされた所定ブロック単位で行う。   The measurement point data generated by the three-dimensional measurement data calculation unit 83 is transferred to the evaluation module 90. As shown in FIG. 6, the evaluation module 90 includes a surface evaluation module 90A and a defect evaluation module 90B. The surface evaluation module 90A applies the above-described alignment algorithm to the transferred measurement point data, aligns the measurement point group with the reference point group, and evaluates the surface shape of the measurement object. The defect evaluation module 90B evaluates the surface defect of the measurement object based on the alignment result of the measurement point group output from the surface evaluation module 90A to the reference point group. The surface evaluation module 90 </ b> A and the defect evaluation module 90 </ b> B perform each evaluation in units of predetermined blocks that are divided in advance in the measurement object.

表面評価モジュール90Aは、測定データ入力部91と、基準点データ格納部92と、点群対応付け部93と、重み演算部94と、収束評価部95と、点群変換部96とを有する。測定データ入力部91は三次元測定データ演算部83から測定点データを受け取る。基準データ格納部92は、測定対象物の表面形状を示す基準点データを格納する。基準点データは、測定対象物において予め区分けされた所定ブロック毎に測定点に対応するように設定された理想的な仕上がり形状を示すデータである。点群対応付け部93は、前記所定ブロック単位で、前述した位置合わせアルゴリズムに基づいて測定点と基準点とを対応させる。重み演算部94は、前述したトータル重み係数:w、つまり隣接点間距離重み係数:γiと対応点間距離重み係数:ρiとを演算し、それらを乗算した値を求める。点群変換部96は、対応付けされた基準点群に測定点群を位置合わせするための合同変換パラメータを求め、この合同変換パラメータを用いて測定点群の位置座標を変換する。収束評価部95は、合同変換パラメータを用いて測定点群が基準点群に収束移動させようとする際にその逐次収束評価値を演算し、測定点群の移動が基準点群に逐次収束していくかどうかを評価する。この実施形態では、点群対応付け部93と重み演算部94と収束評価部95と点群変換部96とは、応する測定点と基準点との間の距離を逐次収束させる逐次収束処理に基づいて前記測定点と前記基準点とを位置合わせする位置合わせ処理手段を構成しており、特に重み演算部94は隣接点間距離重み係数と対応点間距離重み係数の2つの重み係数を決定する重み係数決定手段として構成されている。 The surface evaluation module 90A includes a measurement data input unit 91, a reference point data storage unit 92, a point group association unit 93, a weight calculation unit 94, a convergence evaluation unit 95, and a point group conversion unit 96. The measurement data input unit 91 receives measurement point data from the three-dimensional measurement data calculation unit 83. The reference data storage unit 92 stores reference point data indicating the surface shape of the measurement object. The reference point data is data indicating an ideal finished shape set so as to correspond to the measurement point for each predetermined block previously divided in the measurement object. The point group associating unit 93 associates the measurement point with the reference point based on the above-described alignment algorithm in the predetermined block unit. The weight calculation unit 94 calculates the total weight coefficient w described above, that is, the distance weight coefficient between adjacent points: γ i and the distance weight coefficient between corresponding points: ρ i , and obtains a value obtained by multiplying them. The point group conversion unit 96 obtains a joint conversion parameter for aligning the measurement point group with the associated reference point group, and converts the position coordinates of the measurement point group using the joint conversion parameter. The convergence evaluation unit 95 calculates the successive convergence evaluation value when the measurement point group tries to converge and move to the reference point group using the joint transformation parameter, and the movement of the measurement point group converges sequentially to the reference point group. Evaluate whether or not In this embodiment, the point group association unit 93, the weight calculation unit 94, the convergence evaluation unit 95, and the point group conversion unit 96 perform a sequential convergence process for sequentially converging the distance between the corresponding measurement point and the reference point. Based on this, an alignment processing means for aligning the measurement point and the reference point is configured. In particular, the weight calculation unit 94 determines two weighting factors, a distance weighting factor between adjacent points and a distance weighting factor between corresponding points. It is comprised as a weighting coefficient determination means to do.

表面欠陥評価手段として機能する表面欠陥評価モジュール90Bは、誤対応測定群抽出部97と欠陥判定部98とを有する。誤対応測定群抽出部97は、上述した表面欠陥判定アルゴリズムに基づいて、対応する測定点と基準点との間の対応点間距離が第1閾値より大きい測定点を誤対応測定点とみなしていくことで誤対応測定群(誤対応領域)を抽出する。欠陥判定部98は、抽出された誤対応測定点群に含まれる誤対応測定点のうち隣接する測定点との距離が第2閾値より小さい近傍測定点の数が第3閾値より大きい時に当該近傍測定点を含む領域を表面欠陥と判定する。   The surface defect evaluation module 90 </ b> B that functions as a surface defect evaluation unit includes an erroneous correspondence measurement group extraction unit 97 and a defect determination unit 98. Based on the surface defect determination algorithm described above, the erroneous correspondence measurement group extraction unit 97 regards a measurement point in which the distance between the corresponding measurement points and the reference point is greater than the first threshold as an erroneous correspondence measurement point. As a result, an erroneous correspondence measurement group (incorrect correspondence region) is extracted. When the number of neighboring measurement points whose distance from the adjacent measurement points among the erroneous measurement points included in the extracted erroneous measurement point group is smaller than the second threshold is larger than the third threshold, the defect determination unit 98 A region including the measurement point is determined as a surface defect.

なお、この評価モジュール90では、レーザスリット光の拡散方向、ここではY方向における誤差は無視できるので、図1を用いながら説明したアルゴリズムの適用において、測定点データと基準点データとは、X座標値とZ座標値だけを考慮して実行することができる。   In this evaluation module 90, since the error in the laser slit light diffusion direction, here the Y direction, can be ignored, in the application of the algorithm described with reference to FIG. It can be executed considering only the value and the Z coordinate value.

上述したように構成された表面検査装置を用いた、測定対象物の検査手順を図7に示されたフローチャートを用いて以下に説明する。ここでの測定対象物は、長方形のプレート体に表面に多数の直線状の深溝が形成されたもので、その測定領域は400mm×300mm程度である。この測定領域は100mm×15mmの測定ブロックに区分けされている。1回のX軸方向走査で4つの測定ブロックを走査して、走査ピッチと撮像解像度によって規定される測定単位での直線状深溝の3次元断面形状位置を表す測定点データを取得して、測定ブロック毎に区分けしてメモリに格納する。1回のX軸方向走査が完了する毎に所定ピッチでY軸方向移動を行い、次の測定ブロックに対するX軸方向走査を逆方向で行う。このような、X軸方向走査とY軸方向移動を繰り返すことで、全測定領域おける直線状深溝の測定点データを取得する。さらに、測定死角の発生を考慮して、測定対象物を90度回転させた状態で、再度同じ測定領域における測定を行う。なお、取得した測定点データを用いた測定対象物の測定結果に対する評価は、つまり測定対象物に対する検査は、各ブロック単位で行われ、各ブロック単位での検査結果をまとめて、最終的な総合判定が行われる。   The procedure for inspecting the measurement object using the surface inspection apparatus configured as described above will be described below with reference to the flowchart shown in FIG. The object to be measured here is a rectangular plate having a large number of straight deep grooves formed on the surface thereof, and its measurement area is about 400 mm × 300 mm. This measurement area is divided into 100 mm × 15 mm measurement blocks. Four measurement blocks are scanned in one X-axis direction scan, and measurement point data representing the three-dimensional cross-sectional shape position of the linear deep groove in the measurement unit defined by the scanning pitch and imaging resolution is acquired and measured. Divide into blocks and store in memory. Each time one X-axis direction scan is completed, the Y-axis direction is moved at a predetermined pitch, and the X-axis direction scan for the next measurement block is performed in the reverse direction. By repeating such scanning in the X-axis direction and movement in the Y-axis direction, the measurement point data of the linear deep groove in the entire measurement region is acquired. Further, taking into account the generation of the measurement blind spot, the measurement in the same measurement region is performed again with the measurement object rotated by 90 degrees. In addition, the evaluation of the measurement result of the measurement object using the acquired measurement point data, that is, the inspection of the measurement object is performed in each block unit, and the inspection result in each block unit is put together to make a final synthesis. A determination is made.

上述した検査を行うために、測定対象物が回転テーブル13にセットされる(#01)。図示されていない測定開始ボタンが操作されると(#02Yes分岐)、測定が開始される。まず、光源制御部80によってレーザスリット投光器20がONされ、スリット光が照射される(#03)。測定開始ポイントである1番目の測定ブロックの左エッジがスリット光によって照射されるように、Xステージ14及びYステージ15、回転テーブル13を動作させる(#04)。   In order to perform the above-described inspection, the measurement object is set on the rotary table 13 (# 01). When a measurement start button (not shown) is operated (# 02 Yes branch), measurement is started. First, the laser slit projector 20 is turned on by the light source controller 80, and the slit light is irradiated (# 03). The X stage 14, the Y stage 15, and the rotary table 13 are operated so that the left edge of the first measurement block, which is the measurement start point, is irradiated with the slit light (# 04).

Xステージ14を正方向に定速移動させながらX軸方向走査を行う(#05)。それとともに、撮像ユニット3からの画像データを画像メモリ81に転送する(#06)。このX軸方向走査と画像データの取得は、スリット光が測定対象物の側端に達するまで行われる。スリット光が測定対象物の側端に達すると(#07Yes)、X軸方向走査を停止する(#08)。   X-axis direction scanning is performed while moving the X stage 14 at a constant speed in the positive direction (# 05). At the same time, the image data from the imaging unit 3 is transferred to the image memory 81 (# 06). This X-axis direction scanning and image data acquisition are performed until the slit light reaches the side edge of the measurement object. When the slit light reaches the side edge of the measurement object (# 07 Yes), scanning in the X-axis direction is stopped (# 08).

X軸方向走査を停止すると、画像処理部82は、転送された画像データを処理し、その光切断線画素位置情報を生成する(#09)。この光切断線画素位置情報から、三次元測定データ演算部83は、画素位置とその画素位置から三角測量法に基づいて演算された3次元位置との関係を格納したテーブルを利用して、光切断線画素位置情報に基づき深溝の三次元座標値を読み出し、この値を測定点データとして各測定点に対応付けられたメモリアドレスに転送する(#10)。もちろん、テーブルを用いずに、その都度、光切断線画素位置情報を用いて三角測量法に基づく演算を行い、深溝の三次元座標値を求めて測定点データとしてもよい。   When the scanning in the X-axis direction is stopped, the image processing unit 82 processes the transferred image data and generates the optical cutting line pixel position information (# 09). From this light section line pixel position information, the three-dimensional measurement data calculation unit 83 uses a table storing the relationship between the pixel position and the three-dimensional position calculated from the pixel position based on the triangulation method. The three-dimensional coordinate value of the deep groove is read based on the cutting line pixel position information, and this value is transferred as measurement point data to the memory address associated with each measurement point (# 10). Of course, instead of using the table, the calculation based on the triangulation method may be performed using the light cutting line pixel position information each time, and the three-dimensional coordinate value of the deep groove may be obtained as the measurement point data.

続いて、形状評価モジュール90Aが上述した位置合わせアルゴリズムを用いて測定点データと基準点データの位置合わせを行う位置合わせルーチンを実行する(#11)。この位置合わせルーチンでは、図8に示すように、測定点データ(測定点群の位置データ)が読み出され(#110)、さらにその測定点群に対応する基準点データ(基準点群の位置データ)が読み出される(#111)。最も小さい対応点距離を有するように測定点と基準点を対応付ける(#112)。上述した対応点間距離重み係数:ρiと隣接点間距離重み係数:γiとを算出し、これらを掛け合わせてトータル重み係数:wiを求める(#113)。さらに、このトータル重み係数:wiと対応点間距離をeiとを用いて、逐次収束評価値:Jを算出する(#114)。算出された逐次収束評価値に基づいてこの対応付けられた対応点群が逐次収束しているかどうかチェックされる(#115)。収束しない場合(#115No分岐)、ステップ#112に戻り再度測定点と基準点との対応付けを行う。収束する場合(#115Yes分岐)、対応点群ができる限り一致するような合同変換パラメータを生成する(#116)。生成された合同変換パラメータを用いて測定点群の位置座標を変換し、測定点群を移動させる(#117)。測定点群の移動後と移動前の位置座標からその平均移動量を算出する移動後測定点群と基準点群とから平均対応距離を算出する(#118)。算出された平均対応距離平均移動量と予め設定された閾値を比較して、終了条件が満たされたかどうかチェックされる(#119)。なお、このチェックステップにおいて、上限の繰り返し回数を付加的に設定しておくと好都合である。終了条件が満たされていない場合(#119No分岐)、ステップ#112に戻り、移動後の測定点と基準点との対応付けを行う。終了条件が満たされた場合(#119Yes分岐)、この位置合わせルーチンを終了する。 Subsequently, the shape evaluation module 90A executes an alignment routine for aligning measurement point data and reference point data using the above-described alignment algorithm (# 11). In this alignment routine, as shown in FIG. 8, measurement point data (position data of the measurement point group) is read (# 110), and reference point data corresponding to the measurement point group (position of the reference point group) Data) is read out (# 111). The measurement point and the reference point are associated with each other so as to have the smallest corresponding point distance (# 112). The above-mentioned distance weight coefficient between corresponding points: ρ i and the distance weight coefficient between adjacent points: γ i are calculated and multiplied to obtain a total weight coefficient: w i (# 113). Further, using this total weight coefficient: w i and the distance between corresponding points e i , a successive convergence evaluation value: J is calculated (# 114). Based on the calculated successive convergence evaluation value, it is checked whether or not the associated corresponding point group is successively converged (# 115). If not converged (# 115 No branch), the process returns to step # 112 to associate the measurement point with the reference point again. In the case of convergence (# 115 Yes branch), a joint transformation parameter is generated so that the corresponding point group matches as much as possible (# 116). The position coordinate of the measurement point group is converted using the generated joint conversion parameter, and the measurement point group is moved (# 117). The average corresponding distance is calculated from the measurement point group after movement for calculating the average movement amount from the position coordinates after the movement of the measurement point group and before the movement and the reference point group (# 118). The calculated average corresponding distance average moving amount is compared with a preset threshold value to check whether the end condition is satisfied (# 119). In this check step, it is convenient to additionally set the upper limit number of repetitions. If the end condition is not satisfied (# 119 No branch), the process returns to step # 112 to associate the measurement point after movement with the reference point. When the termination condition is satisfied (# 119 Yes branch), this alignment routine is terminated.

位置合わせルーチンが終了すると、欠陥評価モジュール90Bが上述した表面欠陥判定アルゴリズムを用いて表面欠陥領域の検出を行う欠陥判定ルーチンを実行する(#12)。この欠陥判定ルーチンでは、図9に示すように、まず、対応点間距離閾値である第1閾値を用いて、対応点間距離:eiが第1閾値以上となる測定点の集合体(特定測定点群)を誤対応領域として抽出する処理を実行する(#120)。誤対応領域が抽出されなかった場合(#121No分岐)、この欠陥判定ルーチンを終了する。誤対応領域が抽出された場合(#121Yes分岐)、誤対応領域に含まれている測定点間の隣接点間距離が予め設定されている隣接点間距離閾値としての第2閾値未満となる測定点を近傍測定点として特定していく(#122)。さらに、特定された近傍測定点の数をカウントする(#123)。近傍測定点のカウント数が予め設定されている近傍点数閾値としての第3閾値未満の場合(#124No分岐)、この誤対応領域を表面欠陥領域と判定できないとして、ステップ#120にジャンプして、次の誤対応領域の抽出を行う。近傍測定点のカウント数が第3閾値以上の場合(#124Yes分岐)、この誤対応領域を表面欠陥領域と判定し(#125)、この誤対応領域に含まれる測定点の3次元位置座標値などを表面欠陥領域に関する情報として記録する(#126)。 When the alignment routine ends, the defect evaluation module 90B executes a defect determination routine for detecting a surface defect area using the above-described surface defect determination algorithm (# 12). In this defect determination routine, as shown in FIG. 9, first, using a first threshold that is a distance threshold between corresponding points, a set of measurement points (specific measurement) where the distance between corresponding points: ei is equal to or greater than the first threshold. A process of extracting (point group) as an erroneous correspondence region is executed (# 120). If no erroneous correspondence region is extracted (# 121 No branch), this defect determination routine is terminated. When an erroneous correspondence region is extracted (# 121 Yes branch), the measurement is such that the distance between adjacent points between measurement points included in the erroneous correspondence region is less than a second threshold value as a preset distance threshold between adjacent points. The point is specified as a nearby measurement point (# 122). Further, the number of specified nearby measurement points is counted (# 123). If the count number of the nearby measurement points is less than the third threshold value as a preset neighborhood score threshold (# 124 No branch), jump to step # 120, assuming that this miscorresponding region cannot be determined as a surface defect region, The next erroneous correspondence area is extracted. When the count number of the nearby measurement points is equal to or greater than the third threshold (# 124 Yes branch), this miscorresponding region is determined as a surface defect region (# 125), and the three-dimensional position coordinate values of the measuring points included in this miscorresponding region Are recorded as information on the surface defect area (# 126).

なお、ステップ#08でX軸方向走査を停止すると、位置合わせルーチン及び欠陥判定ルーチンが実行されている間に、Yステージ15が動作され、所定のピッチでY軸方向のシフトが行われる(#13)。つまり、ステップ#09から#12までの評価処理と、Y軸方向のシフト処理が同時に行われる。この評価処理とシフト処理の両方が終わると、X軸方向走査がまだ残っているかどうかのチェックが行われる(#14)。   When scanning in the X-axis direction is stopped in step # 08, the Y stage 15 is operated while the alignment routine and defect determination routine are being executed, and a shift in the Y-axis direction is performed at a predetermined pitch (# 13). That is, the evaluation process from steps # 09 to # 12 and the shift process in the Y-axis direction are performed simultaneously. When both the evaluation process and the shift process are completed, it is checked whether or not the X-axis direction scan still remains (# 14).

ステップ#14のチェックでX軸方向走査がまだ残っている場合(#14Yes分岐)、X軸方向走査の方向を反転し(#15)、ステップ#05に戻ってX軸方向走査を行う。ステップ#14のチェックでX軸方向走査が残っていない場合(#14No分岐)、レーザスリット投光器20がOFFされ、スリット光の照射が停止する(#16)。さらに、回転テーブル13を90度回転測定死角の発生に伴う測定不能箇所の測定データを補完するために、回転テーブル13を90度回転させる必要があるかどうかをチェックする(#17)。回転テーブル13を90度回転させる必要がある場合は(#17Yes分岐)、回転テーブル13の90度回転動作を行い、再びステップ#03に戻り、この測定を繰り返す。なお、この90度の追加回転で不十分な場合には、さらに90度毎のあと2回までの回転(最初の姿勢位置に対する180度位置と270度位置)が行われる。回転テーブル13を90度回転させる必要がない場合は(#17No分岐)、全ての測定ブロックにおける欠陥評価結果に基づいて総合判定を行う(#17)。この総合判定において、欠陥の位置を測定対象物の全体を示す全体図の上でマーキングした欠陥位置表示図をモニタ又はプリントを通じて出力することができる。   If the X-axis direction scan still remains in the check of step # 14 (# 14 Yes branch), the direction of the X-axis direction scan is reversed (# 15), and the process returns to step # 05 to perform the X-axis direction scan. If no X-axis direction scanning remains in the check of step # 14 (# 14 No branch), the laser slit projector 20 is turned off and the slit light irradiation is stopped (# 16). Further, it is checked whether or not it is necessary to rotate the turntable 13 by 90 degrees in order to complement the measurement data of the non-measurable part accompanying the generation of the 90-degree rotation blind spot (# 17). If it is necessary to rotate the rotary table 13 by 90 degrees (# 17 Yes branch), the rotary table 13 is rotated by 90 degrees, and the process returns to step # 03 again to repeat this measurement. If the additional rotation of 90 degrees is not sufficient, another 90 rotations (180 degree position and 270 degree position with respect to the initial posture position) are performed every 90 degrees. When it is not necessary to rotate the turntable 13 by 90 degrees (# 17 No branch), comprehensive determination is performed based on the defect evaluation results in all measurement blocks (# 17). In this comprehensive determination, a defect position display diagram in which the position of the defect is marked on the overall view showing the entire measurement object can be output through a monitor or a print.

以下、別実施形態を例示する。
(1)上記実施形態では、隣接点間距離重み係数は位置合わせルーチンにおいて算出していたが、この隣接点間距離重み係数を基準点群の隣接点間距離に基づいて求める場合、予め算定しておいてテーブル化しておくことで演算速度が高速化する。
(2)上記実施の形態では、判定条件として予め設定された多くの閾値が用いられていたが、この閾値を予め設定されたものではなく、測定対象物の表面形状あるいは測定結果の統計学的な特性から閾値が算定されるような構成を採用してもよい。
Hereinafter, another embodiment is illustrated.
(1) In the above embodiment, the distance weight coefficient between adjacent points is calculated in the alignment routine. However, when the distance weight coefficient between adjacent points is obtained based on the distance between adjacent points of the reference point group, it is calculated in advance. The calculation speed is increased by making a table in advance.
(2) In the above embodiment, many threshold values set in advance are used as determination conditions. However, the threshold values are not set in advance, and the surface shape of the measurement object or the statistical result of the measurement result A configuration in which the threshold value is calculated from various characteristics may be adopted.

本発明で用いられている位置合わせアルゴリズムの説明図Illustration of alignment algorithm used in the present invention 本発明で用いられている表面欠陥判定アルゴリズムの説明図Explanatory diagram of surface defect judgment algorithm used in the present invention 本発明による位置合わせアルゴリズムを採用した表面検査装置の構成を模式的に示す模式図The schematic diagram which shows typically the structure of the surface inspection apparatus which employ | adopted the positioning algorithm by this invention 表面検査装置の測定ヘッドの構成を示す図解斜視図Illustrated perspective view showing configuration of measuring head of surface inspection apparatus 測定ヘッドにおけるスリット光軸と撮像光軸との関係及び撮像光軸と撮像面との関係を模式的に示す展開図Development view schematically showing the relationship between the slit optical axis and the imaging optical axis and the relationship between the imaging optical axis and the imaging surface in the measurement head. 表面検査装置の評価モジュールの構成を示す機能ブロック図Functional block diagram showing the configuration of the evaluation module of the surface inspection device 表面検査装置における表面欠陥検出制御の流れを示すフローチャートFlow chart showing flow of surface defect detection control in surface inspection apparatus 位置合わせルーチンを示すフローチャートFlow chart showing the alignment routine 欠陥判定ルーチンを示すフローチャートFlow chart showing defect determination routine

符号の説明Explanation of symbols

1: 測定装置部
2:スリット光源ユニット
3: 撮像ユニット
20:レーザスリット投光器
21:シリンドリカルレンズ
30:テレセントリックレンズユニット
31:撮像部
31a:撮像面
32:P偏光板
83:評価部
90:評価モジュール
90A:形状評価モジュール
90B:欠陥評価モジュール
91:データ入力部
92:点群対応付け部
93:基準点データ格納部
94:重み演算部
95:収束評価部
96:点群変換部
97:誤対応測定群抽出部
98:欠陥判定部
100:コントローラ
1: Measuring device unit 2: Slit light source unit 3: Imaging unit 20: Laser slit projector 21: Cylindrical lens 30: Telecentric lens unit 31: Imaging unit 31a: Imaging surface 32: P polarizing plate 83: Evaluation unit 90: Evaluation module 90A : Shape evaluation module 90B: defect evaluation module 91: data input unit 92: point group association unit 93: reference point data storage unit 94: weight calculation unit 95: convergence evaluation unit 96: point group conversion unit 97: erroneous correspondence measurement group Extraction unit 98: Defect determination unit 100: Controller

Claims (6)

測定対象物の形状に対応する多数の測定点の位置情報を含む測定点データを入力する測定データ入力部と、前記測定対象物の基準形状に対応する多数の基準点の位置情報を含む基準点データを格納する基準データ格納部と、対応する測定点と基準点との間の距離を逐次収束させる逐次収束処理に基づいて前記測定点と前記基準点とを位置合わせする位置合わせ処理手段と、前記位置合わせ処理手段による位置合わせ処理後の前記測定点データと前記基準点データとに基づいて前記測定対象物の形状を評価する形状評価手段とを含み、前記位置合わせ処理手段は隣接する前記測定点の間の隣接点間距離又は隣接する前記基準点の間の隣接点間距離に基づいて隣接点間距離重み係数を決定する重み係数決定手段を有し、当該隣接点間距離重み係数を前記逐次収束処理における逐次収束評価値を求める際に用いる物体形状評価装置。   A measurement data input unit for inputting measurement point data including position information of a large number of measurement points corresponding to the shape of the measurement object, and a reference point including position information of a large number of reference points corresponding to the reference shape of the measurement object A reference data storage unit for storing data, and an alignment processing means for aligning the measurement point and the reference point based on a sequential convergence process for sequentially converging the distance between the corresponding measurement point and the reference point; Shape evaluation means for evaluating the shape of the measurement object based on the measurement point data and the reference point data after the alignment processing by the alignment processing means, and the alignment processing means is adjacent to the measurement A weight coefficient determining means for determining a distance weight coefficient between adjacent points based on a distance between adjacent points between adjacent points or a distance between adjacent points between adjacent reference points; Object shape evaluation apparatus used to obtain the sequential convergence evaluation value in the sequential convergence process. 前記重み係数決定手段は前記対応する測定点と基準点との間の対応点間距離に基づいて対応点間距離重み係数を決定し、当該対応点間距離重み係数を前記位置合わせ処理手段が前記逐次収束処理における逐次収束評価値を求める際に用いる請求項1に記載の物体形状評価装置。   The weighting factor determining means determines a distance weighting factor between corresponding points based on a distance between corresponding points between the corresponding measurement point and a reference point, and the positioning processing means determines the distance weighting factor between corresponding points. The object shape evaluation apparatus according to claim 1, which is used when obtaining a sequential convergence evaluation value in the sequential convergence processing. 前記隣接点間距離重み係数は前記基準点の隣接点間距離に基づいて決定される請求項1または2に記載の物体形状評価装置。   The object shape evaluation apparatus according to claim 1, wherein the distance weight coefficient between adjacent points is determined based on a distance between adjacent points of the reference point. 前記隣接点間距離重み係数は予め算定され、テーブル化されている請求項3に記載の物体形状評価装置。   The object shape evaluation apparatus according to claim 3, wherein the distance weight coefficient between adjacent points is calculated in advance and tabulated. 前記隣接点間距離重み係数は前記隣接点間距離をパラメータとする閾値関数によって求められる請求項1から4のいずれか一項に記載の物体形状評価装置。   5. The object shape evaluation apparatus according to claim 1, wherein the distance weight coefficient between adjacent points is obtained by a threshold function using the distance between adjacent points as a parameter. 前記測定点データは、スリット光によって照射された前記測定対象物の撮影画像を処理することによって得られた距離画像から取り出された三次元位置データである請求項1から5のいずれか一項に記載の物体形状評価装置。 The measuring point data from claim 1 is a three-dimensional position data retrieved from the acquired distance image by processing the captured image of the measurement object illuminated by the slit light in any one of 5 The object shape evaluation apparatus described.
JP2008278401A 2008-10-29 2008-10-29 Object shape evaluation device Expired - Fee Related JP5049246B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2008278401A JP5049246B2 (en) 2008-10-29 2008-10-29 Object shape evaluation device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2008278401A JP5049246B2 (en) 2008-10-29 2008-10-29 Object shape evaluation device

Publications (2)

Publication Number Publication Date
JP2010107300A JP2010107300A (en) 2010-05-13
JP5049246B2 true JP5049246B2 (en) 2012-10-17

Family

ID=42296892

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2008278401A Expired - Fee Related JP5049246B2 (en) 2008-10-29 2008-10-29 Object shape evaluation device

Country Status (1)

Country Link
JP (1) JP5049246B2 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5445848B2 (en) * 2010-02-05 2014-03-19 アイシン精機株式会社 Object shape evaluation device
JP2012122844A (en) 2010-12-08 2012-06-28 Aisin Seiki Co Ltd Surface inspection device
JP5867787B2 (en) * 2012-03-23 2016-02-24 アイシン精機株式会社 Defect extraction apparatus and defect extraction method
JP6219628B2 (en) * 2013-07-17 2017-10-25 株式会社ディスコ Cutting blade tip shape detection method
JP6242619B2 (en) * 2013-07-23 2017-12-06 株式会社ディスコ Processing equipment
JP6384171B2 (en) * 2014-07-23 2018-09-05 アイシン精機株式会社 Defect extraction apparatus and defect extraction method
CN106802138B (en) * 2017-02-24 2019-09-24 先临三维科技股份有限公司 A kind of 3 D scanning system and its scan method

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5715166A (en) * 1992-03-02 1998-02-03 General Motors Corporation Apparatus for the registration of three-dimensional shapes
JP3465988B2 (en) * 1994-04-27 2003-11-10 松下電器産業株式会社 Method and apparatus for estimating movement and depth
JPH11214289A (en) * 1998-01-29 1999-08-06 Nikon Corp Position measurement method and exposure method using the same
JP4306006B2 (en) * 1999-03-30 2009-07-29 コニカミノルタセンシング株式会社 Three-dimensional data input method and apparatus
JP2003344040A (en) * 2002-05-24 2003-12-03 Canon Inc Error distribution method
JP2007108835A (en) * 2005-10-11 2007-04-26 Keyence Corp Image processor
JP2008002848A (en) * 2006-06-20 2008-01-10 Tateyama Machine Kk Flaw inspection device of rod-shaped rotary tool and flaw detection method of rod-shaped rotary tool
JP4952267B2 (en) * 2007-01-19 2012-06-13 コニカミノルタホールディングス株式会社 Three-dimensional shape processing apparatus, three-dimensional shape processing apparatus control method, and three-dimensional shape processing apparatus control program

Also Published As

Publication number Publication date
JP2010107300A (en) 2010-05-13

Similar Documents

Publication Publication Date Title
JP5049246B2 (en) Object shape evaluation device
JP5252184B2 (en) Uneven surface inspection device
JP2009198440A (en) Correction pattern image generation device, pattern inspection apparatus, and correction pattern image generation method
JP5397537B2 (en) Height measuring method, height measuring program, height measuring device
JP2007256091A (en) Method and apparatus for calibrating range finder
JP5049247B2 (en) Surface defect evaluation system
JP6980631B2 (en) Inspection method and inspection equipment
JP7353757B2 (en) Methods for measuring artifacts
JP2012050013A (en) Imaging apparatus, image processing device, image processing method, and image processing program
JP5403367B2 (en) Object shape evaluation device
JP5445848B2 (en) Object shape evaluation device
KR102177726B1 (en) Method and apparatus for inspecting workpiece
JP6347169B2 (en) Defect extraction apparatus and defect extraction method
JP2007033040A (en) Method and device for calibrating optical head part in three-dimensional shape measuring instrument by optical cutting method
JP4077755B2 (en) POSITION DETECTION METHOD, DEVICE THEREOF, PROGRAM THEREOF, AND CALIBRATION INFORMATION GENERATION METHOD
JP6906177B2 (en) Intersection detection device, camera calibration system, intersection detection method, camera calibration method, program and recording medium
KR102222898B1 (en) Method and apparatus for inspecting workpiece using laser
JP2006023133A (en) Instrument and method for measuring three-dimensional shape
JP4449596B2 (en) Mounting board inspection equipment
JP5867787B2 (en) Defect extraction apparatus and defect extraction method
JP7153514B2 (en) 3D shape inspection device, 3D shape inspection method, 3D shape inspection program, computer
JP2022152480A (en) Three-dimensional measuring device, three-dimensional measuring method, program, system, and method for manufacturing article
JP4650813B2 (en) Reticle defect inspection apparatus and reticle defect inspection method
JP2009229555A (en) Correction pattern image generation device, pattern inspecting device, and correction pattern image generation method
JP2007033039A (en) Method and device for calibrating optical head part in three-dimensional shape measuring instrument by optical cutting method

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20110527

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A821

Effective date: 20110527

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20120412

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20120426

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20120611

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20120628

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20120720

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20150727

Year of fee payment: 3

R150 Certificate of patent or registration of utility model

Free format text: JAPANESE INTERMEDIATE CODE: R150

Ref document number: 5049246

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313117

R360 Written notification for declining of transfer of rights

Free format text: JAPANESE INTERMEDIATE CODE: R360

R360 Written notification for declining of transfer of rights

Free format text: JAPANESE INTERMEDIATE CODE: R360

R371 Transfer withdrawn

Free format text: JAPANESE INTERMEDIATE CODE: R371

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313117

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

LAPS Cancellation because of no payment of annual fees