JP2009236602A - Contact force vector detector and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a contact force vector detector for detecting the strength and the direction of a force applied by a physical object, without utilizing deformation of a contacted element. <P>SOLUTION: This contact force vector detector is equipped with a detection part 101 for detecting a first domain of the physical object contacting with a transmitting plate and a second domain which does not contact the transmitting plate from an input image, acquired by photographing the physical object via the transmitting plate; a characteristic quantity calculating part for calculating a plurality of characteristic quantities for each of images in the first and second domains in the input image, a generating part 107, generating characteristic quantity vectors based on the plurality of characteristic quantities; and a contact-force vector calculating part 109 for calculating contact-force vectors, each showing the strength and direction of contact force, based on a correlation between the quantity vectors and the strengths and directions of contact forces applied to the transmitting plate by the physical object. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a contact force vector detection apparatus and method for detecting the magnitude and direction of a contact force of an object.

In recent years, tactile sensors that detect tactile information have been studied and used as man-machine interfaces. The conventional tactile sensor has problems such as a fragile sensor structure, a large number of wires, and a tactile information detection surface limited to a specific shape.
Further, the conventional tactile sensor has a problem that it cannot acquire any information until the time when the object actually contacts. For example, when a robot equipped with the tactile sensor performs an obstacle avoiding operation, an obstacle existing in a region (a blind spot) where the robot cannot obtain visual information is not recognized until it comes into contact with the tactile sensor. Therefore, the robot is delayed in recognizing the obstacle present in the blind spot, and it is difficult to avoid the obstacle.

  The tactile sensor described in Non-Patent Document 1 realizes a structure with a relatively small number of wires by configuring the tactile information detection surface with pressure-sensitive conductive rubber. In addition, the tactile camera described in Non-Patent Document 2 has two types of markers arranged inside a transparent elastic body, photographs the position change of the marker inside the elastic body, and the tactile sense based on the position change. Information is calculated. The tactile camera described in Non-Patent Document 2 can also be configured with a relatively small number of wires. Both the tactile sensor described in Non-Patent Document 1 and the tactile camera described in Non-Patent Document 2 use the deformation of a contacted element such as a pressure-sensitive conductive rubber and an elastic body in which a marker is arranged to provide tactile information. Calculated.

  On the other hand, the nail-mounted sensor described in Non-Patent Document 3 is an optical sensor that can be mounted on the nail, and the finger is based on the color change accompanying the blood flow change under the nail when the subject applies force to the finger pad. The force applied to the abdomen is calculated. In the pointing device described in Non-Patent Document 4, the amount of movement of the pointer is determined based on the amount of movement of the fingerprint center coordinates of the finger touching the surface of the fingerprint sensor.

Makoto Shimojo, Yuki Taniho, Akio Namiki, Masatoshi Ishikawa, Application to tactile sensors using two-dimensional distributed load measurement method, 5th SICE System Integration Division Annual Conference (SI2004) (Tsukuba, December 19, 2004) / Lecture Proceedings, pp.1144-1145

Kazuyama Kamiyama, Hiroyuki Enomoto, Masahiko Inami, Naoki Kawakami, Tsuji ▲ Show ▼, “Tactile Cameras”, Electrical Engineering E, Vol. 123, No. 1, pp.16-22 (2003)

Hideyuki Ando, Junji Watanabe, Maki Sugimoto, Taro Maeda, “Separation of contact force and bending of nail-mounted sensor by independent component analysis”, Transactions of the Virtual Reality Society of Japan, Vol. 8, No. 4, pp. 379- 387, 2003.

Satoshi Ueda, Atsutoshi Ikeda, Yuichi Kurita, Tsukasa Ogasawara, “Development of a new pointing device using fingerprint deformation”, Proceedings of Human Interface Symposium 2003, p251-254, 2003.

  In the tactile sensor described in Non-Patent Document 1, the detection accuracy is greatly influenced by the characteristics of the pressure-sensitive conductive rubber constituting the detection surface. From the time when the object applies force until the pressure-sensitive conductive rubber is deformed. Since a time lag (delay time) occurs, there is a problem that it is not suitable for real-time tactile information detection, and that wiring is easily cut during detection.

In the tactile camera described in Non-Patent Document 2, the shape of the elastic body provided with the marker is limited to a substantially flat plate shape, and there is a time lag from the point in time when the object applies force until the position of the marker changes. There are problems such as inadequate detection of real-time tactile information.
The nail-mounted sensor described in Non-Patent Document 3 needs to be mounted on the subject's finger. That is, since it is necessary to mount a sensor on the object side and the object is also limited to fingers, it is difficult to use it as a tactile sensor for the robot described above, for example. In addition, the pointing device described in Non-Patent Document 4 detects the amount of movement of the object as tactile information, but does not detect the magnitude and direction of the contact force.

  Therefore, an object of the present invention is to provide a contact force vector detection device capable of detecting the magnitude and direction of a force applied by an object without using deformation of a contacted element.

  The contact force vector detection device according to an aspect of the present invention includes, from an input image obtained by photographing an object through a transmission plate, the first region of the object that is in contact with the transmission plate and the transmission A detection unit that detects a second region that is not in contact with the board; and a feature amount calculation unit that calculates a plurality of feature amounts for each of the image of the first region and the image of the second region in the input image. A generating unit that generates a feature vector based on the plurality of feature values; and the contact based on a correlation between the feature vector and the magnitude and direction of a contact force applied to the transmission plate by the object. A contact force vector calculation unit that calculates a contact force vector indicating the magnitude and direction of the force.

  ADVANTAGE OF THE INVENTION According to this invention, the contact force vector detection apparatus which can detect the magnitude | size and direction of the force which a target object applies without utilizing the deformation | transformation of a to-be-contacted element can be provided.

Embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
As shown in FIG. 1, a contact force vector detection device according to an embodiment of the present invention includes a contour extraction unit 101, a color analysis unit 102, an area calculation unit 103, a center of gravity calculation unit 104, an inertia main axis calculation unit 105, and a curvature calculation. A unit 106, a feature vector generation unit 107, a mapping unit 108, and a contact force vector calculation unit 109.

  In the contact force vector detection device of FIG. 1, for example, an image obtained by photographing the object 201 through the transmission plate 202 in the normal direction to the camera 203 transmission plate 202 as shown in FIG. 2 is input as an input image. The contact force vector indicating the magnitude and direction of the force (contact force) applied by the object 201 to the transmission plate 202 is detected.

  In the contact force vector detection device, the input image is input to the contour extraction unit 101 and the color analysis unit 102. The object 201 is a part of a human body such as a finger and has elasticity. Therefore, as shown in FIG. 3, as the contact force increases, the area of the contact region 301 increases, the color density (pixel value) becomes darker, and the edge becomes sharper. In the following description, it is assumed that the input image is acquired as shown in FIG. 2, but the method for acquiring the input image is not particularly limited. For example, the angle at which the camera 203 captures an image with respect to the transmission plate 202 is not limited to the normal direction, and the shape of the transmission plate 202 may not be flat.

  The contour extraction unit 101 includes an edge detection filter such as a Sobel filter, Roberts filter, or Prewitt filter. The contour extraction unit 101 first calculates mathematical expression (1) below for the differential fx in the horizontal direction (x-axis direction) and the differential fy in the vertical direction (y-axis direction) of the pixel value of each pixel included in the input image. Follow the instructions.

数式（１）において、ｘは入力画像におけるｘ軸方向の座標、ｙは入力画像におけるｙ軸方向の座標を夫々示し、f(x,y)は入力画像の座標(x,y)における画素値を示す。数式（１）において、fxの値は入力画像の座標(x,y)近傍における水平方向の画素値の変化量、fyの値は入力画像の座標(x,y)近傍における垂直方向の画素値の変化量を夫々示す。

In Equation (1), x represents the coordinate in the x-axis direction in the input image, y represents the coordinate in the y-axis direction in the input image, and f (x, y) represents the pixel value at the coordinate (x, y) in the input image. Indicates. In Equation (1), the value of fx is the amount of change in the horizontal pixel value near the coordinates (x, y) of the input image, and the value of fy is the pixel value in the vertical direction near the coordinates (x, y) of the input image. The amount of change is shown respectively.

  Next, the contour extraction unit 101 calculates the sum of absolute values of fx and fy as a change amount of the pixel value in the vicinity of the coordinates (x, y) of the input image. The contour extraction unit 101 detects, as an edge, a portion where the amount of change in the pixel value calculated as described above, that is, a portion where the pixel value changes rapidly. The contour extraction unit 101 extracts the contours of the contact area 301 where the object 201 contacts the transmission plate 202 and the non-contact area 302 where the object 201 does not contact based on the edge detection result. The contour extraction unit 101 passes the extracted contour of the contact region 301 and the contour of the non-contact region 302 to the color analysis unit 102, the area calculation unit 103, the centroid calculation unit 104, the inertia main axis calculation unit 105, and the curvature calculation unit 106.

  The color analysis unit 102 analyzes the color (for example, RGB component) of the contact region 301 based on the input image and the contour of the contact region 301 from the contour extraction unit 101. The color analysis unit 102 calculates the average pixel value of each pixel in the contact area 301 from the analysis result and passes it to the feature vector generation unit 107. In addition, the color analysis unit 102 similarly calculates the average pixel value of each pixel in the non-contact region 302 and passes it to the feature vector generation unit 107.

  The area calculation unit 103 calculates a zero-order moment m00 based on the contour of the contact region 301 from the contour extraction unit 101, and passes it to the feature vector generation unit 107 as the area of the contact region 301. Specifically, the area calculation unit 103 binarizes the pixel value of the pixel inside the contour of the contact region 301 to “1” and the pixel value of the pixel outside the contour to “0”, and follows the following formula (2) Calculate the 0th moment m00.

また、面積算出部１０３は、非接触領域３０２の面積も同様に算出し、特徴量ベクトル生成部１０７に渡す。
重心算出部１０４は、輪郭抽出部１０１からの接触領域３０１の輪郭に基づいて、接触領域３０１の重心を示す座標を算出する。まず、重心算出部１０４は、接触領域３０１の輪郭内側の画素の画素値を「１」、輪郭外側の画素の画素値を「０」に２値化し、以下の数式（３）に従って１次モーメントｍ10及びｍ01を算出する。

The area calculation unit 103 also calculates the area of the non-contact region 302 in the same manner, and passes it to the feature vector generation unit 107.
The center-of-gravity calculation unit 104 calculates coordinates indicating the center of gravity of the contact region 301 based on the contour of the contact region 301 from the contour extraction unit 101. First, the center-of-gravity calculation unit 104 binarizes the pixel value of the pixel inside the contour of the contact area 301 to “1” and the pixel value of the pixel outside the contour to “0”, and first moment according to the following formula (3) m10 and m01 are calculated.

重心算出部１０４は、次に、上記１次モーメントｍ10及びｍ01と、面積算出部１０３によって算出された接触領域３０１の面積（＝ｍ00）とを用いて、以下の数式（４）に従って接触領域３０１の重心を示す座標（ｍx，ｍy）を算出する。

Next, the center-of-gravity calculation unit 104 uses the first moments m10 and m01 and the area (= m00) of the contact region 301 calculated by the area calculation unit 103 according to the following formula (4). The coordinates (mx, my) indicating the center of gravity are calculated.

ここで、ｍxは重心のｘ軸方向の座標、ｍyは重心のｙ軸方向の座標を夫々示す。
また、重心算出部１０４は、非接触領域３０２の重心の座標も同様に算出し、特徴量ベクトル生成部１０７に渡す。
慣性主軸算出部１０５は、輪郭抽出部１０１からの接触領域３０１の輪郭に基づいて、接触領域３０１の慣性主軸を算出する。まず、慣性主軸算出部１０５は、接触領域３０１の輪郭内側の画素の画素値を「１」、輪郭外側の画素の画素値を「０」に２値化し、以下の数式（５）に従って２次モーメントｍ20、ｍ11及びｍ02を算出する。

Here, mx represents the coordinate of the center of gravity in the x-axis direction, and my represents the coordinate of the center of gravity in the y-axis direction.
In addition, the center of gravity calculation unit 104 calculates the coordinates of the center of gravity of the non-contact region 302 in the same manner, and passes it to the feature quantity vector generation unit 107.
The inertia main axis calculation unit 105 calculates the inertia main axis of the contact region 301 based on the contour of the contact region 301 from the contour extraction unit 101. First, the inertial principal axis calculation unit 105 binarizes the pixel value of the pixel inside the contour of the contact area 301 to “1” and the pixel value of the pixel outside the contour to “0”, and performs quadratic according to the following formula (5). Calculate moments m20, m11 and m02.

慣性主軸算出部１０５は、次に、上記２次モーメントｍ20、ｍ11及びｍ20を用いて、以下の数式（６）に従って接触領域３０１の慣性主軸を与える傾きaxisを算出する。

Next, the inertia main axis calculation unit 105 calculates the inclination axis that gives the inertia main axis of the contact region 301 according to the following equation (6) using the second moments m20, m11, and m20.

また、慣性主軸算出部１０５は、非接触領域３０２の慣性主軸も同様に算出し、特徴量ベクトル生成部１０７に渡す。
曲率算出部１０６は、輪郭抽出部１０１からの接触領域３０１の輪郭に基づいて、接触領域３０１の輪郭上の各画素における曲率を算出する。具体的には、曲率算出部１０６は、接触領域３０１の輪郭上の任意の画素Ｐiと、当該画素ＰiからＬ画素離れた輪郭上の２つ画素Ｐi-L及びＰi+Lとが作る２つのベクトルがなす角を曲率θiとして算出する。例えば、Ｌ＝１２とした場合の画素Ｐi、画素Ｐi-L及び画素Ｐi+Lの関係は図４に示す通りである。曲率算出部１０６は、画素Ｐi及び画素Ｐi-Lが作るベクトルが水平方向に対しなす角をθinと、画素Ｐi及び画素Ｐi+Lが作るベクトルが水平方向に対しなす角をθoutとを、以下の数式（７）に従って算出する。

In addition, the inertia main axis calculation unit 105 calculates the inertia main axis of the non-contact region 302 in the same manner, and passes it to the feature vector generation unit 107.
The curvature calculation unit 106 calculates the curvature of each pixel on the contour of the contact region 301 based on the contour of the contact region 301 from the contour extraction unit 101. Specifically, the curvature calculation unit 106 includes two pixels Pi that are formed by an arbitrary pixel Pi on the contour of the contact area 301 and two pixels Pi-L and Pi + L on the contour that are L pixels away from the pixel Pi. The angle formed by the vector is calculated as the curvature θi. For example, when L = 12, the relationship between the pixel Pi, the pixel Pi-L, and the pixel Pi + L is as shown in FIG. The curvature calculation unit 106 defines θin as an angle formed by the vector formed by the pixel Pi and the pixel Pi-L with respect to the horizontal direction, and θout indicates an angle formed by the vector formed by the pixel Pi and the pixel Pi + L with respect to the horizontal direction. This is calculated according to Equation (7).

曲率算出部１０６は、θin及びθoutの差分を、画素Ｐiにおける曲率θiとして特徴量ベクトル生成部１０７に渡す。また、曲率算出部１０６は、非接触領域３０２の輪郭上の各画素における曲率も同様に算出し、特徴量ベクトル生成部１０７に渡す。尚、前述した曲率の算出手法は、文献「計測自動制御学会東北支部 第１８１回研究集会（１９９９．５．２１） 資料番号１８１−１ “画像内の人間領域の抽出”，Ｏｗｅｉ ＨｏｎｇＢｉｎ，梶川伸哉，猪岡光」に詳しい。

The curvature calculation unit 106 passes the difference between θin and θout to the feature vector generation unit 107 as the curvature θi at the pixel Pi. Further, the curvature calculation unit 106 similarly calculates the curvature at each pixel on the contour of the non-contact region 302 and passes it to the feature vector generation unit 107. Note that the curvature calculation method described above is described in the document “The Society of Instrument and Control Engineers Tohoku Branch, 181st Research Meeting (1999.5.21), Material No. 181-1“ Extraction of Human Regions in Images ”, Owei HongBin, Shinya Ninagawa. , "Hikari Tsujioka" is familiar

  The area calculation unit 103, the center of gravity calculation unit 104, the inertia main axis calculation unit 105, and the curvature calculation unit 106 described above may be calculated using other existing methods. Hereinafter, modified examples of the calculation of the center of gravity and the inertia main axis in the center of gravity calculation unit 104 and the inertia main axis calculation unit 105 will be described.

  The center-of-gravity calculation unit 104 and the inertial principal axis calculation unit 105 perform ellipse approximation on the contour of the contact area 301 from the contour extraction unit 101. Specifically, the center-of-gravity calculation unit 104 and the inertial principal axis calculation unit 105 use the coordinates (x, y) of each pixel on the contour of the contact region 301 as a variable, and use the least square method to obtain the following formula (8) ( Approximate to the coordinates (X, Y) satisfying the ellipse formula.

ここで、最小二乗法を用いて近似するとは、接触領域３０１の輪郭上の各画素の座標（ｘ、ｙ）と、上記楕円上の座標（Ｘ，Ｙ）との間の誤差の二乗和が最小となるＸo、Ｙo、ａ、ｂ及びθを導出することを意味する。尚、図５に示すように、座標（Ｘo，Ｙo）が楕円の中心の座標、ａおよびｂが半長軸及び半短軸、θが長軸のｘ軸に対する傾きを夫々示す。

Here, the approximation using the least square method means that the sum of squares of errors between the coordinates (x, y) of each pixel on the contour of the contact region 301 and the coordinates (X, Y) on the ellipse. It means deriving Xo, Yo, a, b and θ which are the minimum. As shown in FIG. 5, the coordinates (Xo, Yo) are the coordinates of the center of the ellipse, a and b are the semi-major axis and the semi-minor axis, and θ is the inclination of the major axis with respect to the x-axis.

  The center of gravity calculation unit 104 calculates the coordinates (Xo, Yo) of the center of the ellipse as coordinates indicating the center of gravity of the contact area. The inertia main axis calculation unit 105 calculates the inclination θ of the major axis of the ellipse with respect to the x axis as the inertia main axis. The center-of-gravity calculation unit 104 and the inertial principal axis calculation unit 105 can similarly calculate the center of gravity and the inertial main axis of the non-contact region 302.

  The feature amount vector generation unit 107 generates a feature amount vector using various feature amounts obtained by the color analysis unit 102, the area calculation unit 103, the centroid calculation unit 104, the inertia principal axis calculation unit 105, and the curvature calculation unit 106 described above. To do. Specifically, the feature quantity vector is multidimensional data, and the feature quantity vector generation unit 107 calculates each component of the feature quantity vector based on the various feature quantities.

Hereinafter, components of the feature vector are exemplified.
The feature vector generation unit 107 may calculate a vector from the center of gravity B of the contact area 301 to the maximum curvature position C as one component of the feature vector as shown in FIG. The component indicates a distortion peak position in the contact region 301.

The feature quantity vector generation unit 107 may calculate a difference between the maximum curvature value and the minimum curvature value of the contact area 301 as one component of the feature quantity vector. The component indicates the degree of distortion of the contact area 301.
Further, the feature quantity vector generation unit 107 may calculate a difference between the coordinates of the center of gravity of the contact area 301 and the coordinates of the center of gravity of the non-contact area 302 as one component of the feature quantity vector. The difference may be a vector or a distance. This component indicates the degree of deviation of the center of gravity of the contact area 301.

  Further, the feature quantity vector generation unit 107 may calculate the deviation of the distance between the four vertices of two ellipses obtained by ellipse approximation of the contact area 301 and the non-contact area 302 as one component of the feature quantity vector. Specifically, the feature vector generation unit 107 sets the ratio of the minimum value to the maximum value of the distances d1, d2, d3, and d4 between the four vertices of the two ellipses shown in FIG. 7 as the deviation. The component indicates the degree of distortion of the contact area 301.

The feature quantity vector generation unit 107 may calculate the ratio of the area of the contact area 301 to the area of the non-contact area 302 as one component of the feature quantity vector. The component indicates the magnitude of the force that the object 202 applies to the transmission plate 203.
In addition, the feature vector generation unit 107 may use the area and average pixel value of the contact region 301 obtained by the area calculation unit 103 and the color analysis unit 102 as they are as one component of the feature vector. The component indicates the magnitude of the force that the object 202 applies to the transmission plate 203.

Further, the feature quantity vector generation unit 107 may calculate a difference between the inertia principal axis of the contact area 301 and the inertia principal axis of the non-contact area 302 as one component of the feature quantity vector. The component indicates the degree of rotation of the contact area 301 with respect to the non-contact area 302.
The feature vector generation unit 107 passes the feature vector generated as described above to the mapping unit 108. The feature vector is correlated with the magnitude and direction of the contact force of the object 202. If the contact force vectors are similar, similar feature vectors can be obtained.

The mapping unit 108 performs mapping of the feature amount vector from the feature amount vector generation unit 107 using a self-organizing mapping (SOM) algorithm. Here, SOM will be described.
According to the SOM, the mapping unit 108 can map multidimensional data such as the feature quantity vector on a two-dimensional space, for example. More specifically, the mapping unit 108 can perform mapping so that similar feature amount vectors approach each other and dissimilar feature amount vectors are separated by SOM.

  The mapping space in which the mapping unit 108 performs mapping is a grid-like two-dimensional space, and the feature vector is mapped to any node (grid point). Each node is assigned a weight vector having the same dimension as the feature vector. The mapping unit 108 maps the input feature vector to a node having a weight vector closest to the feature vector. Here, the Euclidean distance is often used as the distance between vectors. The mapping unit 108 arbitrarily assigns initial values of the respective weight vectors, and approximates them to appropriate values by learning. Learning of the weight vector is performed as follows before the contact force vector detection apparatus in FIG. 1 actually detects the contact force vector.

  A teacher signal is used for weight vector learning. The teacher signal is a feature vector obtained from a plurality of contact force vector patterns. In the following description, it is assumed that teacher signals corresponding to the contact force vectors of a total of 8 (= 4 × 2) patterns are used for each direction (either up, down, left or right) and magnitude (either strong or weak). The mapping unit 108 acquires the feature vector generated by the feature vector generation unit 107 as a teacher signal of “upper strength” with respect to the contact force vector “strong in the upward direction (hereinafter referred to as“ upper strength ”). To do. Similarly, the mapping unit 108 acquires teacher signals of “upper weak”, “lower strong”, “lower weak”, “left strong”, “left weak”, “right strong”, and “right weak”, Do learning.

  First, the mapping unit 108 searches for a node having a weight vector closest to the feature quantity vector as the teacher signal. The node weight vector searched by the mapping unit 108 is called a winner vector. The mapping unit 108 brings the winner vector close to the feature vector. Specifically, the mapping unit 108 updates the current winner vector by adding a vector obtained by multiplying the difference between the feature vector and the winner vector by a predetermined learning rate coefficient. The mapping unit 108 also updates the weight vectors of nodes in the vicinity of the node having the winner vector. Specifically, the mapping unit 108 performs the same calculation by reducing the learning rate coefficient according to the distance between the nodes. Hereinafter, the mapping unit 108 repeats the same processing a plurality of times to perform weight vector learning. The learning rate coefficient is desirably decreased according to the number of repetitions. Next, the mapping unit 108 divides the mapping space into a plurality of regions indicating the plurality of patterns as shown in FIG. 8 based on the node having the weight vector closest to each of the feature vector as the teacher signal. . For example, Voronoi division is used for dividing the mapping space.

As shown in FIG. 9, the mapping unit 108 maps the input feature quantity vector to a Voronoi-divided mapping space (location indicated by “x” in FIG. 9). The mapping unit 108 passes the mapping position to the contact force calculation unit 109.
The contact force vector calculation unit 109 calculates a contact force vector of the object 202 based on the mapping position from the mapping unit 108. For example, the contact force vector calculation unit 109 calculates a contact force vector for the mapping position shown in FIG. 9 described above as follows.

  First, the contact force vector calculation unit 109 determines the main component of the contact force vector from the region to which the mapping position belongs. Since the mapping position shown in FIG. 9 is included in the “strong right” region, the contact force vector calculation unit 109 determines “strong right” as the main component of the contact force vector.

  Next, the contact force vector calculation unit 109 calculates the influence of an area adjacent to the area to which the mapping position belongs (hereinafter referred to as an adjacent area). Here, the adjacent region means a region delimited by a boundary line indicated by a solid line in FIGS. That is, the adjacent regions of “right strong” and “right weak” are “upper strong”, “upper weak”, “lower weak”, and “lower strong”.

  Specifically, the contact force vector calculation unit 109 calculates the distance between the mapping position and the boundary line. In the example of FIG. 9, as shown in FIG. 10, the contact force vector calculation unit 109 has a boundary line i with “upper strength” and a distance li between mapping positions, a boundary line i + 1 with “lower strength”, and A distance l i + 1 between the mapping positions is calculated.

  If the calculated distance is equal to or greater than the predetermined threshold δ, the contact force vector calculation unit 109 ignores the influence of the adjacent region. On the other hand, if the calculated distance is less than the threshold δ, the contact force vector calculation unit 109 considers the influence of the adjacent region. In the example shown in FIG. 10 described above, if li <δ <li + 1, the contact force vector calculation unit 109 considers the influence of “upper strength” and ignores the influence of “lower strength”.

  The contact force vector calculation unit 109 calculates the degree of influence of the adjacent region by a function that decreases according to the distance. An example of the above function is shown in the following formula (9).

数式（９）においてＩiは境界線ｉによって区切られる隣接領域の影響の程度、α及びβは所定の係数、ｌiは境界線ｉからの距離を夫々示す。接触力ベクトル算出部１０９は隣接領域の影響を考慮することによって、特にマッピング位置が境界線付近であった場合に、より正確な接触力ベクトルを算出できる。

In Equation (9), Ii represents the degree of influence of the adjacent region delimited by the boundary line i, α and β represent predetermined coefficients, and l i represents the distance from the boundary line i. The contact force vector calculation unit 109 can calculate a more accurate contact force vector by considering the influence of the adjacent region, particularly when the mapping position is near the boundary line.

  The contact force vector calculation unit 109 adds the influence of the adjacent region calculated as described above to the above-described main component and outputs it as a contact force vector. In the above example, the contact force vector calculation unit 109 outputs ““ right strong ”+ βexp (−αli) ד upper strength ”” as the contact force vector.

Hereinafter, an example of the operation of the contact force vector detection device of FIG. 1 will be described using the flowchart shown in FIG. It is assumed that the weight vector of the mapping unit 108 has already been learned before the processing is started, and the mapping space is divided into regions indicating a plurality of patterns of contact force vectors.
First, the contour extraction unit 101 extracts the contour of the contact area 301 and the contour of the non-contact area 302 in the input image (step S401). Next, the color analysis unit 102, the area calculation unit 103, the center of gravity calculation unit 104, the inertia main axis calculation unit 105, and the curvature calculation unit 106 calculate the average of the contact area 301 and the non-contact area 302 based on the contour extracted in step S401. The pixel value, area, center of gravity, inertial principal axis, and curvature are respectively calculated as feature amounts (step S402). In step S402, calculation of some feature values may be omitted, or other feature values may be calculated.

  Next, the feature vector generation unit 107 generates a feature vector based on the feature calculated in step S402 (step S403). The component of the feature vector is, for example, (a) a vector from the center of gravity of the contact area 301 to the maximum curvature position, (b) a difference between the maximum curvature value and the minimum curvature value of the contact area 301, and (c) the contact area 301 The difference between the coordinates of the center of gravity and the coordinates of the center of gravity of the non-contact region 302, (d) the deviation of the distance between the four vertices of two ellipses that approximate the contact region 301 and the non-contact region 302 by an ellipse, and (e) the non-contact The ratio of the area of the contact region 301 to the area of the region 302, (f) the area of the contact region 301, (g) the average pixel value of the contact region 301, and (h) the inertia principal axis of the contact region 301 and the inertia principal axis of the non-contact region 302 The difference between them. The feature vector generated by the feature vector generation unit 107 may include all of the above (a) to (h) as components, may not include some as components, or may include other components. .

  Next, the mapping unit 108 performs mapping of the feature vector generated in step S403 (step S404). As described above, the mapping space is divided into regions indicating a plurality of patterns of contact force vectors. Next, the contact force vector calculation unit 109 calculates a contact force vector based on the mapping position in step S404 (step S405) and outputs it.

  As described above, the contact force vector detection device according to the present embodiment includes the feature amount vector generated based on the feature amounts of the contact area and the non-contact area of the target object, and the contact force of the feature quantity vector and the target object. The contact force vector is calculated on the basis of the correlation between the size and direction. More specifically, the correlation is obtained by learning the SOM weight vector from the teacher signal. According to the contact force vector detection device according to the present embodiment, the magnitude and direction of the force applied by the object can be detected without using the deformation of the contacted element. Accordingly, there is no problem with the time lag from when the force is applied until the contacted element is deformed.

  Further, if the input image is acquired by photographing the object through the transmission plate as the contacted element, the detection apparatus main body does not directly touch the object, and thus is not easily broken. In addition to the video signal cable and power cable of the camera that acquires the input image, no wiring is required and the number of wiring is small. Furthermore, the shape of the transmission plate is not limited to a flat plate shape, and the degree of freedom of the shape of the detection surface is high.

  Further, according to the contact force vector detection device according to the present embodiment, it is possible to relatively easily detect the slip of the object by analyzing the input image, or to acquire information before the contact point of the object. it can. That is, both functions of a visual sensor and a tactile sensor can be realized. Further, since the input image is acquired by the camera, there is an advantage that the entire sensor can be reduced in size, and if the wireless camera is used as the camera, the detection device main body can be configured separately.

  Note that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. Further, for example, a configuration in which some components are deleted from all the components shown in each embodiment is also conceivable. Furthermore, you may combine suitably the component described in different embodiment.

The block diagram which shows the contact force vector detection apparatus which concerns on one Embodiment. Explanatory drawing of the mode of acquisition of the input image in the contact force vector detection apparatus of FIG. Explanatory drawing of the change accompanying the increase in the contact force of the target object of the input image acquired by the camera of FIG. Explanatory drawing of calculation of the curvature in the curvature calculation part of FIG. FIG. 3 is an explanatory diagram of calculation of a center of gravity and an inertia main axis in a center of gravity calculation unit and an inertia main axis calculation unit in FIG. 1. Explanatory drawing of one component of the feature-value vector produced | generated by the feature-value vector production | generation part of FIG. Explanatory drawing of another example of FIG. The figure which shows an example of the mapping space where the mapping part of FIG. 1 maps a feature-value vector. The figure which shows an example of the mapping result by the mapping part of FIG. Explanatory drawing of calculation of the contact force vector in the contact force vector calculation part of FIG. The flowchart which shows an example of operation | movement of the contact force vector detection apparatus of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Contour extraction part 102 ... Color analysis part 103 ... Area calculation part 104 ... Gravity center calculation part 105 ... Inertial principal axis calculation part 106 ... Curvature calculation part 107 ... Feature-value vector Generation unit 108 ... Mapping unit 109 ... Contact force vector calculation unit 201 ... Object 202 ... Transmission plate 203 ... Camera 301 ... Contact region 302 ... Non-contact region

Claims (10)

Detection that detects a first region of the object that is in contact with the transmission plate and a second region that is not in contact with the transmission plate from an input image obtained by photographing the object through the transmission plate And
A feature amount calculation unit that calculates a plurality of feature amounts for each of the first region image and the second region image in the input image;
A generating unit that generates a feature vector based on the plurality of feature values;
A contact force vector calculation unit that calculates a contact force vector indicating the magnitude and direction of the contact force based on a correlation between the feature amount vector and the magnitude and direction of the contact force applied to the transmission plate by the object. And a contact force vector detecting device.   The feature amount calculation unit calculates at least one of an average pixel value, an area, a center of gravity, a principal axis of inertia, and a curvature of each of the image of the first region and the image of the second region as the plurality of feature amounts. The contact force vector detection device according to claim 1. A mapping unit that maps the feature vector to any one of a plurality of nodes to which a weight vector learned by a teacher signal corresponding to a plurality of patterns of the contact force vector is assigned;
The contact force vector detection device according to claim 1, wherein the contact force vector calculation unit calculates the contact force vector based on a node to which the feature vector is mapped.   The feature vector is (a) a vector from the center of gravity of the first region to the maximum curvature position, (b) a difference between the maximum curvature value and the minimum curvature value of the first region, and (c) the first region. The difference between the coordinates of the center of gravity of one region and the coordinates of the center of gravity of the second region, (d) the four vertices of an ellipse that approximates the first region to an ellipse and the ellipse that approximates the second region to an ellipse (E) the ratio of the area of the first region to the area of the second region, (f) the area of the first region, and (g) the average pixel value of the first region. And (h) at least one of the differences between the inertial principal axis of the first region and the inertial principal axis of the second region. Vector detection device. Detection that detects a first region of the object that is in contact with the transmission plate and a second region that is not in contact with the transmission plate from an input image obtained by photographing the object through the transmission plate Steps,
A feature amount calculating step of calculating a plurality of feature amounts for each of the first region image and the second region image in the input image;
Generating a feature vector based on the plurality of features;
A contact force vector calculation step of calculating a contact force vector indicating the magnitude and direction of the contact force based on a correlation between the feature vector and the magnitude and direction of the contact force applied by the object to the transmission plate. And a contact force vector detection method comprising:   The feature amount calculating step calculates at least one of an average pixel value, an area, a center of gravity, an inertia main axis, and a curvature of each of the image of the first region and the image of the second region as the plurality of feature amounts. The method of detecting a contact force vector according to claim 1. A mapping step of mapping the feature quantity vector to any one of a plurality of nodes to which a weight vector learned by a teacher signal corresponding to a plurality of patterns of the contact force vector is assigned;
3. The contact force vector detecting method according to claim 1, wherein the contact force vector calculating step calculates the contact force vector based on a node to which the feature vector is mapped.   The feature vector is (a) a vector from the center of gravity of the first region to the maximum curvature position, (b) a difference between the maximum curvature value and the minimum curvature value of the first region, and (c) the first region. The difference between the coordinates of the center of gravity of one region and the coordinates of the center of gravity of the second region, (d) the four vertices of an ellipse that approximates the first region to an ellipse and the ellipse that approximates the second region to an ellipse (E) the ratio of the area of the first region to the area of the second region, (f) the area of the first region, and (g) the average pixel value of the first region. And (h) at least one of the differences between the inertial principal axis of the first region and the inertial principal axis of the second region. Vector detection method.   The program for making a computer perform the contact force vector detection method of any one of Claim 5 thru | or 9.   A computer-readable recording medium recording a program for causing a computer to execute the contact force vector detection method according to claim 5.

Images