WO2013172491A1 - Procédé et dispositif pour reconnaître un objet au moyen de données de profondeur - Google Patents

Procédé et dispositif pour reconnaître un objet au moyen de données de profondeur Download PDF

Info

Publication number
WO2013172491A1
WO2013172491A1 PCT/KR2012/003840 KR2012003840W WO2013172491A1 WO 2013172491 A1 WO2013172491 A1 WO 2013172491A1 KR 2012003840 W KR2012003840 W KR 2012003840W WO 2013172491 A1 WO2013172491 A1 WO 2013172491A1
Authority
WO
WIPO (PCT)
Prior art keywords
recognition
clusters
color
depth
candidate
Prior art date
Application number
PCT/KR2012/003840
Other languages
English (en)
Korean (ko)
Inventor
최병호
김제우
황영배
배주한
Original Assignee
전자부품연구원
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 전자부품연구원 filed Critical 전자부품연구원
Priority to PCT/KR2012/003840 priority Critical patent/WO2013172491A1/fr
Publication of WO2013172491A1 publication Critical patent/WO2013172491A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Definitions

  • the present invention relates to an object recognition method and apparatus, and more particularly, to an object recognition method and apparatus using depth information.
  • Feature point matching is mainly used as a method for recognizing whether an object to be found exists in a captured image.
  • This technique is able to find out the existence of the object to be found on the image as well as the location of the object relatively accurately, but it reveals the problem of inaccuracy in some cases.
  • the present invention has been made to solve the above problems, and an object of the present invention is to more reliably improve the object recognition rate even in the case of a feature point-based object recognition or an object having a complex background. It is to provide a method and apparatus for recognizing an object that can be increased.
  • an object recognition method includes: obtaining a color image and a depth image; Clustering the color-image into a plurality of clusters based on the depth information indicated in the depth-image; And performing object recognition based on the clustered clusters.
  • the object recognition may include: performing feature point matching on each of the clusters and each of the recognition objects; Selecting some regions in the clusters as candidate regions; And eliminating feature point matching made outside clusters including the selected candidate regions, and performing object recognition.
  • performing object recognition may include selecting some regions in the clusters as candidate regions; Performing feature point matching on each of the candidate areas and each of the recognition objects; And performing object recognition using the feature point matching result.
  • the candidate region selecting step may include: a first selecting step of selecting some regions in the clusters as candidate regions by comparing color histograms for each of the clusters and color histograms for each of the recognition objects; And comparing the depth statistics of the candidate region selected in the first selection step with the recognition object, And a second selecting step of selecting a candidate region and a recognition object having a difference less than a threshold.
  • the object recognition step may further include: a first selecting step of selecting some areas in the clusters as candidate areas; Performing feature point matching on each of the candidate regions selected in the first selecting step and each of the recognition objects; A second selecting step of selecting again some of the candidate areas selected in the first selecting step; And eliminating feature point matching made outside the candidate regions selected in the second selection step, and performing object recognition.
  • the method may further include processing the recognized object recognized in the plurality of clusters as not recognized.
  • the object recognition apparatus for acquiring a color-image and a depth-image; A storage unit in which images of recognition objects are DBized; And a processor that clusters the color-image into a plurality of clusters based on the depth information shown in the depth-image, and performs object recognition based on the clustered clusters.
  • clustering an image using depth information selecting candidate regions in a cluster through color-histogram comparison, and using the image recognition to increase the object recognition rate, as well as the amount of computation required for feature point matching and subsequent processing. It can be greatly reduced.
  • the candidate area can be verified, thereby further increasing the object recognition rate.
  • the recognition rate may be further increased.
  • FIG. 1 is a flowchart provided to explain an object recognition method using depth information according to an embodiment of the present invention
  • 3 is a diagram showing three candidate regions selected from clusters in a color image.
  • 5 and 6 are views illustrating a case where one recognition object is recognized in a plurality of clusters
  • FIG. 7 and 8 illustrate a case where a plurality of recognition objects are recognized in a color-image. Illustrated drawing ,
  • FIG. 9 is a flowchart provided to explain an object recognition method using depth information according to an embodiment of the present invention.
  • FIG. 10 is a flowchart provided to explain an object recognition method using depth information according to another embodiment of the present invention.
  • FIG. 11 is a diagram illustrating an object recognition apparatus capable of performing the object recognition method illustrated in FIGS. 1, 9, and 10.
  • the object recognition method using depth information uses depth information in clustering color-images and verifying a region selected as a candidate having a high probability of the object to be recognized in the cluster.
  • FIG. 1 is a flowchart provided to explain an object recognition method using depth information according to an exemplary embodiment of the present invention.
  • a color-image and a depth-image are obtained (S110).
  • Color- and depth-images can be generated using RGB cameras and D-cameras, respectively.
  • the color-image is clustered into a plurality of clusters based on the depth information indicated in the depth-image (S120). Strictly speaking, clustering in the S120 phase In addition to the depth information, location information is further reflected, and the reflectance is greater in the depth information.
  • the position information refers to information on the position (x, y) on the plane perpendicular to the depth (d).
  • Clustering performed in step S120 is to cluster pixels having similar depths and close distances to one cluster. .
  • the 'object to be recognized' (hereinafter, abbreviated as 'aware object') existing on the color image is very likely to belong to any one cluster. This is because the pixels of one recognition object have similar depths and close distances.
  • Recognition object is the object to find in the color-image, and the image is stored in DB.
  • step S130 feature point matching is performed on each of the clusters and each of the recognition objects (S130). If k clusters are clustered and the number of objects recognized in the DB is n , step S130 is performed.
  • the clustering clusters may first select the cluster clusters as candidate regions (S 140).
  • Step S140 is performed for each of the clustered clusters in step S120, and candidate region selection is performed by comparing the color and histogram of the cluster and the recognition object.
  • the region is selected as a candidate region, 2n) If there is an area with a -histogram similar to the color-histogram of recognition object -n in cluster -2, the area is selected as a candidate area, and kl) with the color-histogram of recognition object -1 in cluster -k. If there is an area with a similar color-histogram, the area is selected as a candidate area, and k2) if there is an area with a color- histogram similar to the color- histogram of the recognition object -2 in the cluster -k, the area.
  • step S140 the depth standard deviations between the candidate region firstly selected in step S140 and the recognition object are compared, and the candidate regions are secondarily selected (S150).
  • the depth standard deviation is the standard deviation of the depth values for the pixels
  • the depth standard deviation of the candidate area is the standard deviation of the depth values of the pixels constituting the candidate area
  • the depth standard deviation of the recognition object is an image of the recognition object.
  • the depth standard deviation of the recognition object is preferably calculated in advance and stored in the DB.
  • step S140 1) Some areas of cluster -1 were selected as 'candidate area -1' because 'recognition object -1' and color-histogram are similar.
  • a partial region of the cluster -3 was selected as the candidate region -2 because the recognition object - ⁇ and the color histogram are similar.
  • the candidate area -2 and the recognition object # 1 are selected.
  • the candidate area -3 and the recognition object -2 are selected.
  • the candidate area -4 and the recognition object -2 are selected.
  • the candidate area -5 and the recognition object-3 are selected.
  • the candidate area # 7 and the recognition object -4 are selected.
  • the object recognition is performed by excluding feature point matching for clusters that do not include candidate regions selected in step S150 from the feature point matching result in step S130 (S160). That is, the object recognition is performed while leaving only the feature point matching for the clusters including the candidate regions selected in step S150 among the feature point matching results in step S130.
  • step S150 'candidate area -1 and recognition object -1', 'candidate area -2 and recognition object -1', 'candidate area -3 and recognition object -2', 'candidate area -5 and recognition object -3' 'And' candidate area— 6 and recognition object -4 'are selected,
  • Candidate region -1 is included in cluster -1
  • candidate region -3 is included in cluster -2
  • candidate region -2 and candidate region -5 are included in cluster -3
  • candidate region -6 is included in cluster -5 Included
  • step S160 only feature point matching for cluster-1, cluster-2, cluster-3, and cluster # 5 is left, except for feature point matching for the remaining clusters, Perform object recognition
  • the recognized object when one recognized object is recognized in a plurality of clusters, the recognized object may be treated as not recognized (S170). Specifically, a recognition object in which the ratio of the feature point matching number between the first cluster with the highest feature point matching number and the second largest cluster within a specific range is treated as not recognized.
  • the second matching number is the same as in the case where the recognition object—1 has a feature point matching number (1) of cluster ⁇ 1 is “100” and a feature point matching number (2) of cluster ⁇ 3 is “90”. If (2) is 80% or more of the first matching number (1) and the recognition object -1 can be treated as recognized in both cluster -1 and cluster -3, the recognition object -1 is treated as not recognized. .
  • the same recognition object is recognized in multiple clusters because it is more likely to be due to an error in the image recognition process than it actually is, and to prevent false recognition in advance.
  • the feature point matching number 3 of the recognition object -2 and the cluster -2 is "90”
  • the feature point matching number (4) of the recognition object -3 and the cluster -3 is "60”
  • the recognition object -4 The second matching number (5) is more than 70% of the first matching number (3), as in the case where the characteristic point matching number (5) of the cluster-5 is " 70 " Object -4 All of them can be treated as recognized.
  • the color-correlation of recognition object -2 and cluster-2 is compared with the color-correlation of recognition object -4 and cluster -5. It's what's recognized in the image.
  • the feature point matching number 3 of the recognition object—2 and the cluster-2 is “90”
  • the feature point matching number 4 of the recognition object ⁇ 3 and the cluster ⁇ 3 is “40”
  • the recognition object ⁇ 4 and the cluster If the second matching number (4) is less than 70% of the first matching number (3), such as the case where the characteristic point matching number (5) of -5 is "30”, the matching number is the most without the color correlation.
  • Many recognition objects -2 treat as recognized in cluster -2 of color-images.
  • Depth Proximity Clustering refers to clustering in step S120.
  • color-image is divided into an object and a background and clustered.
  • Clustering may be performed using K-means clustering all, and similarly close and close distances may be clustered in one cluster with reference to depth information and location information. Specifically, the clustering may be performed according to Equation 1 below.
  • k is the number of clustering, is the geometric center of 3 ⁇ 4 of the i th cluster, and Xj is a vector expressed by [cx, a 2 y, a 3 d].
  • x and y are the x and y coordinates of the pixel, and d is the depth value.
  • FIG. 2 In the upper left part of FIG. 2, a color-image of a specific space is shown (in case of inevitably converted to a hog-image due to the limitation of drawing standards), and in the lower left, a depth-image of the specific space is shown. Shows the Depth Proximity Clustering results.
  • the object detection algorithm using color histogram back projection is used to select a region where a recognition object is likely to exist even in a cluster as a candidate region.
  • the Equality Test of Standard Deviation can be understood as a procedure for verifying or reselecting candidate areas where recognition objects are likely to exist by comparing depth standard deviations between candidate areas selected through Color Histogram based Object Detection and recognition objects. have.
  • Figure 4 shows the depth standard deviation for the recognition objects, books, dolls, cushions.
  • the depth standard deviation for an object mostly follows a Gaussian distribution regardless of the shape of the object. Therefore, assuming that the random sample of the recognition object and the candidate area follows a normal distribution, the ratio of the standard deviation follows the F distribution.
  • the reliability is ⁇
  • the degrees of freedom are ⁇ ⁇ -1, n y -l
  • the degrees of freedom have the following confidence interval when set to the number of random samples, matching beyond the confidence interval is removed and the rest is reliable It is considered a match.
  • a feature point matching in step S130 it is preferable to use a feature point matching technique that is invariant to scale and rotation, and to use another type of feature point matching technique. It is not excluded.
  • the relationship between the two images can be expressed as a matrix.
  • Homography when there are several large points on the same plane as the building wall, it can be approximated to exist on one plane and this relationship is called Homography.
  • the homography between the cluster and the recognition object generated through Depth Proximity Clustering is more accurate than the homography between the color-image and the recognition object because the matching of the feature point of the recognition object and the background is excluded.
  • Final decision with consistency check is a procedure for final object recognition when one recognition object is recognized in multiple clusters and when multiple recognition objects are recognized in a color-image.
  • FIGS. 5 and 6 The case where one recognition object is recognized in a plurality of clusters is illustrated in FIGS. 5 and 6. As shown in FIGS. 5 and 6, it can be seen that the recognition object "cushion" shows a large number of feature point matchings in different clusters. As mentioned above As such, if the ratio of the number of feature point matches between both is within a certain range, the "cushion" in the color-image is treated as not recognized. 7 and 8 illustrate a case where a large number of recognition objects are recognized in a color image (represented by inevitably converting to a hog-image due to limitations in drawing standards). Feature point matching for "book” and "doll" with the top two matching numbers is shown.
  • the ratio of feature point matching between the two is outside the specified range, for example, the feature number matching between "book” and cluster is equal to "doll". If the number of feature point matches between clusters is significantly greater than the number of feature points matching between clusters, the "book” is considered recognized, whereas if the ratio of the number of feature points matching between the two is within a certain range, the color_correlation between cluster and "book” and the cluster and "doll” By comparing the color-correlation between the two objects, a recognition object with a large color-correlation is recognized as being recognized in a cluster 7.
  • a description of an object recognition method using depth information will be described with reference to FIG. 9, and the same parts as the above description will be briefly described.
  • a color-image and a depth-image are obtained (S210).
  • the color-image is clustered into a plurality of clusters based on the depth information indicated in the depth-image (S220).
  • each of the clustered clusters is performed in operation S220, and the candidate region selection is performed by comparing the color and histogram of the cluster and the recognition object.
  • step S240 of FIG. 9 is a feature point matching between the "candidate area” and the recognition object, which is different from the feature point matching between the "cluster” and the recognition object in step S130 of FIG.
  • candidate regions similar in depth standard deviation between the candidate region firstly selected in step S230 and the object to be recognized are secondarily selected (S250).
  • the feature point matching is excluded from the feature point matching in step S240 and the object point matching performed outside the "candidate regions" selected in step S250 is performed (S260). That is, object recognition is performed while leaving only the feature point matchings included in the candidate areas selected in step S250 among the feature point matching results in step S240.
  • Feature point matching excluded in step S260 is different from step S160 in FIG. 1 to exclude feature point matching made outside clusters including the second selected candidate areas in that the feature point matchings are made outside the second selected candidate areas. .
  • the recognized object when one recognized object is recognized in a plurality of "candidate areas", the recognized object may be treated as not recognized (S270). Specifically, a recognition object in which the ratio of the feature point matching number between the first candidate region with the greatest number of feature point matches and the second candidate region is within a specific range is treated as not recognized.
  • a plurality of recognition objects If it is recognized, only one recognition object is selected by comparing the color-correlation between the candidate regions having the top two feature point matching numbers and the recognition objects, and the selected recognition object is processed as being recognized (S280).
  • step S280 only the candidate region having the ratio of the feature point matching number between the first recognition object with the highest number of feature points and the second recognition object within a specific range is performed. Otherwise, the recognition object with the highest number of feature point matches is treated as recognized.
  • steps S270 and S280 are different from those in steps S170 and S180 of FIG. 1, in which the processing criteria are clusters.
  • FIG. 11 may perform the object recognition method illustrated in FIGS. 1, 9, and 10. It is a figure which shows the object recognition apparatus. As shown in FIG. 11, the object recognition apparatus to which the present invention is applicable includes an RGB camera 210, a D-camera 420, a processor 430, a storage 440, and a display 450. .
  • RGB-camera 410 produces a color-image
  • D-camera 420 produces a depth-image.
  • the RGB-camera 410 and the D-camera 420 may be configured separately as shown in FIG. 11, but may also be configured as one camera.
  • images of recognition objects are DBized.
  • the storage unit 440 is also a DB of the depth standard deviations for the recognition objects.
  • Processor 430 is a color-image generated by the RGB camera 410,
  • the depth-image generated by the D-camera 420 and the storage unit 440 perform the object recognition method illustrated in FIGS. 1, 9, and 10 by using the stored recognition objects.
  • Display 450 shows a color-image generated by RGB camera 410, which may be displayed as a stereo-image using depth-image generated by D-camera 420.
  • the display 450 displays an object recognition result performed by the processor 430.
  • the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims Of course, various modifications can be made by those skilled in the art, and these modifications can be individually understood from the technical spirit or the prospect of the present invention. I will not.

Abstract

L'invention concerne un procédé et un dispositif pour reconnaître un objet au moyen de données de profondeur. Le procédé pour reconnaître l'objet consiste à obtenir une image couleur et une image de profondeur; à agglomérer l'image couleur en une pluralité de blocs sur la base des données de profondeur représentées sur l'image de profondeur; puis à exécuter la reconnaissance de l'objet sur la base des blocs agglomérés. Ainsi, même si un objet présente peu de points caractéristiques dans la reconnaissance d'objet par points caractéristiques ou qu'il présente un arrière-plan complexe, il est possible d'augmenter de manière plus fiable le taux de reconnaissance d'objet.
PCT/KR2012/003840 2012-05-16 2012-05-16 Procédé et dispositif pour reconnaître un objet au moyen de données de profondeur WO2013172491A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/KR2012/003840 WO2013172491A1 (fr) 2012-05-16 2012-05-16 Procédé et dispositif pour reconnaître un objet au moyen de données de profondeur

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/KR2012/003840 WO2013172491A1 (fr) 2012-05-16 2012-05-16 Procédé et dispositif pour reconnaître un objet au moyen de données de profondeur

Publications (1)

Publication Number Publication Date
WO2013172491A1 true WO2013172491A1 (fr) 2013-11-21

Family

ID=49583886

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2012/003840 WO2013172491A1 (fr) 2012-05-16 2012-05-16 Procédé et dispositif pour reconnaître un objet au moyen de données de profondeur

Country Status (1)

Country Link
WO (1) WO2013172491A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10255057A (ja) * 1997-03-06 1998-09-25 Matsushita Electric Ind Co Ltd 移動物体抽出装置
JP2010514064A (ja) * 2006-12-21 2010-04-30 本田技研工業株式会社 ラベル付けを用いた人体姿勢の推定および追跡
JP2010257267A (ja) * 2009-04-27 2010-11-11 Nippon Telegr & Teleph Corp <Ntt> 物体領域検出装置、物体領域検出方法および物体領域検出プログラム
WO2011080282A1 (fr) * 2009-12-28 2011-07-07 Softkinetic Procédé de suivi

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10255057A (ja) * 1997-03-06 1998-09-25 Matsushita Electric Ind Co Ltd 移動物体抽出装置
JP2010514064A (ja) * 2006-12-21 2010-04-30 本田技研工業株式会社 ラベル付けを用いた人体姿勢の推定および追跡
JP2010257267A (ja) * 2009-04-27 2010-11-11 Nippon Telegr & Teleph Corp <Ntt> 物体領域検出装置、物体領域検出方法および物体領域検出プログラム
WO2011080282A1 (fr) * 2009-12-28 2011-07-07 Softkinetic Procédé de suivi

Similar Documents

Publication Publication Date Title
WO2017092431A1 (fr) Procédé et dispositif de détection de main humaine basés sur une couleur de peau
US8600162B2 (en) Image identification device, image identification method and recording medium
CN102902959B (zh) 基于二代身份证存储证件照的人脸识别方法及系统
EP3249579B1 (fr) Appareil de reconnaissance d&#39;objet, procédé de reconnaissance d&#39;objection et programme
CN103207898B (zh) 一种基于局部敏感哈希的相似人脸快速检索方法
CN111368683B (zh) 基于模约束CenterFace的人脸图像特征提取方法及人脸识别方法
Ok et al. Circular oil tank detection from panchromatic satellite images: A new automated approach
JP5929896B2 (ja) 画像認識システム、画像認識方法および画像認識用プログラム
JP5521676B2 (ja) 画像処理装置及び画像処理プログラム
US20140192158A1 (en) Stereo Image Matching
WO2011042601A1 (fr) Reconnaissance de visages dans des images numériques
JP2005316973A (ja) 赤目検出装置および方法並びにプログラム
CN108710916B (zh) 图片分类的方法及装置
JP2003030667A (ja) イメージ内で目を自動的に位置決めする方法
CN111191655A (zh) 对象识别方法和装置
Vignesh et al. Face image quality assessment for face selection in surveillance video using convolutional neural networks
CN110765976B (zh) 人脸特征点的生成方法、数据网络的训练方法及相关装置
KR101916460B1 (ko) 깊이 정보를 이용한 물체 인식 방법 및 장치
Lu et al. Contour box: Rejecting object proposals without explicit closed contours
CN111832405A (zh) 一种基于hog和深度残差网络的人脸识别方法
CN107832359B (zh) 一种图片检索方法及系统
Gankin et al. Iris image segmentation based on approximate methods with subsequent refinements
Zeng et al. Automated extraction of PCB components based on specularity using layered illumination
JP2013164832A (ja) オブジェクト認識装置および方法
WO2017101380A1 (fr) Procédé, système et dispositif pour la reconnaissance d&#39;une main

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12876607

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12876607

Country of ref document: EP

Kind code of ref document: A1