WO2022239202A1 - 画像処理装置 - Google Patents
画像処理装置 Download PDFInfo
- Publication number
- WO2022239202A1 WO2022239202A1 PCT/JP2021/018278 JP2021018278W WO2022239202A1 WO 2022239202 A1 WO2022239202 A1 WO 2022239202A1 JP 2021018278 W JP2021018278 W JP 2021018278W WO 2022239202 A1 WO2022239202 A1 WO 2022239202A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- detection
- unit
- image processing
- matching
- parameter
- Prior art date
Links
- 238000001514 detection method Methods 0.000 claims abstract description 115
- 238000004364 calculation method Methods 0.000 claims abstract description 31
- 238000000605 extraction Methods 0.000 claims abstract description 23
- 239000000284 extract Substances 0.000 claims abstract description 6
- 238000003384 imaging method Methods 0.000 claims description 10
- 230000007717 exclusion Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 238000000034 method Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Definitions
- the present invention relates to an image processing device.
- image processing technology that identifies the position of objects by analyzing images taken by imaging devices (cameras) is widely used.
- image processing technique in general, when feature points in a photographed image are extracted and the arrangement pattern of the extracted feature points matches a pre-stored model pattern, If the numerical value of the degree of matching is equal to or greater than the threshold, it is determined that the object exists.
- the matching threshold is too low, erroneous detection of objects other than the intended target will occur. Conversely, if the threshold value for the degree of matching is too high, non-detection occurs in which the intended object cannot be detected as an object. Therefore, the user needs to check the detection results in a plurality of captured images and set the matching threshold so as to reduce erroneous detection and non-detection.
- the brightness gradient of the captured image may change depending on the brightness of the surroundings. For this reason, it has been proposed to statistically process the distribution of luminance gradients in a captured image and set a luminance gradient threshold for extracting feature points based on the average value and standard deviation (see, for example, Patent Document 1). ).
- An image processing device is an image processing device that detects an object in an image captured by an imaging device, and includes a model storage unit that stores a model pattern and extracts feature points from the captured image. a feature point extraction unit; an original matching degree calculation unit that calculates a degree of matching between the model pattern and the arrangement of the feature points; and detecting the object in the captured image by comparing the degree of matching with a detection threshold.
- a simple matching degree calculation for calculating the degree of matching based on the detection information storage unit to store and the detection information stored in the detection information storage unit when the detection parameter is changed, based on the changed detection parameter and the detection information stored in the detection information storage unit and
- FIG. 1 is a block diagram showing a configuration of image processing provided with an image processing device according to a first embodiment of the present disclosure
- FIG. 11 is a block diagram showing the configuration of an image processing device according to a second embodiment of the present disclosure
- FIG. 3 is a diagram exemplifying a display screen of a user interface unit of the image processing apparatus of FIG. 2;
- FIG. 1 is a block diagram showing the configuration of an image processing device 1 according to the first embodiment of the present disclosure.
- the image processing device 1 acquires the data of the captured image from the imaging device C and detects the object in the captured image.
- the imaging device C acquires an image of a subject within a predetermined field of view.
- the imaging device C may be a visual sensor that acquires a visible light image, or a sensor that acquires an infrared image, a distance image, a point cloud image, or the like.
- the image processing apparatus 1 includes a model storage unit 11, a feature point extraction unit 12, a feature point exclusion unit 13, an original matching degree calculation unit 14, an object detection unit 15, a parameter setting unit 16, and detection information storage.
- a unit 17 , a simple matching degree calculation unit 18 , and a parameter change unit 19 are provided.
- the image processing device 1 can be realized by causing one or more computer devices having a memory, a CPU, an input/output interface, etc. to execute appropriate programs.
- Each component of the image processing apparatus 1 described above is a categorization of the functions of the image processing apparatus 1, and may not be clearly distinguishable in terms of physical configuration and program configuration.
- the model storage unit 11 stores model patterns in which contours of objects are converted into data.
- the model pattern may be theoretically generated from the shape data of the object, or may be generated from the contour data of the object specified in the image of the object actually photographed, the coordinates of the feature points, etc. good.
- the feature point extraction unit 12 extracts feature points, which are points that change discontinuously from adjacent image regions from the captured image and that may be the contours of the target object.
- the feature point extraction unit 12 can be configured to extract, as feature points, points whose pixel attribute values are equal to or greater than the extraction threshold value set in the parameter setting unit 16 or are within the extraction range.
- Attribute values for discriminating feature points preferably include at least one of color, brightness, magnitude of brightness gradient, and direction of brightness gradient of feature points.
- the luminance gradient means the difference in luminance between adjacent pixels or unit regions, that is, the luminance change rate with respect to position. means the direction
- By using such attribute values as indexes it is possible to extract points that are highly likely to be the contours of some object or pattern in the image. In particular, points with large luminance gradients are highly likely to be the contours of the object.
- the feature point exclusion unit 13 excludes feature points whose distance from the model pattern exceeds the distance threshold from the calculation targets of the original matching degree calculation unit 14 . By excluding noise that clearly does not constitute the outline of the object in this way, the object detection accuracy can be improved.
- the original matching degree calculation unit 14 calculates the degree of matching between the model pattern and the arrangement of the feature points.
- the model patterns are arranged so that the total value of the distances or the squares of the distances to the model pattern of all the feature points excluding those excluded by the feature point exclusion unit 13 is the smallest, and the feature points match the model patterns.
- Quantify the degree of A matching degree calculation method can be the same as that of the conventional image processing apparatus.
- the object detection unit 15 detects the object in the captured image by comparing the degree of matching calculated by the original degree of matching calculation unit 14 with the detection threshold set in the parameter setting unit. That is, the object detection unit 15 determines that the feature point is the contour point of the object when the degree of coincidence is equal to or greater than the detection threshold, and determines that the object exists when the degree of coincidence is less than the detection threshold. It is judged as undetected.
- the parameter setting unit 16 sets detection parameters including at least the detection threshold used by the object detection unit 15 . That is, the parameter setting unit 16 performs initial setting of the detection parameter, and stores the value of the initially set detection parameter or the value of the detection parameter changed by the parameter changing unit 19 after the initial setting.
- the parameter setting unit 16 of this embodiment also sets an extraction threshold used by the feature point extraction unit 12 and a detection threshold used by the object detection unit 15 in addition to the distance threshold.
- the parameter setting unit 16 may be configured to set the initial value of the detection parameter to a predetermined value, or may be configured to wait for input by the user and then set the initial value to the value input by the user. good.
- the initial value of the extraction threshold set by the parameter setting unit 16 is preferably set to a sufficiently low value or a wide range, that is, a value that extracts more than an appropriate number of feature points.
- detection information of a sufficient number of feature points is stored in the detection information storage unit 17, and the matching degree after the detection parameter is changed can be calculated relatively accurately by the simple matching degree calculation unit 18.
- the parameter setting unit 16 may cause the feature point extraction unit 12 to try to extract feature points using the captured image, search for a value that allows a certain number of feature points to be extracted, and use it as the initial value of the extraction threshold.
- the detection information storage unit 17 stores detection information including at least the positions of the feature points of the target object extracted by the feature point extraction unit 12 .
- the detection information stored in the detection information storage unit 17 includes attributes of feature points to be compared with the extraction threshold. It preferably further contains a value.
- the simple match calculator 18 calculates the match based on the changed detection parameter and the detection information stored in the detection information storage 17. In this way, by providing the simple matching degree calculation unit 18 that recalculates the matching degree when the detection parameter is changed, it is possible to easily determine whether or not the change in the detection parameter is appropriate. In addition, since the simple matching degree calculation unit 18 calculates the matching degree from the detection information, it is possible to check the matching degree after changing the detection parameter without extracting feature points with a high computational load. Parameters can be optimized.
- the parameter changing unit 19 changes the detection parameters so that the matching degree calculated by the simple matching degree calculation unit 18 based on the detection information increases. In other words, the parameter changing unit 19 automatically optimizes the detection parameters so that the number of undetected objects is reduced.
- the parameter changing unit 19 temporarily changes the detection parameter, causes the simple matching degree calculation unit 18 to recalculate the degree of matching, and confirms the value of the detection parameter at which the degree of matching becomes sufficiently large. , may be configured to determine the value of the detection parameter.
- the parameter changing unit 19 may be configured to change the detection parameter to a plurality of preset values, confirm the degree of matching, and finally select the value with the highest degree of matching. Further, the parameter changing unit 19 may be configured to search for a detection parameter value that can sufficiently increase the degree of matching based on any algorithm such as a binary search method or a hill-climbing method.
- the parameter changing unit 19 uses the simple matching degree calculation unit 18, which recalculates the degree of matching using the detection information stored in the detection information storage unit 17, to reduce the image calculation load. Since the detection parameters are optimized without repeatedly extracting feature points by the feature point extracting unit 12 having a large value, the detection parameters can be rapidly set to optimum values.
- FIG. 2 is a block diagram showing the configuration of an image processing device 1A according to the second embodiment of the present disclosure.
- the same components as those of the image processing apparatus 1 of FIG. are identical to those of the image processing apparatus 1 of FIG.
- the image processing device 1A acquires the data of the captured image from the imaging device C and detects the object in the captured image. Further, the image processing apparatus 1A receives input from the user via the input device E, and uses the display device D to display information to the user. The image processing apparatus 1A of this embodiment allows the user to independently set the value of the detection parameter.
- any one or a plurality of devices such as a keyboard and a mouse can be used.
- the display device D an image display device such as a CRT, a liquid crystal panel, an organic EL panel, etc., which has the ability to display an image captured by the imaging device C can be used.
- the image processing apparatus 1A includes a model storage unit 11, a feature point extraction unit 12, a feature point exclusion unit 13, an original matching degree calculation unit 14, an object detection unit 15, a parameter setting unit 16A, and detection information storage.
- a unit 17 a simple matching degree calculator 18 , a parameter changer 19A, and a user interface unit 20 .
- the parameter setting unit 16A sets detection parameters according to the user's input through the user interface unit 20.
- the parameter setting unit 16A may be configured to set the initial value of the detection parameter based on a preset initial setting value or a past setting value.
- the parameter changing unit 19A changes the detection parameters according to the user's input through the user interface unit 20.
- the user interface unit 20 receives the image captured by the imaging device C, the matching degree calculated by the original matching degree calculation unit 14, the detection result by the target object detection unit 15, the detection parameters set in the parameter setting unit 16A, and simple matching degree calculation.
- the degrees of matching and the like calculated by the unit 18 are displayed on the display device D singly or in combination as necessary.
- the user interface unit 20 causes the display device D to display an image such as a text box prompting the user to input the set values of the detection parameters through the input device E in accordance with these displays.
- FIG. 3 exemplifies the display screen of the display device D performed by the user interface unit 20.
- This display screen includes an area for displaying a photographed image (upper left), an area for displaying for the user to set detection parameters (interrupted on the right), a feature point extraction unit 12, an original matching degree calculation unit 14, and and an area (lower stage) for displaying the calculation result of the simple matching degree calculation unit 18 .
- the degree of matching is displayed as a "score”
- the detection threshold is displayed as a "score threshold”
- the detection threshold for the luminance gradient is displayed as a "contrast threshold”.
- the feature points extracted by the feature point extraction unit 12 are overlaid on the photographed image, with the feature points excluded by the feature point exclusion unit 13 and those excluded by the feature point exclusion unit 13 separated by color.
- the simple matching degree calculation unit 18 is stored in the detection information storage unit 17 without extracting feature points by the feature point extraction unit 12. Since the degree of matching is recalculated using the detection information, the color of the feature point and the value of the degree of matching (score) change without time lag. Therefore, the user can confirm not only the numerical value of the degree of coincidence but also the correspondence relationship between the photographed image and the feature points adopted for the degree of coincidence, and can set more appropriate detection parameters.
- the parameter changing unit has a function of automatically changing detection parameters as in the first embodiment described above, and a detection parameter according to user input as in the second embodiment. and a function to change , both of which can be switched.
- Reference Signs List 1 1A image processing device 11 model storage unit 12 feature point extraction unit 13 feature point exclusion unit 14 original matching degree calculation unit 15 target object detection unit 16, 16A parameter setting unit 17 detection information storage unit 18 simple matching degree calculation unit 19, 19A Parameter changing unit 20 User interface unit C Imaging device D Display device E Input device
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
Description
11 モデル記憶部
12 特徴点抽出部
13 特徴点除外部
14 原一致度算出部
15 対象物検出部
16,16A パラメータ設定部
17 検出情報記憶部
18 簡易一致度算出部
19,19A パラメータ変更部
20 ユーザインターフェイス部
C 撮像装置
D 表示装置
E 入力装置
Claims (5)
- 撮像装置の撮影画像中の対象物を検出する画像処理装置であって、
モデルパターンを記憶するモデル記憶部と、
前記撮影画像から特徴点を抽出する特徴点抽出部と、
前記モデルパターンと前記特徴点の配置との一致度を算出する原一致度算出部と、
前記一致度と検出閾値との比較により前記撮影画像中の前記対象物を検出する対象物検出部と、
少なくとも前記検出閾値を含む検出パラメータを設定するパラメータ設定部と、
前記対象物検出部が検出した前記対象物の前記特徴点について、少なくとも前記特徴点の位置を含む検出情報を記憶する検出情報記憶部と、
前記検出パラメータが変更された場合に、変更後の前記検出パラメータ及び前記検出情報記憶部に記憶されている検出情報に基づいて前記一致度を算出する簡易一致度算出部と、
を備える、画像処理装置。 - 前記特徴点抽出部は、属性値が抽出閾値以上である点を前記特徴点として抽出し、
前記パラメータ設定部に設定される前記検出パラメータは、前記抽出閾値を含み、
前記検出情報記憶部が記憶する前記検出情報は、前記属性値を含む、請求項1に記載の画像処理装置。 - 前記属性値は、前記特徴点の色、輝度、輝度勾配の大きさ及び輝度勾配の方向の少なくともいずれかを含む、請求項2に記載の画像処理装置。
- 前記簡易一致度算出部により前記検出情報に基づいて算出される前記一致度が大きくなるよう、前記検出パラメータを変更するパラメータ変更部をさらに備える、請求項1から3のいずれかに記載の画像処理装置。
- 前記簡易一致度算出部が算出した前記一致度を表示するユーザインターフェイス部をさらに備える、請求項1から4のいずれかに記載の画像処理装置。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202180097150.7A CN117203665A (zh) | 2021-05-13 | 2021-05-13 | 图像处理装置 |
PCT/JP2021/018278 WO2022239202A1 (ja) | 2021-05-13 | 2021-05-13 | 画像処理装置 |
DE112021007225.0T DE112021007225T5 (de) | 2021-05-13 | 2021-05-13 | Bildverarbeitungsvorrichtung |
JP2023520697A JPWO2022239202A1 (ja) | 2021-05-13 | 2021-05-13 | |
TW111115476A TW202244779A (zh) | 2021-05-13 | 2022-04-22 | 圖像處理裝置 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2021/018278 WO2022239202A1 (ja) | 2021-05-13 | 2021-05-13 | 画像処理装置 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022239202A1 true WO2022239202A1 (ja) | 2022-11-17 |
Family
ID=84028056
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2021/018278 WO2022239202A1 (ja) | 2021-05-13 | 2021-05-13 | 画像処理装置 |
Country Status (5)
Country | Link |
---|---|
JP (1) | JPWO2022239202A1 (ja) |
CN (1) | CN117203665A (ja) |
DE (1) | DE112021007225T5 (ja) |
TW (1) | TW202244779A (ja) |
WO (1) | WO2022239202A1 (ja) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015133049A (ja) * | 2014-01-15 | 2015-07-23 | オムロン株式会社 | 画像照合装置、画像センサ、処理システム、画像照合方法 |
JP2018036528A (ja) * | 2016-08-31 | 2018-03-08 | キヤノン株式会社 | 位置検出方法、位置検出装置、リソグラフィ装置および物品製造方法 |
JP2020181290A (ja) * | 2019-04-24 | 2020-11-05 | 株式会社日立製作所 | 物品認識システムおよび物品認識方法 |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6348093B2 (ja) | 2015-11-06 | 2018-06-27 | ファナック株式会社 | 入力データから検出対象物の像を検出する画像処理装置および方法 |
-
2021
- 2021-05-13 DE DE112021007225.0T patent/DE112021007225T5/de active Pending
- 2021-05-13 JP JP2023520697A patent/JPWO2022239202A1/ja active Pending
- 2021-05-13 CN CN202180097150.7A patent/CN117203665A/zh active Pending
- 2021-05-13 WO PCT/JP2021/018278 patent/WO2022239202A1/ja active Application Filing
-
2022
- 2022-04-22 TW TW111115476A patent/TW202244779A/zh unknown
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015133049A (ja) * | 2014-01-15 | 2015-07-23 | オムロン株式会社 | 画像照合装置、画像センサ、処理システム、画像照合方法 |
JP2018036528A (ja) * | 2016-08-31 | 2018-03-08 | キヤノン株式会社 | 位置検出方法、位置検出装置、リソグラフィ装置および物品製造方法 |
JP2020181290A (ja) * | 2019-04-24 | 2020-11-05 | 株式会社日立製作所 | 物品認識システムおよび物品認識方法 |
Also Published As
Publication number | Publication date |
---|---|
TW202244779A (zh) | 2022-11-16 |
CN117203665A (zh) | 2023-12-08 |
DE112021007225T5 (de) | 2024-01-18 |
JPWO2022239202A1 (ja) | 2022-11-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7948479B2 (en) | Method and system for distinguishing multiple touch points | |
US10885311B2 (en) | Face recognition system, face recognition method, display control apparatus, display control method, and display control program | |
CN111986178A (zh) | 产品缺陷检测方法、装置、电子设备和存储介质 | |
AU2016225841B2 (en) | Predicting accuracy of object recognition in a stitched image | |
JP2010097616A (ja) | タッチポイント検出方法及び装置 | |
JP2022059013A (ja) | 情報処理装置、認識支援方法およびコンピュータプログラム | |
CN105469427B (zh) | 一种用于视频中目标跟踪方法 | |
US9286669B2 (en) | Image processing apparatus, image processing method and program | |
JP2013215549A (ja) | 画像処理装置、画像処理プログラムおよび画像処理方法 | |
KR101105872B1 (ko) | 적외선 카메라와 모니터를 이용한 손 인식 방법 및 장치 | |
WO2022239202A1 (ja) | 画像処理装置 | |
KR101021015B1 (ko) | 3차원 사용자 인터페이스 방법 | |
JP2019149119A (ja) | 画像処理装置、画像処理方法及びプログラム | |
KR20120107698A (ko) | 동적 프로그래밍과 캠쉬프트 알고리즘을 이용한 영상 센서 기반 항공기 탐지 장치 및 방법 | |
JP2015045919A (ja) | 画像認識方法及びロボット | |
US9842406B2 (en) | System and method for determining colors of foreground, and computer readable recording medium therefor | |
Uma et al. | Marker based augmented reality food menu | |
US11461585B2 (en) | Image collection apparatus, image collection system, image collection method, image generation apparatus, image generation system, image generation method, and program | |
JP2008226176A (ja) | 熱目標追跡装置及び熱目標追跡方法 | |
US20210004644A1 (en) | Store apparatus, store system, image acquisition method and program | |
Laureano et al. | A topological approach for detection of chessboard patterns for camera calibration | |
JP7345355B2 (ja) | 物体識別装置 | |
JP7211496B2 (ja) | 教師データ生成装置 | |
US9852352B2 (en) | System and method for determining colors of foreground, and computer readable recording medium therefor | |
JP2013246674A (ja) | 図形検出処理装置、図形検出処理方法及び図形検出処理プログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21941935 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2023520697 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202180097150.7 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18555884 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 112021007225 Country of ref document: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21941935 Country of ref document: EP Kind code of ref document: A1 |