JP2006115265A - Image processing method of visual device, and image processing apparatus - Google Patents

Image processing method of visual device, and image processing apparatus Download PDF

Info

Publication number
JP2006115265A
JP2006115265A JP2004301083A JP2004301083A JP2006115265A JP 2006115265 A JP2006115265 A JP 2006115265A JP 2004301083 A JP2004301083 A JP 2004301083A JP 2004301083 A JP2004301083 A JP 2004301083A JP 2006115265 A JP2006115265 A JP 2006115265A
Authority
JP
Japan
Prior art keywords
image
camera
resolution
image processing
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2004301083A
Other languages
Japanese (ja)
Other versions
JP4414859B2 (en
Inventor
Hiroshi Osada
拓 長田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honda Motor Co Ltd
Original Assignee
Honda Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honda Motor Co Ltd filed Critical Honda Motor Co Ltd
Priority to JP2004301083A priority Critical patent/JP4414859B2/en
Publication of JP2006115265A publication Critical patent/JP2006115265A/en
Application granted granted Critical
Publication of JP4414859B2 publication Critical patent/JP4414859B2/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Processing (AREA)
  • Studio Circuits (AREA)
  • Studio Devices (AREA)

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method of a visual device, which obtains a planar image with required resolution from an image, using a wide viewing angle obtained by a fish-eye lens, and to provide an image processing apparatus. <P>SOLUTION: In the visual device including a camera 1, provided with the fish-eye lens and a camera direction changing means (actuator 2), image processing is performed through a step (S2) of setting an area with a prescribed resolution, a step (S2) of imaging with the camera, a step (S3) of converting the imaged by the camera to a planar image with resolution based on the prescribed resolution, a step (S6) of determining whether an object exists within an area with prescribed resolution of the plane image, and a step (S8) of changing the direction of the camera so as to capture the object within the area with prescribed resolution, when it is determined that the object does not exist within the area of the prescribed resolution. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

本発明は、視覚装置の画像処理方法および画像処理装置に関するものである。   The present invention relates to an image processing method and an image processing apparatus for a visual device.

カメラで撮影した画像から物体認識を行うには、画像の全域にわたって一様な解像度が得られるピンホールカメラ画像であることが望ましい。しかしこのピンホールカメラ画像は、解像度が一様である反面、広い視野を得ようとすると解像度が低下して対象物の情報を十分に得ることができなくなり、解像度を高めると限られた狭い視野しか得られなくなって探索範囲が狭くなるという欠点がある。   In order to perform object recognition from an image captured by a camera, a pinhole camera image that can obtain a uniform resolution over the entire area of the image is desirable. However, this pinhole camera image has a uniform resolution, but if you try to obtain a wide field of view, the resolution will drop and you will not be able to get enough information about the object. However, there is a drawback that the search range is narrowed because it can only be obtained.

つまりピンホールカメラ画像は、目的物の探査や障害物回避を迅速に行う上には好適とは言い難く、これを自律移動ロボットの視覚情報として用いて対象物を十分な精度で測定するには、対象物を捉えるために視線の向きを頻繁に変えなければならなくなる。   In other words, pinhole camera images are not suitable for expediting exploration of objects or avoiding obstacles, and it is necessary to use this as visual information for autonomous mobile robots to measure objects with sufficient accuracy. , The direction of the line of sight must be changed frequently in order to capture the object.

このような不都合に対処し得る技術として、魚眼（超広角）レンズの画像から物体を認識するシステムが特許文献１に提案されている。
特開平６−１６７５６４号公報 As a technique that can cope with such inconvenience, a system that recognizes an object from an image of a fish-eye (super wide-angle) lens is proposed in Patent Document 1.
JP-A-6-167564

しかるに、特許文献１に記載のような魚眼レンズで得た画像は、視野角が広い反面、画像歪みが大きく、画角の周辺へ行くに従って解像度が低下するため、平面画像による物体認識に関する一般的な手法を適用し難い。   However, an image obtained with a fisheye lens as described in Patent Document 1 has a wide viewing angle, but has a large image distortion and decreases in resolution as it goes to the periphery of the angle of view. It is difficult to apply the method.

本発明は、このような従来技術の欠点を解消すべく案出されたものであり、その主な目的は、魚眼レンズにより得られる広い視野角の画像から必要な解像度の平面画像が得られるようにするための視覚装置の画像処理方法ならびに画像処理装置を提供することにある。   The present invention has been devised to eliminate such drawbacks of the prior art, and its main purpose is to obtain a planar image having a necessary resolution from a wide viewing angle image obtained by a fisheye lens. An object of the present invention is to provide an image processing method for a visual device and an image processing device.

このような課題を解決するため、本発明の請求項１は、魚眼レンズを備えたカメラ（１）と、該カメラの向きの変更手段（アクチュエータ２）とを備えた視覚装置の画像処理方法として、所定解像度の領域を設定して（ステップ１）前記カメラで撮像する過程（ステップ２）と、カメラで撮像した画像を所定解像度に基づく解像度の平面画像に変換する過程（ステップ３）と、平面画像における所定解像度の領域内に対象物が存在するか否かを判別する過程（ステップ６）と、所定解像度の領域内に対象物が存在しないと判定された場合に所定解像度の領域内に対象物を捉えるようにカメラの向きを変更する過程（ステップ８）とを有することを特徴とする画像処理方法を提供することとした。特に、請求項２においては、前記所定解像度の領域内に対象物が存在すると判定された場合には、所定解像度の領域を狭める過程（ステップ７）を経た上で再び平面画像に変換する過程へ移行することとした。   In order to solve such a problem, claim 1 of the present invention provides an image processing method for a visual device including a camera (1) having a fisheye lens and means for changing the orientation of the camera (actuator 2). A process of setting an area with a predetermined resolution (step 1) and capturing with the camera (step 2), a process of converting an image captured with the camera into a planar image with a resolution based on the predetermined resolution (step 3), and a planar image The process of determining whether or not the object exists in the predetermined resolution area (step 6) and the object in the predetermined resolution area when it is determined that the object does not exist in the predetermined resolution area An image processing method characterized by having a process (step 8) of changing the direction of the camera so as to capture the image. In particular, in the second aspect, when it is determined that an object is present in the predetermined resolution area, a process of narrowing the predetermined resolution area (step 7) and then converting to a planar image again is performed. I decided to migrate.

また本発明の請求項３は、魚眼レンズを備えたカメラと、該カメラの向きの変更手段とを備えた視覚装置の画像処理装置において、前記カメラで撮像した画像を所定解像度の平面画像に変換する手段（画像変換部１１）と、平面画像における所定解像度の領域内に対象物が存在するか否かを判別する手段（対象物抽出部１２）と、所定解像度の領域内に対象物が存在しないと判定された場合に所定解像度の領域内に対象物を捉えるようにカメラの向きの変更指令を発する手段（運動制御部９）とを有することを特徴とするものとした。   According to a third aspect of the present invention, in an image processing apparatus of a visual device including a camera equipped with a fisheye lens and a direction changing unit of the camera, an image captured by the camera is converted into a planar image having a predetermined resolution. Means (image conversion unit 11), means (object extraction unit 12) for determining whether or not an object is present in an area of a predetermined resolution in a planar image, and no object is present in an area of a predetermined resolution And a means (motion control unit 9) for issuing a camera orientation change command so as to capture the object within a predetermined resolution area.

このような本発明によれば、魚眼レンズで得た広い視野角の画像を、所望の解像度でのピンホールカメラ画像に変換し、一般的な平面画像上での認識手法を適用して対象物を的確に認識することができるので、カメラの向きの変更を最小限にして広範囲に対象物の探査を行うことができる。   According to the present invention, an image with a wide viewing angle obtained by a fisheye lens is converted into a pinhole camera image at a desired resolution, and a recognition method on a general plane image is applied to apply the object. Since it can be accurately recognized, it is possible to search for an object in a wide range with minimal change in the direction of the camera.

以下に添付の図面を参照して本発明について詳細に説明する。   Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

図１は、本発明が適用されるロボットの概略構成を示すブロック図である。このロボットは、魚眼レンズカメラ１及びカメラの向きを変えるアクチュエータ２を備えた頭部３、移動のための脚部４、及び作業を行うための腕部５を備えたロボット本体６と、これらを制御する制御ユニット７とを備えている。   FIG. 1 is a block diagram showing a schematic configuration of a robot to which the present invention is applied. This robot has a fish body camera 1 and a head 3 having an actuator 2 for changing the direction of the camera, a leg portion 4 for movement, and a robot body 6 having an arm portion 5 for performing work, and controls these. The control unit 7 is provided.

制御ユニット７は、魚眼レンズカメラ１で撮像した魚眼画像から対象物の位置を認識する視覚制御部８と、視覚制御部８が認識した対象物の位置に基づいてロボットの頭部３、脚部４、および腕部５の運動を制御する運動制御部９とを備えている。   The control unit 7 includes a visual control unit 8 that recognizes the position of an object from a fisheye image captured by the fisheye lens camera 1, and the head 3 and legs of the robot based on the position of the object recognized by the visual control unit 8. 4 and a movement control unit 9 for controlling the movement of the arm unit 5.

視覚制御部８は、魚眼レンズカメラ１が撮像した元画像を取り込む画像入力部１０と、入力画像を平面画像に変換する画像変換部１１と、平面画像から対象物を抽出する対象物抽出部１２と、抽出された対象物の空間内での位置を算出する対象物位置演算部１３とを備えている。   The visual control unit 8 includes an image input unit 10 that captures an original image captured by the fisheye lens camera 1, an image conversion unit 11 that converts the input image into a plane image, and an object extraction unit 12 that extracts an object from the plane image. And an object position calculation unit 13 for calculating the position of the extracted object in the space.

次に本発明が関わる画像変換部１１の処理について図２を参照して詳細に説明する。   Next, the processing of the image conversion unit 11 according to the present invention will be described in detail with reference to FIG.

本発明が対象とする魚眼レンズは、広範囲な画角で撮像できる反面、周辺部へ行くに従って歪みが増大し、かつ解像度も低下するという不都合がある。そのため、元画像を単に平面画像に変換しただけでは、対象物の識別に必要な特徴量の抽出に十分な解像度が得られないことがある。そこで本発明においては、以下に詳述する処理を行うこととした。   The fisheye lens targeted by the present invention can capture images in a wide range of angles, but has the disadvantages that distortion increases and resolution decreases as it goes to the periphery. Therefore, simply converting the original image into a planar image may not provide sufficient resolution for extracting feature quantities necessary for object identification. Therefore, in the present invention, the processing described in detail below is performed.

魚眼レンズカメラ１で得た画像から存在位置の不明な対象物を探索する場合、先ず、対象物が存在する可能性のある範囲を推定し、その範囲に応じて適宜な初期等解像度円を設定し（ステップ１）、その初期等解像度円内に対象物を捉えられるようにカメラの初期位置を設定した上で対象物が存在する可能性のある範囲を撮像する（ステップ２）。   When searching for an object whose location is unknown from an image obtained with the fisheye lens camera 1, first, a range where the object may exist is estimated, and an appropriate initial equal resolution circle is set according to the range. (Step 1) After setting the initial position of the camera so that the object can be captured within the initial equal-resolution circle, the range in which the object may exist is imaged (Step 2).

このようにして得られた魚眼画像は、図３に示すように、ボウ歪み、スキュー歪みが共に大きく、その中心部の解像度は高いが、画角の周辺へ行くに従って解像度が著しく低下する特性を有している。そこで図３に破線の同心円で示した等解像度円のうちから適宜な解像度の円（例えば最内周の円）を設定し、その内側の画像は圧縮処理し、外側の画像は伸長処理して画像全体の歪み補正を行い、図４に示したようなピンホールカメラ画像、つまり平面画像に変換する（ステップ３）。   As shown in FIG. 3, the fish-eye image obtained in this way has large bow distortion and skew distortion, and the resolution at the center is high, but the resolution is remarkably reduced as it goes to the periphery of the angle of view. have. Therefore, an appropriate resolution circle (for example, the innermost circle) is set from the equal resolution circles shown by the concentric circles in FIG. 3, and the inner image is compressed and the outer image is expanded. The distortion of the entire image is corrected and converted into a pinhole camera image as shown in FIG. 4, that is, a planar image (step 3).

このようにして得られたピンホールカメラ画像上にて対象物の判別処理を行い（ステップ４）、対象物の画像解像度が特徴量の抽出処理に十分な解像度であるか否かを判別する（ステップ５）。ここで解像度が十分と判断された場合はこのフローを終了し、必要な解像度に不足すると判断された場合は、等解像度円の内外いずれに対象物があるかを判別し（ステップ６）、対象物が内側にあると判断された場合は、必要な解像度が得られるところまでズームして等解像度円の半径を小さくし（ステップ７）、ステップ３で再度平面画像変換処理を行った上で対象物の認識処理を行う。   An object discrimination process is performed on the pinhole camera image thus obtained (step 4), and it is determined whether or not the image resolution of the object is sufficient for the feature amount extraction process ( Step 5). If it is determined that the resolution is sufficient, the flow is terminated. If it is determined that the required resolution is insufficient, it is determined whether the target is inside or outside the equal resolution circle (step 6). If it is determined that the object is inside, zoom to the required resolution to reduce the radius of the iso-resolution circle (step 7), and after performing the planar image conversion process again in step 3, the object Perform object recognition processing.

他方、ステップ６で対象物が等解像度円の外側にあると判断された場合は、対象物の認識に十分な解像度が得られないと判断し、対象物が等解像度円内に入るように改めてカメラの向きを修正し（ステップ８）、ステップ２へ戻る。   On the other hand, if it is determined in step 6 that the object is outside the iso-resolution circle, it is determined that sufficient resolution for recognition of the object cannot be obtained, and the object is re-entered within the iso-resolution circle. The camera orientation is corrected (step 8), and the process returns to step 2.

この一連の処理を経ることにより、魚眼レンズの広い視野角と、魚眼画像の中心付近の高い解像度とを共に好適に利用することができることとなり、存在位置の不明な障害物あるいは目標物を早期に認識し、無駄のない迅速な回避行動あるいは接近行動をとることが可能となる。   Through this series of processing, both the wide viewing angle of the fisheye lens and the high resolution near the center of the fisheye image can be suitably used, and an obstacle or target whose location is unknown can be quickly used. It is possible to recognize and take a quick avoiding action or approaching action without waste.

本発明装置の概略構成を示すブロック図である。It is a block diagram which shows schematic structure of this invention apparatus. 本発明方法の処理フロー図である。It is a processing flow figure of a method of the present invention. 魚眼画像の概念図である。It is a conceptual diagram of a fisheye image. 変換後の平面画像の概念図である。It is a conceptual diagram of the planar image after conversion.

符号の説明Explanation of symbols

１ 魚眼レンズカメラ
２ アクチュエータ
９ 運動制御部
１１ 画像変換部
１２ 対象物抽出部 DESCRIPTION OF SYMBOLS 1 Fisheye lens camera 2 Actuator 9 Motion control part 11 Image conversion part 12 Object extraction part

Claims (3)

魚眼レンズを備えたカメラと、該カメラの向きの変更手段とを備えた視覚装置の画像処理方法であって、
所定解像度の領域を設定して前記カメラで撮像する過程と、
前記カメラで撮像した画像を前記所定解像度に基づく解像度の平面画像に変換する過程と、
前記平面画像における前記所定解像度の領域内に対象物が存在するか否かを判別する過程と、
前記所定解像度の領域内に前記対象物が存在しないと判定された場合に前記所定解像度の領域内に対象物を捉えるように前記カメラの向きを変更する過程とを有することを特徴とする視覚装置の画像処理方法。 An image processing method for a visual device comprising a camera equipped with a fisheye lens and means for changing the orientation of the camera,
A process of setting an area of a predetermined resolution and imaging with the camera;
Converting an image captured by the camera into a planar image having a resolution based on the predetermined resolution;
Determining whether or not an object exists in the area of the predetermined resolution in the planar image;
And a step of changing the direction of the camera so as to capture the object in the region of the predetermined resolution when it is determined that the object does not exist in the region of the predetermined resolution. Image processing method. 前記所定解像度の領域内に対象物が存在すると判定された場合には、前記所定解像度の領域を狭める過程を経た上で再び前記平面画像に変換する過程へ移行することを特徴とする請求項１に記載の視覚装置の画像処理方法。   2. If it is determined that an object is present in the predetermined resolution area, the process proceeds to a process of narrowing the predetermined resolution area and then converting to the planar image again. An image processing method for a visual device according to claim 1. 魚眼レンズを備えたカメラと、該カメラの向きの変更手段とを備えた視覚装置の画像処理装置であって、
前記カメラで撮像した画像を所定解像度の平面画像に変換する手段と、
前記平面画像における所定解像度の領域内に対象物が存在するか否かを判別する手段と、
前記所定解像度の領域内に前記対象物が存在しないと判定された場合に前記所定解像度の領域内に対象物を捉えるように前記カメラの向きの変更指令を発する手段とを有することを特徴とする視覚装置の画像処理装置。 An image processing device for a visual device comprising a camera equipped with a fisheye lens and means for changing the orientation of the camera,
Means for converting an image captured by the camera into a planar image having a predetermined resolution;
Means for determining whether or not an object is present in an area of a predetermined resolution in the planar image;
Means for issuing an instruction to change the orientation of the camera so as to capture the object in the area of the predetermined resolution when it is determined that the object does not exist in the area of the predetermined resolution. An image processing device for a visual device.
JP2004301083A 2004-10-15 2004-10-15 Image processing method and image processing apparatus for visual device Expired - Fee Related JP4414859B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2004301083A JP4414859B2 (en) 2004-10-15 2004-10-15 Image processing method and image processing apparatus for visual device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2004301083A JP4414859B2 (en) 2004-10-15 2004-10-15 Image processing method and image processing apparatus for visual device

Publications (2)

Publication Number Publication Date
JP2006115265A true JP2006115265A (en) 2006-04-27
JP4414859B2 JP4414859B2 (en) 2010-02-10

Family

ID=36383390

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2004301083A Expired - Fee Related JP4414859B2 (en) 2004-10-15 2004-10-15 Image processing method and image processing apparatus for visual device

Country Status (1)

Country Link
JP (1) JP4414859B2 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014091877A1 (en) * 2012-12-12 2014-06-19 日産自動車株式会社 Moving object location/attitude angle estimation device and moving object location/attitude angle estimation method
CN104135608A (en) * 2014-05-21 2014-11-05 张家港恩达通讯科技有限公司 Flattening method and system for fish-eye lens image

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014091877A1 (en) * 2012-12-12 2014-06-19 日産自動車株式会社 Moving object location/attitude angle estimation device and moving object location/attitude angle estimation method
JP5962771B2 (en) * 2012-12-12 2016-08-03 日産自動車株式会社 Moving object position / posture angle estimation apparatus and moving object position / posture angle estimation method
JPWO2014091877A1 (en) * 2012-12-12 2017-01-05 日産自動車株式会社 Moving object position / posture angle estimation apparatus and moving object position / posture angle estimation method
US9740942B2 (en) 2012-12-12 2017-08-22 Nissan Motor Co., Ltd. Moving object location/attitude angle estimation device and moving object location/attitude angle estimation method
CN104135608A (en) * 2014-05-21 2014-11-05 张家港恩达通讯科技有限公司 Flattening method and system for fish-eye lens image
CN104135608B (en) * 2014-05-21 2017-07-14 张家港恩达通讯科技有限公司 The flattening method and system of a kind of fisheye image

Also Published As

Publication number Publication date
JP4414859B2 (en) 2010-02-10

Similar Documents

Publication Publication Date Title
JP4451122B2 (en) Video tracking system and method
US7783076B2 (en) Moving-object tracking control apparatus, moving-object tracking system, moving-object tracking control method, and program
US8754945B2 (en) Image capturing device and motion tracking method
CN111683204A (en) Unmanned aerial vehicle shooting method and device, computer equipment and storage medium
WO2017215351A1 (en) Method and apparatus for adjusting recognition range of photographing apparatus
JP2008507229A (en) Automatic expansion of zoom function of wide-angle video camera
JP6335701B2 (en) Information processing apparatus, information processing method, and program
WO2017215108A1 (en) Method for adjusting projected image, and projector
US10397485B2 (en) Monitoring camera direction control
WO2018168406A1 (en) Photography control device, photography system, and photography control method
JP2017138947A (en) Flight altitude control device, unmanned aerial vehicle, flight altitude control method, and flight altitude control program
JP6065629B2 (en) Object detection device
JP2017022574A (en) Correction information output device, image processing device, correction information output method, imaging control system and movable body control system
JP2013021551A (en) Detection device, detection method, and program
JP2017204795A (en) Tracking apparatus
JP4414859B2 (en) Image processing method and image processing apparatus for visual device
JP2006127083A (en) Image processing method, and image processor
JP2012086285A (en) Tracking robot device, tracking robot control method, tracking robot control program, homography matrix acquisition device, homography matrix acquisition method, and homography matrix acquisition program
JP2008060988A (en) Apparatus for acquiring travel environment information
US11070719B2 (en) Image capture assist device and image capture assist method
JP2006285403A (en) Image processing method, and image processing device
JP2013179614A (en) Imaging apparatus
Wang et al. A vision location system design of glue dispensing robot
JPH07162730A (en) Mobile object image pickup device
CN110595443A (en) Projection device

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20061201

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20090303

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20090331

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20090526

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20090818

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20091015

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20091110

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20091120

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20121127

Year of fee payment: 3

R150 Certificate of patent or registration of utility model

Ref document number: 4414859

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20131127

Year of fee payment: 4

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

LAPS Cancellation because of no payment of annual fees