WO2019107679A1 - 다종센서 캘리브래이션 시스템 및 다종센서 캘리브래이션 방법 - Google Patents

다종센서 캘리브래이션 시스템 및 다종센서 캘리브래이션 방법 Download PDF

Info

Publication number
WO2019107679A1
WO2019107679A1 PCT/KR2018/005475 KR2018005475W WO2019107679A1 WO 2019107679 A1 WO2019107679 A1 WO 2019107679A1 KR 2018005475 W KR2018005475 W KR 2018005475W WO 2019107679 A1 WO2019107679 A1 WO 2019107679A1
Authority
WO
WIPO (PCT)
Prior art keywords
depth
image information
plate
calibration
photographing
Prior art date
Application number
PCT/KR2018/005475
Other languages
English (en)
French (fr)
Korean (ko)
Inventor
엄태영
최영호
배기덕
이석재
Original Assignee
한국로봇융합연구원
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 한국로봇융합연구원 filed Critical 한국로봇융합연구원
Priority to CN201880077060.XA priority Critical patent/CN111433570B/zh
Publication of WO2019107679A1 publication Critical patent/WO2019107679A1/ko

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
    • G01D18/00—Testing or calibrating apparatus or arrangements provided for in groups G01D1/00 - G01D15/00
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
    • G01D21/00—Measuring or testing not otherwise provided for
    • G01D21/02—Measuring two or more variables by means not covered by a single other subclass

Definitions

  • the present invention relates to a multi-sensor calibration system and a multi-sensor calibration method, and more particularly, to a multi-sensor calibration system, And to a method of calibrating various kinds of sensors.
  • various sensors are used to detect the surrounding environment in a robot or an autonomous vehicle.
  • radar, radar, and camera are widely used to detect the environment around a robot or an autonomous vehicle.
  • sensor information using multiple sensors is fused and utilized rather than one single sensor.
  • the relationship between the sensors is the rotation and translation of the sensor.
  • calibrate between the sensors In order to improve the accuracy of information fusion, the error of calibration should be minimized.
  • Patent Document 1 Korean Patent Registration No. 10-1473736
  • Patent Document 2 Korean Patent Publication No. 10-2013-0098040
  • a color pattern plate including: a color pattern plate having a grid pattern formed by alternating a black square having a predetermined size and a white square having a predetermined size; A hot wire plate in which a checker pattern is formed and a hot wire is provided at an edge portion of the checker pattern; A photographing sensor unit for photographing the color pattern plate and the hot plate; A depth pattern plate having a square pattern formed by at least one square having a predetermined depth and a predetermined depth; And a depth sensor for photographing the depth pattern plate as an image; And a photographing sensor unit to generate corrected image information, correct the image information photographed by the depth sensor unit to generate depth correction information, integrate the depth correction information and the corrected image information, And a calibration unit for generating a plurality of sensor calibration systems.
  • the photographing sensor unit photographs the color pattern plate to generate color image information, and photographs the heat ray plate to generate thermal image information.
  • the depth sensor unit is characterized by photographing a depth pattern plate to generate depth image information.
  • the calibration section is characterized in that the calibration image information is generated by correcting the distorted portions of the color pattern plate and the hot-wire plate in the image information photographed by the photographing sensor section.
  • a method of manufacturing a color image sensor A second step of calibrating the image information generated by the photographing sensor unit to generate corrected image information; A third step of photographing the depth pattern plate as a depth sensor image; And a fourth step of calibrating the image information photographed by the photographing sensor unit to generate depth calibration information, and integrating the depth calibration information and the calibration image information to generate final image information.
  • the second step is characterized in that the image information generated by the photographing sensor unit is column image information or color image information.
  • a multi-type sensor calibration system and a multi-type sensor calibration method are provided, so that the multi-sensor can be calibrated at the same time.
  • FIG. 1 is a block diagram of a first embodiment of a multi-sensor calibration system according to the present invention.
  • Figure 2 is a front view of the color pattern plate in Figure 1;
  • FIG. 3 is a front view of the hot wire plate in Fig.
  • Figure 4 is a front view of the depth pattern plate in Figure 1;
  • FIG. 5 is a cross-sectional view taken along line A-A 'of the depth pattern plate shown in Fig.
  • FIG. 7 is a flowchart showing a multiple sensor calibration method according to the present invention as a first embodiment.
  • first and second components are intended to distinguish one element from another, and the scope of the right should not be limited by these terms.
  • the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.
  • first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.
  • FIG. 1 is a block diagram of a first embodiment of a multi-sensor calibration system of the present invention
  • Fig. 2 is a front view of the color pattern plate of Fig. 1
  • Fig. 3 is a front view of the heat ray plate of Fig. 1
  • 4 is a front view of the depth pattern plate in Fig. 1
  • Fig. 5 is an AA 'sectional view of the depth pattern plate in Fig. 1
  • Fig. 6 is an actual photograph of the heat ray plate in Fig.
  • the multiple sensor calibration system 10 includes a color pattern plate 100, a heat ray plate 200, a photographing sensor unit 300, a depth pattern plate 400, a depth sensor unit 500, .
  • the color pattern plate 100 is formed with a lattice pattern formed by alternately arranging a black rectangle 11 and a white rectangle 12 of a constant size on one side (see FIG. 2).
  • the black rectangle 11 and the white rectangle 12 may be formed in a rectangular shape (preferably square).
  • the black rectangle 11 and the white rectangle 12 may be formed in the same size and shape.
  • a checkerboard pattern is formed on one side of the heat ray plate 200, and a heat ray 13 is provided on the corner of the checkerboard pattern (see FIG. 3).
  • the checkerboard pattern may be formed in the same size and shape as the grid pattern of the color pattern plate 100.
  • the photographing sensor unit 300 photographs the color pattern plate 100 and the heat ray plate 200 as images.
  • the photographing sensor unit 300 may photograph the color pattern plate 100 to generate color image information, and may photograph the heat ray plate 200 to generate thermal image information.
  • the photographing sensor unit 300 may transmit the color image information and the column image information to the calibrator 600.
  • the photographing sensor unit 300 includes a camera (for example, a color camera) for photographing the color pattern plate 100 or a camera (for example, an infrared camera, a night vision camera, a 3D Lidar Etc.), and the like.
  • An infrared camera is a device that tracks and detects heat and displays it on the screen at a glance. Since it shows the screen according to the degree of heat, the object can be checked irrespective of existence of an obstacle such as smoke or light.
  • the night vision camera uses the infrared region to secure a field of view and detects an object ahead based on the secured field of view.
  • a night vision camera can be divided into a near infrared ray method and a far infrared ray method.
  • the near-infrared method is a method of detecting reflected near-infrared rays (wavelength: 800 nm to 1,000 nm) through a camera after infrared rays emitted through an IR generator reach the object.
  • the far infrared ray method refers to a method of detecting heat (wavelength: 1,000 nm or more) emitted by an object using a far infrared ray camera.
  • 3D Lidar is a sensor that detects the surroundings of an object regardless of object motion.
  • the depth pattern plate 400 is formed with a square pattern 14 formed of at least one square of a predetermined size having a predetermined depth (see FIG. 4).
  • the rectangle may be formed to have the same size as the black rectangle 11 or the white rectangle 12.
  • the rectangular patterns 14 may be provided at positions spaced apart from each other by a width of the white square 12 in the horizontal direction and may be installed at positions spaced apart from each other by a vertical length of the white square 12 in the direction of each other.
  • the depth sensor unit 500 photographs the depth pattern plate 400 as an image.
  • the depth sensor unit 500 can photograph the depth pattern plate 400 to generate depth image information.
  • the depth image information may store depth information of the photographed depth pattern plate 400.
  • the depth sensor unit 500 may transmit the depth image information to the calibration unit 600.
  • the depth sensor unit 500 is formed of various types of depth sensors, and can be used as an active depth sensor type and a passive depth sensor type.
  • the active depth sensor method is a method of acquiring three-dimensional depth information of an object directly using a laser sensor, an infrared sensor, and a pattern sensor.
  • a laser sensor an infrared sensor
  • a pattern sensor an infrared sensor
  • the passive depth sensor method is a method of acquiring three-dimensional depth information indirectly from the multi-viewpoint and stereo image. In this case, it takes a long time to acquire the depth information, and there is a disadvantage in providing inaccurate depth information in a relatively occluded region and a non-textured region. However, it provides a high resolution depth image, There are advantages.
  • the calibration unit 600 calibrates the image information captured by the image sensor unit 300 to generate calibration image information, calibrates the image information captured by the depth sensor unit 500 to generate depth calibration information, And the final image information is generated by integrating the information and the corrected image information.
  • the calibration unit 600 may calibrate the distorted portions of the color pattern plate 100 and the hot plate 200 in the image information photographed by the photographing sensor unit 300 to generate corrected image information.
  • the calibration unit 600 receives the color image information from the photographing sensor unit 300 and calibrates a distorted portion (for example, a peripheral portion of the color pattern plate 100) of the captured color pattern plate 100 . In other words, the calibration unit 600 can correct the distortion in the image coordinate system of the color pattern plate 100 in the color image information.
  • the calibration unit 600 may receive the thermal image information from the image sensor unit 300 and calibrate the distorted portion of the captured heat ray plate 200 (e.g., the peripheral portion of the heat ray plate 200) . In other words, the calibration unit 600 can correct distortion in the image coordinate system of the hot plate 200 in the thermal image information.
  • the calibration unit 600 may compare the color image information and the thermal image information to generate the corrected image information by correcting the distorted part.
  • the calibration unit 600 receives the depth image information from the depth sensor unit 500 and confirms the depth of the photographed depth pattern plate 400 so that the depth of the photographed depth pattern plate 400 and the depth of the actual depth pattern plate 400 ) Can be compared and corrected.
  • the calibration unit 600 can correct a difference of 5 mm when the depth of the depth pattern plate 400 stored in the depth image information is 5 mm and the depth of the actual depth pattern plate 400 is 10 mm .
  • the calibration unit 600 may store in advance the actual pattern information of the color pattern plate 100, the actual pattern information of the heat ray plate 200, and the actual depth information of the depth pattern plate 400.
  • the photographing sensor unit 300 and the depth sensor unit 500 are mounted on the color pattern plate 100, the heat ray plate 200 and the depth pattern plate 400, (Calibrating operation) simultaneously with a plurality of multi-sensors (for example, a thermal imaging camera, a night vision camera, a 3D Lidar, etc.).
  • a plurality of multi-sensors for example, a thermal imaging camera, a night vision camera, a 3D Lidar, etc.
  • FIG. 7 is a flowchart showing an embodiment of the multi-sensor calibration method of the present invention.
  • the multiple sensor calibration method includes a first step S100, a second step S200, a third step S300, and a fourth step S400.
  • the photographing sensor unit 300 photographs the color pattern plate 100 and the heat ray plate 200 as an image (S100).
  • the photographing sensor unit 300 calibrates the photographed image information to generate the corrected image information (S200).
  • the depth sensor plate 500 captures the depth pattern plate 400 as an image (S300).
  • the corrected image information and the image information photographed by the depth sensor unit 500 are corrected (S400).
  • the multiple sensor calibration system includes a color pattern and a hot wire plate (not shown in the drawings for convenience of explanation), a photographing sensor unit 300, a depth pattern plate 400, a depth sensor unit 500, .
  • a photographing sensor unit 300, the depth pattern plate 400, the depth sensor unit 500, and the calibration unit 600 will not be described below with respect to components similar to those of FIG. 1, Explain it.
  • the color pattern and the heat ray plate are formed with a grid pattern formed by alternately arranging a black square of a certain size and a white square of a certain size on one surface, and a heat ray is installed at a corner portion of the grid pattern.
  • the photographing sensor unit 300 photographs one side of the color pattern and the heat ray plate as an image.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Spectrometry And Color Measurement (AREA)
PCT/KR2018/005475 2017-11-30 2018-05-14 다종센서 캘리브래이션 시스템 및 다종센서 캘리브래이션 방법 WO2019107679A1 (ko)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201880077060.XA CN111433570B (zh) 2017-11-30 2018-05-14 多种传感器校准系统以及多种传感器校准方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020170163474A KR102012568B1 (ko) 2017-11-30 2017-11-30 다종센서 캘리브래이션 시스템 및 다종센서 캘리브래이션 방법
KR10-2017-0163474 2017-11-30

Publications (1)

Publication Number Publication Date
WO2019107679A1 true WO2019107679A1 (ko) 2019-06-06

Family

ID=66664081

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2018/005475 WO2019107679A1 (ko) 2017-11-30 2018-05-14 다종센서 캘리브래이션 시스템 및 다종센서 캘리브래이션 방법

Country Status (3)

Country Link
KR (1) KR102012568B1 (zh)
CN (1) CN111433570B (zh)
WO (1) WO2019107679A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10104033A (ja) * 1996-09-27 1998-04-24 Komatsu Ltd 画像処理装置のキャリブレーションプレート、これを用いたキャリブレーション装置及び３次元位置計測装置
JP2009236533A (ja) * 2008-03-26 2009-10-15 Asahi Corp キャリブレーションプレート
KR20130123891A (ko) * 2012-05-04 2013-11-13 전자부품연구원 홀 발생 억제를 위한 3ｄ­워핑 방법 및 이를 적용한 영상 처리 장치
KR101390882B1 (ko) * 2013-02-27 2014-04-30 한국표준과학연구원 카메라용 캘리브레이션 플레이트
KR101681105B1 (ko) * 2016-06-28 2016-12-02 주식회사 파인에스엔에스 카메라용 캘리브레이션 장치

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3177293B2 (ja) 1992-04-17 2001-06-18 三井化学株式会社 塗工紙用組成物及び塗工紙
US6591125B1 (en) * 2000-06-27 2003-07-08 Therasense, Inc. Small volume in vitro analyte sensor with diffusible or non-leachable redox mediator
JP5388417B2 (ja) * 2007-02-16 2014-01-15 株式会社Ｉｈｉ 赤外線カメラ調整装置及び赤外線カメラ調整方法
JP2010014986A (ja) * 2008-07-04 2010-01-21 Konica Minolta Business Technologies Inc カラー画像形成装置及びその制御方法
DE102011084264A1 (de) * 2011-10-11 2013-04-11 Robert Bosch Gmbh Verfahren und Vorrichtung zum Kalibrieren eines Umfeldsensors
KR20130098040A (ko) 2012-02-27 2013-09-04 삼성전자주식회사 깊이 이미지를 자동으로 캘리브레이션하는 이미지 시스템
JP3177293U (ja) * 2012-05-16 2012-07-26 株式会社エイチアンドエフ カメラキャリブレーション用ボード
KR101473736B1 (ko) 2013-12-20 2014-12-18 국방과학연구소 폐루프 기반의 다중 센서 캘리브레이션 장치 및 그 방법
CN204924273U (zh) * 2015-08-31 2015-12-30 中国科学院沈阳自动化研究所 一种全景三维激光传感器数据校准装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10104033A (ja) * 1996-09-27 1998-04-24 Komatsu Ltd 画像処理装置のキャリブレーションプレート、これを用いたキャリブレーション装置及び３次元位置計測装置
JP2009236533A (ja) * 2008-03-26 2009-10-15 Asahi Corp キャリブレーションプレート
KR20130123891A (ko) * 2012-05-04 2013-11-13 전자부품연구원 홀 발생 억제를 위한 3ｄ­워핑 방법 및 이를 적용한 영상 처리 장치
KR101390882B1 (ko) * 2013-02-27 2014-04-30 한국표준과학연구원 카메라용 캘리브레이션 플레이트
KR101681105B1 (ko) * 2016-06-28 2016-12-02 주식회사 파인에스엔에스 카메라용 캘리브레이션 장치

Also Published As

Publication number Publication date
KR20190064708A (ko) 2019-06-11
KR102012568B1 (ko) 2019-08-20
CN111433570B (zh) 2022-04-08
CN111433570A (zh) 2020-07-17

Similar Documents

Publication Publication Date Title
KR102166691B1 (ko) 객체의 3차원 형상을 산출하는 장치 및 방법
WO2016153100A1 (ko) 카메라로부터 획득한 영상에 대한 자동보정기능을 구비한 영상처리장치 및 그 방법
CN103024347B (zh) 摄像装置、监视用摄像机以及摄像机画面的掩模方法
WO2014051196A1 (ko) 구강 스캐너
US8593536B2 (en) Image pickup apparatus with calibration function
WO2012176945A1 (ko) 차량 주변 시각화를 위한 3차원 영상 합성장치 및 그 방법
WO2018049699A1 (zh) 一种基于微型摄像头矩阵的图像获取装置
CN108596982A (zh) 一种简易的车载多目摄像机环视系统标定方法及装置
WO2012148025A1 (ko) 복수의 카메라를 이용한 3차원 물체 검출장치 및 방법
CN111429531A (zh) 标定方法、标定装置和非易失性计算机可读存储介质
US8125520B2 (en) System for establishing references for multi-image stitching
WO2011078615A2 (en) Distance adaptive 3d camera
JP2012198031A (ja) 画像補正方法及び画像補正装置
Starr et al. Application of thermal infrared stereo vision in fire environments
CN106813595B (zh) 三相机组特征点匹配方法、测量方法及三维检测装置
JP3666348B2 (ja) 距離認識装置
JP5487946B2 (ja) カメラ画像の補正方法およびカメラ装置および座標変換パラメータ決定装置
WO2017218206A1 (en) Method and apparatus of lens alignment for capsule camera
WO2019107679A1 (ko) 다종센서 캘리브래이션 시스템 및 다종센서 캘리브래이션 방법
WO2021261642A1 (ko) 측정온도 보정시스템
WO2014185578A1 (ko) 직교식 입체 카메라의 광축 정렬 방법 및 직교식 입체 카메라
JP6949086B2 (ja) カメラ装置、運転支援システムおよび車両
CN115018923A (zh) 一种全景环带镜头参数的检测方法及系统
WO2014046325A1 (ko) 3차원 계측 시스템 및 그 방법
WO2019124750A1 (ko) 타임슬라이스 촬영을 위한 카메라 캘리브레이션 방법 및 이를 위한 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18884541

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18884541

Country of ref document: EP

Kind code of ref document: A1