WO2019019172A1 - Traitement adaptatif d'image dans un véhicule robotisé - Google Patents

Traitement adaptatif d'image dans un véhicule robotisé Download PDF

Info

Publication number
WO2019019172A1
WO2019019172A1 PCT/CN2017/094965 CN2017094965W WO2019019172A1 WO 2019019172 A1 WO2019019172 A1 WO 2019019172A1 CN 2017094965 W CN2017094965 W CN 2017094965W WO 2019019172 A1 WO2019019172 A1 WO 2019019172A1
Authority
WO
WIPO (PCT)
Prior art keywords
estimated
pixels
pixel
processing device
difference
Prior art date
Application number
PCT/CN2017/094965
Other languages
English (en)
Inventor
Yin Huang
Xiaoyi Zhu
Leslie Chiming Chan
Ruowei WANG
Original Assignee
Qualcomm Incorporated
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Incorporated filed Critical Qualcomm Incorporated
Priority to PCT/CN2017/094965 priority Critical patent/WO2019019172A1/fr
Publication of WO2019019172A1 publication Critical patent/WO2019019172A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
    • H04N7/185—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source from a mobile camera, e.g. for remote control
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/242—Aligning, centring, orientation detection or correction of the image by image rotation, e.g. by 90 degrees
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6812—Motion detection based on additional sensors, e.g. acceleration sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/683—Vibration or motion blur correction performed by a processor, e.g. controlling the readout of an image memory
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/53—Control of the integration time
    • H04N25/531—Control of the integration time by controlling rolling shutters in CMOS SSIS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Definitions

  • Various embodiments include methods that may be implemented on a processing device of a robotic vehicle for processing an image captured by an image sensor of the robotic vehicle to adaptively track regions of interest within images for captured by an image sensor of a robotic vehicle without the need for a physical gimbal.
  • Various embodiments improve the efficiency and accuracy of image processing of such images captured using a rolling shutter type image sensor in a robotic vehicle subject to pitch, yaw, and roll.
  • Various embodiments further improve efficiency and accuracy of correcting for rolling shutter distortion and lens distortion by images captured by a robotic vehicle in motion, particularly an aerial robotic vehicle.
  • a processing device of a robotic vehicle may utilize tracking information obtained from various sensors on a robotic vehicle (e.g., inertial measurement units) to estimate the relative motion of individual pixels of image sensor output.
  • the exposure time for each pixel may be determined or calculated by the processing device based on this estimated motion.
  • the exposure time, position, and motion of pixels may be used by the processing device to calculate a correction matrix, which may be applied to the image sensor output to produce a corrected image.
  • the power module 230 may include one or more batteries that may provide power to various components, including the processing device 220, the sensors 240, the payload-securing unit (s) 244, the image sensor (s) 245, the output module 250, the input module 260, and the radio module 270.
  • the power module 230 may include energy storage components, such as rechargeable batteries.
  • the processing device 220 may be configured with processor-executable instructions to control the charging of the power module 230 (i.e., the storage of harvested energy) , such as by executing a charging control algorithm using a charge control circuit.
  • the power module 230 may be configured to manage its own charging.
  • the processing device 220 may be coupled to the output module 250, which may output control signals for managing the motors that drive the rotors 202 and other components.
  • the rolling-shutter correction and warp unit 426 may crop the image information, correct for distortions in the image caused by the lens 404, and may apply the transformation matrices/map to the image information.
  • the rolling-shutter correction and warp unit 426 may provide as output a corrected image 428 based on the cropping, distortion correction, and/or application of the transformation matrix.
  • the corrected image may include an image having a corrected horizontal orientation or horizontal rotation.
  • the corrected image may include a stabilized video output.
  • FIG. 6B illustrates rolling shutter distortion that may be cause by a pitch and a yaw of a motion sensor.
  • Image sensor rotation e.g., caused by pitch and yaw of a platform of the image sensor, e.g., a robotic vehicle
  • changes in yaw during exposure of a frame may cause vertical lines to develop a diagonal skew 606.
  • changes in pitch during exposure of a frame may change a separation 608 between horizontal lines and may lead to a perception of residual motion along a Y-axis (e.g., horizontal axis) of the image.
  • a processing device may correct rolling shutter distortion and lens distortion by modeling a motion of pixels within the image or frame.
  • the image sensor capture may be mapped to the matrix K.
  • a point (X, Y, Z) in 3D space may be mapped to an image plane (x, y) based on pin-hole.
  • the image sensor capture may be represented as:

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne des dispositifs et des procédés destinés à un traitement adaptatif d'image dans un véhicule robotisé. Un processeur de véhicule robotisé peut prendre une image par un capteur d'image du véhicule robotisé. Le processeur peut effectuer une correction de distorsion de lentille sur des pixels centraux de chaque ligne de l'image capturée pour produire des informations de mouvement estimé. Le processeur peut produire une carte de pixels, la carte de pixels comprenant un mappage de positions de pixels d'entrée permettant d'obtenir des positions de pixels en fonction, au moins en partie, des informations de mouvement estimé. Le processeur peut effectuer une correction de distorsion de lentille sur les pixels en fonction, au moins en partie, des informations de mouvement estimé, pour produire des informations de mouvement de pixel. Le processeur peut effectuer des corrections d'obturateur roulant sur les pixels en fonction, au moins en partie, des informations de mouvement de pixel, pour produire des pixels corrigés. Le processeur peut produire une image corrigée à l'aide des pixels corrigés et de la carte produite de pixels.
PCT/CN2017/094965 2017-07-28 2017-07-28 Traitement adaptatif d'image dans un véhicule robotisé WO2019019172A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/094965 WO2019019172A1 (fr) 2017-07-28 2017-07-28 Traitement adaptatif d'image dans un véhicule robotisé

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/094965 WO2019019172A1 (fr) 2017-07-28 2017-07-28 Traitement adaptatif d'image dans un véhicule robotisé

Publications (1)

Publication Number Publication Date
WO2019019172A1 true WO2019019172A1 (fr) 2019-01-31

Family

ID=65039894

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/094965 WO2019019172A1 (fr) 2017-07-28 2017-07-28 Traitement adaptatif d'image dans un véhicule robotisé

Country Status (1)

Country Link
WO (1) WO2019019172A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR3106402A1 (fr) * 2020-01-16 2021-07-23 Continental Automotive Procédé de calibration d’un support rotatif de dispositif d’imagerie de véhicule automobile
CN113538283A (zh) * 2021-07-22 2021-10-22 浙江赫千电子科技有限公司 一种冗余鱼眼摄像头拍摄图像的畸变矫正方法
WO2022112221A3 (fr) * 2020-11-27 2022-07-21 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Évaluation d'informations de balayage à l'aide d'informations de position
CN115150546A (zh) * 2021-03-30 2022-10-04 本田技研工业株式会社 信息处理装置和方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070092157A1 (en) * 2005-10-20 2007-04-26 Alpha Imaging Technology Corp. Image acquiring apparatus and image processing method thereof
US20140247358A1 (en) * 2011-11-24 2014-09-04 Aisin Seiki Kabushiki Kaisha Image generation device for monitoring surroundings of vehicle
CN105447853A (zh) * 2015-11-13 2016-03-30 深圳市道通智能航空技术有限公司 飞行装置、飞行控制系统及方法
US20160189350A1 (en) * 2014-12-30 2016-06-30 Texas Instruments Incorporated System and method for remapping of image to correct optical distortions
KR20170077994A (ko) * 2015-12-29 2017-07-07 전자부품연구원 카메라 영상 보정 및 렉티피케이션을 위한 효율적인 데이터 좌표 맵 생성 방법

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070092157A1 (en) * 2005-10-20 2007-04-26 Alpha Imaging Technology Corp. Image acquiring apparatus and image processing method thereof
US20140247358A1 (en) * 2011-11-24 2014-09-04 Aisin Seiki Kabushiki Kaisha Image generation device for monitoring surroundings of vehicle
US20160189350A1 (en) * 2014-12-30 2016-06-30 Texas Instruments Incorporated System and method for remapping of image to correct optical distortions
CN105447853A (zh) * 2015-11-13 2016-03-30 深圳市道通智能航空技术有限公司 飞行装置、飞行控制系统及方法
KR20170077994A (ko) * 2015-12-29 2017-07-07 전자부품연구원 카메라 영상 보정 및 렉티피케이션을 위한 효율적인 데이터 좌표 맵 생성 방법

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR3106402A1 (fr) * 2020-01-16 2021-07-23 Continental Automotive Procédé de calibration d’un support rotatif de dispositif d’imagerie de véhicule automobile
WO2022112221A3 (fr) * 2020-11-27 2022-07-21 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Évaluation d'informations de balayage à l'aide d'informations de position
CN115150546A (zh) * 2021-03-30 2022-10-04 本田技研工业株式会社 信息处理装置和方法
CN113538283A (zh) * 2021-07-22 2021-10-22 浙江赫千电子科技有限公司 一种冗余鱼眼摄像头拍摄图像的畸变矫正方法
CN113538283B (zh) * 2021-07-22 2024-04-30 浙江赫千电子科技有限公司 一种冗余鱼眼摄像头拍摄图像的畸变矫正方法

Similar Documents

Publication Publication Date Title
US20210192764A1 (en) Method and system for detecting and tracking objects using characteristic points
US11604479B2 (en) Methods and system for vision-based landing
US10979615B2 (en) System and method for providing autonomous photography and videography
US10873702B2 (en) Adaptive motion filtering in an unmanned autonomous vehicle
US10447912B2 (en) Systems, methods, and devices for setting camera parameters
US10917561B2 (en) Image processing in an unmanned autonomous vehicle
US10778967B2 (en) Systems and methods for improving performance of a robotic vehicle by managing on-board camera defects
EP3123260B1 (fr) Traitement sélectif de données de capteur
US11288824B2 (en) Processing images to obtain environmental information
US20200117210A1 (en) Auto-Exploration Control of a Robotic Vehicle
WO2018053809A1 (fr) Traitement d'image adaptatif dans un véhicule autonome sans pilote
WO2019019172A1 (fr) Traitement adaptatif d'image dans un véhicule robotisé
US11080890B2 (en) Image sensor initialization in a robotic vehicle
US11244468B2 (en) Image output adjustment in a robotic vehicle
WO2019019139A1 (fr) Étalonnage de capteur d'image dans un véhicule robotique

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17918999

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17918999

Country of ref document: EP

Kind code of ref document: A1