WO2015083857A1 - Appareil pour un matériel de caractéristique robuste accélérée (surf) et procédé pour gérer une image intégrale - Google Patents

Appareil pour un matériel de caractéristique robuste accélérée (surf) et procédé pour gérer une image intégrale Download PDF

Info

Publication number
WO2015083857A1
WO2015083857A1 PCT/KR2013/011270 KR2013011270W WO2015083857A1 WO 2015083857 A1 WO2015083857 A1 WO 2015083857A1 KR 2013011270 W KR2013011270 W KR 2013011270W WO 2015083857 A1 WO2015083857 A1 WO 2015083857A1
Authority
WO
WIPO (PCT)
Prior art keywords
integrated image
image
sub
memory
integrated
Prior art date
Application number
PCT/KR2013/011270
Other languages
English (en)
Korean (ko)
Inventor
최병호
장성준
이상설
Original Assignee
전자부품연구원
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 전자부품연구원 filed Critical 전자부품연구원
Publication of WO2015083857A1 publication Critical patent/WO2015083857A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to a speeded up robust feature (SURF) hardware device and an integrated image memory management method.
  • SURF speeded up robust feature
  • the SURF (Speeded Up Robust Feature) algorithm is one of representative algorithms for extracting feature points of objects.
  • the SURF algorithm performs object and scene recognition by regenerating an integrated image based on the input black and white image.
  • the integrated image memory is frequently accessed because it calculates the Hessian by reading the integral image value pixel by pixel.
  • performance deterioration is severe in terms of speed, so most implementations configure it as dual port-based memory internally.
  • An object of the present invention is to provide a Speed Up Robust Feature (SURF) hardware device and an integrated image memory management method operating at low power by minimizing integrated image memory power consumption.
  • SURF Speed Up Robust Feature
  • the SURF hardware device is a Speed Up Robust Feature (SURF) hardware device for calculating the feature points and descriptors of the image
  • the integrated image generation unit for generating an integrated image based on the input black and white image
  • the integral Multi-integrated image memory including a plurality of sub-integrated images
  • the clock and power of the plurality of sub-integrated image memories are sequentially managed based on the result of real-time monitoring of the pixel values of the integrated image.
  • a memory management unit a Speed Up Robust Feature
  • the plurality of sub-integrated image memories may be in a bit group unit.
  • the memory manager includes
  • the pixel value of the integrated image is stored, and the pixel value of the integrated image output from the integrated image generator is real-time monitored to satisfy a predetermined threshold value.
  • the power may be turned on by applying a clock to a second sub-integration image memory of the first sub-integration image memory.
  • an integrated image memory management method is a method of managing an integrated image memory by a SURF (Speeded Up Robust Feature) hardware device that calculates a feature point and a descriptor of an image. Generating and sequentially managing clocks and powers of the plurality of sub-integrated image memories based on a result of real-time monitoring of pixel values of the integrated image.
  • SURF Speeded Up Robust Feature
  • the plurality of sub-integrated image memories may include a multi-integrated image memory in units of bit groups.
  • the power consumption of the integrated image memory is minimized to operate the SURF hardware device at low power.
  • 1 is an exemplary diagram of an integral value for an arbitrary image applied to an embodiment of the present invention.
  • FIG. 2 illustrates a typical single integrated image memory structure.
  • FIG. 3 illustrates a bit group multi-integrated image memory structure according to an embodiment of the present invention.
  • FIG. 4 is a configuration diagram of a speeded up robust feature (SURF) hardware device according to an embodiment of the present invention.
  • SURF speeded up robust feature
  • FIG. 5 is a flowchart illustrating an integrated image memory management method according to an exemplary embodiment of the present invention.
  • FIG. 1 is a diagram illustrating an integrated value of an arbitrary image applied to an embodiment of the present invention
  • FIG. 2 illustrates a general single integrated image memory structure
  • FIG. 3 illustrates a bit group multi-integrated image memory structure according to an embodiment of the present invention
  • 4 is a configuration diagram of a speeded up robust feature (SURF) hardware device according to an embodiment of the present invention.
  • SURF speeded up robust feature
  • an image received by a SURF hardware device is an 8-bit gray-scale image.
  • the value of one pixel is represented by 8 bits, that is, it is between 0 and 255. At this time, the maximum value of one pixel is 255 when white, and the minimum value is 0 when black. In other words, the more colors close to black are in the frame, the smaller the total sum (integral value) becomes.
  • the integral value of the input image of FIG. 1A is '0xd1b2b0', and the integral value of the input image of FIG. 1B is '0x6529ab'.
  • the image of (B) has more colors closer to black than the image of (A). Accordingly, it can be seen that the integral value of FIG. 1B having more black pixels is smaller than that of FIG. 1A.
  • the hardware design always considers the worst case, i.e. for a 640x480 image, when the value of every pixel is 255, the final pixel value of the integrated image is 78,336,000, which is allocated 27 bits. However, in the general situation (picture), most of them need less than 27 bits.
  • an embodiment of the present invention manages the bit group multi-integrated image memory structure as shown in FIG. 3.
  • an image signal processor (ISP) 1 and a SURF feature extraction IP 3 are connected to a central processing processor (CPU) 7 and a high speed interface via a bus 5.
  • ISP image signal processor
  • CPU central processing processor
  • the central processing processor (CPU) 33 drives an operating system (OS) and firmware (Firmware) for operating the internal register setting and SURF in the SURF hardware device.
  • OS operating system
  • Firmware firmware for operating the internal register setting and SURF in the SURF hardware device.
  • High Speed Interfaces 35 are controller modules for providing high speed external interfaces such as USB.
  • Low Speed Interfaces 37 are controller modules for providing low speed external interfaces such as Universal Asynchronous Receiver / Transmitter (UART) and Serial Peripheral Interface (SPI).
  • UART Universal Asynchronous Receiver / Transmitter
  • SPI Serial Peripheral Interface
  • the external memory 13 includes an external memory PHY 15 in which a black and white image is stored and an external memory controller 17 that controls the operation of the external memory pie.
  • the image signal processing unit (ISP) 1 generates an RGB (Red-Green-Blue) / Y (luminance) U (difference between the luminance signal and the blue component) V (difference between the luminance signal and the red component) image.
  • the image signal processor (ISP) 1 includes a gray-scale image converter (Gray-Scale Image Converter) 19 and a write direct memory access (W-DMA) module 1 (21).
  • ISP image signal processor
  • W-DMA write direct memory access
  • the black and white image converter 19 converts the original image into a black and white image.
  • the converted black and white image is stored in the external memory 13 through the W-DMA module 1 (21).
  • the integrated image memory 23 is a dedicated memory for extracting feature points, and stores the integrated image generated by the integrated image generator 27.
  • the integrated image memory 23 is a memory shared by a plurality of Hessian calculators 31.
  • the memory manager 25 monitors the pixel integrated value in units of frames in real time and manages the clock and power of the integrated image memory 23 based on the real time monitoring.
  • the integrated image memory 23 is implemented as a bit group multi-integrated image memory structure as shown in FIG. 3. That is, the memory manager 25 manages a single integrated image memory as shown in FIG. 2 in a structure in which three integrated image memories are layered in bit groups (8-bit units) as shown in FIG. 3.
  • the memory manager 25 initially applies a clock only to the first integrated image memory 23-1 and turns on the power to store the integrated value.
  • a clock is applied to the next integrated image memory 23-3 and the power is turned on.
  • the pixel integrated value is monitored in real time on a frame basis, and when the pixel integrated value satisfies a predetermined threshold value, the clock is applied to the next integrated image memory 23-5 and the power is turned on.
  • the memory manager 25 monitors the integral value generated by the integrated image generator 23 in real time, and based on the threshold value, each sub-integral image memory (Sub-Integral Image MEM0, Sub-Integral Image MEM1,...) Manages the clock and power of Sub-Integral Image MEMn sequentially (Integral Image Monitor & Integral Image Memory Clock / Power Manager). At this time, the turn-on of the clock and the power supply and the settling time after the change are set to a margin.
  • the integrated image generator 27 reads a black and white image from the external memory 13 using a read direct memory access (R-DMA) module 29 to generate an integrated image.
  • R-DMA read direct memory access
  • the integrated image generator 27 regenerates and uses an integrated image obtained by integrating a monochrome image, which is an input image, for a quick box filtering operation.
  • a monochrome image which is an input image
  • the maximum value of one pixel can be 255, and when all pixels are 255, the maximum value of the integrated image is 78,336,000, which requires a maximum of 27 bits.
  • the plurality of Hessian calculation units 31 patch the integrated image pixel values from the integrated image memory 23 through the data fatching unit 33 according to the box filter pattern to perform a box filter operation. Perform.
  • the Hessian values calculated by the plurality of Hessian calculation units 31 are stored in each Hessian memory 35.
  • the feature point extractor 37 extracts the feature points based on the result of the box filtering operation stored in each of the Hessian memories 35. At this time, the pixel having the largest Hessian value is extracted as the feature point, compared to the result of the box filtering operation having another size.
  • the main controller 39 controls the added block, that is, the integrated image memory 23 and the memory manager 25, in addition to the function of controlling the entire block.
  • the integrated image memory 23, the memory manager 25, the integrated image generator 27, the R-DMA module 29, the Hessian calculator 31, the data patching unit 33, the Hessian memory ( 35), the SURF feature extraction IP including the feature extractor 37, the main controller 39, and the W-DMA module 2 41 may be implemented as a system-on-chip.
  • the integrated image memory management method is as follows.
  • FIG. 5 is a flowchart illustrating an integrated image memory management method according to an exemplary embodiment of the present invention.
  • the memory manager 25 monitors the pixel integrated value generated by the integrated image generator 23 in real time (S101).
  • the memory manager 25 determines whether the pixel integrated value satisfies a predefined threshold (S103).
  • the threshold is determined by how many sub-integrated image memories the integral image memory 23 is divided into.
  • the integrated image memory having a data width of 27 bits may be divided into two groups, a 7-bit memory (upper bit) and a 20-bit memory (lower bit). At this time, up to a value of 2 20 , that is, 0xFFFFF, may be stored in the 20- bit memory for the lower bit. If the integral is greater than that, you are turning on 7-bit memory for the higher bits.
  • the threshold may include a turn on margin. In other words, the time to stabilize by turning on a clock or power can be turned on earlier by margining at a threshold.
  • step S101 is restarted.
  • the power is turned on by applying the clock of the next sub-integrated image memory (S105).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un appareil pour un matériel de caractéristique robuste accélérée (SURF) et un procédé pour gérer des images intégrales. L'appareil pour un matériel SURF est un appareil pour calculer des points-clés et des descripteurs pour une image et comprend : une unité de génération d'image intégrale pour générer une image intégrale sur la base d'une image en noir et blanc entrée ; de multiples mémoires d'image intégrale, comprenant une pluralité de sous-images intégrales, pour mémoriser des valeurs de pixel des images intégrales ; et une unité de gestion de mémoire pour gérer l'horloge et la source d'alimentation de la pluralité de mémoires de sous-images intégrales séquentiellement sur la base de résultats de surveillance en temps réel des valeurs de pixel de l'image intégrale.
PCT/KR2013/011270 2013-12-05 2013-12-06 Appareil pour un matériel de caractéristique robuste accélérée (surf) et procédé pour gérer une image intégrale WO2015083857A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020130150958A KR101531038B1 (ko) 2013-12-05 2013-12-05 Ｓｕｒｆ 하드웨어 장치 및 적분 이미지 메모리 관리 방법
KR10-2013-0150958 2013-12-05

Publications (1)

Publication Number Publication Date
WO2015083857A1 true WO2015083857A1 (fr) 2015-06-11

Family

ID=53273598

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2013/011270 WO2015083857A1 (fr) 2013-12-05 2013-12-06 Appareil pour un matériel de caractéristique robuste accélérée (surf) et procédé pour gérer une image intégrale

Country Status (2)

Country Link
KR (1) KR101531038B1 (fr)
WO (1) WO2015083857A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016208806A1 (fr) * 2015-06-26 2016-12-29 광운대학교 산학협력단 Dispositif de génération d'image intégrale utilisant une structure de bloc et procédé associé
CN108830831A (zh) * 2018-05-11 2018-11-16 中南大学 一种基于改进surf匹配的锌浮选泡沫自然速度特征提取方法
CN110570444A (zh) * 2019-09-06 2019-12-13 合肥工业大学 一种基于Box Filter算法的阈值计算方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100048791A (ko) * 2008-10-31 2010-05-11 삼성전자주식회사 영상처리 장치 및 방법
US20130272575A1 (en) * 2011-11-01 2013-10-17 Intel Corporation Object detection using extended surf features

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6510525B1 (en) * 1999-04-26 2003-01-21 Mediaq, Inc. Method and apparatus to power up an integrated device from a low power state
JP2003068076A (ja) * 2001-08-27 2003-03-07 Elpida Memory Inc 半導体記憶装置の電力制御方法及び半導体記憶装置

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100048791A (ko) * 2008-10-31 2010-05-11 삼성전자주식회사 영상처리 장치 및 방법
US20130272575A1 (en) * 2011-11-01 2013-10-17 Intel Corporation Object detection using extended surf features

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ERIC LI ET AL.: "SURF Cascade Face Detect ion Acceleration on Sandy Bridge Processor", COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 2012 IEEE COMPUTER SOCIETY CONFERENCE, 16 June 2012 (2012-06-16), PROVIDENCE, RHODE ISLAND, pages 41 - 47 *
KIM, SOO JIN ET AL.: "Design of High-performance Pedestrian and Vehicle Detect ion Circuit Using Haar-like Features", TENCON 2012 - 2012 IEEE REGION 10 CONFERENCE, 19 November 2012 (2012-11-19), CEBU *
NICO CORNEL IS ET AL.: "Fast Scale Invariant Feature Detect ion and Matching on Programmable Graphics Hardware", COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW), 2008, IEEE COMPUTER SOCIETY CONFERENCE, 23 June 2008 (2008-06-23), ANCHORAGE, AI KSKA *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016208806A1 (fr) * 2015-06-26 2016-12-29 광운대학교 산학협력단 Dispositif de génération d'image intégrale utilisant une structure de bloc et procédé associé
CN108830831A (zh) * 2018-05-11 2018-11-16 中南大学 一种基于改进surf匹配的锌浮选泡沫自然速度特征提取方法
CN110570444A (zh) * 2019-09-06 2019-12-13 合肥工业大学 一种基于Box Filter算法的阈值计算方法
CN110570444B (zh) * 2019-09-06 2023-03-21 合肥工业大学 一种基于Box Filter算法的阈值计算方法

Also Published As

Publication number Publication date
KR20150065532A (ko) 2015-06-15
KR101531038B1 (ko) 2015-06-23

Similar Documents

Publication Publication Date Title
US10212339B2 (en) Image generation method based on dual camera module and dual camera apparatus
CN110602488B (zh) 一种日夜型摄像装置切换异常检测方法、装置及摄像装置
WO2021133001A1 (fr) Procédé et dispositif d'inférence d'image sémantique
WO2014069822A1 (fr) Appareil et procédé de reconnaissance de visage
WO2015083857A1 (fr) Appareil pour un matériel de caractéristique robuste accélérée (surf) et procédé pour gérer une image intégrale
CN102111640B (zh) 一种图像有效区域检测方法及系统
US10621464B1 (en) Block based non-maximum suppression
WO2020197268A1 (fr) Procédé et dispositif électronique de traitement d'images faciales
WO2014208817A1 (fr) Appareil d'égalisation d'histogramme utilisant une compression d'histogramme
WO2024007948A1 (fr) Procédé et appareil de traitement d'image stroboscopique, dispositif électronique et support de stockage lisible
EP2953340B1 (fr) Dispositif de traitement d'image, procédé de traitement d'image, programme, et support d'enregistrement
WO2020067615A1 (fr) Procédé de commande d'un dispositif d'anonymisation vidéo permettant d'améliorer les performances d'anonymisation, et dispositif associé
CN113132639A (zh) 一种图像处理方法、装置、电子设备及存储介质
CN109934783B (zh) 图像处理方法、装置、计算机设备和存储介质
WO2017086522A1 (fr) Procédé de synthèse d'image d'incrustation couleur sans écran d'arrière-plan
KR20160053632A (ko) 로컬 톤 맵핑 회로와 이를 포함하는 모바일 컴퓨팅 장치
CN113391779B (zh) 类纸屏幕的参数调节方法、装置及设备
CN104917969A (zh) 一种图像处理的方法及移动终端
WO2015108218A1 (fr) Dispositif et procédé de traitement graphique
US20150046962A1 (en) Method of controlling physically separated network computers in one monitor and security system using the same.
CN106776393A (zh) 一种无中断的串口数据接收方法和装置
TWI482099B (zh) 影像處理紀錄系統及其影像處理紀錄方法
CN114973344A (zh) 人脸检测方法、装置、终端设备和计算机可读存储介质
JP6798690B2 (ja) 画像処理装置、画像処理方法、及びプログラム
TWI754496B (zh) 圖像融合方法、裝置、電子設備及存儲介質

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13898844

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13898844

Country of ref document: EP

Kind code of ref document: A1