US20080273815A1 - Method and device for retrieving a test block from a blockwise stored reference image - Google Patents

Method and device for retrieving a test block from a blockwise stored reference image Download PDF

Info

Publication number
US20080273815A1
US20080273815A1 US12/148,692 US14869208A US2008273815A1 US 20080273815 A1 US20080273815 A1 US 20080273815A1 US 14869208 A US14869208 A US 14869208A US 2008273815 A1 US2008273815 A1 US 2008273815A1
Authority
US
United States
Prior art keywords
block
corona
test
generating
reference image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/148,692
Other languages
English (en)
Inventor
Malte Borsum
Klaus Gaedke
Marco Georgi
Original Assignee
Thomson Licensing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing filed Critical Thomson Licensing
Assigned to THOMSON LICENSING reassignment THOMSON LICENSING ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BORSUM, MALTE, GAEDKE, KLAUS, GEORGI, MARCO
Publication of US20080273815A1 publication Critical patent/US20080273815A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/55—Motion estimation with spatial constraints, e.g. at image or region borders
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/223—Analysis of motion using block-matching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/43—Hardware specially adapted for motion estimation or compensation
    • H04N19/433—Hardware specially adapted for motion estimation or compensation characterised by techniques for memory access
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/563—Motion estimation with padding, i.e. with filling of non-object values in an arbitrarily shaped picture block or region for estimation purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/28—Indexing scheme for image data processing or generation, in general involving image processing hardware
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the invention relates to retrieving a test block from a blockwise stored reference image.
  • Storing a reference image as blocks of two or more pixels in a blockwise accessible storage device has the advantage of direct access to image blocks at regular positions. This advantage comes along with the drawback that blocks at positions other than the regular positions are not directly accessible.
  • test block retrieval at arbitrary positions is required.
  • a given block of a first image is compared with a variety of test blocks at different positions in a second image. Then, motion of said given block between said first and second image is estimated by help of the position of the given block and the position of the most similar test block.
  • Motion estimation is useful in a number of technical applications varying from robot vision to video compression. Earlier video compression techniques were restricted to comparing said given block with test blocks which are positioned entirely within the reference image. But, MPEG4 also allows for comparing said given block with a test block which intersects the reference image boundary.
  • test block is partially outside the reference image. It comprises pixels which do not belong to the reference image.
  • the reference image is expanded with a corona.
  • the corona surrounds the reference image and its pixels are generated by replicating pixels of the reference image.
  • the corona is stored together with the reference image.
  • the corona requires additional storage capacity.
  • Said method comprises the steps of retrieving a reference block and generating from the reference block a first corona block. Then, the test block is composed from a subsection of the reference block and from a subsection of the first corona block.
  • the method further comprises retrieving a block adjacent to the reference block and generating from the adjacent block an adjacent corona block. Then, the test block is also composed from a subsection of the adjacent block and from a subsection of the adjacent corona block.
  • generating the first corona block comprises replicating a boundary column of the reference block and generating the adjacent corona block comprises replicating a boundary column of the adjacent block.
  • generating the first corona block comprises replicating a boundary row of the reference block and generating the adjacent corona block comprises replicating a boundary row of the adjacent block.
  • a second corona block and a corner corona block are generated from the reference block. Then, the test block is composed also from a subsection of the second corona block and from a subsection of the corner corona block.
  • Said yet another embodiment preferably comprises generating the first corona block by replicating a boundary row of the reference image, generating the second corona block by replicating a boundary column of the reference image, and generating the corner corona block by replicating a corner pixel of the reference block wherein the corner pixel is comprised in said boundary row and in said boundary column.
  • Generating a corona block from a boundary column, a boundary row or a corner pixel gives a good estimation of the corona.
  • Preferably generating any of the corona blocks comprises generating pixels in parallel.
  • the invention is further related to a method which comprises the steps of retrieving a reference block and three adjacent blocks and forming the test block by combining a subsection of the reference block with a subsection of each of the three adjacent blocks.
  • test block retrieved according to the inventive method can be used advantageously if it is compared with a block from another image for motion estimation.
  • the invention is further related to a device for retrieving a test block from a blockwise stored reference image with the features of claim 10 .
  • Said device comprises means for retrieving a block and means for generating from the retrieved block a corona block.
  • the device further comprises means for composing the test block from a subsection of the retrieved block and from a subsection of the generated corona block.
  • the means for generating a corona block are adapted to generate pixels of the corona block in parallel by replicating a column, a row or one pixel of the retrieved block.
  • Said device allows for fast test block retrieval.
  • FIG. 1 depicts an exemplary embodiment of a device for retrieving a test block positioned at a test block position in a blockwise stored reference image
  • FIG. 2 depicts a section of a reference image plus surrounding corona blocks and a test block at a test block position.
  • FIG. 1 depicts a blockwise accessible storage device 10 , a block retriever 20 , a self-completing depository 30 and a composer 40 forming an exemplary embodiment of the inventive device for retrieving a test block from a blockwise stored reference image.
  • the block retriever 20 receives a test position as input. It determines a reference block in the reference image wherein the reference block intersects a test block positioned at the test position 50 . Then, the block retriever 20 retrieves the reference block from the blockwise accessible storage device 10 . If the test position corresponds to a test block which intersects adjacent blocks in the reference image, these adjacent blocks are retrieved as well. The reference block and the adjacent blocks, if there are any, are deposited in the self-completing depository 30 .
  • the self-completing depository 30 is adapted to generate pixels of different types of corona block in parallel. The corona blocks differ in being generated by replicating a row, a column or a corner pixel of a deposited block.
  • test position corresponds to a test block which intersects one or more corona blocks surrounding the reference image
  • this is detected by the self-completing depository 30 by help of the received test position and/or by help of the number of adjacent blocks deposited.
  • the self-completing depository 30 generates the intersected corona blocks.
  • Generated corona blocks are deposited as well in the self-completing depository 30 .
  • the blocks deposited in the self-completing depository 30 are accessed by the composer 40 .
  • the composer 40 takes the test position as input and composes a test block by combining the respective intersections of the deposited blocks.
  • the blockwise accessible storage device 10 allows only for direct access of blocks at predefined periodic positions by help of the exemplary block retrieval device a test block at an arbitrary position can be retrieved from said blockwise accessible storage device 10 .
  • the set of possible positions include positions of blocks which intersect the boundary of the image.
  • FIG. 2 depicts a section of a reference image plus surrounding corona blocks.
  • the corona blocks are not stored permanently but generated on demand and deposited temporarily.
  • the section comprises a boundary column 100 and a boundary row 101 of the reference image wherein the boundary column 100 and the boundary row 101 meet at a corner.
  • a reference block 110 Positioned in said corner is a reference block 110 with a boundary column and a boundary row which coincide partly with the boundary column 100 , respectively the boundary row 101 of the reference image.
  • the corner pixel of the reference image and the reference block 110 coincide, also.
  • the reference block 110 is surrounded by three adjacent blocks 120 which form part of the reference image and five corona blocks.
  • the corona blocks comprise a row corona block 130 , a column corona block 140 , a corner corona block 150 , an adjacent column corona block 160 with a boundary column which coincides partly with the boundary of the reference image and an adjacent row corona block 170 with a boundary row which coincides partly with the boundary of the reference image.
  • the corona blocks 130 , 140 , 150 , 160 , 170 are not kept in the blockwise accessible storage device 10 but generated when needed as described below. It also should be noted, that only the reference block is retrieved from the storage device 10 for sure while retrieval of the adjacent blocks 120 depends on the test position 50 .
  • FIG. 2 further shows a test block 180 at a position where it intersects the reference block 110 , one of the adjacent blocks 120 , the column corona block 140 and the adjacent column corona block 160 .
  • the block retriever 20 determines the number of blocks to retrieve. This may include determining which of the required blocks are already comprised in the self-completing depository 30 and omitting the determined blocks from retrieval.
  • the test position 50 refers to a test block 180 which overlaps the reference image only partly. That is, the test block is positioned partly outside the reference image. Given further, the test block comprises a corner pixel of the reference image. Then, only the reference block 110 is retrieved. This retrieved reference block 110 has a boundary column which is part of the boundary column 100 of the reference image and a boundary row which is part of the boundary row 101 of the reference image.
  • the test position 50 refers to a test block 180 which overlaps the reference image only partly but does not comprise any corner pixel of the reference image.
  • the reference block 110 and one block 120 adjacent to the reference block 110 are retrieved.
  • the retrieved reference block 110 and the retrieved adjacent block 120 have a boundary column, each, which is part of the boundary column 100 of the reference image.
  • the retrieved reference block 110 and the retrieved adjacent block 120 have a boundary row, each, which is part of the boundary row 101 of the reference image.
  • test position 50 refers to a test block 180 which overlaps the reference image completely. That is the test block is positioned completely inside the reference image. Then, a reference block 110 and three adjacent blocks 120 are retrieved. The reference block 110 and the three adjacent blocks 120 form a square of two times two blocks which comprises the test block the test position 50 refers to.
  • the retrieved blocks 110 , 120 are deposited in the self-completing depository 30 .
  • the self-completing depository 30 Dependent on the number of retrieved blocks 110 , 120 and/or the test position 50 the self-completing depository 30 generates corona blocks 130 , 140 , 150 , 160 , 170 such that in total four blocks are present.
  • the self-completing depository 30 completes itself either upon reception of the test position 50 or upon detection of the number and positions of the retrieved blocks 110 , 120 deposited by the block retriever 20 . If the self-completing depository 30 is adapted to detect the number and positions of the retrieved and deposited blocks 110 , 120 , the test position 50 is not necessarily provided as input to the self-completing depository 30 .
  • test position 50 refers to a test block 180 which overlaps the reference image only partly and comprises a corner pixel of the reference image or if only the reference block 110 is deposited in the self-completing depository 30 , three corona blocks 130 , 140 , 150 are generated:
  • test position 50 refers to a test block 180 which overlaps the reference image only partly but does not comprise any corner pixel of the reference image or if the reference block 110 and exactly one adjacent block 120 is deposited in the self-completing depository 30 , two corona blocks 130 , 170 or 140 , 160 are generated:
  • each, which is part of the boundary column 100 of the reference image two corona blocks 140 , 160 are generated.
  • a column corona block 140 is generated, which comprises columns which equal said boundary column of the reference block 110 .
  • an adjacent column corona block 160 is generated, which comprises columns which equal said boundary column of the adjacent block 120 .
  • each, which is part of the boundary row 101 of the reference image and two row corona blocks 130 , 170 are generated.
  • a row corona block 130 is generated, which comprises rows which equal said boundary row of the reference block 110 .
  • an adjacent row corona block 170 is generated, which comprises rows which equal said boundary row of the adjacent block 120 .
  • test position 50 refers to a test block 180 which overlaps the reference image completely or if the reference block 110 and three adjacent blocks 120 is deposited in the self-completing depository 30 , no corona blocks are generated.
  • generation of a corona block may be realised by zero-padding, i.e. generation of a corona block with pre-defined values, or by mirroring, i.e. replication of all rows respectively columns of the block the corona block is generated from in reverse or opposite order.
  • the self-completing depository 30 is adapted to generate some or all pixels of a corona block in parallel by replicating pixels of a column, of a row or a single pixel of the block from which the corona block is generated.
  • the composer 40 forms the desired test block according to the provided test position 50 . This is done either by selecting one of the blocks in the self-completing depository 30 or by combining subsections of two or more of said blocks in the self-completing depository 30 . Then, the composer 40 outputs the formed block 60 .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
US12/148,692 2007-05-04 2008-04-22 Method and device for retrieving a test block from a blockwise stored reference image Abandoned US20080273815A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP07107509.7 2007-05-04
EP07107509A EP1988502A1 (fr) 2007-05-04 2007-05-04 Procédé et dispositif de récupération d'un bloc de test dans une image de référence stockée par blocs

Publications (1)

Publication Number Publication Date
US20080273815A1 true US20080273815A1 (en) 2008-11-06

Family

ID=38268934

Family Applications (1)

Application Number Title Priority Date Filing Date
US12/148,692 Abandoned US20080273815A1 (en) 2007-05-04 2008-04-22 Method and device for retrieving a test block from a blockwise stored reference image

Country Status (4)

Country Link
US (1) US20080273815A1 (fr)
EP (2) EP1988502A1 (fr)
JP (1) JP2008278490A (fr)
CN (1) CN101299273A (fr)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010258741A (ja) * 2009-04-24 2010-11-11 Sony Corp 画像処理装置および方法、並びにプログラム
EP2346254A1 (fr) * 2009-11-26 2011-07-20 Research In Motion Limited Décodeur vidéo et procédé pour la compensation du mouvement pour les pixels hors limite
JP5908605B2 (ja) 2011-12-30 2016-04-26 インテル コーポレイション 動き推定を用いたオブジェクト検出
WO2015059880A1 (fr) * 2013-10-22 2015-04-30 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Procede de compensation de mouvement, procede de codage d'image, procede de decodage d'image, dispositif de codage d'image et dispositif de decodage d'image
JP7171770B2 (ja) 2018-06-28 2022-11-15 ホアウェイ・テクノロジーズ・カンパニー・リミテッド 動きベクトル精密化および動き補償のためのメモリアクセスウィンドウおよびパディング

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6298144B1 (en) * 1998-05-20 2001-10-02 The United States Of America As Represented By The National Security Agency Device for and method of detecting motion in an image
US6317460B1 (en) * 1998-05-12 2001-11-13 Sarnoff Corporation Motion vector generation by temporal interpolation
US6483538B2 (en) * 1998-11-05 2002-11-19 Tektronix, Inc. High precision sub-pixel spatial alignment of digital images
US20040252762A1 (en) * 2003-06-16 2004-12-16 Pai R. Lakshmikanth System, method, and apparatus for reducing memory and bandwidth requirements in decoder system
US20060002474A1 (en) * 2004-06-26 2006-01-05 Oscar Chi-Lim Au Efficient multi-block motion estimation for video compression
US7412080B2 (en) * 2002-07-14 2008-08-12 Apple Inc Adaptive motion estimation
US7693219B2 (en) * 2006-01-04 2010-04-06 Freescale Semiconductor, Inc. System and method for fast motion estimation

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6339656B1 (en) * 1997-12-25 2002-01-15 Matsushita Electric Industrial Co., Ltd. Moving picture encoding decoding processing apparatus
US7440500B2 (en) * 2003-07-15 2008-10-21 Lsi Logic Corporation Supporting motion vectors outside picture boundaries in motion estimation process
EP1503597A3 (fr) * 2003-07-28 2007-01-03 Matsushita Electric Industrial Co., Ltd. Dispositif de décodage vidéo
KR100621137B1 (ko) * 2004-02-27 2006-09-13 세이코 엡슨 가부시키가이샤 동화상 부호화 장치 및 동화상 처리장치

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6317460B1 (en) * 1998-05-12 2001-11-13 Sarnoff Corporation Motion vector generation by temporal interpolation
US6298144B1 (en) * 1998-05-20 2001-10-02 The United States Of America As Represented By The National Security Agency Device for and method of detecting motion in an image
US6483538B2 (en) * 1998-11-05 2002-11-19 Tektronix, Inc. High precision sub-pixel spatial alignment of digital images
US7412080B2 (en) * 2002-07-14 2008-08-12 Apple Inc Adaptive motion estimation
US20040252762A1 (en) * 2003-06-16 2004-12-16 Pai R. Lakshmikanth System, method, and apparatus for reducing memory and bandwidth requirements in decoder system
US20060002474A1 (en) * 2004-06-26 2006-01-05 Oscar Chi-Lim Au Efficient multi-block motion estimation for video compression
US7693219B2 (en) * 2006-01-04 2010-04-06 Freescale Semiconductor, Inc. System and method for fast motion estimation

Also Published As

Publication number Publication date
CN101299273A (zh) 2008-11-05
EP1988502A1 (fr) 2008-11-05
EP1988503A1 (fr) 2008-11-05
JP2008278490A (ja) 2008-11-13

Similar Documents

Publication Publication Date Title
US20080273815A1 (en) Method and device for retrieving a test block from a blockwise stored reference image
CN101059949B (zh) 海量图像数据的压缩、存储和显示方法
TWI441092B (zh) 具漫畫圖像分割功能的電子裝置及方法
US8249388B2 (en) Identifying portions of an image for cropping
KR100804898B1 (ko) 이미지 회전을 위한 방법 및 장치
CN102411620B (zh) 浏览相关图像搜索结果集合
CN106203277B (zh) 基于sift特征聚类的固定镜头实时监控视频特征提取方法
US9131227B2 (en) Computing device with video analyzing function and video analyzing method
US10242506B2 (en) Visualization method and system, and integrated data file generating method and apparatus for 4D data
CA2742586C (fr) Systeme, structure de donnees et methode d'extraction simultanee de donnees multidimensionnelles sans conflit de donnees
CN103167036A (zh) 基于分布式多级缓存系统的栅格数据存取方法
US20070009181A1 (en) Efficient and high speed 2D data transpose engine for SOC application
US20120170866A1 (en) Electronic device and image comparison method thereof
US20160148649A1 (en) Method and apparatus for visualizing information of a digital video stream
US7505048B2 (en) Estimation of overlap of polygons
US20080095519A1 (en) Image playback apparatus providing smart search for motion and method of using the same
US9679205B2 (en) Method and system for displaying stereo image by cascade structure and analyzing target in image
US20170270996A1 (en) Semiconductor memory deivce and accessing method thereof
US20100066900A1 (en) Image processing method
JP2004149024A5 (fr)
KR101104883B1 (ko) 당구 동영상 검색 시스템 및 방법
CN109598182A (zh) 指纹比对方法和装置
JP2006217510A5 (fr)
US10277912B2 (en) Methods and apparatus for storing data related to video decoding
TWI746987B (zh) 卷積神經網路系統

Legal Events

Date Code Title Description
AS Assignment

Owner name: THOMSON LICENSING, FRANCE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BORSUM, MALTE;GAEDKE, KLAUS;GEORGI, MARCO;REEL/FRAME:020895/0733

Effective date: 20080212

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION