WO2012081367A1 - Dispositif de conversion d'image - Google Patents

Dispositif de conversion d'image Download PDF

Info

Publication number
WO2012081367A1
WO2012081367A1 PCT/JP2011/076998 JP2011076998W WO2012081367A1 WO 2012081367 A1 WO2012081367 A1 WO 2012081367A1 JP 2011076998 W JP2011076998 W JP 2011076998W WO 2012081367 A1 WO2012081367 A1 WO 2012081367A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
pixel
coordinate
reference coordinates
region
Prior art date
Application number
PCT/JP2011/076998
Other languages
English (en)
Japanese (ja)
Inventor
祐司 國米
秀貴 佐々木
田中　慎一
宏 酒井
Original Assignee
株式会社ニコン
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社ニコン filed Critical 株式会社ニコン
Publication of WO2012081367A1 publication Critical patent/WO2012081367A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation

Definitions

  • the present invention relates to an image conversion apparatus.
  • Patent Document 1 In an image conversion apparatus that converts an equidistant projection image photographed with a fisheye lens into a central projection image without distortion, an image conversion method as shown in Patent Document 1 is used.
  • image conversion apparatus disclosed in Patent Document 1 pixel values of pixels around a position on the equidistant projection image corresponding to the pixel of the central projection image are weighted according to the position and added. Thus, high-precision interpolation is performed, and the contour of the subject in the converted central projection image is smoothed.
  • each coordinate (output coordinate) of the central projection image, a plurality of coordinates of the equidistant projection image for obtaining the pixel value at this output coordinate, and each weight are stored in association with each other. There is a problem that the storage area becomes large because it is necessary.
  • the present invention has been made in view of such problems, and an object of the present invention is to provide an image conversion apparatus capable of performing high-precision interpolation with a small storage capacity in image conversion processing.
  • an image conversion apparatus is an image conversion apparatus that converts a first image into a second image, an image storage unit that stores the first image, and a second Information on the position of the first image for calculating the pixel value of the pixel of the second image for each pixel of the image of the first image, the reference coordinate indicating the pixel of the first image and the reference coordinate from the reference coordinate
  • a table storage unit that stores a coordinate conversion table composed of relative positions, a pixel value of a pixel indicated by the reference coordinates of the first image, and a pixel value of a pixel of the first image located in the vicinity of the reference coordinates
  • An image processing unit that performs image conversion from the first image to the second image by calculating the pixel value of the pixel of the second image by calculating with an arithmetic expression corresponding to the relative position. It is characterized by.
  • the relative position of the coordinate conversion table is obtained by dividing the first image into a plurality of regions including the pixel for each pixel of the first image, and further dividing the region into a plurality of sub-regions. It is preferable to indicate a sub-region in the region including the reference coordinates when divided into two.
  • the pixel of the first image located in the vicinity of the reference coordinates for calculating the pixel value of the second image by the image processing unit is a pixel adjacent to the pixel of the reference coordinates. Preferably there is.
  • the reference coordinates of the coordinate conversion table are stored as a difference from the adjacent reference coordinates in the coordinate conversion table.
  • the first image is an equidistant projection image and the second image is a center projection image.
  • the image conversion apparatus is an image conversion apparatus that converts a first image, which is an image of a predetermined projection method, into a projection method different from the first image and outputs it as a second image.
  • Information on a position in the first image corresponding to the pixel of the second image when the projection method is converted, and the reference coordinates indicating the pixel of the first image in the vicinity of the position and the position A table storage unit for storing a set of relative positions from the reference coordinates as a coordinate conversion table for the pixels of the second image, and for each pixel of the second image, the pixel of the second image of the coordinate conversion table
  • the pixel value of the pixel indicated by the reference coordinate of the first image and the pixel value of the pixel adjacent to the pixel of the reference coordinate are calculated using an arithmetic expression corresponding to the relative position.
  • Compute pixel of second image By calculating the pixel value, and having an image processing unit that performs image conversion from the first image to the second image.
  • the relative position of the coordinate conversion table is obtained by dividing the first image into a plurality of regions including the pixel for each pixel of the first image, and further dividing the region into a plurality of sub-regions. It is preferable to indicate a sub-region in the region including the reference coordinates when divided into two.
  • the pixel of the first image located in the vicinity of the reference coordinates for calculating the pixel value of the second image by the image processing unit is a pixel adjacent to the pixel of the reference coordinates. Preferably there is.
  • the reference coordinates of the coordinate conversion table are stored as a difference from the adjacent reference coordinates in the coordinate conversion table.
  • the first image is an equidistant projection image and the second image is a center projection image.
  • the capacity of the coordinate conversion table stored in the table storage unit is reduced, so that high-precision interpolation can be performed with a small storage capacity in the image conversion process. .
  • the image conversion apparatus 10 includes a fisheye lens, and includes an imaging optical system 1 that forms an image of a subject, an imaging element 2 that includes a CCD, a CMOS, and the like, and that detects an image formed by the imaging optical system 1; An image processing unit 3 that converts a first image detected by the image sensor 2 (hereinafter described as an “equal distance projection image”) into a second image (hereinafter described as a “center projection image”); An image storage unit 4 that temporarily stores the equidistant projection image detected by the image sensor 2 and captured by the image processing unit 3, and information (coordinate conversion) used when changing from the equidistant projection image to the central projection image A table storage unit 5 for storing (table).
  • the image processing unit 3 includes a CPU, and may be configured to execute the above-described image conversion by executing a program having the image conversion method according to the present embodiment, or may be configured as an ASIC or DSP,
  • the image conversion method according to this embodiment may be implemented as a logic circuit.
  • the image storage unit 4 is composed of a volatile memory such as SRAM, and the table storage unit 5 is composed of a nonvolatile memory such as flash memory. Further, the central projection image converted by the image processing unit 3 is passed to and used by a processing unit (not shown).
  • FIG. 2 shows one image, and it is assumed that the width is W pixels (pixels) and the vertical is H pixels (pixels).
  • data whose coordinates are represented by (x, y) are set as one-dimensional data and managed as (y ⁇ W + x) -th data.
  • Such a data structure is, for example, an image obtained by sequentially extracting values (pixel values) detected by each pixel of the image sensor 2 in the horizontal direction from the upper left to the lower right and arranging them one-dimensionally. It is.
  • This image has a pixel value (luminance value) at coordinates (x, y) as P (x, As y), this value is stored one-dimensionally in the order described above for the number of pixels of the image sensor 2.
  • the coordinates on the equidistant projection image corresponding to each pixel of the central projection image are as follows. , It does not always coincide with the position of the pixel. For this reason, the pixel value (luminance value) at a specific position of the equidistant projection image needs to be obtained by interpolation using the arithmetic expression from the pixel values acquired at the surrounding pixels. This interpolation method will be described with reference to FIG.
  • each of nine pixels centered on a pixel at a coordinate (x, y) in the equidistant projection image is surrounded by a rectangular area Z.
  • each pixel is located at the center of the region Z, and each region Z is further divided into nine sub-regions Z1 to Z9.
  • the pixel value (luminance value) of each pixel is represented by P (x, y)
  • the interpolated pixel values (luminance values) Pz1 to Pz9 in the sub-regions Z1 to Z9 are expressed by the following equation (1). It is obtained by (9).
  • Pz1 (P (x-1, y-1) + P (x, y-1) + P (x-1, y) + P (x, y)) / 4 (1)
  • Pz2 (P (x, y-1) + P (x, y)) / 2 (2)
  • Pz3 (P (x, y-1) + P (x + 1, y-1) + P (x, y) + P (x + 1, y)) / 4
  • Pz4 (P (x-1, y) + P (x, y)) / 2 (4)
  • Pz5 P (x, y) (5)
  • Pz6 (P (x, y) + P (x + 1, y)) / 2 (6)
  • Pz7 (P (x-1, y) + P (x, y) + P (x-1, y + 1) + P (x, y + 1)) / 4 (7)
  • Pz8 (P (x, y) + P (x, y +
  • the reference coordinates (x, y) that are the coordinates of the pixel in the center of the region, and the subregions Z1 to Z9
  • the pixel value of the pixel on the reference coordinate of the equidistant projection image and the pixel value of the pixel in the vicinity of the reference coordinate (adjacent to the reference coordinate in this embodiment)
  • the pixel value (luminance value) of the output coordinates in the central projection image can be obtained by using the above-described arithmetic expressions (1) to (9).
  • FIG. 4 shows the data structure of the coordinate conversion table 50 stored in the table storage unit 5.
  • the coordinate conversion table 50 includes reference coordinates (coordinates of pixels of the original equidistant projection image) 51 for calculating the output coordinates for each output coordinate (coordinates of the central projected image after conversion). And a sub-region (identifier) 52 in the region Z including the reference coordinates is stored. Since the sizes of the equidistant projection image and the central projection image (the number of pixels in the horizontal direction ⁇ the number of pixels in the vertical direction) are known, the output coordinates are stored in the coordinate conversion table 50 as long as the storage order is determined. There is no need to keep it.
  • the image processing unit 3 reads one image (equal distance projection image) from the image sensor 2 and stores it in the image storage unit 4 (step S100).
  • the values of the reference coordinates 51 and the sub area 52 are read out from the coordinate conversion table 50 stored in the table storage unit 5 in the stored order (step S110). At this time, one record may be read out or a plurality of records may be read out collectively.
  • the pixel value of the output coordinate is obtained for each read reference coordinate and sub-region using the above-described arithmetic expressions (1) to (9) (step S120). Specifically, the pixel values of nine coordinates including the reference coordinates are read from the image storage unit 4 based on the reference coordinates, and further, the sub area is selected from the above-described arithmetic expressions (1) to (9). Is selected, and the pixel value of the sub-region is calculated based on the calculation formula. Note that an arithmetic expression may be determined from the sub-region, and a pixel value of a pixel necessary for the expression may be extracted from the image storage unit 4 and the calculation may be performed.
  • the calculated result is output from the image processing unit 3 to the outside as a pixel value at the output coordinates of the central projection image (step S130).
  • the conversion of one equidistant projection image extracted from the image sensor 2 into the central projection image is terminated. Note that, when images are continuously captured from the image sensor 2 and processed as moving images, it can be realized by repeatedly executing the above steps S100 to S140.
  • image conversion is performed by processing images output from the respective imaging elements 2 according to the above procedure. It is possible.
  • the records of the coordinate conversion table 50 are arranged in the order of the predetermined output coordinates, the reference coordinates and sub-regions are extracted in that order, and the pixel values of the output coordinates are calculated and output. Thus, a desired central projection image can be obtained.
  • the table stored in advance in the image conversion apparatus 10 is only the coordinate conversion table 50 stored in the table storage unit 5 as described above. Since the coordinate conversion table 50 stores only a set of reference coordinates and sub-region information for each output coordinate, it can have a very small data capacity. Therefore, an ASIC or DSP having a small storage capacity can be used for the image processing unit 3 (and the table storage unit 5) of the image conversion apparatus 10 according to the present embodiment.
  • the method of obtaining the interpolation value by dividing each pixel of the equidistant projection image acquired by the image sensor 2 into the rectangular region Z including the pixel as the center has been described.
  • the arrangement of the region Z is not limited to the above description.
  • the region A is determined so that each pixel is positioned at the vertex of a rectangle, and any coordinate (for example, the upper left coordinate) is used as a reference coordinate, and this region is divided into a plurality of sub-regions (
  • the above-described coordinate conversion table 50 can be configured with a set of reference coordinates and sub-regions even if divided into 4 ⁇ 4 16 sub-regions Z1 to Z16).
  • the pixel value (luminance value) in each sub-region is calculated from the pixel values of four pixels surrounding the region Z based on a predetermined arithmetic expression. Also, the arithmetic expression is determined according to the arrangement of the region Z, the number of sub-regions divided, and the size. Further, the shape of the sub-region may be changed for each region as shown in FIG. 3, or may be divided by the same size as shown in FIG.
  • the reference coordinate 51 of the coordinate conversion table 50 stores the value of the corresponding coordinate. However, if the difference from the coordinate of the previous record is stored, Since the data size is reduced, the capacity of the coordinate conversion table 50, that is, the storage capacity of the table storage unit 5 can be further reduced.
  • the coordinates of the first record can be a difference from the reference coordinates stored separately.
  • the record of the coordinate conversion table 50 (a set of the reference coordinate 51 and the sub-region 52) is uniquely determined corresponding to the output coordinate.
  • the output coordinate is, for example, from the upper left to the lower right of the image.
  • the coordinates of the pixels in the horizontal direction are arranged one-dimensionally in order. For this reason, the reference coordinates corresponding to the output coordinates also have coordinates that are not far apart from adjacent coordinates, and the difference value often does not increase.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

L'invention concerne un dispositif de conversion d'image susceptible d'une interpolation de grande précision à l'aide d'une quantité extrêmement faible de mémoire pour un traitement de conversion d'image. Le dispositif de conversion d'image convertit une première image en une seconde image et comprend : une unité de mémoire d'image pour stocker la première image ; une unité de mémoire de table pour stocker une table de conversion de coordonnées comprenant des coordonnées de référence, qui constituent des informations concernant des positions dans la première image utilisées pour calculer des valeurs de pixel pour chaque pixel dans la seconde image et qui indiquent des pixels dans la première image, et des positions relatives (sous-régions) qui sont produites à partir desdites coordonnées de référence ; et un processeur d'image pour convertir la première image en la seconde image par le calcul des valeurs de pixel des pixels indiqués par les coordonnées de référence de la première image et des valeurs de pixel des pixels dans la première image situés dans le voisinage des coordonnées de référence au moyen d'expressions arithmétiques correspondant aux positions relatives, calculant ainsi les valeurs de pixel des pixels dans la seconde image.
PCT/JP2011/076998 2010-12-13 2011-11-24 Dispositif de conversion d'image WO2012081367A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2010-276698 2010-12-13
JP2010276698A JP2012128476A (ja) 2010-12-13 2010-12-13 画像変換装置

Publications (1)

Publication Number Publication Date
WO2012081367A1 true WO2012081367A1 (fr) 2012-06-21

Family

ID=46244480

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/076998 WO2012081367A1 (fr) 2010-12-13 2011-11-24 Dispositif de conversion d'image

Country Status (2)

Country Link
JP (1) JP2012128476A (fr)
WO (1) WO2012081367A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007079708A (ja) * 2005-09-12 2007-03-29 Konica Minolta Holdings Inc 画像処理装置及び画像処理方法
WO2007108081A1 (fr) * 2006-03-20 2007-09-27 Fujitsu Limited Instrument de photographie, procede et programme de photographie, dispositif de conception de tableau et procede relatif a un instrument de photographie, processeur video et procede de traitement
JP2007251914A (ja) * 2006-02-20 2007-09-27 Matsushita Electric Works Ltd 映像信号処理装置、仮想現実感生成システム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007079708A (ja) * 2005-09-12 2007-03-29 Konica Minolta Holdings Inc 画像処理装置及び画像処理方法
JP2007251914A (ja) * 2006-02-20 2007-09-27 Matsushita Electric Works Ltd 映像信号処理装置、仮想現実感生成システム
WO2007108081A1 (fr) * 2006-03-20 2007-09-27 Fujitsu Limited Instrument de photographie, procede et programme de photographie, dispositif de conception de tableau et procede relatif a un instrument de photographie, processeur video et procede de traitement

Also Published As

Publication number Publication date
JP2012128476A (ja) 2012-07-05

Similar Documents

Publication Publication Date Title
KR101921672B1 (ko) 이미지 처리 방법 및 장치
US9532018B2 (en) Projection system, device and method for the output of calibration projection scenes
JP4657367B2 (ja) 画像処理装置、撮像装置、および画像歪み補正方法
JP2020511022A (ja) デュアルコアフォーカシングイメージセンサ、そのフォーカシング制御方法、および電子装置
JP5535053B2 (ja) 画像処理装置、及び画像処理方法
JP5141245B2 (ja) 画像処理装置、補正情報生成方法、および撮像装置
JP6027777B2 (ja) 画素情報管理装置およびそれを用いた撮像装置
JP5984493B2 (ja) 画像処理装置、画像処理方法、撮像装置およびプログラム
JP2006270918A (ja) 画像補正方法、撮影装置、画像補正装置およびプログラム並びに記録媒体
US20170330311A1 (en) Image processing device and method, image capturing device, program, and record medium
JP4866147B2 (ja) 画像シフト検出方法および装置および画像入力装置
JP6557499B2 (ja) 焦点検出装置およびその制御方法、撮像装置、プログラム、ならびに記憶媒体
JP6153318B2 (ja) 画像処理装置、画像処理方法、画像処理プログラム、および、記憶媒体
JP2007079708A (ja) 画像処理装置及び画像処理方法
US10122939B2 (en) Image processing device for processing image data and map data with regard to depth distribution of a subject, image processing system, imaging apparatus, image processing method, and recording medium
JP5446285B2 (ja) 画像処理装置及び画像処理方法
KR100733505B1 (ko) 해상도변환방법
WO2012081367A1 (fr) Dispositif de conversion d'image
JP5676233B2 (ja) 画像処理装置、画像処理方法、及び撮像装置
JP6604783B2 (ja) 画像処理装置、撮像装置および画像処理プログラム
US20170034423A1 (en) Image capturing apparatus
US10200674B2 (en) Image processing apparatus, image capturing apparatus, and recording medium
JP2019121972A (ja) 画像処理方法、画像処理装置、撮像装置、画像処理プログラム、および、記憶媒体
JP2019061035A (ja) 撮像装置及び撮像方法
JP6468751B2 (ja) 画像処理装置、画像処理方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11848348

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11848348

Country of ref document: EP

Kind code of ref document: A1