WO2012081367A1

WO2012081367A1 - Image conversion device

Info

Publication number: WO2012081367A1
Application number: PCT/JP2011/076998
Authority: WO
Inventors: 祐司國米; 秀貴佐々木; 田中　慎一; 宏酒井
Original assignee: 株式会社ニコン
Priority date: 2010-12-13
Filing date: 2011-11-24
Publication date: 2012-06-21
Also published as: JP2012128476A

Abstract

Provided is an image conversion device capable of highly accurate interpolation using an extremely small amount of memory for image conversion processing. The image conversion device converts a first image to a second image and comprises: an image memory unit for storing the first image; a table memory unit for storing a coordinate conversion table comprising reference coordinates, which constitute information regarding positions in the first image used for calculating pixel values for each pixel in the second image and which indicate pixels in the first image, and relative positions (sub-regions) which are produced from said reference coordinates; and an image processor for converting the first image into the second image by calculating the pixel values of the pixels indicated by the reference coordinates of the first image and the pixel values of the pixels in the first image located in the vicinity of the reference coordinates by means of arithmetic expressions corresponding to the relative positions, thereby calculating the pixel values of the pixels in the second image.

Description

Image converter

The present invention relates to an image conversion apparatus.

In an image conversion apparatus that converts an equidistant projection image photographed with a fisheye lens into a central projection image without distortion, an image conversion method as shown in Patent Document 1 is used. In the image conversion apparatus disclosed in Patent Document 1, pixel values of pixels around a position on the equidistant projection image corresponding to the pixel of the central projection image are weighted according to the position and added. Thus, high-precision interpolation is performed, and the contour of the subject in the converted central projection image is smoothed.

JP 2009-146099 A

However, according to the conventional method, each coordinate (output coordinate) of the central projection image, a plurality of coordinates of the equidistant projection image for obtaining the pixel value at this output coordinate, and each weight are stored in association with each other. There is a problem that the storage area becomes large because it is necessary.

The present invention has been made in view of such problems, and an object of the present invention is to provide an image conversion apparatus capable of performing high-precision interpolation with a small storage capacity in image conversion processing.

In order to solve the above problems, an image conversion apparatus according to the present invention is an image conversion apparatus that converts a first image into a second image, an image storage unit that stores the first image, and a second Information on the position of the first image for calculating the pixel value of the pixel of the second image for each pixel of the image of the first image, the reference coordinate indicating the pixel of the first image and the reference coordinate from the reference coordinate A table storage unit that stores a coordinate conversion table composed of relative positions, a pixel value of a pixel indicated by the reference coordinates of the first image, and a pixel value of a pixel of the first image located in the vicinity of the reference coordinates An image processing unit that performs image conversion from the first image to the second image by calculating the pixel value of the pixel of the second image by calculating with an arithmetic expression corresponding to the relative position. It is characterized by.

In such an image conversion apparatus, the relative position of the coordinate conversion table is obtained by dividing the first image into a plurality of regions including the pixel for each pixel of the first image, and further dividing the region into a plurality of sub-regions. It is preferable to indicate a sub-region in the region including the reference coordinates when divided into two.

In such an image conversion apparatus, the pixel of the first image located in the vicinity of the reference coordinates for calculating the pixel value of the second image by the image processing unit is a pixel adjacent to the pixel of the reference coordinates. Preferably there is.

Further, in such an image conversion apparatus, it is preferable that the reference coordinates of the coordinate conversion table are stored as a difference from the adjacent reference coordinates in the coordinate conversion table.

In such an image conversion apparatus, it is preferable that the first image is an equidistant projection image and the second image is a center projection image.

The image conversion apparatus according to the present invention is an image conversion apparatus that converts a first image, which is an image of a predetermined projection method, into a projection method different from the first image and outputs it as a second image. Information on a position in the first image corresponding to the pixel of the second image when the projection method is converted, and the reference coordinates indicating the pixel of the first image in the vicinity of the position and the position A table storage unit for storing a set of relative positions from the reference coordinates as a coordinate conversion table for the pixels of the second image, and for each pixel of the second image, the pixel of the second image of the coordinate conversion table Based on the corresponding reference coordinate and relative position, the pixel value of the pixel indicated by the reference coordinate of the first image and the pixel value of the pixel adjacent to the pixel of the reference coordinate are calculated using an arithmetic expression corresponding to the relative position. Compute pixel of second image By calculating the pixel value, and having an image processing unit that performs image conversion from the first image to the second image.

When the image conversion apparatus according to the present invention is configured as described above, the capacity of the coordinate conversion table stored in the table storage unit is reduced, so that high-precision interpolation can be performed with a small storage capacity in the image conversion process. .

It is a block diagram which shows the structure of an image converter. It is explanatory drawing for demonstrating the management method of the pixel value in an image. It is explanatory drawing for demonstrating the area | region and sub area | region for calculating | requiring by interpolating a pixel value. It is explanatory drawing which shows the data structure of a coordinate conversion table. It is a flowchart for demonstrating the image conversion method which concerns on this embodiment. It is explanatory drawing which shows other embodiment about the definition of an area | region and a sub area | region.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. First, the configuration of the image conversion apparatus according to the present embodiment will be described with reference to FIG. The image conversion apparatus 10 includes a fisheye lens, and includes an imaging optical system 1 that forms an image of a subject, an imaging element 2 that includes a CCD, a CMOS, and the like, and that detects an image formed by the imaging optical system 1; An image processing unit 3 that converts a first image detected by the image sensor 2 (hereinafter described as an “equal distance projection image”) into a second image (hereinafter described as a “center projection image”); An image storage unit 4 that temporarily stores the equidistant projection image detected by the image sensor 2 and captured by the image processing unit 3, and information (coordinate conversion) used when changing from the equidistant projection image to the central projection image A table storage unit 5 for storing (table). The image processing unit 3 includes a CPU, and may be configured to execute the above-described image conversion by executing a program having the image conversion method according to the present embodiment, or may be configured as an ASIC or DSP, The image conversion method according to this embodiment may be implemented as a logic circuit. The image storage unit 4 is composed of a volatile memory such as SRAM, and the table storage unit 5 is composed of a nonvolatile memory such as flash memory. Further, the central projection image converted by the image processing unit 3 is passed to and used by a processing unit (not shown).

Here, the data structure when processing the equidistant projection image and the central projection image will be described with reference to FIG. FIG. 2 shows one image, and it is assumed that the width is W pixels (pixels) and the vertical is H pixels (pixels). In the image conversion apparatus 10 according to the present embodiment, as shown in FIG. 2, data whose coordinates are represented by (x, y) are set as one-dimensional data and managed as (y × W + x) -th data. Such a data structure is, for example, an image obtained by sequentially extracting values (pixel values) detected by each pixel of the image sensor 2 in the horizontal direction from the upper left to the lower right and arranging them one-dimensionally. It is. The image storage unit 4 shown in FIG. 1 stores an equidistant projection image acquired from the image sensor 2. This image has a pixel value (luminance value) at coordinates (x, y) as P (x, As y), this value is stored one-dimensionally in the order described above for the number of pixels of the image sensor 2.

By the way, when converting from an equidistant projection image to a central projection image, if the central projection image is expressed in units of pixels, the coordinates on the equidistant projection image corresponding to each pixel of the central projection image are as follows. , It does not always coincide with the position of the pixel. For this reason, the pixel value (luminance value) at a specific position of the equidistant projection image needs to be obtained by interpolation using the arithmetic expression from the pixel values acquired at the surrounding pixels. This interpolation method will be described with reference to FIG.

In FIG. 3, each of nine pixels centered on a pixel at a coordinate (x, y) in the equidistant projection image is surrounded by a rectangular area Z. Here, each pixel is located at the center of the region Z, and each region Z is further divided into nine sub-regions Z1 to Z9. Now, when the pixel value (luminance value) of each pixel is represented by P (x, y), the interpolated pixel values (luminance values) Pz1 to Pz9 in the sub-regions Z1 to Z9 are expressed by the following equation (1). It is obtained by (9).

Pz1 = (P (x-1, y-1) + P (x, y-1) + P (x-1, y) + P (x, y)) / 4 (1)
Pz2 = (P (x, y-1) + P (x, y)) / 2 (2)
Pz3 = (P (x, y-1) + P (x + 1, y-1) + P (x, y) + P (x + 1, y)) / 4 (3)
Pz4 = (P (x-1, y) + P (x, y)) / 2 (4)
Pz5 = P (x, y) (5)
Pz6 = (P (x, y) + P (x + 1, y)) / 2 (6)
Pz7 = (P (x-1, y) + P (x, y) + P (x-1, y + 1) + P (x, y + 1)) / 4 (7)
Pz8 = (P (x, y) + P (x, y + 1)) / 2 (8)
Pz9 = (P (x, y) + P (x + 1, y) + P (x, y + 1) + P (x + 1, y + 1)) / 4 (9)

As described above, when a point in a certain region Z is specified by belonging to any of the subregions Z1 to Z9, the reference coordinates (x, y) that are the coordinates of the pixel in the center of the region, and the subregions Z1 to Z9 Based on the information belonging to Z9, the pixel value of the pixel on the reference coordinate of the equidistant projection image and the pixel value of the pixel in the vicinity of the reference coordinate (adjacent to the reference coordinate in this embodiment) Thus, the pixel value (luminance value) of the output coordinates in the central projection image can be obtained by using the above-described arithmetic expressions (1) to (9). Thus, by interpolating using the pixel value of the coordinates (reference coordinates) in the vicinity of the position where the pixel value is to be obtained and the pixel values of the adjacent (adjacent) pixels, for example, a central projection image The edge portion such as the contour of the subject can be smoothed.

FIG. 4 shows the data structure of the coordinate conversion table 50 stored in the table storage unit 5. As described above, the coordinate conversion table 50 includes reference coordinates (coordinates of pixels of the original equidistant projection image) 51 for calculating the output coordinates for each output coordinate (coordinates of the central projected image after conversion). And a sub-region (identifier) 52 in the region Z including the reference coordinates is stored. Since the sizes of the equidistant projection image and the central projection image (the number of pixels in the horizontal direction × the number of pixels in the vertical direction) are known, the output coordinates are stored in the coordinate conversion table 50 as long as the storage order is determined. There is no need to keep it.

5 is an image conversion method according to the present embodiment, in which an equidistant projection image is read from the image sensor 2, converted into a center distance projection image, and output. The processing executed by the image processing unit 3 explain. First, the image processing unit 3 reads one image (equal distance projection image) from the image sensor 2 and stores it in the image storage unit 4 (step S100). Next, the values of the reference coordinates 51 and the sub area 52 are read out from the coordinate conversion table 50 stored in the table storage unit 5 in the stored order (step S110). At this time, one record may be read out or a plurality of records may be read out collectively.

Then, the pixel value of the output coordinate is obtained for each read reference coordinate and sub-region using the above-described arithmetic expressions (1) to (9) (step S120). Specifically, the pixel values of nine coordinates including the reference coordinates are read from the image storage unit 4 based on the reference coordinates, and further, the sub area is selected from the above-described arithmetic expressions (1) to (9). Is selected, and the pixel value of the sub-region is calculated based on the calculation formula. Note that an arithmetic expression may be determined from the sub-region, and a pixel value of a pixel necessary for the expression may be extracted from the image storage unit 4 and the calculation may be performed. The calculated result is output from the image processing unit 3 to the outside as a pixel value at the output coordinates of the central projection image (step S130). When the calculation results are output, it is determined whether or not the calculation of all the pixels of the central projection image has been completed (step S140). If there is a pixel that has not been calculated yet, the process returns to step S110 and the above processing is performed. repeat. When it is determined that the calculation (conversion) has been completed for all the pixels, the conversion of one equidistant projection image extracted from the image sensor 2 into the central projection image is terminated. Note that, when images are continuously captured from the image sensor 2 and processed as moving images, it can be realized by repeatedly executing the above steps S100 to S140. When a subject is stereoscopically viewed using a set of two or more imaging optical systems 1 and imaging elements 2, image conversion is performed by processing images output from the respective imaging elements 2 according to the above procedure. It is possible.

As described above, since the records of the coordinate conversion table 50 are arranged in the order of the predetermined output coordinates, the reference coordinates and sub-regions are extracted in that order, and the pixel values of the output coordinates are calculated and output. Thus, a desired central projection image can be obtained. Further, the table stored in advance in the image conversion apparatus 10 is only the coordinate conversion table 50 stored in the table storage unit 5 as described above. Since the coordinate conversion table 50 stores only a set of reference coordinates and sub-region information for each output coordinate, it can have a very small data capacity. Therefore, an ASIC or DSP having a small storage capacity can be used for the image processing unit 3 (and the table storage unit 5) of the image conversion apparatus 10 according to the present embodiment.

In the above description, the method of obtaining the interpolation value by dividing each pixel of the equidistant projection image acquired by the image sensor 2 into the rectangular region Z including the pixel as the center has been described. The arrangement of the region Z is not limited to the above description. For example, as shown in FIG. 6, the region A is determined so that each pixel is positioned at the vertex of a rectangle, and any coordinate (for example, the upper left coordinate) is used as a reference coordinate, and this region is divided into a plurality of sub-regions ( For example, in FIG. 6, the above-described coordinate conversion table 50 can be configured with a set of reference coordinates and sub-regions even if divided into 4 × 4 16 sub-regions Z1 to Z16). In the case shown in FIG. 6, the pixel value (luminance value) in each sub-region is calculated from the pixel values of four pixels surrounding the region Z based on a predetermined arithmetic expression. Also, the arithmetic expression is determined according to the arrangement of the region Z, the number of sub-regions divided, and the size. Further, the shape of the sub-region may be changed for each region as shown in FIG. 3, or may be divided by the same size as shown in FIG.

In the above description, the reference coordinate 51 of the coordinate conversion table 50 stores the value of the corresponding coordinate. However, if the difference from the coordinate of the previous record is stored, Since the data size is reduced, the capacity of the coordinate conversion table 50, that is, the storage capacity of the table storage unit 5 can be further reduced. In this case, the coordinates of the first record can be a difference from the reference coordinates stored separately. As described above, the record of the coordinate conversion table 50 (a set of the reference coordinate 51 and the sub-region 52) is uniquely determined corresponding to the output coordinate. The output coordinate is, for example, from the upper left to the lower right of the image. Thus, the coordinates of the pixels in the horizontal direction are arranged one-dimensionally in order. For this reason, the reference coordinates corresponding to the output coordinates also have coordinates that are not far apart from adjacent coordinates, and the difference value often does not increase.

3 image processing unit 4 image storage unit 5 table storage unit 10 image conversion device 50 coordinate conversion table

Claims

An image conversion device for converting a first image into a second image,
An image storage unit for storing the first image;
For each pixel of the second image, information on the position of the first image for calculating the pixel value of the pixel of the second image, the reference coordinates indicating the pixel of the first image, and A table storage unit for storing a coordinate conversion table composed of relative positions from the reference coordinates;
The pixel value of the pixel indicated by the reference coordinate of the first image and the pixel value of the pixel of the first image located in the vicinity of the reference coordinate are calculated by an arithmetic expression corresponding to the relative position. And an image processing unit that performs image conversion from the first image to the second image by calculating a pixel value of a pixel of the second image.
The relative position of the coordinate conversion table is determined by dividing the first image into a plurality of regions including the pixel for each pixel of the first image, and further dividing the region into a plurality of sub-regions. The image conversion apparatus according to claim 1, wherein the image conversion device indicates the sub-region in the region including the reference coordinates.
The pixel of the first image located in the vicinity of the reference coordinate for calculating the pixel value of the second image by the image processing unit is a pixel adjacent to the pixel of the reference coordinate. The image conversion apparatus according to claim 1 or 2.
The image conversion apparatus according to any one of claims 1 to 3, wherein the reference coordinates of the coordinate conversion table are stored as a difference from the adjacent reference coordinates in the coordinate conversion table.
5. The image conversion apparatus according to claim 1, wherein the first image is an equidistant projection image, and the second image is a center projection image.
An image conversion device that converts a first image that is an image of a predetermined projection method into a projection method different from the first image and outputs the second image as a second image,
Information about a position in the first image corresponding to a pixel of the second image when the projection method is converted, and reference coordinates indicating a pixel of the first image in the vicinity of the position; A table storage unit that stores a set of relative positions of the positions from the reference coordinates as a coordinate conversion table for pixels of the second image;
For each pixel of the second image, based on the reference coordinate and the relative position corresponding to the pixel of the second image in the coordinate conversion table, the pixel indicated by the reference coordinate of the first image By calculating a pixel value of a pixel of the second image by calculating a pixel value and a pixel value of a pixel adjacent to the pixel of the reference coordinate using an arithmetic expression corresponding to the relative position, the first image And an image processing unit that performs image conversion from the first image to the second image.
The relative position of the coordinate conversion table is determined by dividing the first image into a plurality of regions including the pixel for each pixel of the first image, and further dividing the region into a plurality of sub-regions. The image conversion apparatus according to claim 6, wherein the sub-region in the region including the reference coordinates is indicated.
The pixel of the first image located in the vicinity of the reference coordinate for calculating the pixel value of the second image by the image processing unit is a pixel adjacent to the pixel of the reference coordinate. The image conversion apparatus according to claim 6 or 7.
9. The image conversion apparatus according to claim 6, wherein the reference coordinates of the coordinate conversion table are stored as a difference from the adjacent reference coordinates in the coordinate conversion table.
10. The image conversion apparatus according to claim 6, wherein the first image is an equidistant projection image, and the second image is a center projection image.