JP2013005140A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a satisfactory image quality even if images having different aspect ratios are inputted, while a circuit scale is small.SOLUTION: An image processing apparatus is provided with: an output image coordinate generator 10 for generating the coordinates of an output image; an input image coordinate generator 20 which generates the coordinates of an input image corresponding to the coordinates of the output image generated by the output image coordinate generator 10, by performing reverse perspective projection transformation and reverse aspect ratio transformation; and an image data processing part 30 which interpolates image data on the basis of the coordinates of the input image generated by the input image coordinate generator 20.

Description

  The present invention relates to an image processing apparatus and an image processing method for processing an image, and more particularly to geometric distortion correction in a projection display apparatus such as a liquid crystal projector.

  Projection display devices such as liquid crystal projectors are widely used for business purposes such as presentations and conferences, and home uses such as home theaters. For this reason, there are various places where the projection display device is installed, and if the projection display device and the projection surface are not facing each other, the image is distorted into a trapezoid. For example, as shown in FIG. 6A, when the projection display device 2 is disposed below the projection plane 1, the display image on the projection plane 1 is displayed from the lower side as shown in FIG. The upper side is distorted into a trapezoidal shape. Therefore, a method is known in which the input image is deformed before projection so that the shape of the input image looks the same on the projection plane. For example, in the projector described in Patent Document 1, as shown in FIGS. 7A and 7B, a rectangular input image is transformed in advance into a trapezoidal output image opposite to the projection surface before projection. The output image transformed in this way is displayed in a rectangular shape on the projection plane as shown in FIG.

  On the other hand, the aspect ratio of the conventional projection display device and the image input to the projection display device is mainly 4: 3, but an image with an aspect ratio of 16: 9 such as recent high-definition broadcasting has appeared. Thus, a projection display device having an aspect ratio of 16: 9 has been commercialized. Therefore, there are combinations in which an image with an aspect ratio of 16: 9 is displayed on a projection display device with an aspect ratio of 4: 3, or an image with an aspect ratio of 4: 3 is displayed on a projection display device with an aspect ratio of 16: 9. To do. In such a combination, when the input image is displayed in full, the subject in the output image looks distorted as shown in FIG. For example, when an image with an aspect ratio of 4: 3 is displayed on a projection display device with an aspect ratio of 16: 9, the subject in the output image appears to extend in the horizontal direction as shown in FIG. Conversely, when an image with an aspect ratio of 16: 9 is displayed on a projection display device with an aspect ratio of 4: 3, the subject in the output image appears to contract in the horizontal direction as shown in FIG. 8B. . Therefore, Patent Document 2 describes an aspect ratio conversion method that alleviates such a sense of incongruity. In this method, as shown in FIG. 9, the aspect ratio of the center portion of the display screen is substantially matched with the input image, and the aspect ratio is greatly increased in the horizontal direction as the distance from the center of the display screen to the peripheral portion increases. As a result, it is possible to suppress an uncomfortable appearance at the center of the display screen while displaying the input image in full.

JP 2009-225226 A JP-A-6-6634

  It is desired to obtain good image quality even when an image having an aspect ratio different from the aspect ratio of the projection display apparatus is input to the projection display apparatus. For this purpose, it is necessary to perform both aspect ratio conversion and trapezoidal distortion correction with little image quality degradation. However, according to the prior art, the circuit scale of the projection display device is greatly increased, which increases the cost of the projection display device. End up. Hereinafter, this point will be described in detail.

  FIG. 10 is a block diagram of a conventional trapezoidal distortion correction circuit 302. The trapezoidal distortion correction circuit 302 is a circuit for realizing trapezoidal distortion correction, and includes an output image coordinate generation unit 101, an input image coordinate generation unit 102, and an image data processing unit 103. The output image coordinate generation unit 101 generates pixel positions (output image coordinates) to be displayed and supplies them to the input image coordinate generation unit 102. The input image coordinate generation unit 102 obtains the position (input image coordinate) of the input pixel after trapezoid conversion corresponding to the output image coordinate, and supplies the position to the image data processing unit 103. The image data processing unit 103 obtains an output pixel by interpolating pixels near the input pixel coordinates.

  FIG. 11 is a block diagram of a conventional aspect ratio conversion circuit 301. The aspect ratio conversion circuit 301 is a circuit for realizing aspect ratio conversion with little image quality deterioration, and includes a horizontal counter 201, an input image coordinate generation unit 202, and an image data processing unit 203. The horizontal counter 201 designates a horizontal pixel position (output image horizontal coordinate) to be displayed, and supplies it to the input image coordinate generation unit 202. The input image coordinate generation unit 202 obtains the position (input image coordinate) of the input pixel after aspect ratio conversion corresponding to the pixel position (output image coordinate) to be displayed, and supplies it to the image data processing unit 203. The image data processing unit 203 obtains an output pixel by interpolating pixels near the input pixel coordinates.

  Here, in order to perform both aspect ratio conversion and trapezoidal distortion correction with little image quality degradation, as shown in FIG. 12, the aspect ratio conversion circuit 301 in FIG. 11 is connected in front of the trapezoidal distortion correction circuit 302 in FIG. It is possible to do. However, according to such a method, the circuit scale of the projection display device is greatly increased, and the cost of the projection display device is increased.

  The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide an image processing apparatus capable of obtaining a good image quality even when images having different aspect ratios are input, while having a small circuit scale. An image processing method is provided.

  In order to solve the above problems, an image processing apparatus according to an embodiment of the present invention supports an output image coordinate generation unit that generates coordinates of an output image, and coordinates of an output image generated by the output image coordinate generation unit. An input image coordinate generation unit that generates the input image coordinates by performing reverse perspective projection conversion and reverse aspect ratio conversion, and image data interpolation processing based on the input image coordinates generated by the input image coordinate generation unit And an image data processing unit for performing the above.

  Specifically, the input image generation unit includes the inverse perspective projection in a stage subsequent to the inverse perspective projection conversion unit that obtains coordinates before perspective projection conversion corresponding to the coordinates of the output image generated by the output image coordinate generation unit. In this configuration, an inverse aspect ratio coordinate conversion unit that calculates coordinates before aspect ratio conversion corresponding to coordinates before perspective projection conversion obtained by the conversion unit is connected.

  The inverse aspect ratio coordinate conversion unit may divide the intermediate processed image before the perspective projection conversion into a plurality of regions, and set an enlargement / reduction ratio so that an arbitrary aspect ratio is obtained for each region.

  The inverse aspect ratio coordinate conversion unit sets the enlargement / reduction ratio so that the aspect ratio of the horizontal central portion on the intermediate processed image before perspective projection conversion is equal to the input image, and the aspect ratio changes as the distance from the central portion increases. May be.

  In order to solve the above problems, an image processing method according to an embodiment of the present invention corresponds to an output image coordinate generation step for generating coordinates of an output image, and coordinates of an output image generated in the output image coordinate generation step. An input image coordinate generation step for generating coordinates of the input image to be performed by performing reverse perspective projection conversion and reverse aspect ratio conversion, and interpolation processing of image data based on the coordinates of the input image generated in the input image coordinate generation step And an image data processing step.

  According to the present invention, it is possible to provide an image processing apparatus and an image processing method capable of obtaining a good image quality even when an image having a different aspect ratio is input while having a small circuit scale.

1 is a block diagram of an image processing apparatus according to an embodiment of the present invention. It is a figure which shows the flow of the image data in embodiment of this invention, and its process sequence. 6 is a flowchart illustrating an operation of the image processing apparatus according to the embodiment of the present invention. It is explanatory drawing of the reverse aspect-ratio conversion method in embodiment of this invention. It is explanatory drawing of another reverse aspect-ratio conversion method in embodiment of this invention. It is explanatory drawing of the display image on the conventional projection surface. 10 is an explanatory diagram of a display image on a projection surface described in Patent Document 1. FIG. It is explanatory drawing of the conventional aspect-ratio conversion method. It is explanatory drawing of the aspect-ratio conversion method of patent document 2. FIG. It is a block diagram of the conventional trapezoid distortion correction circuit. It is a block diagram of a conventional aspect ratio conversion circuit. It is a block diagram of the conventional projection type display apparatus.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram of an image processing apparatus according to an embodiment of the present invention. This image processing apparatus is an apparatus applied to a projection display device or the like, and includes an output image coordinate generation unit 10, an input image coordinate generation unit 20, and an image data processing unit 30, as shown in FIG. I have.

  The output image coordinate generation unit 10 is a processing unit that generates the coordinates of the output image, and includes a horizontal counter 11 and a vertical counter 12. The horizontal counter 11 counts the number of pixels in the horizontal direction. The vertical counter 12 counts the number of pixels in the vertical direction.

  The input image coordinate generation unit 20 is a processing unit that generates the coordinates of the input image corresponding to the coordinates of the output image generated by the output image coordinate generation unit 10 by performing reverse perspective projection conversion and reverse aspect ratio conversion. And a reverse perspective projection conversion circuit 21 and an inverse aspect ratio coordinate conversion circuit 22. The reverse perspective projection conversion circuit 21 obtains coordinates before the trapezoidal distortion correction corresponding to the coordinates generated by the output image coordinate generation unit 10. The inverse aspect ratio coordinate conversion circuit 22 obtains coordinates before aspect ratio conversion corresponding to the coordinates obtained by the reverse perspective projection conversion circuit 21.

  The image data processing unit 30 is a processing unit that performs image data interpolation processing based on the coordinates of the input image generated by the input image coordinate generation unit 20, and includes an input image buffering 31 and an input image data reading control circuit. 32 and an image data interpolation processing circuit 33. The input image buffering 31 holds input image data. The input image data reading control circuit 32 determines whether or not the coordinates generated by the input image coordinate generation unit 20 are areas where images exist, and controls the image data to be inserted. The image data interpolation processing circuit 33 calculates a pixel value corresponding to the position of the coordinate generated by the input image coordinate generation unit 20.

Hereinafter, a flow of image data and a processing procedure thereof will be described with reference to FIG. Here, it is assumed that an input image IMG _i having an aspect ratio of 16: 9 is input to a 4: 3 projection display device. In this case, by performing aspect ratio conversion and perspective projection conversion (trapezoidal distortion correction) in the direction of the black arrow and deforming the image, a projection image IMG _s with less distortion on the projection plane can be obtained. In order to obtain each pixel value IMG _k (X _k , Y _k ) of the output image IMG _k of the projection display device just before the projection thus converted, which coordinate (X _i , Y on the input image IMG _i is obtained. _It is necessary to know whether the pixel value IMG _k (X _k , Y _k ) is generated using the pixel _i ). Therefore, the coordinates of the input image corresponding to the coordinates of each pixel of the output image IMG _k are guided to the reverse procedure (in the direction of the white arrow). First, the coordinates (X _a , Y _a ) on the intermediate processed image IMG _a before the trapezoidal distortion correction corresponding to the coordinates (X _k , Y _k ) of the output image IMG _k are _obtained . Hereinafter, this process is called reverse perspective projection conversion. Subsequently, coordinates (X _i , Y _i ) on the input image IMG _i corresponding to the coordinates (X _a , Y _a ) on the intermediate processed image IMG _a are obtained. Hereinafter, this process is referred to as inverse aspect ratio conversion. By performing inverse perspective projection transformation and inverse aspect ratio transformation in this manner, the coordinates (X _i , Y _i ) of the input image IMG _i corresponding to the coordinates (X _k , Y _k ) of the output image IMG _k can be obtained. it can.

  FIG. 3 is a flowchart showing the operation of the image processing apparatus according to the embodiment of the present invention. Hereinafter, the configuration of the image processing apparatus will be described together with the operation with reference to FIG.

First, the output image coordinate generation unit 10 determines the coordinates (X _k , Y _k ) of the target pixel of the output image using the horizontal counter 11 and the vertical counter 12 (step S1), and the inverse perspective projection conversion circuit 21. To supply. The perspective projection conversion circuit 21 uses coordinates (X _a , Y _a ) on the intermediate processed image IMG _a before trapezoidal distortion correction corresponding to the output image coordinates (X _k , Y _k ) using the following equation (1). Is obtained (step S4). Here, since A to I are conversion coefficients determined by the user, it is necessary to derive A to I in advance (steps S2 to S3).

The conversion coefficients A to H can be obtained by fixing the conversion coefficient I to 1 and solving the simultaneous equations of the following expression (2) using the coordinates of each of the four points before and after correcting the trapezoidal distortion (step S3). ). Here, (X _a0 , Y _a0 ), (X _a1 , Y _a1 ),... _Are coordinates on the intermediate processed image IMG _a , and (X _k0 , Y _k0 ), (X _k1 , Y _k1 ) _,. The coordinates on the output image IMG _k .

The coordinates of each of the four points before and after trapezoidal distortion correction can be determined, for example, by the following procedure (step S2). First, the user manually moves the pixel positions of the four corners while viewing the projected image IMG _s on the projection plane. Next, the movement is stopped at a position where the trapezoidal distortion of the image IMG _s on the projection plane is considered improved. The pixel positions before and after the movement at this time correspond to the coordinates of the four points as they are. The inverse perspective projection conversion circuit 21 supplies the obtained coordinates (X _a , Y _a ) before the trapezoidal distortion correction to the inverse aspect ratio coordinate conversion circuit 22.

As shown in FIG. 4A, the inverse aspect ratio coordinate conversion circuit 22 divides the intermediate processed image IMGa into _a plurality of strip-shaped areas in advance so that each area has an arbitrary aspect ratio. The enlargement / reduction magnifications k1, k2,... Are set (step S5). Then, as shown in Expression (3), it is determined in which region the coordinates (X _a , Y _a ) exist, and the intermediate processing image IMG _a is used using the enlargement / reduction magnification k in that region. The coordinates (X _i , Y _i ) on the input image IMG _i corresponding to the coordinates (X _a , Y _a ) are obtained (step S6). Here, as shown in FIG. 4B, X _i1 , X _i2 , and X _i3 are horizontal coordinates on the input image IMG _{i at} coordinates X _a1 , X _a2 , and X _a3 .

The above is an example of the reverse aspect ratio conversion method, and other methods can be adopted. That is, in the aspect ratio conversion, it is often desirable that the aspect ratio of the central portion in the horizontal direction on the intermediate processed image IMG _a is equal to the input image IMG _i and the aspect ratio changes as the distance from the central portion increases. Therefore, the magnification k may be changed for each pixel in the horizontal direction. An example is shown in equation (4).

Here, N is the number of horizontal pixels, and a and b are coefficients that can be arbitrarily determined by the user. In Expression (4), FIG. 5 shows the relationship between the horizontal pixel position _Xa and the magnification k when a = −3 and b = 1. As shown in this figure, the magnification k is equal (k = 1) at the center of the coordinate, decreases as the distance from the center increases, and becomes the smallest at both ends of the coordinate. Thus, the aspect ratio is changed for each horizontal pixel by changing the magnification k for each pixel. The inverse aspect ratio coordinate conversion circuit 22 supplies the obtained coordinates (X _i , Y _i ) to the input image data read control circuit 32 and the image data interpolation processing circuit 33.

The input image data reading control circuit 32 determines whether or not the coordinates (X _i , Y _i ) obtained by the inverse aspect ratio coordinate conversion circuit 22 are included in the input image data (step S7). If it is included, the image data of several pixels around the coordinates (X _i , Y _i ) is read out from the input image buffer ring 31 holding the input image data (step S8). If not included, image data prepared in advance as a background is read (step S9). The image data read out in this way is supplied to the image data interpolation processing circuit 33.

The image data interpolation processing circuit 33 performs interpolation processing using the image data obtained by the input image data reading control circuit 32 (step S10), and obtains output image data. That is, the coordinates (X _i , Y _i ) of the input image IMG _i corresponding to the coordinates (X _k , Y _k ) of the output image IMG _k obtained by the inverse aspect ratio coordinate conversion circuit 22 are coordinates at which pixels can exist. (X _i = integer, Y _i = integer) is not established, and usually the coordinates between pixels (X _i or Y _i is a decimal number) are obtained. Therefore, interpolation processing is performed using the image data obtained by the input image data read control circuit 32 by an interpolation method such as bilinear convolution or bicubic convolution shown in Expression (5). Thereby, pixels near the input pixel coordinates are interpolated to obtain output pixel data.

  As described above, the image processing apparatus according to the embodiment of the present invention can obtain a good image quality even when images having different aspect ratios are input, although the circuit scale is small. That is, since the coordinates of the input image corresponding to the coordinates of the output image are generated by performing the reverse perspective projection conversion and the reverse aspect ratio conversion, the trapezoidal distortion correction circuit 302 of FIG. 10 and the aspect ratio conversion of FIG. The image data processing units existing in both of the circuits 301 can be shared into one. As a result, the circuit scale can be reduced, and the cost increase of the projection display apparatus can be suppressed.

  In addition, since the image processing apparatus according to the embodiment of the present invention employs a configuration in which the inverse aspect ratio coordinate conversion circuit 22 is connected to the subsequent stage of the reverse perspective projection conversion circuit 21, it is possible to efficiently obtain good image quality. . That is, the image data processing unit can be shared by a configuration in which the connection order is reversed (a configuration in which the reverse perspective projection conversion circuit 21 is connected to the subsequent stage of the reverse aspect ratio coordinate conversion circuit 22). However, according to such a configuration, there is a problem that the image is distorted into a barrel shape, and it is necessary to correct this distortion. Therefore, in order to efficiently obtain good image quality, it is desirable that the inverse aspect ratio coordinate conversion circuit 22 is connected to the subsequent stage of the reverse perspective projection conversion circuit 21 as described above.

  Here, an example is shown in which the aspect ratio at the center of the screen is substantially matched with the aspect ratio of the input image in order to perform aspect ratio conversion with little image quality degradation. However, the important part of the image is not always near the center of the screen. Therefore, the aspect ratio of an arbitrary part in the screen may be made to substantially match the aspect ratio of the input image. That is, it is possible to adopt a configuration in which the screen is divided into a plurality of areas and the user specifies the enlargement / reduction magnification k of the screen for each area.

  In addition, the present invention can be realized not only as an image processing apparatus, but also as an image processing method that uses a characteristic processing unit included in such an image processing apparatus as steps, or executes these steps in a computer. It can also be realized as a program to be executed. It goes without saying that such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

DESCRIPTION OF SYMBOLS 10 ... Output image coordinate generation part 20 ... Input image coordinate generation part 21 ... Reverse perspective projection conversion circuit (inverse perspective projection conversion part)
22 ... Inverse aspect ratio coordinate conversion circuit (inverse aspect ratio coordinate conversion unit)
30: Image data processing section

Claims (5)

An output image coordinate generation unit for generating coordinates of the output image;
An input image coordinate generation unit that generates the coordinates of the input image corresponding to the coordinates of the output image generated by the output image coordinate generation unit by performing reverse perspective projection conversion and reverse aspect ratio conversion;
An image data processing unit that performs interpolation processing of image data based on the coordinates of the input image generated by the input image coordinate generation unit;
An image processing apparatus comprising:   The input image generation unit is obtained by the inverse perspective projection conversion unit after the inverse perspective projection conversion unit for obtaining coordinates before the perspective projection conversion corresponding to the coordinates of the output image generated by the output image coordinate generation unit. 2. The image processing apparatus according to claim 1, further comprising an inverse aspect ratio coordinate conversion unit for obtaining coordinates before aspect ratio conversion corresponding to coordinates before perspective projection conversion.   The inverse aspect ratio coordinate conversion unit divides an intermediate processed image before perspective projection conversion into a plurality of regions, and sets an enlargement / reduction ratio so as to have an arbitrary aspect ratio for each region. The image processing apparatus according to claim 2.   The inverse aspect ratio coordinate conversion unit sets the enlargement / reduction ratio so that the aspect ratio of the horizontal central portion on the intermediate processed image before perspective projection conversion is equal to the input image, and the aspect ratio changes as the distance from the central portion increases. The image processing apparatus according to claim 2, wherein: An output image coordinate generation step for generating coordinates of the output image;
An input image coordinate generation step for generating coordinates of the input image corresponding to the coordinates of the output image generated in the output image coordinate generation step by performing reverse perspective projection conversion and reverse aspect ratio conversion;
An image data processing step for performing interpolation processing of image data based on the coordinates of the input image generated in the input image coordinate generation step;
An image processing method comprising:

Images