JP2728426B2 - Image interpolation circuit - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a configuration of an interpolation circuit that converts a spatially sampled image into an image having a different resolution.

[Conventional technology]

As a conventional interpolation circuit used for image signal processing, a method of weight-adding a two-dimensional operator to a two-dimensional image signal has been often used (a so-called two-dimensional convolution method).

This interpolation method is as follows: (1) When the two-dimensional operator is relatively small, for example, 3 × 3.

(2) When the positional relationship between the position of the pixel determined by interpolation and the position of the pixel of the original image is regular. (For example, when the former is located at the middle point of the latter), the amount of calculation is small and it is effective as a general interpolation method.

[Problems to be solved by the invention]

However, there is a disadvantage that the amount of calculation increases and the circuit scale increases in the following cases.

(1) A case where a two-dimensional operator becomes large, for example, in a case where a block average is smoothed and displayed in hierarchical coding of an image.

(2) In the case where the position of the pixel to be obtained by interpolation and the pixel of the original image do not have a simple positional relationship such as a midpoint, such as a change between the PAL and NTSC signals.

An object of the present invention is to realize an interpolation circuit capable of performing an interpolation process with a relatively small amount of calculation even in such a case.

[Means for solving the problem]

In the present invention, the interpolation processing is calculated by temporarily converting the signal of the original image into a two-dimensional frequency component using an orthogonal transformation matrix, and then using an interpolation matrix to convert it again to the spatial domain. The amount has been reduced.

Further, the product of the orthogonal transformation matrix and the interpolation matrix is provided in the conversion constant memory in advance as a new transformation matrix, and these are appropriately switched in accordance with the position of the pixel to be interpolated and the type of interpolation, thereby achieving processing. Higher speed has been achieved.

〔Example〕

Hereinafter, a first embodiment of the present invention will be described with reference to FIG.

FIG. 1 is a configuration diagram of a circuit that converts an original pixel of m × n pixels into an interpolated image of M × N pixels.

In FIG. 1, a digital image signal captured from a television camera 1 and an A / D converter 2, a still image file 3, or a still image transmission device 4 is once written in a frame memory 5. From this frame memory, mx
Signal V (i, j) of n pixels (i = 1,2, m) (j = 1,2, n)
Is read out, and the product-sum operation is performed by the product-sum operation circuit 6 with the signal read from the conversion constant memory 8, and the coefficient C (i, j) (i = 1,2) corresponding to the frequency component is obtained. ,, m) (j =
1,2,, n). This transformation is a linear transformation, and the transformation matrix D1 (i, j) (i = 1,2,, n) D2 (i, j) (i, j
= 1,2,, m) and is expressed as follows.

C = D1, Y, D2 (1) Although the transformation matrix includes a KL transformation and a Hadamard transformation, the DCT (D
iscrete Cosine Transform) is often used. In the case of DCT, D1 and D2 are as follows.

D1 and D2 are orthonormal transform matrices (unitary matrices). When m = n, the product is a unit matrix.

The coefficient C (j, i) converted by the product-sum operation circuit 6
(I = 1,2, m) (j = 1,2,, n) is a component corresponding to a kind of spatial frequency. For example, C (1,1) is a DC coefficient corresponding to the average value of a block, and the larger the value of i, j, the higher the spatial frequency. These coefficients are temporarily stored in the buffer memory 10.

Next, the interpolation processing matrix is stored in the conversion constant memory 8.
, And the multiplication with the above conversion coefficient is performed by the product-sum operation circuit 6 to generate the interpolated image F. This interpolated image is written to the frame memory 9 and the D / A converter 1
It is displayed on a monitor (display device) 12 via 1. These circuits are controlled by a control circuit 7.

 Here, the following equation holds for the above conversion.

F = H1 · C · H2 (4) H1 and H2 are as follows.

By the above operation, the original image of m × n pixels is converted into an interpolated image of M × N pixels by the DCT matrices D1 and D2 and the interpolation matrices H1 and H2.

In the above description, the operation for obtaining the orthogonal transform coefficient and the operation for generating the interpolated image from the coefficient are described in two stages, but in the present embodiment, this is realized by one operation. Next, the details will be described.

 Substituting equation (1) into equation (2) gives the following equation.

F = H1 / D1 / Y / D2 / H2 = (H1 / D1) / Y / (D2 / H2) (7) Here, (H1 / D1) and (D2 / H2) are (n ×
N) and a matrix of (M × m). When the positional relationship between the original image and the pixel to be interpolated is fixed, it can be determined uniquely regardless of the content of the original image.

Therefore, by pre-writing the contents of these matrices in the conversion constant memory 8, the interpolation processing can be realized by an extremely high-speed and simple operation.

When no interpolation is performed (M = m, N = n), (H1
・ D1) and (D2 ・ H2) are (n × n) (mx
m), and the original image Y is reproduced as it is.

The relationship between the orthogonal transform coefficients and the image in the above interpolation processing will be described.

When the conversion coefficient of Expression (1) is made to correspond to the original image area,
Appears as a cosine surface corresponding to the order of those coefficients. For example, the coefficient C (2,1) is a curved surface that changes in the horizontal direction at a half cycle of the cosine. This curved surface is continuous, and the value of an arbitrary point on the surface is significant as an interpolated image.

Therefore, in equation (5), i and j are defined as integers.
It is not always necessary to be an integer. When the interpolation points are unequally spaced, an arbitrary real number can be used.

Further, in addition to the above-mentioned interpolation operation, it is also possible to simultaneously realize image filtering processing and the like.

For example, from the left and right of the original image Y or the interpolation image F,
By multiplying arbitrary matrices W and W ', low-pass and other filtering operations can be realized. That is, instead of (H1 · D1) and (D2 · H2) in equation (7), (W · H1 · D1) and (D2 · H2 · W) or (H1
・ D1 ・ W) and (W ・ D2 ・ H2) or (H1 ・ W ・ D)
By writing 1) and (D2.W.H2) in the variable constant memory 8 in advance and multiplying them by the original image Y, it is possible to easily increase the speed of the calculation.

In these matrices, conversion constants may be arranged very regularly, and a high-speed calculation method called FFT may be applied.

Although examples of interpolation and filtering of some images have been described above, it is clear that similar effects can be obtained with higher-speed processing and complicated combinations thereof.

The circuits required for such processing are only a memory circuit for recording the above-mentioned matrix, and a few control circuits that switch according to the type of conversion, and can be realized extremely easily.

Furthermore, in the description so far, the image signal taken from a television camera, a still image file, a still image transmission device, or the like is handled. Each time, a transform such as DCT is performed, and the coefficients are entropy-encoded and transmitted. Therefore, when the coefficients are inversely transformed on the receiving side, it is possible to share the circuit for the inverse transformation with the arithmetic circuit shown in equation (4) of this embodiment.

Further, it is also possible to realize the interpolation processing shown in this embodiment by using a digital signal processor having an extremely high-speed product-sum operation function.

Further, a method of adaptively rewriting the contents of the memory circuit by using these processors or the like can be easily realized.

In this way, for any linear transformation represented by a matrix, the product of the matrix and the inverse orthogonal transformation matrix is read out from the memory circuit and multiplied from the left and right sides of the coefficient matrix, thereby simultaneously performing image transformation and inverse orthogonal transformation. And the calculation time can be reduced to half or less.

Next, a second embodiment of the present invention will be described with reference to FIG.

In the first embodiment, a method of obtaining an interpolated image of M × N pixels from an original image of m × n pixels has been described. For example, if an interpolation process is performed using the average of m × n blocks as it is, The average value of each block is not always guaranteed.

Therefore, in the present embodiment, an interpolation method for storing a DC value (average value) of an image divided into blocks will be specifically described.

FIG. 2 shows an example in which interpolation is performed using, for example, an average value of a block to be interpolated and 24 blocks around the block.

When the orthogonal transformation is performed on the original image Y (5,5) composed of 5 × 5 blocks by the DCT method described in the first embodiment, the number of transformations C (5,5) of 5 × 5 is obtained. From the coefficients using equation (4).
Can be requested.

Here, assuming that each block is composed of m × n pixels, F = H1 · C · H2 M = 1,2,, 5m, N = 1,2,, 5n. Here, H1 and H2 are matrices defined by the equations (5) and (6).

That is, the interpolated image F (M, N) can be represented using 25 unknown conversion variables C (i, j) (i = 1,2,5) (j = 1,2,5). it can.

Therefore, the average of each block is calculated as A (i, j) (i = 1,
2,, 5) (j = 1,2,, 5), the following equation holds.

As a result, 25 linear equations equal to the number of unknowns are generated. By solving the simultaneous equations, the above-described variable coefficient C (i, j) can be obtained.

 That is, the following equation is established.

a = W · c (9) c = W · a (10) where c, a is a vector composed of 25 terms obtained by rearranging the 5 × 5 matrix C, A into one dimension, and the matrix W is By performing the calculation in advance without depending on the original image and writing the calculated value in the conversion constant memory (8), the interpolation processing can be realized using the same method as in the first embodiment.

That is, the matrix C is determined from the vector c, and the interpolation image F in which the block average is stored is calculated by using the equation (4).
(M, N) can be obtained.

In the above description, the original image Y composed of 5 × 5 blocks
Having mentioned (5,5), it is clear that it can be extended to blocks of any size.

Further, similarly to the first embodiment, low-pass and other filtering operations can be simultaneously realized.

Also, in the transmission apparatus of a moving image or a still image, which performs an orthogonal transformation such as DCT for each block, a difference from an interpolation pixel F (M, N) obtained from a DC value is transmitted, thereby obtaining a transmission time. Can also be shortened. This is because the deviation from the average value increases around the block. That is, the portion F ′ (K, L) (K, L = 1,2, m) (L = 1,2, n) of the interpolated image to be transmitted is extracted and the orthogonality is calculated using equation (1). When converted, the following equation is established.

C ′ = D1 · F ′ · D2 = (D1 · H1) · C · (H2 · D2) (11) Here, the difference from the interpolated pixel F ′ (M, N) obtained from the DC value is orthogonally transformed. The difference is also expressed in the area of the coefficient C ', and a method of transmitting the difference between the coefficient obtained by orthogonally transforming the block of the original picture and C' is effective for improving the transmission efficiency. (D1 ・ H
1) A method of calculating the value of (H2 · D2) in advance and reading out from the conversion constant memory 8 is also applicable. The above transmission efficiency is expected to be improved by about 8% by simulation.

The above-described interpolation and filtering processes can be realized by exactly the same operation as the content of the conversion constant memory only by increasing the space of the memory for conversion constants, which is extremely flexible.

In the present embodiment, a product-sum operation unit is used for these operations. However, if there is a relatively long processing time, a signal processor can be used.

〔The invention's effect〕

As is apparent from the above description, according to the present invention, the image signal is once converted into a two-dimensional spatial frequency component using orthogonal transform, and then converted again into the spatial domain by inverse orthogonal transform, thereby obtaining the image signal. Can be implemented very flexibly and easily.

Furthermore, the product of the above-described orthogonal transformation, inverse orthogonal transformation, and matrix for performing any linear transformation is calculated in advance, written in a conversion constant memory, and these are appropriately selected and used. Interpolation and filtering can be performed.

Further, in image communication using orthogonal transformation, the transmission time can be shortened by transmitting the block average and the like first, and transmitting the difference from the curved surface obtained by interpolation. At this time, the above difference is a difference between the orthogonal transform coefficients themselves, and can be realized very easily without requiring complicated processing.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of an image interpolation circuit according to the present invention, and FIG. 2 is a conceptual diagram showing a relationship between a DC value of a block-divided image and an interpolated image. 1 ... TV camera, 2 ... A / D converter, 3 ... Image file, 4 ... Image transmission device, 5 ... Frame memory,
6: product-sum operation circuit, 7: control circuit, 8: memory for conversion constant, 9: frame memory, 10: buffer memory, 11: D / A converter, 12: monitor television.

Claims (2)

(57) [Claims] An image signal is divided into blocks of m × n pixels, and a matrix-like image signal belonging to the block is provided with a first orthogonal transformation matrix of n × n dimensions and a second orthogonal transformation matrix of m × m dimensions. Means for multiplying the orthogonal transformation matrices from the left and right, respectively, into an n × m-dimensional coefficient matrix corresponding to a two-dimensional frequency component, a first interpolation matrix of N × n dimensions and a second interpolation matrix of m × M dimensions Is multiplied from the left and right, respectively, and the image signal (N
≠ n or M ≠ m). 2. An N × n-dimensional first interpolation matrix and m
A memory means preliminarily holding a plurality of types of × M-dimensional second interpolation matrices, and a product-sum operation means for performing a matrix multiplication process, wherein the product-sum operation means comprises:
Any one of the n × m-dimensional coefficient matrices corresponding to two-dimensional frequency components obtained by orthogonally transforming matrix image signals belonging to a block of m × n pixels is read out of the memory means. Multiplying one of the first interpolation matrices from the left and one of the second interpolation matrices from the right to output an image signal (N ≠ n or M ≠ m) of a block composed of M × N pixels. Characteristic image interpolation circuit.