JPH10134175A - Processor and method for image processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enlarge or reduce an image at an arbitrary conversion rate. SOLUTION: A residue circuit 11 outputs a phase variation component Pd supplied from a specific device and the decimal part of the sum of the value of a register 12 to the register 12. An approximation circuit 13 outputs a filter select signal Pi whose phase (x) corresponds to a filter coefficient set corresponding to the phase closest to the value of the register 12 to a coefficient memory 1A. Thus, an optimum filter coefficient set among a specific number of filter coefficient sets is selected for the interpolation of specific pixel data. Then product sum operations between the four filter coefficient sets and four pixel data are performed by multipliers 3-1 to 3-4 and an adder by a Cubic approximating method to calculate an interpolated value of pixels.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method, and more particularly, to a filter coefficient set corresponding to each phase when a pixel interval of an original image is divided by a predetermined number of divisions. The present invention relates to an image processing apparatus and method for enlarging or reducing an image at an arbitrary magnification by performing an interpolation operation on pixel data using a filter coefficient set closest to the phase of a pixel to be subjected to an interpolation operation.

[0002]

2. Description of the Related Art Many image displays (displays) such as television receivers use a CRT (Cathode Ray Tube). In such a display, when an image signal is handled in an analog manner corresponding to various image systems and an image is displayed, the display is often handled by changing a horizontal scanning frequency.

On the other hand, when image signals are handled digitally, the resolution of the images differs according to the broadcast transmission system such as NTSC or PAL. The number differs for each broadcast transmission system. Also, there are various broadcast transmission systems including HDTV, and the standards for the number of pixels (resolution) are various. Therefore, in a system that performs digital processing on image data, in order to support all of these transmission methods, the number of pixels is set to an “interpolation filter”.
Needs to be converted.

Further, in a liquid crystal display or a plasma display which has recently become widespread, the number of pixels of a display image is fixed to a predetermined number. To
Interpolation filters are required to convert to the number of pixels corresponding to these displays.

Next, an example of an interpolation filter for converting the number of pixels of an image will be described.

First, the enlargement or reduction of an image and the conversion of the sampling frequency (the number of pixels) will be described.

[0007] Both enlargement or reduction of an image and conversion of the sampling frequency (the number of pixels) of the image (conversion between image standards having different resolutions) are performed with respect to each pixel position of the original image in the original image. This is realized by performing an operation for obtaining data of a pixel that did not exist. Therefore, the above-described two processes can be performed by using an interpolation filter that performs the same calculation operation.

FIG. 21 shows an example of a part of the original image. The circles in the figure represent the positions of the pixels. This portion includes eight pixels in the horizontal direction and six pixels in the vertical direction (the number of pixels is set to a small value here for convenience).

Next, a case where this original image is enlarged, for example, by (10/7) will be described. The magnification is expressed not by the area but by the length ratio. In the case of enlarging the image of FIG. 21, the display image standard is not changed, and the pixel arrangement (that is, the pixel interval or the like) is kept the same as in FIG. When the enlargement process is performed in this manner, the resulting image is as shown in FIG. In this case, the magnification is 1.42
9 (= 10/7), the length of one side of the image is
1.429 times, the number of pixels increases about 1.429 ² times.

For example, in the horizontal direction (horizontal scanning direction), the number of pixels in the original image is 8, but after enlargement, 11 or 12 (8 × 10/7 = an integer close to 11.429) pixels become. Therefore, since the positional relationship of each pixel corresponding to the same part of the image in the similar image after enlargement is different from the positional relationship in the original image, the value of the data (expressing luminance and color) of each pixel after enlargement is Will be different from that of the original image.

FIG. 23 shows the positional relationship between horizontal pixels in the original image and the enlarged image when the image is enlarged at a magnification of (10/7).

In the figure, the upper Ri (i = 1, 2,...)
Represents the pixels of the original image, and the lower Qi (i = 1,
2,...) Represent data of the interpolated pixels after the enlargement. The pixels corresponding to Ri are arranged at an interval (10/7) times the interval between the pixels corresponding to Qi. Note that FIG.
3 shows only the state of enlargement in the horizontal direction, but the same applies to the vertical direction, and a description thereof will be omitted.

The value of the data of each pixel after the enlargement is calculated from the values of the pixels of some surrounding original images in accordance with the correspondence relationship with the position of each pixel of the original image as shown in FIG. It is calculated by performing an operation, that is, a convolution operation of an interpolation function (described later).

Next, consider the case where the sampling frequency is increased, for example, by (10/7) without changing the size of the image.
This sampling frequency conversion is equivalent to conversion to an image standard whose resolution is higher by (10/7) times. That is, the number of pixels in the horizontal direction is changed to (10/7) times. In this case, as shown in FIG. 24, the original image of FIG. 21 is one-dimensionally converted into an image having about 1.429 times the number of pixels, that is, an area density of 1.429 ² times.

The correspondence between each pixel in FIG. 21 and each pixel in FIG. 22 and the correspondence between each pixel in FIG. 21 and each pixel in FIG. 24 are as shown in FIG. Therefore, the operation for converting to an image standard having a large number of pixels is performed in the same manner as the above-described operation for enlarging an image.

Next, the original image shown in FIG.
3) The case of reducing the size by a factor of 2 will be described.

When the image is reduced, the standard of the image is not changed. Therefore, the arrangement of pixels in the reduced image, that is, the pixel interval becomes the same as that of the original image shown in FIG.

FIG. 25 shows the original image of FIG.
3) shows an image reduced by a factor of two. In this case, the magnification is 0.769 (= 10/13).
The length of the side is reduced to 0.769 times, and the number of pixels forming the reduced screen is reduced to about 0.769 ² .

For example, in the original image, the number of pixels in the horizontal direction is 8, but in the reduced image, the number of pixels in the horizontal direction is an integer close to 6 or 7 (8 × 10/13 = 6.154). )become. Accordingly, since the positional relationship of each pixel corresponding to the same portion of the image in the reduced similar image is different from the positional relationship of each pixel in the original image, data (expressing luminance and color) of each pixel after reduction is obtained. Is different from that of the original image.

FIG. 26 shows the positional relationship between pixels in the horizontal direction between the original image and the reduced image when the image is reduced by a factor of (10/13).

In the figure, the upper Ri (i = 1, 2,...)
Represents the pixels of the original image, and the lower Qi (i = 1,
2,...) Represent the data of the interpolated pixels after reduction. The pixels corresponding to Ri are arranged at an interval (10/13) times the interval between the pixels corresponding to Qi. Although FIG. 26 shows only the state of reduction in the horizontal direction, the same applies to the vertical direction, and a description thereof will be omitted.

The value of the data of each pixel after reduction is calculated from the values of some peripheral pixels of the original image according to the correspondence relationship with each pixel of the original image as shown in FIG. That is, it is calculated by performing a convolution operation of the interpolation function.

Next, consider a case where the sampling frequency is increased, for example, by (10/13) without changing the size of the image. This sampling frequency conversion has a resolution of (10/13)
This is equivalent to converting to an image standard that is twice as low. That is, the number of pixels is changed to (10/13) times. In this case, the original image of FIG. 21, as shown in FIG. 27, the one-dimensional and is converted to about 0.769 times the number of pixels, i.e., an image having 0.769 ² times the surface density.

The correspondence between each pixel in FIG. 21 and each pixel in FIG. 25 and the correspondence between each pixel in FIG. 21 and each pixel in FIG. 27 are as shown in FIG. 26, and are the same. Therefore, the calculation operation for converting to a low-resolution image standard is performed in the same manner as the above-described calculation operation for image reduction.

As described above, when performing enlargement or reduction of an image and conversion of the sampling frequency (the number of pixels), an interpolation filter for calculating pixel data at a position not existing in the original image is required. .

Next, the operation performed in the interpolation filter will be described.

As shown in FIG. 28, the sampling interval of the original image is S, and the position (phase) P away from the position of the pixel R of the original image is the position of the pixel Qi generated by interpolation (the interpolation point). ), The value of the pixel Qi is calculated by a convolution operation on the pixel value R of the surrounding original image.

According to the "sampling theorem", when ideal "interpolation" is performed, the sinc function as shown in equation (1) and FIG. The convolution operation from the pixel of to the pixel in the future for infinite time is performed. f (x) = sinc (π × x) = sin (π × x) / (π × x) (1)

Here, π represents the pi.

However, in practice, it is necessary to calculate an interpolated value within a finite time, so an interpolation function that approximates a sinc function within a finite range is used.

As an approximation method, a nearest neighbor approximation method, a bilinear approximation method, a Cubic approximation method and the like are known.

In the nearest neighbor approximation method, data of one pixel after interpolation is calculated from data of one pixel of an original image by using an interpolation function as shown in equation (2) and FIG. 29 (B). . Note that the variable x in Equation (2) and FIG.
It is assumed that the displacement in the horizontal direction from the pixel position of the original image represents an amount normalized by the sample interval of the original image.

(Equation 1)

In the bilinear approximation method, data of one pixel after interpolation is calculated from data of two pixels of an original image by using an interpolation function as shown in equation (3) and FIG. 29 (C). . It should be noted that the variable x in Equation (3) and FIG.
It is assumed that the displacement in the horizontal direction from the pixel position of the original image represents an amount normalized by the sample interval of the original image. The bilinear approximation method is well known as linear interpolation, and a weighted average is calculated.

(Equation 2)

In the Cubic approximation method, data of one pixel after interpolation is calculated from data of four pixels of the original image using an interpolation function as shown in equation (4) and FIG. 29 (D). The variable x in equation (4) and FIG.
Represents the amount of displacement in the horizontal direction from the pixel position of the original image normalized by the sample interval of the original image.

(Equation 3)

These convolution operations can be performed using a so-called FIR digital filter. In this case, the center of the interpolation function is set to the interpolation point, and a value obtained by sampling the interpolation function at the sampling points of the original image nearby by a predetermined number of pixels is used as an interpolation filter coefficient set.

For example, in the case of performing an interpolation operation by the bilinear approximation method, when the phase P is 0.0, two weights (filter coefficients) constituting the filter coefficient set are 1.0 and 0.0. , The coefficient set is such that the pixel values of the original image at the same position are output as they are.

When the phase P is 0.5, the two filter coefficients are 0.5 and 0.5, and when P is 0.3, they are 0.7 and 0.3.

When the interpolation operation is performed by the Cubic approximation method, when the phase P is 0.0, the four weights (filter coefficients) constituting the filter coefficient set are 0.0, 1.
The coefficient set is 0, 0.0, and 0.0, and is a coefficient set that directly outputs the data value of the original image pixel at the same position.

When the phase P is 0.5, the four filter coefficients are -0.125, 0.625, 0.62
5, and -0.125, and when P is 0.3, -0.063, 0.847, 0.36
3, and -0.147.

At this time, since the phase P with the pixel of the original image is different for each interpolation point for calculating the data,
A plurality of sets of filter coefficients corresponding to different phases are required.

Next, a conventional interpolation filter operation device will be described.

FIG. 30 shows an example of the configuration of an arithmetic unit using an FIR digital filter for performing an interpolation operation, that is, a convolution operation of an interpolation function. The arithmetic device in FIG. 30 performs a convolution operation using the Cubic approximation method.

The coefficient memory 1 holds a plurality of filter coefficients corresponding to each interpolation point (or each phase), and stores four filter coefficients corresponding to a filter selection signal supplied from a predetermined device (not shown). Filter coefficients FC1, FC2, FC
3 and FC4 are output to multipliers 3-1 to 3-4, respectively.

The register 2-1 holds data supplied from a predetermined device (not shown) and outputs the data to the register 2-2 in response to a control signal. Register 2-2, 2-3
Holds the data supplied from the registers 2-1 and 2-2, respectively, and stores the data in the registers 2-3 and 2 according to the control signal.
-4. The register 2-4 holds the data supplied from the register 2-3.

The registers 2-1 to 2-4 are connected in series and operate as a four-stage shift register. To this shift register, a horizontally scanned input image data sequence is sequentially input in word units,
The pixel data of one original image is stored.

Further, in the Cubic approximation method, data of an interpolation point is calculated from data of a total of four pixels, that is, two pixels on the left and right sides sandwiching the interpolation point, using a four-stage shift register.

The multiplier 3-i (i = 1,..., 4)
The value held in the register 2-i is multiplied by the value (filter coefficient) FCi supplied from the coefficient memory 1 and the operation result is output to the adder 4.

The adder 4 calculates the sum of the values supplied from the multipliers 3-1 to 3-4 and outputs the result as an interpolated value.

As described above, the data and filter coefficients input to the register 2-1 in time series are combined with the multiplier 3
-1 to 3-4 and a product-sum operation in the adder 4
The data of the interpolation points, which are the calculation results, are output in chronological order.

Next, the operation of the arithmetic unit shown in FIG. 30 when the original image is enlarged to (10/7) times by Cubic approximation will be specifically described.

In the image magnification of (10/7) times, as described above, the positional relationship of each pixel with respect to each interpolation point is expressed by:
What is necessary is just to set as shown in FIG. 28 and to perform an interpolation filter operation.

FIG. 31 shows the values of each part of the arithmetic unit of FIG. 30 in each cycle.

In an apparatus which performs processing in hardware as shown in FIG. 30, latency (delay for realizing high-speed operation) usually occurs due to pipeline processing in the multiplication and summation operations. For convenience, it is assumed here that there is no latency.

In the first cycle of FIG. 31, input data R1 as image data for one pixel of the original image is supplied from a predetermined device. At this time, the registers 2-1 to 2-4 store data Rm0, which is one before the input data R1,
Data Rm1 two before the input data R1, data Rm2 three before the input data R1, and input data R
The data Rm3 four data before four are held.

At this time, since the value of the control signal is "H", the registers 2-1 to 2-4 shift the data at the timing of the next rising edge of the clock.

Since each data is shifted, the second
In the cycle, registers 2-1 to 2-4 store R
1, Rm0, Rm1, and Rm2 are held. At this time, since the value of the control signal is “H”, the registers 2-1 to 2-4 shift the data at the timing of the next rising edge of the clock.

Since the data are shifted respectively, the third
In the cycle, registers 2-1 to 2-4 store R
2, R1, Rm0 and Rm1, respectively. At this time, since the value of the control signal is “H”, the registers 2-1 to 2-4 shift the data at the timing of the next rising edge of the clock.

Similarly, since the data is shifted, in the fourth cycle, the registers 2-1 to 2-
4 holds R3, R2, R1, and Rm0, respectively. Also, Rm0, R1, R2, R in FIG.
Filter selection signal P indicating the phase of interpolation value Q1 for
0 is supplied to the coefficient memory 1. As described above, the filter selection signal is supplied corresponding to the phase P of the output signal Qi.

In this case, Ra in FIG.
, Rb corresponds to R1, Rc corresponds to R2, Rd corresponds to R3, and Q in FIG. 28 corresponds to the interpolation value Q1.

The coefficient memory 1 stores ten types of filter coefficient sets shown in FIG. 32, and four coefficients FC1, FC2, FC3, and FC3 according to the supplied filter selection signal Pi.
Select and output FC4. In the case of (10/7) times image enlargement, the phase of the pixel in the interpolation calculation is as shown in FIG.
Since there are only 10 types as shown in FIG.
Have as many filter coefficient sets as their phases.

That is, the filter selection signal Pi corresponds to a filter coefficient set when the phase is i / 10, out of ten types of phases corresponding to each position where S is divided into ten equal parts in FIG. . The decimal point expression coefficient (filter coefficient) in FIG. 32 is a value calculated by substituting the phase corresponding to the filter selection signal Pi into x in Expression (4), and the 8-bit expression coefficient is the decimal point expression coefficient. Is limited to 8 bits in word length (here, the maximum amplitude is set to 128).

In this case, since the filter selection signal is P0, the coefficient memory 1 sets the filter coefficient set (0.0, 1.0, 0.0, 0.0) corresponding to the phase P0 in FIG.
(In the case of 8-bit representation, (0,128,0,0)) is
These are output to the multipliers 3-1 to 3-4 as four filter coefficients FC1, FC2, FC3, and FC4, respectively.

Then, the above-described product-sum operation is performed by the multipliers 3-1 to 3-4 and the adder 4, and the operation result is output as output data Q1.

At this time (in the fourth cycle), since the value of the control signal is "L", in the next clock,
The registers 2-1 to 2-4 do not shift the held data.

In the fifth cycle, the register 2-1
Nos. To 2-4 continuously hold the data held in the fourth cycle. At this time, R in FIG.
A filter selection signal P7 indicating the phase of the output data Q2 for m0, R1, R2, and R3 is supplied to the coefficient memory 1.

In this case, as shown in FIG.
Since 1 and Q1 have the same phase, and the interval between Q1 and Q2 is 7/10 of the interval S between R1 and R2, the phase is 7/10, and the filter selection signal P7 is supplied.

Since the filter selection signal is P7, the coefficient memory 1 sets the filter coefficient set (-0.147, 0.363,
0.847, -0.063) (in the case of 8-bit representation,
(−19, 46, 108, −8)) as the four filter coefficients FC1, FC2, FC3, and FC4 as multiplier 3-
1 to 3-4.

Then, the multiplier-sum operation is performed by the multipliers 3-1 to 3-4 and the adder 4, and the operation result is output as output data Q2.

When the pixels of the original image used to calculate the output data Q1 in the fourth cycle and the output data Q2 in the fifth cycle are the same as in this case, the value of the control signal in the fourth cycle is Is set to “L”, the registers 2-1 to 2-4 are not shifted when shifting from the fourth cycle to the fifth cycle.

At this time (in the fifth cycle), since the value of the control signal is "H", the registers 2-1 to 2-
No. 4 shifts the data at the timing of the next rising edge of the clock.

Next, in the sixth cycle, the registers 2-1 to 2-4 store R4, R3, R2, and R1.
Are respectively held. Further, R4, R in FIG.
3, a filter selection signal P4 indicating the phase of the output data Q3 with respect to R2 and R1 is supplied to the coefficient memory 1.

The current phase is the previous phase 7/10.
Is added to 7/10, and becomes 14/10. Since the phase (= 10/10) corresponding to one original image data is subtracted, the current phase is 4/10 (= 7/10 + 7). / 1
0-10 / 10).

That is, the phase originally changes by 7/10 every cycle. Then, since the integer is treated as a data delay, the phase is eventually subjected to a modulo operation.

The coefficient memory 1 stores the filter selection signal P4
Therefore, the filter coefficient set (−0.096, 0.744, 0.496, −0.1) corresponding to P4 in FIG.
44) (In the case of the 8-bit representation, (-12, 95, 63,
−18)) with four filter coefficients FC1, FC2, F
Output to multipliers 3-1 to 3-4 as C3 and FC4.

The multiplier-sum operation is performed by the multipliers 3-1 to 3-4 and the adder 4, and the operation result is output as output data Q3.

At this time, since the value of the control signal is "H", the registers 2-1 to 2-4 shift the data at the timing of the next rising edge of the clock.

In the same manner, as shown in FIG. 31, the processing proceeds, and output data Qi is sequentially output.

When the number of pixels of an image is converted using the above-described apparatus, the input data rate and the output data rate change due to a change in the number of pixels.

For example, in the conversion in which the number of pixels increases as described above, although the output data rate is constant, the supply of the input data string may be stopped as in the fifth cycle in FIG. is there. Further, in the case of conversion in which the number of pixels is reduced, the output of output data may be stopped although the input is constant.

Therefore, in practice, the data rate is kept constant by providing a buffer memory for temporarily storing the input / output data of the arithmetic unit shown in FIG.

As described above, (in the horizontal direction)
The enlargement or reduction of the image and the conversion of the resolution are performed in hardware (that is, using an electronic circuit configured for each operation in the processing).

However, as described above, when a device for enlarging or reducing an image and converting the resolution is used in terms of hardware, various types of image processing that are desired to be performed simultaneously during image conversion are desired. In addition, in order to perform television signal processing, noise removal, and the like, a device corresponding to each processing is separately required.

Therefore, for example, Japanese Patent Application No.
No. 246627, SIMD (Si
A method has been considered in which the above-described computation is performed by software using a parallel processor of the "ngle Instruction Multiple Data stream" type.

FIG. 33 shows an example of the configuration of such a parallel processor. This parallel processor includes an input pointer 21, an input SAM (serial access memory) unit 2
2. Data memory unit 23, ALU array unit 24, output S
It comprises an AM unit 25, an output pointer 26, and a program control unit 27.

The input SAM unit 22, the data memory unit 23,
The ALU array unit 24 and the output SAM unit 25 constitute a group of element processors parallelized in a linear array (linear array) type. These element processors 31
In accordance with one program of the program control unit 27, it is controlled in an interlocked manner (that is, SIMD controlled). The program control unit 27 includes a program memory,
It has a sequence control circuit or the like that advances the program, and generates various control signals according to the program written in advance in the program memory to control the various circuits.

The input SAM 22, the data memory 23, and the output SAM 25 are mainly composed of memories. Although not described in detail, in the apparatus of FIG. 33, it is assumed that “row” address decoders for these memories are included in the program control unit 27.

The parallelized element processors 31 (for a single element) correspond to the hatched portions in FIG. 33, and a plurality of element processors 31 are arranged in the horizontal direction in the figure. That is, only the hatched portions in FIG. 33 have components corresponding to one processor.

Next, the operation of the linear array type parallel processor for image processing of FIG. 33 will be described.

The input data (image data for one pixel) supplied to the input terminal DIN is supplied to the input SAM unit 22.

The input pointer 21 stores the value “H” of 1 for only one element processor 31 for one input data.
It outputs a bit signal, that is, an input pointer signal (SIP). Then, the element processor 31 specified by the value “H”
Is input to the input SAM unit 22 (input SAM cell).

The designation of the element processor 31 that supplies the data by the input pointer signal sequentially moves from the leftmost element processor 31 to the rightmost element processor 31 in the figure at each clock of the input data. Are sequentially supplied from the input SAM unit 22 (input SAM cells) of the leftmost element processor 31 to the input SAM cells of the rightmost element processor 31.

Since the number of element processors 31 is designed to be equal to or greater than the number of pixels H in one horizontal scanning period of the image signal, pixel data for one horizontal scanning period of the image signal can be stored in the input SAM unit 22. it can. Such an input operation is
It is repeated every horizontal scanning period.

The program control section 27 receives the data of the image signal for one horizontal scanning period in the input SAM section 22 in this manner.
Each time the data is accumulated, the input SAM unit 22, the data memory unit 23, the ALU array unit 24, and the output SAM unit 25 are SIMD-controlled as described below to execute processing.

This program control is repeated every horizontal scanning period. Therefore, it is possible to process a program corresponding to the number of steps calculated by dividing the time corresponding to the horizontal scanning period by the instruction cycle period of the processor. Because of the SIMD control, the following operations are similarly executed in parallel in all the element processors 31.

The input data for one horizontal scanning period stored in the input SAM unit 22 is transferred from the input SAM unit 22 to the data memory unit 23 as needed in the next horizontal scanning period, and is used for subsequent arithmetic processing. used.

The input SAM unit 22 to the data memory unit 23
In the transfer of data to the
Is input SA based on the input SAM read signal (SIR).
After selecting and accessing data of a predetermined bit of the M section 22, a memory access signal (SWA) is issued, and the data is written to a predetermined memory cell (described later) of the data memory section 23.

Next, the program control unit 27 operates the data memory unit 2 of each element processor 31 according to the program.
3 is stored in the element processor 31
To the ALU array unit 24 to perform an arithmetic operation or a logical operation on the data. Then, the calculation result is written to a predetermined address of the data memory unit 23.

Note that all the operations in the ALU array section 24 are performed in units of bits, so that the processing proceeds one bit per cycle. For example, it takes at least eight cycles to perform a logical operation between 8-bit data. In addition, it takes at least 9 cycles to perform addition of 8-bit data. 8
When multiplying bits of data, the multiplication is 64
Since this is equivalent to one bit addition, it takes at least 64 cycles.

The element processor 31 is connected to a nearby element processor 31 and can perform inter-processor communication. However, the neighboring element processors 31
To access the data memory unit 23 of the SIM
Due to the D control, for example, the element processor 31 on the right
, All the element processors 31 access the data memory section 23 of the element processor 31 on the right.

[0100] The operation as described above is performed by the FIR
There is no particular problem in realizing a digital filter. When reading data from the element processor 31 that is not directly connected, the number of program steps is slightly increased.
Data is read out by repeating communication between nearby processors.

By utilizing such communication, the FIR digital filter operation in the horizontal direction of the image can be realized using the data held by the neighboring element processors 31.

In such a parallel processor, the data of the pixel at the same position in the horizontal direction of the screen is processed by one predetermined element processor 31 during all horizontal scanning periods. Part 2
2 is transferred to the data memory unit 23, the data storage address is changed for each horizontal scanning period, so that the input data of the past horizontal scanning period is transferred to the data memory unit 23 until the subsequent horizontal scanning period. 23. By doing so, the data necessary for the calculation of the FIR digital filter in the vertical direction of the image can be sequentially stored in the data memory unit 23.

In this way, each element processor 31 stores a predetermined continuous number of pixel data in the vertical direction (perpendicular to the horizontal scanning direction) in the data memory unit 2.
3 to realize a vertical FIR digital filter operation.

When the operation assigned to one horizontal scanning period is completed as described above, the data calculated during the horizontal scanning period during the horizontal scanning period is output SA.
The data is transferred to the M unit 25.

As described above, during one horizontal scanning period, the transfer of the input data stored in the input SAM section 22 to the data memory section 23, the calculation by the ALU array section 24, and the data transfer to the output SAM section 25 Is executed according to a SIMD control program in units of bits.
These processes are performed on a horizontal scanning period basis.
It is repeated sequentially.

The output data transferred to the output SAM unit 25 is the output data in the next horizontal scanning period.
It is output from the AM unit 25.

As described above, the input processing for writing the input data to the input SAM unit 22, the transfer of the input data accumulated in the input SAM unit 22 to the data memory unit 23 by the program control unit 27, and the processing by the ALU array unit 24 Three processes of a calculation and a process of transferring the output data to the output SAM unit 25 and an output process of outputting the output data from the output SAM unit 25 are performed on each horizontal scan input data. Note that these three processes are:
This is executed as pipeline processing in units of one horizontal scanning period of the image signal.

Focusing on the input data for one horizontal scanning period, the time corresponding to one horizontal scanning period elapses in each of the three processes for the input data. Although it takes a corresponding time, since the three processes are executed in parallel as pipeline processes, the process can be performed in a time corresponding to one horizontal scanning period per input data for one horizontal scanning period on average. .

[0109]

However, in the above-described apparatus, if the conversion ratio of a predetermined image is K: L, L filter coefficient sets, which are the number of pixels after conversion, are required. Is not a simple integer ratio, many filter coefficient sets are required, and a large-capacity storage unit for storing the filter coefficient sets is used, so that the cost and circuit size of the apparatus can be reduced. It has the problem of being difficult.

When the conversion ratio is changed in response to, for example, a user operation, it is conceivable to use a set of filter coefficients corresponding to a plurality of conversion ratios. And the number of filter coefficient sets corresponding to the product of the number of pixels after conversion
Since a large-capacity storage unit is required, there is a problem that it is difficult to reduce the cost and the circuit scale of the device.

Further, although it is possible to realize a general FIR digital filter like the parallel processor described above,
When performing enlargement or reduction of an image that requires interpolation calculation (similarly, when performing resolution conversion), interpolation calculation is performed using F
Although it is a kind of IR digital filter, the input SA
The number of data held in the M unit 22 and the output SAM unit 25
Input SAM unit 2
In the second or output SAM unit 25, the input data Ri or the output data Qi are not densely arranged.

Therefore, when the element processor 31 acquires input pixel data necessary for the interpolation processing from a predetermined number of other element processors 31, the element processor 31
And the positional relationship with other element processors 31 is different for each element processor 31, so that all the element processors 3
1 has a problem that it is difficult to obtain necessary data in a SIMD parallel processor in which the same operation is performed.

For example, in the Cubic approximation, a convolution operation is required for four data of continuous input data. For example, when the image is enlarged by (10/7) times, as shown in FIG. 34, the input data Ri is not densely arranged, and thus the input data R1 and R2 required when calculating the output data Q3, for example. , R3, R4
R1, R3, and R4 are adjacent to the left two and to the right one from the element processor 31 that calculates the output data Q3,
And are held by the three element processors 31 adjacent to the right.

On the other hand, of the input data R2, R3, R4, and R5 necessary for calculating the output data Q4, R2, R4, R5
5 is held by the element processors 31 that are immediately adjacent to the left, adjacent to the right, and adjacent to the right.
Also, input data R2, necessary for calculating output data Q5,
R3, R4, and R5 are held in the element processors 31 that are two adjacent left, one adjacent left, one adjacent right, and two adjacent right of the element processor 31 that is the base point.

As described above, the positional relationship between the element processor 31 holding the input data necessary for calculating each output data and the element processor 31 calculating the output data changes for each output data.

FIG. 35 shows a pattern of the element processor 31 holding input data necessary for calculating each output data when the image is enlarged by (10/7). As shown in FIG. 35, in this case, it is classified into five patterns.

When the above-described parallel processor is used, each element processor 31 calculates output data corresponding to one pixel.
It has the problem that different filter coefficients need to be supplied.

The present invention has been made in view of such a situation, and includes a filter coefficient set corresponding to each phase when the pixel interval of the original image is divided by a predetermined number of divisions.
The interpolation operation of the image data is performed by using the filter coefficient set whose phase is closest to the phase of the pixel to be subjected to the interpolation operation, so that the image having an arbitrary conversion ratio can be enlarged or reduced. Things.

[0119]

An image processing apparatus according to claim 1 stores a filter coefficient set corresponding to each phase when a pixel interval of an original image is divided by a predetermined number of divisions. A storage unit that outputs a filter coefficient set whose phase is closest to the phase of the pixel data to be interpolated among the coefficient sets to the operation unit, and an operation unit that performs the interpolation operation of the pixel data by using the filter coefficient set And characterized in that:

According to a third aspect of the present invention, in the image processing method according to the first aspect of the present invention, the storage unit storing the filter coefficient set corresponding to each phase when the pixel interval of the original image is divided by a predetermined number of divisions, A filter coefficient set closest to the phase of the pixel data to be subjected to the interpolation calculation is output to the calculation unit, and the calculation unit performs the interpolation calculation of the pixel data using the filter coefficient set.

According to the image processing apparatus of the present invention, in the filter coefficient set corresponding to each phase when the pixel interval of the original image is divided by a predetermined number of divisions, the phase is determined by the pixel data of the pixel data to be processed. The filter coefficient set closest to the phase is
To the element processors, respectively.
It is characterized in that pixel data interpolation processing is performed using a filter coefficient set.

In the image processing method according to the nineteenth aspect, among the filter coefficient sets corresponding to the respective phases when the pixel interval of the original image is divided by a predetermined number of divisions, the phase is determined by the pixel data of the pixel data to be processed. A filter coefficient set closest to the phase is supplied to each of the element processors, and each of the element processors performs a pixel data interpolation process using the filter coefficient set.

In the image processing apparatus according to the first aspect, the storage unit is configured such that, of the filter coefficient sets corresponding to the respective phases when the pixel interval of the original image is divided by a predetermined number of divisions, the phase is determined by interpolation. The filter coefficient set closest to the phase of the pixel data to be calculated is output to the calculation unit, and the calculation unit performs the interpolation calculation of the pixel data using the filter coefficient set.

In the image processing method according to the third aspect, the storage unit storing the filter coefficient sets corresponding to the respective phases when the pixel interval of the original image is divided by the predetermined number of divisions is used. Then, the filter coefficient set closest to the phase of the pixel data to be interpolated is output to the calculation unit, and the calculation unit performs the interpolation calculation of the pixel data using the filter coefficient set.

In the image processing apparatus according to the fourth aspect, of the filter coefficient set corresponding to each phase when the pixel interval of the original image is divided by a predetermined number of divisions, the phase is determined by the pixel data to be processed. Are supplied to the element processors, respectively, and the element processors use the filter coefficient sets to perform the respective pixel data interpolation processes.

In the image processing method according to the nineteenth aspect, of the filter coefficient set corresponding to each phase when the pixel interval of the original image is divided by a predetermined number of divisions, the phase is determined by the pixel data to be processed. Are supplied to the element processors, respectively, and the element processors use the filter coefficient sets to perform the respective pixel data interpolation processes.

[0127]

FIG. 1 shows a configuration example of a first embodiment of an image processing apparatus according to the present invention.

The coefficient memory 1A stores a filter coefficient set corresponding to each phase when the pixel interval of the original image is divided by a predetermined number of divisions.

For example, when the number of divisions is set to 16, the coefficient memory 1A stores 16 filter coefficient sets (FC1, FC2, FC2) corresponding to the normalized phase amount x and the filter selection signal Pi as shown in FIG. FC3, FC4) are stored in advance.

The control circuit 5 is supplied from a predetermined device (not shown) with a phase change Pd corresponding to the pixel interval after interpolation, which is associated with the image conversion, and in accordance with the phase change Pd, A filter selection signal Pi is generated so that a filter coefficient set corresponding to a pixel to be interpolated is selected, and the filter selection signal Pi is output to the coefficient memory 1A.

The control circuit 5 generates a control signal to be supplied to the registers 2-1 to 2-4 in accordance with the accumulated value of the phase change Pd.

The remainder circuit 11 of the control circuit 5 includes the register 1
The remainder (that is, the fractional part of the sum) obtained by dividing the sum of the value stored in 2 and the phase change Pd by 1 is output to the register 12. Also, the remainder circuit 1
1 outputs a predetermined signal to the control signal generation circuit 14 and the adjustment circuit 15 when the sum of the value stored in the register 12 and the phase change Pd is 1 or more.

The register 12 of the control circuit 5 stores the remainder circuit 1
The value supplied from 1 is held, and the value is output to the remainder circuit 11 and the approximation circuit 13. Further, the register 12 resets the held value to 0 in response to the clear signal supplied every one horizontal scanning period.

The approximation circuit 13 of the control circuit 5 includes the register 1
The normalized phase amount x closest to the value supplied from 2 (FIG. 2)
Is output to the coefficient memory 1A.

When a predetermined signal is supplied from the remainder circuit 11, the control signal generation circuit 14 of the control circuit 5 outputs a control signal having a value of "L" to the registers 2-1 to 2-4, When a predetermined signal is not supplied from 11, a control signal having a value of "H" is output.

The adjusting circuit 15 of the control circuit has a built-in buffer memory (not shown), and outputs supplied input pixel data to the register 2-1 at a predetermined timing. When a predetermined signal is supplied from the remainder circuit 11, the adjustment circuit 15 stops outputting pixel data in that cycle.

The registers 2-1 to 2-4, the multipliers 3-1 to 3-4, and the adder 4 are configured in the same manner as in FIG. 30, and the description thereof will be omitted.

Next, the operation of the image processing apparatus of FIG. 1 will be described.

First, the phase change Pd is supplied to the remainder circuit 11 of the control circuit 5. Then, the remainder circuit 11 outputs the fractional part of the sum of the supplied value and the value of the register 12 to the register 12. At this time, if the sum of the supplied value and the value of the register 12 is 1 or more, the remainder circuit 11 outputs a predetermined signal to the control signal generation circuit 14 and the adjustment circuit 15.

Then, the approximation circuit 13 stores the filter selection signal Pi corresponding to the filter coefficient set whose phase x corresponds to the phase closest to the value of the register 12 to the coefficient memory 1A.
Output to

As described above, in interpolation of predetermined pixel data, an optimum filter coefficient set is selected from a predetermined number of filter coefficient sets.

The operation of calculating the interpolation value using the filter coefficient set is the same as that of the apparatus shown in FIG. 30, and the description thereof will be omitted.

Next, referring to FIG. 3, a control circuit for enlarging an image by (10/7) times using, for example, a set of 16 filter coefficients stored in the coefficient memory 1A in advance. Operation 5 will be described.

First, when calculating the first interpolation value Q1 (FIG. 23) having a phase of 0, the register 1
2 is set to an initial value 0, and the value is supplied to the approximation circuit 13. The approximation circuit 13 sets the phase x of the filter selection signal Pi shown in FIG. A close filter selection signal P0 (x = 0.0) is selected, and the selected filter selection signal P0 is output to the coefficient memory 1A.
At this time, since the phase x corresponding to the filter selection signal P0 is 0.0, which is the same as the supplied value, the error relating to the phase is 0.

At this time, the phase change Pd (in this case, Pd = 0.7) is supplied to the remainder circuit 11, and the fractional part of the sum of the value 0 of the register 12 and the phase change 0.7 is obtained.
That is, 0.7 is output to the register 12.

Next, when calculating the second interpolation value Q2 (FIG. 23) having a phase of 0.7, the value of the register 12 at this time is set to 0.7, and the value is approximated. The circuit 13 supplies the filter selection signal P11 (x = 0.6875) whose phase x is closest to the supplied value 0.7 among the filter selection signals Pi shown in FIG.
And the filter selection signal P11 is stored in the coefficient memory 1
Output to A. At this time, the filter selection signal P1
Since the phase x corresponding to 1 is 0.6875, which is different from the supplied value 0.7, an error relating to the phase of 0.0125 occurs.

At this time, the phase change Pd (Pd = 0.7) is supplied to the remainder circuit 11,
Of the sum of the value 0.7 and the phase change amount 0.7, that is,
0.4 is output to the register 12.

When calculating the third interpolation value Q3 (FIG. 23) having the phase of 0.4, the value of the register 12 at this time is set to 0.4, and the value is set to the approximation circuit. 13, the approximation circuit 13 sets the phase x of the filter selection signal Pi shown in FIG.
Is selected, and the filter selection signal P6 is output to the coefficient memory 1A. At this time, the phase x corresponding to the filter selection signal P6 is 0.375, and the supplied value 0.4
Therefore, an error relating to the phase of 0.025 occurs.

At this time, the phase change Pd (Pd = 0.7) is supplied to the remainder circuit 11 and the register 12
Of the sum of the value 0.4 and the phase change amount 0.7, that is,
0.1 is output to the register 12.

Similarly, the fourth to tenth interpolation values Qi are calculated in the same manner. Eleventh interpolated value Q
When calculating ₁₁ , the phase of the pixel to be interpolated returns to 0. In this case, a total of 10 filter coefficient sets out of 16 filter coefficient sets are used.

Since the phase change Pd is determined by L of the conversion ratio K: L, it is possible to enlarge or reduce an image having a different conversion ratio only by changing the value of the phase change Pd.

When a predetermined number of filter coefficient sets are used irrespective of the conversion ratio, a phase error occurs as described above.
Since the approximation of the sinc function is used, such a phase error hardly causes a problem. Note that the number of divisions of the pixel interval may be increased as necessary.

As described above, in the first embodiment, it is possible to enlarge or reduce an image by hardware at an arbitrary conversion ratio.

In the above description, the number of divisions of the pixel interval is set to 16, but it is needless to say that another division number may be used.

The above-mentioned phase change Pd is expressed as a decimal number. However, an integer value obtained by dividing the decimal number by the unit phase amount obtained by dividing the pixel interval by the number of divisions may be used as the phase change. Good. In this case, the process of calculating the remainder by the remainder circuit 11 is simplified by setting the number of divisions to a power of two. That is, in that case, the remainder circuit 11 can be realized by a binary adder that ignores the upper bits. In this case, the approximation circuit 13 may be a circuit that performs rounding for simple word length limitation.

Further, as described above, out of the filter coefficient sets corresponding to the number of divisions (in this case, 16),
When only a predetermined number (in this case, 10) of filter coefficient sets are used, the coefficient memory 1A may store only the filter coefficient sets to be used. In that case, the storage area of the coefficient memory 1A can be saved.

FIG. 4 shows the configuration of an image processing apparatus according to a second embodiment of the present invention.

The input pointer 21 is the element processor 31
Each time, an input pointer signal indicating whether or not to receive input data is output to the input SAM unit 22. Note that this input pointer 21 is, for example,
Input data can be selectively supplied to the element processor 31 in the same manner as described in JP-A-123683.

The input SAM unit 22 has the element processor 31
Each storage device has a storage unit for holding predetermined input data, and stores the input data corresponding to the SIP signal supplied from the input pointer 21. Also, input S
When receiving the SIR signal from the program control unit 27A, the AM unit 22 outputs the held data to the data memory unit 23.

The data memory unit 23 stores the element processor 3
Each memory has a storage unit for holding predetermined data, and upon receiving a SWA signal from the program control unit 27A, stores the data supplied from the input SAM unit 22 or the ALU array unit 24 in the storage unit. When a read access signal (SRAA, SRBA) is received, the data is transferred to A
The data is output to the LU array unit 24.

The ALU array unit 24 includes the element processor 3
An arithmetic unit (ALU (Arithmetic and Logical
Unit) 81), and an operation corresponding to the ALU control signal (SALU-CONT) supplied from the program control unit 27A is performed on the data supplied from the data memory unit 23.

The output SAM unit 25 is connected to the element processor 31
Each memory has a storage unit for holding predetermined output data, and upon receiving an output SAM write signal (SOW) supplied from the program control unit 27A, stores the output data from the ALU array unit 24 in the storage unit. It has been made like that. The output SAM unit 25 outputs the held data in accordance with the output pointer signal (SOP) supplied from the output pointer 26.

The output pointer 26 corresponds to the element processor 31
Each time, an SOP signal indicating whether to output output data is output to the output SAM unit 25. The output pointer 26 is described in, for example, Japanese Patent Application Laid-Open No. 8-123.
Data can be selectively output from the element processor 31 in the same manner as described in Japanese Patent No. 683.

The program control section 27A controls each section in accordance with a predetermined program to perform various operations described later.

FIG. 5 shows a configuration example of the element processor 31. The element processor 31 of FIG. 5 is configured as a general-purpose processor that can be used for various purposes. The input buffer memory (IQ) 41 stores the input S of FIG.
Input data is stored corresponding to one element processor of the AM unit 22. The data memory (RF) 42 is a three-port memory corresponding to one element processor of the data memory unit 23 in FIG. The output buffer memory (OQ) 44 corresponds to the output SAM unit 2 in FIG.
Output data is stored corresponding to one of the five element processors.

The operation unit (ALU) 43 corresponds to one element processor of the ALU array unit 24 in FIG. 4 and performs various operations on the data supplied from the data memory 42.
The calculation result is output to the data memory 42 or the output buffer memory 44.

In the element processor 31 shown in FIG. 5, input data is once input to the input buffer memory 41,
Thereafter, the data is transferred to the data memory 42. Arithmetic unit 43
Performs various operations on newly stored data, data stored in the past, data in the middle of operation, and the like, which are supplied as necessary from the data memory 42, and writes the data back into the data memory 42. Repeat for the program. The calculation result is output buffer memory 4
4 and output at a predetermined speed and format.

In the element processor 31, the input SAM unit 22, the data memory unit 23, and the output S
The AM unit 25 forms a “column” of the memory. The ALU array unit 24 is a 1-bit ALU and has a circuit configuration mainly using a full adder (full adder). Therefore, unlike a processor that performs processing in word units used in a general so-called personal computer or the like, the element processor 31 is a bit processing processor and performs processing in units of bits.

Since the bit processing processor has a small hardware scale per processor, the number of parallel processing can be increased. Therefore, the parallel processor for image processing is designed such that the number of parallel elements in the linear array of the element processors 31 is equal to or greater than the number H of pixels for one horizontal scanning period of the image signal.

FIG. 6 shows an example of a detailed circuit configuration of the element processor 31 described above. In addition, the structure of each cell in FIG. 6 is described as a very general structure for easy understanding. A portion where a plurality of the same circuits are arranged is described as a single circuit (a circuit for one bit).

The portion of the input SAM unit 22 corresponding to one element processor 31 is controlled by the input pointer 21.
According to the number of bits ISB of the input data,
Input SAM cells 22-1 to 22- storing one bit
It is composed of ISB. In FIG. 6, IS
Instead of the B input SAM cells 22-1 to 22-ISB, one cell 22-i is described.

In the input SAM cell 22-i, the gate terminal of the transistor Tr1 is connected to the input pointer 21, and the other two terminals of the transistor Tr1 are connected to the input data bus 65 or the capacitor C1 for storing 1-bit data. Is connected to one end.

The gate terminal of the transistor Tr2 is connected to the program control unit 27A,
The signal is supplied, and the other two terminals of the transistor Tr2 are connected to the write bit line 63 and one end of the capacitor C1, respectively.

One end of the capacitor C1 is connected to the transistor T
r1 and Tr2, and the other end is grounded.

The part corresponding to one element processor 31 of the data memory unit 23 is composed of MB data memory cells 23-1 corresponding to the number of bits MB required as a working memory.
To 23-MB. In FIG. 6, MB data memory cells 23-1 to 23-MB
, One cell 23-i is described.

Data memory cell 2 of data memory unit 23
3-i (i = 1,..., MB) is a three-port memory having two read bit lines 61 and 62 and one write bit line 63.

In the data memory cell 23-i, the gate terminal of the transistor Tr11 is connected to the program control unit 27A, supplied with the SWA signal, and the other two terminals of the transistor Tr11 are connected to the write bit line 63 and Capacitor C for storing 1-bit data
11 are respectively connected to one end.

One end of the capacitor C11 is connected to the gate terminal of the transistor Tr12 and the transistor Tr11, and the other end is grounded.

The other two terminals of the transistor Tr12 are connected to a power supply (not shown) via a ground point and a resistor R.
Connected to each other. Note that the resistor R may be omitted.

The gate terminal of the transistor Tr13 is connected to the program control unit 27A and supplied with the SRAA signal. The remaining two terminals of the transistor Tr13 are connected to the transistor Tr12, the resistor R, and the read bit line 61, respectively. It is connected.

The gate terminal of the transistor Tr14 is connected to the program control unit 27A and supplied with the SRBA signal. The remaining two terminals of the transistor Tr14 are connected to the transistor Tr12, the resistor R and the read bit line 62, respectively. It is connected.

The part corresponding to one element processor 31 of the ALU array unit 24 is the ALU cell 24 shown in FIG.
A (ALU unit). ALU81 of ALU cell 24A
Is a 1-bit ALU, has a circuit configuration such as a full adder (full adder), and includes flip-flops 82-1 to 82-1.
An operation is performed on the 1-bit value supplied from 2-3, and the operation result is output to the selector 83.

The ALU cell 24A is a flip-flop 8 holding a 1-bit value input to the ALU 81.
2-1 to 82-3, flip-flops 82-1 to 8-8
Selector (SEL) 8 for selecting a value supplied to 2-3
4-1 to 84-3.

A portion of the output SAM unit 25 corresponding to one element processor 31 is controlled by an output pointer 26,
It is composed of OSB output SAM cells 25-1 to 25-OSB corresponding to the number of output signal bits (OSB). In FIG. 6, one cell 25-i is described instead of the output SAM cells 25-1 to 25-OSB.

In the output SAM cell 25-i, the gate terminal of the transistor Tr7 is connected to the program control unit 27.
A, is supplied with the SOW signal, and the other two terminals of the transistor Tr7 are connected to the write bit line 63.
A and a capacitor C for storing 1-bit data
4 is connected to one end.

One end of the capacitor C4 is connected to the transistor T
r7 and Tr8, and the other end is grounded.

The gate terminal of the transistor Tr8 is connected to the output pointer 26, and the other two terminals of the transistor Tr8 are connected.
One terminal of the two terminals is connected to the capacitor C4 and the transistor Tr7, and the other terminal is connected to the output data bus 6
6 is connected.

All of the word lines connected to the element processor 31 are also connected to the other element processors 31 arranged therein, and the SIR signal, the SWA signal, and the memory read access signals (SRAA, SRB) are provided.
A), the SOW signal and the like are transmitted to all the element processors 31. These word lines are address-decoded in the program control unit 27A of FIG.

The input data bus 65 is connected to the input SAM cells 22-i of all the element processors 31, and the output data bus 66 is connected to all the element processors 3.
1 SAM cell 25-i.

The memory 28 (storage means) in FIG.
At the time of startup, during a horizontal retrace period, a vertical retrace period, and the like, data of all interpolation filter coefficients required for a filter operation in all the element processors 31 and supplied from an external control CPU (not shown), The numbers are stored in the order of the element processors 31. That is, the memory 2
Reference numeral 8 stores filter coefficient sets of a predetermined number of divisions, similarly to the coefficient memory 1A of the first embodiment.

Next, data transfer and calculation in element processor 31 will be described.

In the input SAM cell 22-i of the element processor 31 specified by the input pointer 21, the transistor Tr1 is turned on, and the terminal voltage of the capacitor C1 changes to the input data bus 65 (and the buffer 7).
The voltage becomes a voltage corresponding to the input data supplied via 1).

Thus, the input data is stored in the input SAM unit 22 of the specified element processor 31.

Next, the input SAM cell 22- selected by the SIR signal supplied from the program control unit 27A.
In i, the transistor Tr2 is turned on,
A transfer data signal corresponding to the voltage of the capacitor C1 is generated on the write bit line 63.

At this time, the SBC signal is supplied to the buffer 72, and the SWA signal is supplied to the transistor Tr11 of the predetermined data memory cell 23-i, turning on the transistor Tr11. The voltage becomes a voltage corresponding to the data stored in the capacitor C1 of the input SAM cell 22-i.

When writing data from the ALU cell 24A to the data memory cell 23-i, the SBCA signal is supplied to the buffer 73.

In this data transfer, the write bit line 63
Through one bit per cycle. Input S
The SIR signal used to read data from each input SAM cell 22-i of the AM unit 22 and the SWA signal used to write data to each data memory cell 23-i of the data memory unit 23 are the same. Addresses in the address space are shown, each being decoded by a row decoder and given as a word line.

The ALU cell 24A stores, in the data memory unit 23, the input data written in the above-described manner, the data being calculated, or the flip-flop 82.
Using the data stored in -1 to 82-3, the arithmetic processing in bit units is sequentially advanced.

For example, the data of the data memory cell 23-i corresponding to a predetermined bit of the data memory unit 23 and the data of the data memory cell 23-i corresponding to another bit are added, and further added to another bit. When writing the addition result to the corresponding data memory cell 23-i, the following operation is performed.

The program control unit 27A operates the data memory cell 23 corresponding to a predetermined bit of the data memory unit 23.
-I to supply the SRAA signal to turn on the transistor Tr13 of the cell to output the data stored in the capacitor C11 to one of the read bit lines 61.

At the same time, program control unit 27A stores SRBA in data memory cell 23-i corresponding to another bit.
A signal is supplied, the transistor Tr14 of the cell is turned on, and the data stored in the capacitor C11 is output to the other read bit line 62.

These two read data are AL
The data is supplied to the ALU 81 via the selectors 84-1 to 84-3 of the U cell 24A. Then, the ALU 81 performs a predetermined operation on the data, and supplies the operation result to the flip-flop 85 via the selector 83.

Then, the program control unit 27A
The CA signal is supplied, the operation result is output from the flip-flop 85 to the write bit line 63, and the SWA signal is
The data is supplied to the data memory cell 23-i corresponding to the predetermined bit, the transistor Tr11 of the cell 23-i is turned on, and the terminal voltage of the capacitor C11 is set to a voltage corresponding to the operation result.

The arithmetic operation in ALU cell 24A is performed according to an ALU control signal (SALU-CONT) supplied from program control unit 27A. ALU
The operation result in the cell 24A is written into the data memory unit 23 as described above, or is stored in the flip-flop 82-3 of the ALU cell 24A as necessary. When the operation in the ALU 81 is addition,
The ALU cell 24A outputs the carry in the operation result to the flip-flop 82-3 and outputs the sum to the data memory unit 23.

Next, when outputting data from the data memory cell 23-i, the program control unit 27A sets the data memory cell 23-i storing the data to be output.
And a memory access signal (SRAA or SRBA)
And the transistor Tr13 of the cell 23-i is supplied.
Alternatively, Tr14 is turned on to read the data stored in the capacitor C11 and read the bit line 61 or 6
Output to 2.

Then, the program control unit 27A
A predetermined control signal is supplied to the U cell 24A, and the data from the data memory cell 23-i is output to the output SAM cell 25-A.
i. At this time, the program control unit 27A
Outputs an SOW signal so that data is supplied to the capacitor C4 of the output SAM cell 25-i, turns on the transistor Tr17 of the cell, and changes the terminal voltage of the capacitor C4 according to the data. To voltage.

Data is transferred bit by bit via the write bit line 63. At this time, the ALU 81 may perform some processing on the data.

The SOW used when storing data in each output SAM cell 25-i of the output SAM unit 25 is used.
Signal and each data memory cell 23 of the data memory unit 23
Memory access signals (SRAA, SRBA) used when reading data from -i are addresses in the same address space, and are decoded by a row decoder and supplied via word lines.

In output SAM cell 25-i of element processor 31 designated by output pointer 26, transistor Tr8 is turned on in response to the output pointer signal, and an output signal corresponding to the potential of capacitor C4 is output to the output data bus. It is output to 66.

Then, the value of the output pointer 26 is "H".
Is output to the leftmost element processor 31.
To the rightmost element processor 31 in accordance with the clock signal.
From the output SAM cell 25-i.

In this manner, the output SAM cell 25-i
Is output to the output terminal DOUT via the output data bus 66.

Since the element processor 31 is provided with more than the number of pixels H in one horizontal scanning period of the image signal, by performing this operation, the data of the output image signal for one horizontal scanning period is output to the output SAM. Output from the unit 25. This output operation is repeated every horizontal scanning period.

As described above, each element processor 31
Performs processes such as data input, data transfer, calculation, and data output.

In the second embodiment, all the filter coefficient sets are supplied to the data memory units 23 of all the element processors 31 at the time of startup, during the horizontal retrace period or the vertical retrace period. . At this time, the filter coefficient set is supplied from the memory 28 to the input SAM unit 22 via a part (predetermined bit width) of the input data bus 65, and is transferred to the data memory unit 23. The operation at this time is the same as the operation of supplying the input data Ri to the data memory unit 23, which will be described next, and thus the description thereof is omitted.

Next, referring to the flowchart of FIG.
The operation of the second embodiment will be described.

First, in step S1, predetermined L-bit input data Ri (= 1) for one horizontal scanning period
{ _Ri0 , ..., _{ri (L-1)} }) are input to the input SAM unit 22.

In the case of performing (10/7) magnification of an image,
As described above, the positional relationship between the element processor 31 that holds input data necessary for calculating each output data and the element processor 31 that calculates the output data changes for each output data. For example, when calculating the output data of 10 pixels corresponding to the input data of 7 pixels, the pattern of the element processor 31 holding the input data necessary for the calculation of each output data is 5 as shown in FIG. Are classified into three patterns.

Therefore, at this time, as shown in FIG.
Any of the input data is overlapped, and the seven input data are densely supplied to the ten element processors 31. That is, among the element processors 31 shown in FIG. 34 to which input data is not supplied, the same input data as the element processor 31 on the left side is supplied.

This input data supply procedure may be realized in the same manner as the data input in the apparatus shown in FIG. 30, or may be performed by using a method described in Japanese Patent Application Laid-Open No. 8-123683. Alternatively, data may be supplied once so as to be in a sparse state, and then predetermined data may be copied according to a program.

In FIG. 8, the input data Ri and the output data Qi are actually about 8 bits, but are represented by 4 bits for convenience. Also,
In the input SAM unit 22, the data memory unit 23, and the output SAM unit 25, only the memory capacities necessary for explanation are shown.

By supplying the input data in this manner, as shown in FIG. 9, for example, in the case of the pattern of type 1 in FIG. 9, the element processor 31 adjacent to the left two elements and the element processor 31 adjacent to the left are used. The same input data is supplied, and the same input data is supplied to the element processor 31 on the right side and the element processor 31 on the two right side. Therefore, the pattern of type 1 in FIG. Can be handled in the same way as the pattern of

In the case of the type 3 pattern shown in FIG. 9, the same input data is supplied to the predetermined element processor 31 and the element processor 31 on the left side thereof, so that the type 3 pattern shown in FIG. It can be handled in the same way as the type 4 pattern.

Further, in the case of the pattern of type 5 in FIG. 9, the same input data is supplied to the element processor 31 on the right side and the element processor 31 on the right two sides. Can be handled in the same manner as the type 2 pattern.

Therefore, by supplying input data as shown in FIG. 8, the above-mentioned five patterns are reduced to two patterns (type 2 and type 4) shown in FIG.

In the case of a conversion ratio other than (10/7), by calculating in advance the input data supply method that minimizes the number of patterns described above, the positional relationship as described above is obtained. Can be degenerated.

Then, the program control unit 27A supplies a one-bit value (0 or 1) indicating a pattern corresponding to the element processor 31 to each of the two patterns together with the input data.

Next, in steps S2 to S5, the program control unit 27A
1 is supplied from the input SAM unit 22 to the data memory unit 2 via the write bit line 63.
3 is transferred one bit at a time.

In this case, for the sake of convenience, the input data Ri is 4
The bits are set in bits and are stored at addresses 0 to 4 of the input SAM unit 22. Therefore, as shown in FIG. 8, the contents of the address 0 of the input SAM unit 22 are transferred to the address 8 of the data memory unit 23, and similarly, the input SA
The contents of addresses 1 to 3 of the M section 22 are transferred to addresses 9 to 11 of the data memory section 23, respectively.

In step S6, each element processor 31 performs signal processing (interpolation calculation) described later.

At steps S7 to S10, the program control unit 27A
The calculation result (output data Qi) calculated in is read out from the data memory unit 23 by the read bit lines 61, 62,
Then, the output SAM unit 25 is transferred one bit at a time via the ALU cell 24A.

In this case, the output data Qi (= q _i0 ,...)
., Q _i3 ) is set to 4 bits for convenience, and stored in addresses 16 to 19 of the data memory unit 23. Therefore, as shown in FIG.
Are transferred to the address 20 of the output SAM unit 25, and similarly, the contents of the addresses 17 to 19 of the data memory unit 23 are transferred to the addresses 21 to 23 of the output SAM unit 25, respectively.

Next, in step S11, the calculated output data Qi for one horizontal scanning period is output to the output SAM unit 2.
5 is output.

As described above, the filter operation is performed for each image data for one horizontal scanning period. The operation of step S1, the operations of steps S2 to S10, and the operation of step S11 are performed in parallel, and the operations of steps S2 to S10 are performed on image data for one predetermined horizontal scanning period. When the operation is performed, the operation of step S11 is performed on the image data of one horizontal scanning period one line before, and the operation of step S1 is performed on the image data of one horizontal scanning period after one line. .

Next, the details of the signal processing in step S6 in FIG. 7 will be described with reference to the flowcharts in FIGS.

First, in step S21, each element processor 31 holds the supplied data and copies it to the element processor 31 on the left. In addition,
In the present embodiment, the element processor 31 is configured to be able to communicate only with the element processor immediately adjacent to the left and right or two elements adjacent to the left and right,
When performing an interpolation operation, the element processors 31 adjacent to the right three elements
Is used, the data is supplied to step S21.
, The data is stored in the element processor 31 on the left.
Is copied in advance.

Hereinafter, the data supplied to the predetermined element processor 31 is R ₀ , the data supplied to the left adjacent element processor 31 is R _−1, and the data supplied to the left two adjacent element processors 31 is R _−1. The obtained data is _defined as R- ₂ . Further, the data supplied to the right adjacent element processor 31 is R _{+ 1} , the data supplied to the right two adjacent element processors 31 is R _{+ 2} , and the three right adjacent element processors 31 are R _{+ 2.
Is} R _{+ 3} .

Next, in step S22, the element processor 31 outputs the data R _-1 of the element processor 31 on the left.
If, calculates the product of the filter coefficients FC1 being supplied advance, and substitutes the result to Y _1A. The operation of this product is
It is executed by performing a predetermined number of bit operations.

In the filter coefficient sets (FC1, FC2, FC3, FC4) used for the interpolation operation, the phase corresponding to the filter coefficient set is the one closest to the phase of the pixel processed by the element processor. Is selected.

As described above, a predetermined number of filter coefficient sets are prepared, and the same filter coefficient set corresponding to the phase of the pixel to be processed is used. In operation, an optimal set of filter coefficients will be selected.

In step S23, the element processor 31 calculates the product of the data R ₀ supplied thereto and the filter coefficient FC2, and substitutes the result of the calculation into Y _2A .

Then, in step S24, the element processor 31 calculates the sum of Y _1A and Y _2A and substitutes the result of the calculation into Y _1A . The calculation of the sum is performed by performing a predetermined number of bit calculations.

[0242] Next, in step S25, the element processor 31 has a data R ₊₂ element processors 31 of the right two neighboring calculates the product of the filter coefficients FC3, and substitutes the result to Y _2A.

Then, in step S26, the element processor 31 calculates the sum of Y _1A and Y _2A and substitutes the result of the calculation into Y _1A .

Next, in step S27, the element processor 31 outputs the data R of the three right next element processors 31 that the two right next element processors 31 have.
_+3, calculates the product of the filter coefficients FC4, and substitutes the result to Y _2A.

Then, in step S28, the element processor 31 calculates the sum of Y _1A and Y _2A and substitutes the result of the calculation into Y _1A . At this time, the value of Y _1A is R ₋₁ × F
C1 + _R0 * FC2 + R _{+ 2} * FC3 + R _{+ 3} * FC4, which corresponds to the type 2 pattern in FIG.

[0246] Next, in step S29, the element processor 31 has a data R _-2 element processors 31 of the left two neighboring, calculates the product of the filter coefficients FC1, and substitutes the result to Y _1B.

In step S30, the element processor 31 calculates the product of the data R ₀ supplied thereto and the filter coefficient FC2, and substitutes the result of the calculation into Y _2B .

Then, in step S31, the element processor 31 calculates the sum of Y _1B and Y _2B and substitutes the calculation result for Y _1B .

Next, in step S32, the element processor 31 sets the data R ₊₁ of the element processor 31 on the right.
, And the product of the filter coefficient FC3 and the result of the operation is substituted into Y _2B .

Then, in step S33, the element processor 31 calculates the sum of Y _1B and Y _2B and substitutes the calculation result into Y _1B .

Next, in step S34, the element processor 31 calculates the product of the data R ₊ 2 of the two adjacent right element processors 31 and the filter coefficient FC4, and substitutes the calculation result into Y _2B .

In step S35, the element processor 31 calculates the sum of Y _1B and Y _2B and calculates the result of the calculation as Y _1B
Substitute for At this time, the value of Y _1B is R ₋₂ × FC1 + R
₀ × FC2 + R _{+ 1} × FC3 + R _{+ 2} × FC4, and FIG.
0 corresponds to the type 4 pattern.

Then, in step S36, the element processor 31 supplies the input data Ri with
With reference to the value (0 or 1) indicating the above-described positional relationship, it is determined whether or not the value is a first value (a value corresponding to type 2 in FIG. 10). If it is determined that the value, the process proceeds to step S37, the Y _1A in step S28 the calculation result, a value indicating a positional relationship described above, when it is determined that it is not the first value (i.e., the type of FIG. 10 If the value is a value corresponding to 4), Y _1B in step S35 is set as the calculation result.

As described above, the filter operation (interpolation operation) is performed using the data of the neighboring element processors 31 corresponding to the two types of positional relationships. By selecting the filter coefficient set so that the phase error is minimized as described above, image processing at an arbitrary conversion ratio can be performed even in a SIMD parallel processor.

By providing a selector so that either the filter coefficient set or the input data from the memory 28 is supplied to the input SAM unit 22, for example, the input SAM unit such as a vertical blanking period is supplied. The filter coefficient set may be supplied in the same manner as the input data during a period when the input data 22 is not used for supplying the input data Ri.

By doing so, the filter coefficient can be supplied using the bus 65 having the same number of bits as the input data, so that the filter coefficient having a large number of bits (long word length) can be shortened. Can be supplied in time.

For example, when the number of bits of the filter coefficient is 10, the set of four filter coefficients is data of a total of 40 bits, and therefore, for example, the data memory section via the input data bus 65 and the input SAM section 22 of 16 bits. It is entirely possible to supply the filter coefficients to H.23 within the vertical retrace interval.

In the second embodiment, the filter coefficient set is supplied to the input SAM unit 22 in a pattern different from the input data Ri. In the case where filter coefficients are supplied in parallel with input data, two circuits for pointer control of the input SAM unit 22 are provided, and pointer control for input data Ri is performed.
Pointer control for filter coefficients is performed independently.

The input data supply procedure may be realized in the same manner as the data input in the apparatus shown in FIG. 30, or a method described in Japanese Patent Application Laid-Open No. 8-123683 may be used. Then, the data may be supplied once so as to be in a sparse state, and then predetermined data may be copied according to a program.

FIG. 13 shows the configuration of the third embodiment of the present invention. In the third embodiment, the memory 29 (storage means) holds data of a filter coefficient set corresponding to a predetermined number of divisions, similarly to the memory 28 of FIG. Also, the memory 29 is controlled by the program control unit 27B, and at startup, in a horizontal retrace period, a vertical retrace period, and the like, the filter coefficient set is used to calculate the value of a pixel having a phase corresponding to the filter coefficient set. The data is supplied to the data memory unit 23 via the ALU array unit 24 of the processor 31.

The memory 28A stores, for each element processor 31, a filter selection number i (ie, phase x) corresponding to the pixel phase calculated by the element processor 31.
Holds the number i) corresponding to the filter selection signal Pi of the filter coefficient set closest to the phase of the pixel to be interpolated. The filter selection number i is supplied to the data memory unit 23 together with the input data Ri via the input data bus 65, similarly to the filter coefficient set of the second embodiment.

It is assumed that the data stored in the memories 28A and 29 is supplied in advance by the external control CPU at the time of startup or the like.

The program control section 27B controls each section to perform the operation described later.

The other components are the same as those of the second embodiment, and the description thereof will be omitted.

Note that the filter selection number i held in the memory 28A is, for example, the value of the input data bus 6 at the time of startup.
5 and the data memory unit 23 via the input SAM unit 22
Is supplied in advance.

For example, if the number of divisions of the pixel interval (ie, the number of filter coefficient sets) is 16, the memory 28A
Suffices to store 16 filter selection numbers i corresponding to 16 types of phases, regardless of the number H of pixels in the horizontal direction. That is, when there are 16 filter selection numbers i, the filter selection numbers can be expressed by a 4-bit binary number, so that the memory 28A stores 4-bit data as the filter selection numbers i.

Even if there are 1,000 types of filter selection numbers i, they can be represented by a 10-bit binary number, so that the filter coefficient is input to the input SAM unit 22 as in the second embodiment. The capacity of the memory 28A can be reduced as compared with the case where the power is supplied via the memory 28A.

FIG. 14 shows a filter selection number i (=) stored in the data memory unit 23 of each element processor 31.
{Φ _i0 ,..., Φ _i3 }). FIG.
Of the 16 types of filter selection numbers, 10 types of filter selection numbers i (i =
0,..., 9) are stored as 4-bit data. For example, the data memory unit 23 of the element processor 31 whose number is 6 stores 4-bit data {φ ₃₀ ,..., Φ ₃₃ } whose filter selection number i is 3.

Next, the operation of each unit when the filter coefficient set is supplied to the data memory unit 23 of each element processor 31 in the third embodiment will be described with reference to the flowchart of FIG.

First, in step S41, the program control section 27B sets the value of the counter j for counting the filter selection number i corresponding to the supplied filter coefficient set to 0.

Next, in step S42, the program control unit 27B sets the value of the counter m used when supplying the value of the counter j in bit units to 1.

Then, the program control unit 27B outputs the value of the m-th bit of the value of the counter j to the ALU cells 24A of all the element processors 31, and the ALU cell 24A of each element processor 31 Receives the value of

In step S44, the program control unit 27B determines whether or not the value of the counter m is equal to or greater than the bit length of the counter j.
Is determined to be smaller than the bit length of
After the value of the counter m is increased by 1, the flow returns to step S43 to supply the next bit.

As described above, the value of the counter j is supplied to each element processor 31 bit by bit.

On the other hand, in step S44, the counter m
Is determined to be greater than or equal to the bit length of the counter j, it means that the value of the counter j has been supplied. In step S46, each element processor 31 stores the value of the received counter j in the memory It is determined whether or not the value of the filter selection number i supplied from 28A is the same,
If they are the same, for example, a flag is set according to the determination, and the process proceeds to step S47.

In step S47, each element processor 31 sets the value of the counter k for counting the number of bits of the supplied filter coefficient set to 1 in accordance with the flag.

In step S48, each element processor 31 stores the value of the k-th bit of the filter coefficient set output from the memory 29 in the ALU cell 24.
A, and the data is stored in the data memory unit 23.

In the memory 29, a filter coefficient set corresponding to each phase (ie, filter selection number i) is stored for each coefficient in order from the most significant bit (MSB) or the least significant bit (LSB). And the filter coefficient set is transmitted via the 1-bit line as described above.
The bits are sequentially output to the ALU cell 24A of the element processor 31 bit by bit.

[0279] In step S49, each element processor 31 determines whether the value of the counter k is equal to or greater than the bit length of the filter coefficient set, and determines that the value of the counter k is smaller than the bit length of the filter coefficient set. If it is determined that the value of the counter k is 1 in step S50.
Then, the process returns to step S48 to receive the next bit of the filter coefficient set.

On the other hand, in step S49, the counter k
Is determined to be greater than or equal to the bit length of the filter coefficient set, it means that the supply of the filter coefficient set corresponding to the value of the counter j has been completed.
Proceed to.

On the other hand, in step S46, when the element processor 31 determines that the value of the counter j is not the same as the value of the filter selection number i supplied from the memory 28A in advance (that is, when the flag is not set) ), The element processor 31 does not receive the filter coefficient set output from the memory 29, that is, does not store it in the data memory unit 23, and skips steps S47 to S50. In practice, it is difficult to skip processing in SIMD control.
1 performs processing so that the same result as in the case where steps S47 to S50 are skipped is obtained.

Next, in step S51, the program control unit 27B determines whether or not the value of the counter j is equal to or greater than a value obtained by subtracting 1 from the number N of the filter coefficient sets. When it is determined that the value is equal to or greater than the value obtained by subtracting 1 from the number N of the filter coefficient sets (j ≧ N−1), one of the N filter coefficient sets has been supplied to each element processor 31. Therefore, the process of supplying the filter coefficient set ends.

On the other hand, when the program control unit 27B determines that the value of the counter j is smaller than the value obtained by subtracting 1 from the number N of the filter coefficient sets (j <N−1), in step S52, the value of the counter j is determined. The value is increased by 1 and the process returns to step S42 to supply a filter coefficient set corresponding to the next filter selection number i.

Thus, each element processor 31
Receives the filter coefficient corresponding to the filter selection number i supplied in advance from the memory 29 and stores it in the data memory unit 23.

As described above, by supplying the filter coefficient set through a path different from the input data Ri, it is necessary to use a number of program steps to selectively supply the filter coefficient set to the element processor 31. And can be easily realized.

When any one of, for example, 16 types of filter coefficient sets stored in the memory 29 is supplied to each of the element processors 31, one filter coefficient set is set to about 1/16 of all the element processors 31. Are simultaneously supplied to the element processors 31, and when the filter coefficient set is, for example, 40 bits, regardless of the number of element processors 31, the filter operation is performed on all the element processors 31 by the operation of 640 (= 40 × 16) steps. A set of coefficients can be provided.

Since the operation when processing image data is the same as that of the second embodiment, the description is omitted.

As described above, in the third embodiment, the filter coefficient set is supplied through a different path from the input data, so that the filter coefficient set is supplied regardless of the operation status of the input SAM unit 22. be able to.

In the third embodiment, when changing the conversion ratio, the filter selection number i corresponding to each element processor 31 stored in the memory 28A may be changed.

Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, each element processor 31 calculates a filter coefficient set in each element processor 31 corresponding to the filter selection number i.

Note that the configuration of the fourth embodiment and the operation at the time of filter operation are the same as those of the second embodiment, and a description thereof will be omitted. However, the memory 28
It is assumed that the filter selection number i is stored in the same manner as in the memory 28A of the embodiment.

Therefore, when changing the conversion ratio, the filter selection number i corresponding to each element processor 31 stored in the memory 28 may be changed.

Next, the operation of each unit when calculating the filter coefficient set in the fourth embodiment will be described with reference to the flowcharts of FIGS. Here, a filter coefficient set in the Cubic approximation method of Expression (4) is calculated. Of course, a filter coefficient set in another approximation method may be calculated.

First, in step S61, the element processor 31 sets the image conversion ratio to K / L times,
Than previously supplied by that filter selection number i and K, and calculates the phase i / K pixel for calculating the value, stored as X _0. Note that K and L are supplied from the program control unit 27A.

Next, the element processor 31 determines in step S
In step 62, X ₀ is substituted for X. In step S63, the square of X (X × X) is calculated, and the calculation result is expressed as X
Remember as ₂ .

In step S64, the element processor 31 calculates the product of X ₂ and X (that is, the cube of X), and stores the calculation result as X ₃ .

Then, in step S65, the element processor 31 calculates the filter coefficient FC3 from X, X ₂ and X ₃ using the equation (4) according to the following equation. _{FC3 = -X 3 + 5X 2 -8X} + 4 (5)

Next, in step S66, the element processor 31 substitutes X for a value obtained by adding 1 to X ₀ (= i / K).

Then, in step S67, the element processor 31 calculates the square of X (X × X) and substitutes the calculation result in X _{2. In} step S68, the product of X ₂ and X (ie, X raised to the third power), and the result of the calculation is X
Substitute for ₃ .

[0300] In step S69, the element processor 31 obtains X, X ₂ and X ₃ by using the equation (4).
The filter coefficient FC4 is calculated according to the following equation. FC4 = X ₃ -2X ₂ +1 (6)

Next, in step S70, the element processor 31 substitutes a value obtained by subtracting X ₀ from 1 into X.

Then, in step S71, the element processor 31 calculates the square of X (X × X), substitutes the calculation result for X ₂ , and in step S72, calculates the product of X ₂ and X (ie, X raised to the third power), and the result of the calculation is X
Substitute for ₃ .

In step S73, the element processor 31 obtains X, X ₂ and X ₃ from equation (4) using
The filter coefficient FC2 is calculated according to the following equation. _{FC2 = -X 3 + 5X 2 -8X} + 4 (7)

Next, in step S74, the element processor 31 calculates a value (2-i / K) obtained by adding 1 to X, and substitutes the calculation result for X.

Then, in step S75, the element processor 31 calculates the square of X (X × X), substitutes the calculation result for X ₂ , and in step S76, calculates the product of X ₂ and X (ie, X raised to the third power), and the result of the calculation is X
Substitute for ₃ .

In step S77, the element processor 31 obtains X, X ₂ and X ₃ using the equation (4).
The filter coefficient FC1 is calculated according to the following equation. _{FC1 = X 3 -2X 2 +1 (} 8)

As described above, in the fourth embodiment, the filter coefficient sets (FC1, FC2, F
C3, FC4) are calculated.

As described above, by calculating the filter coefficient set by each of the element processors 31, there is no need to supply the filter coefficient set from memories (such as the memories 28 and 29) external to the element processor 31. There is no need to consider supply timing and the like.

FIG. 18 shows the configuration of the fifth embodiment of the present invention.

In the fifth embodiment, the memory 28A of the third embodiment is removed, and the filter selection number i
Is calculated by each element processor 31.

[0311] The program of the program control unit 27C has been modified so as to perform the operation described later.
The other components, the operation at the time of supplying the filter coefficient, and the operation at the time of the filter operation are the same as those of the third embodiment, and thus the description thereof is omitted.

Next, the operation for calculating the filter selection number i will be described with reference to the flowchart of FIG.

First, in step S81, the element processor 31 sets the registers ZA ₀ , ZB ₀ ,
ZC ₀ is secured respectively.

Next, in step S82, each element processor 31 substitutes zero for ZA ₀ , ZB ₀ , and ZC ₀ respectively.

In step S83, each element processor 31 determines the value ZA ₀ of the element processor 31 on the left side.
The sum of A _-1 and L when the conversion ratio is K / L times (that is, K: L) is calculated, and the calculation result is stored in ZA ₀ . Since the leftmost element processor 31 has no element processor 31 on the left side, the calculation is performed with ZA- _{1 set} to 0.

In step S84, each element processor 31 determines whether or not the value of ZA ₀ is greater than K.
If it is determined that the value of ZA ₀ is larger than K,
At 85, the remainder when the value of ZA ₀ is divided by K is calculated, and the calculation result is substituted into ZA ₀ .

On the other hand, when each element processor 31 determines that the value of ZA ₀ is equal to or smaller than K, it skips step S85. In practice, it is difficult to skip the process in the SIMD control, so the element processor 31 performs the process so as to obtain the same result as when step S85 is skipped.

Then, in step S86, each element processor 31 executes steps S83 to S85
Is determined to have been repeated more times than the number of pixels in the horizontal direction of the image format currently being handled, and the operations of steps S83 to S85 are repeated only for the number of pixels in the horizontal direction of the currently handled image format. If it is determined that there is not, the process returns to step S83, and the operations of steps S83 to S85 are performed again.

On the other hand, when each element processor 31 determines that the operations of steps S83 to S85 have been repeated more times than the number of pixels in the horizontal direction of the image format currently being handled, the process proceeds to step S87.

[0320] In step S87, each element processor 31 determines the value ZB ₀ of the element processor 31 on the left.
The sum of B _-1 and L is calculated, and the calculation result is stored in ZC ₀ . Note that the leftmost element processor 31 does not have the element processor 31 on the left side, and therefore performs the operation with ZB ₋₁ as 0.

[0321] Next, in step S88, the element processor 31 determines whether the value of the ZC ₀ is greater than 2 times the value of K, determines the value of the ZC ₀ is greater than 2 times the value of K If you, at step S90, and substitutes a value obtained by subtracting K from the value of ZB ₀ to ZB _0.

On the other hand, if the element processor 31 determines that the value of ZC ₀ is equal to or less than twice the value of K, the process proceeds to step S.
In 89 substitutes the value obtained by subtracting K from the values of ZC ₀ to ZB _0.

Then, in step S91, each element processor 31 executes steps S87 to S90.
Is determined to have been repeated more times than the number of pixels in the horizontal direction of the currently handled image format, and the operations of steps S87 to S90 are repeated only for the number of pixels in the horizontal direction of the currently handled image format. If it is determined that there is not, the process returns to step S87, and the operations of steps S87 to S90 are performed again.

On the other hand, when each element processor 31 determines that the operations of steps S87 to S90 have been repeated more times than the number of pixels in the horizontal direction of the currently handled image format, the process proceeds to step S92.

In step S92, each element processor 31 determines whether or not K is larger than L, that is, whether or not the process is for image enlargement. If it is determined that K is larger than L, the process proceeds to step S94. If the value of ZA ₀ is used as the filter selection number i and it is determined that K is equal to or less than L, the value of ZB ₀ is used as the filter selection number i in step S93.

The filter selection number i is calculated as described above. Although division (calculation of remainder) is performed in step S85, subtraction is repeatedly performed in practice. Although the above-described processing has many processing steps, there is no particular problem if the processing is performed before performing the real-time processing or during the vertical blanking period.

Steps S84 and S8
8, corresponding to the input data or output data and the element processor 31 (Ri in FIG. 34).
May be set. That is, in step S85, the same processing as the above-described modulo operation of the phase is performed, so that the element processor 31 in which the modulo operation has occurred corresponds to the determination in step S84.
Is determined to be a place where there is no input data in FIG.

FIG. 20 shows the configuration of the sixth embodiment of the present invention.

In the sixth embodiment, a filter selection number i
And the corresponding filter coefficient sets are calculated by each element processor 31 in the same manner as in the fourth or fifth embodiment, so that the memories 28, 28
A and 29 are unnecessary.

[0330] The program control unit 27D controls each element processor 31, and, like the program control unit of the fourth or fifth embodiment, stores the filter selection number i and the filter coefficient set corresponding thereto. It is made to calculate.

Since the other components are the same as those of the fifth embodiment, the description will not be repeated. The operation when calculating the filter coefficient set is the same as that of the fourth embodiment, and the other operation is the same as that of the fifth embodiment.

In the above embodiment, when the number of divisions is increased in order to reduce the phase error, the processing can be simplified by performing the filter operation in two stages.

For example, instead of changing the Cubic approximation having the division number of 16 to the Cubic approximation having the division number of 256, first, the Cubic approximation having the division number of 16 is performed, and the interpolated value which is the operation result is obtained. By performing, for example, a bilinear approximation of 16 divisions from two points in the vicinity of, the entire interpolation process is simplified.

The pixel data is usually composed of luminance data including luminance information and color data including color information. The interpolation process may be performed by a number.

[0335] In the above embodiment, the enlargement of an image is mainly described, but it is of course possible to reduce the image. In the case of image reduction, the input S
The input data is supplied densely to the AM unit 22 in order, and the output data is sparsely output to the output SAM unit 25.

In the element processor 31 near the end (right end, left end) of the above-described embodiment, there is a case where there is no peripheral element processor 31 having input data used for calculation. , With the value of the input data set to 0.

In addition, there are various methods for processing at the edge of the image, for example, assuming that the data at the edge is continuous outside the edge or that the data is symmetric about the edge. Can be considered. Of these methods, the method can be realized by writing a program corresponding to a predetermined method.

The capacity of the above-mentioned memories 28, 28A and 29 depends on the type of the phase of the pixel and the numerator or denominator of the fraction corresponding to the conversion ratio, and is not so large. It doesn't grow.

[0339]

As described above, according to the image processing apparatus according to the first aspect and the image processing method according to the third aspect,
From a storage unit storing a filter coefficient set corresponding to each phase when the pixel interval of the original image is divided by a predetermined division number, a filter coefficient set whose phase is closest to the phase of the pixel data to be interpolated Is output to the calculation unit, and the calculation unit performs the interpolation calculation of the pixel data using the filter coefficient set, so that the image with an arbitrary conversion ratio can be enlarged or reduced.

According to the image processing apparatus of the fourth aspect and the image processing method of the nineteenth aspect, a filter coefficient set corresponding to each phase when a pixel interval of an original image is divided by a predetermined number of divisions The filter coefficient set whose phase is closest to the phase of the pixel data to be processed is supplied to each of the element processors, and each of the element processors performs the pixel data interpolation processing using the filter coefficient set. Therefore, an image having an arbitrary conversion ratio can be enlarged or reduced using a parallel processor of the SIMD format.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of a correspondence relationship between a filter selection signal Pi and a phase amount x, and a filter coefficient set;

FIG. 3 shows an interpolation value Qi and a filter selection signal Pi in each cycle of a filter operation performed in the apparatus of FIG.
It is a figure which shows an example of the correspondence of.

FIG. 4 is a block diagram illustrating a configuration of an image processing apparatus according to a second embodiment of the present invention.

FIG. 5 is a block diagram illustrating a configuration example of an element processor.

FIG. 6 is a circuit diagram illustrating a detailed configuration example of an element processor.

FIG. 7 is a flowchart illustrating an operation of the image processing apparatus of FIG. 4;

8 is a diagram illustrating an example of data stored in each unit of the image processing apparatus in FIG.

FIG. 9 is a diagram illustrating an example of a positional relationship with an element processor having data necessary for processing.

FIG. 10 is a diagram illustrating an example of a positional relationship in which the positional relationship of FIG. 9 is degenerated.

FIG. 11 is a flowchart illustrating a filter calculation process in the image processing apparatus of FIG. 4;

FIG. 12 is a flowchart illustrating a filter calculation process in the image processing apparatus in FIG. 4;

FIG. 13 is a block diagram illustrating a configuration of an image processing apparatus according to a third embodiment of the present invention.

FIG. 14 is a diagram illustrating an example of a filter selection number stored in a data memory unit.

FIG. 13 when supplying a filter coefficient set;
5 is a flowchart for explaining the operation of the image processing apparatus of FIG.

FIG. 16 is a flowchart illustrating an operation when each element processor calculates a filter coefficient set in the fourth embodiment.

FIG. 17 is a flowchart illustrating an operation when each element processor calculates a filter coefficient set in the fourth embodiment.

FIG. 18 is a block diagram illustrating a configuration of an image processing apparatus according to a fifth embodiment of the present invention.

FIG. 19 is a flowchart illustrating an operation of the image processing apparatus of FIG. 18 when each element processor calculates a filter selection number.

FIG. 20 is a block diagram illustrating a configuration of an image processing apparatus according to a sixth embodiment of the present invention.

FIG. 21 is a diagram illustrating an example of an original image.

FIG. 22 is a diagram illustrating an example of an image obtained by enlarging an original image.

FIG. 23 is a diagram illustrating an example of a positional relationship between pixels of an original image and pixels of an enlarged image.

FIG. 24 is a diagram illustrating an example of an image in which the resolution of an original image is increased.

FIG. 25 is a diagram illustrating an example of an image obtained by reducing an original image.

FIG. 26 is a diagram illustrating an example of a positional relationship between pixels of an original image and pixels of a reduced image.

FIG. 27 is a diagram illustrating an example of an image in which the resolution of an original image is reduced.

FIG. 28 is a diagram illustrating an example of a positional relationship between pixels of an original image and pixels generated by interpolation.

FIG. 29 is a diagram illustrating an example of an interpolation function.

FIG. 30 is a block diagram illustrating a configuration example of a device that performs a filter operation in hardware.

31 is a diagram illustrating an example of a signal of each unit in each cycle of a filter operation performed in the device in FIG. 30.

FIG. 32 is a diagram illustrating an example of a correspondence relationship between a filter selection signal and a filter coefficient set.

FIG. 33 is a block diagram illustrating a configuration example of a device that performs a filter operation using software.

34 is a diagram illustrating an example of a supply pattern of input data when an image is enlarged in the apparatus in FIG. 33.

FIG. 35 is a diagram illustrating an example of a positional relationship with an element processor having data necessary for processing.

[Explanation of symbols]

5 control circuits, 11 remainder circuits, 12 registers,
13 approximate circuit, 21 input pointer, 22 input SAM part, 23 data memory part, 24 ALU array part, 25 output SAM part, 26 output pointer,
27, 27A, 27B, 27C, 27D Program control unit, 28, 28A, 29 Memory

 ────────────────────────────────────────────────── ─── Continued from the front page (72) Inventor Kenichiro Nakamura 6-7-35 Kita Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation

Claims (19)

[Claims] 1. An arithmetic unit for performing an interpolation operation of pixel data accompanying enlargement or reduction of an image by hardware corresponding to the interpolation operation, and a storage unit for storing a filter coefficient set used for the interpolation operation. In the image processing apparatus provided, the storage unit stores a filter coefficient set corresponding to each phase when the pixel interval of the original image is divided by a predetermined number of divisions, among the filter coefficients, the phase, the phase, An image, wherein a filter coefficient set closest to the phase of the pixel data to be subjected to the interpolation calculation is output to the calculation unit, and the calculation unit performs the interpolation calculation of the pixel data using the filter coefficient set. Processing equipment. 2. The image processing apparatus according to claim 1, wherein the number of divisions is a power of two. 3. An operation unit for performing an interpolation operation on pixel data accompanying enlargement or reduction of an image by hardware corresponding to the interpolation operation, and a storage unit for storing a filter coefficient set used for the interpolation operation. In the image processing method in the image processing apparatus provided with, from the storage unit that stores a filter coefficient set corresponding to each phase when the pixel interval of the original image is divided by a predetermined division number, the phase is the interpolation Image processing, wherein a filter coefficient set closest to the phase of the pixel data to be calculated is output to the calculation unit, and the calculation unit performs an interpolation calculation of the pixel data using the filter coefficient set. Method. 4. A method for supplying pixel data to a plurality of element processors, performing SIMD control on the plurality of element processors,
An image processing apparatus in which each element processor uses pixel data of peripheral element processors to perform pixel data interpolation processing in parallel with image enlargement or reduction in parallel. Of the filter coefficient sets corresponding to each phase when the interval is divided, the phase supplies a filter coefficient set closest to the phase of the pixel data to be processed, to the element processor, respectively. An image processing apparatus, wherein each of the pixel data interpolation processes is performed using a filter coefficient set. 5. The image processing apparatus according to claim 4, wherein the number of divisions is a power of two. 6. The image processing apparatus according to claim 4, wherein the element processor is a one-bit processor that processes data one bit at a time. 7. The pixel data is supplied to the plurality of element processors such that a type of a pattern of a positional relationship between the peripheral element processor and the predetermined element processor is minimized. Item 5. The image processing device according to Item 4. 8. The filter coefficient set is supplied to the element processor via a circuit used when the pixel data is supplied.
An image processing apparatus according to claim 1. 9. The image processing apparatus according to claim 8, further comprising a storage connected to the element processor and storing the filter coefficient set. 10. The image processing apparatus according to claim 9, wherein the storage unit stores the filter coefficient set according to a phase order corresponding to the filter coefficient set. 11. The element processor includes a storage unit for storing the filter coefficient set, and an ALU unit for performing an operation, wherein the filter coefficient set corresponding to phase information of pixel data assigned to each element processor is stored in the element processor. The image processing apparatus according to claim 4, wherein the image processing apparatus is supplied to the storage unit via an ALU unit. 12. The image processing apparatus according to claim 11, wherein said element processors each calculate said phase information. 13. A storage unit connected to the ALU unit for storing the filter coefficient set, wherein the storage unit stores the filter coefficient set in accordance with an order of phases corresponding to the filter coefficient set. The image processing apparatus according to claim 11, wherein: 14. The image according to claim 4, wherein the element processor calculates a filter coefficient set used for interpolation in accordance with the phase information of the pixel data assigned to the element processor. Processing equipment. 15. The image processing apparatus according to claim 13, wherein said element processors each calculate said phase information. 16. The image processing apparatus according to claim 4, wherein the interpolation operation is an operation corresponding to Cubic approximation. 17. The method according to claim 1, wherein after performing a first interpolation operation with a first number of divisions according to a first interpolation method, a second interpolation method is performed on the calculation result of the first interpolation operation. The image processing apparatus according to claim 4, wherein the second interpolation calculation is performed with the second division number according to the following equation. 18. The pixel data is composed of luminance data and color data, and the color data has a smaller number of divisions than the number of divisions corresponding to a filter coefficient set used when performing the interpolation of the luminance data. The image processing apparatus according to claim 4, wherein interpolation is performed using a filter coefficient set corresponding to each phase when the pixel interval of the original image is divided. 19. Pixel data is supplied to a plurality of element processors, and the plurality of element processors are subjected to SIMD control. Each of the element processors utilizes pixel data of peripheral element processors to enlarge or reduce an image. In an image processing method of performing pixel data interpolation processing in parallel with the above, the phase of the filter coefficient set corresponding to each phase when the pixel interval of the original image is divided by a predetermined number of divisions is processed. A filter coefficient set closest to the phase of the pixel data is supplied to each of the element processors, and the element processor performs the interpolation processing of the pixel data using the filter coefficient set. Image processing method.