JP7292903B2 - Image processing device and image processing method - Google Patents

Description

The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing technique for performing filter operation processing on two-dimensional data.

For example, in the field of image processing, a two-dimensional data operation such as a two-dimensional convolution filter operation (convolution operation) is frequently used because an image has a two-dimensional pixel array. FIG. 14 is a diagram for explaining an example of convolution filter calculation, which is one of two-dimensional data calculations. In FIG. 14, (a) shows a target image 1401 for filter operation, (b) shows a filter kernel 1402 used for filter operation, and (c) shows an operation output image 1403 which is the result of filter operation. showing.

FIG. 14 shows an example of a 3×3 filter kernel 1402 in which filter coefficients are arranged two-dimensionally. be.

Here, “D _i,j ” indicates the pixel value at the coordinates (j, i) of the operation target image, and “R _i,j ” indicates the filter operation result at the coordinates (j, i) of the operation output image. show. "W _s,t " indicates the value of the filter kernel (filter coefficient value) applied to the pixel value at the coordinates (j+t, i+s) of the calculation target image. "columnSize" indicates the horizontal size of the filter kernel, and "rowSize" indicates the vertical size of the filter kernel.

By performing the above-described calculation while scanning the filter kernel 1402 in the calculation target image 1401, a calculation output image 1403 of the convolution filter calculation can be obtained. At this time, if the size of the calculation target image 1401, which is the original image, is vertical size A×horizontal size B, the size of the calculation output image 1403, which is the result of the filter calculation, is (A−rowSize+1)×(B−columnSize+1). becomes.

For such calculations, in Patent Documents 1 and 2, a plurality of calculators such as sum-of-products calculators are prepared, and the pixel data of the calculation target image supplied to the calculators are processed by the plurality of calculators. shared between them to perform parallel processing. As a result, it is attempted to achieve high-speed arithmetic processing and efficient use of pixel data of the arithmetic target image.

JP-A-2004-13873 JP 2010-134697 A

However, if an attempt is made to increase the number of arithmetic units capable of performing arithmetic processing in parallel using the methods described in Patent Documents 1 and 2, the parallel arithmetic is extended in one direction (for example, the horizontal direction) with respect to the image to be arithmetically operated. . In the configurations described in Patent Documents 1 and 2, once the "number of sum-of-products computing units" is determined, the "distribution of filter computation output pixels computed in parallel" is uniquely determined. For example, when the number of sum-of-products calculators is eight, eight pixels arranged one-dimensionally are calculated in parallel. Therefore, the following problems occur.

First, when the image to be computed is small, there arises a problem that parallel computation cannot be effectively utilized. For example, if the horizontal size of the target image is smaller than the number of multiply-accumulate calculators prepared for parallel calculation, some multiply-accumulator calculators that do not contribute to the filter calculation will occur, resulting in lower calculation efficiency. It is conceivable that it will lead to a decline.

Conversely, if you try to increase the degree of parallelism (the number of multiply-accumulate calculators prepared for parallel calculation) in order to improve the calculation efficiency, for example, the horizontal size of the calculation target image will be exceeded. Parallelism cannot be increased. In other words, even if the number of calculators is increased any further, there arises a problem that the calculation efficiency is not improved.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and an object of the present invention is to enable a filter operation to be performed on an operation target image of any size with good operation efficiency.

An image processing apparatus according to the present invention is an image processing apparatus for performing filter arithmetic processing by scanning a filter kernel with respect to pixel data of an image stored in an image storage means, wherein the filter arithmetic processing is performed in parallel. a plurality of temporary storage means for temporarily storing the plurality of pixel data read from the image storage means, the filter coefficients in the filter kernel; Filter operation processing is performed in parallel using the plurality of pixel data stored in the plurality of temporary storage means, and one or more operation groups are formed according to the arrangement of pixels in which the filter operation processing is performed in parallel. and controlling transfer of the pixel data used for the filtering operation between the plurality of arithmetic means grouped into a plurality of temporary storage means , wherein the plurality of arithmetic means are grouped into a plurality of arithmetic groups. In the first transfer mode, the pixel data are transferred so as to be used by the arithmetic means belonging to the same arithmetic group, and in the second transfer mode, the pixel data are transferred so as to be used by the arithmetic means belonging to another arithmetic group. and transfer control means for controlling to transfer the pixel data .

According to the present invention, it is possible to perform filter calculation with good calculation efficiency on a calculation target image of any size.

1 is a diagram illustrating a configuration example of an image processing apparatus according to a first embodiment; FIG. It is a figure explaining the example of control of the parallelization shape in this embodiment. It is a figure which shows the structural example of the arithmetic processing part in 1st Embodiment. 4 is a flowchart showing an example of arithmetic processing in the first embodiment; 4 is a time chart showing an example of arithmetic processing in the first embodiment; 4 is a time chart showing an example of arithmetic processing in the first embodiment; 4 is a time chart showing an example of arithmetic processing in the first embodiment; FIG. 10 is a diagram illustrating a configuration example of an image processing apparatus according to a second embodiment; FIG. It is a figure which shows the structural example of the arithmetic processing part in 2nd Embodiment. 9 is a flowchart showing an example of arithmetic processing in the second embodiment; 9 is a time chart showing an example of arithmetic processing in the second embodiment; 9 is a time chart showing an example of arithmetic processing in the second embodiment; FIG. 12 is a diagram illustrating a configuration example of an image processing apparatus according to a third embodiment; FIG. It is a figure explaining a convolution filter operation.

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings.

(First embodiment)
A first embodiment of the present invention will be described. In the image processing apparatus according to the first embodiment, when performing filter computation processing such as convolution filter computation in parallel, the arrangement of filter computation output pixels calculated in parallel according to the shape of the computation target region of the filter computation is to be able to change In this way, it is possible to change the arrangement of the filter operation output pixels to be processed in parallel according to the shape of the area subject to the filter operation, so that the efficiency of the parallel operation does not decrease even if the shape of the area subject to the operation changes. do.

Here, the arrangement of filter operation output pixels indicates the arrangement of output pixels that are simultaneously output in parallel operation. For example, when the image processing apparatus in this embodiment has eight arithmetic units (eight parallel units), it is possible to simultaneously output eight pixels as filter arithmetic outputs. Further, whether the arrangement of the 8 pixels is 1 pixel in the vertical direction and 8 pixels in the horizontal direction, 2 pixels in the vertical direction and 4 pixels in the horizontal direction, or 4 pixels in the vertical direction and 2 pixels in the horizontal direction. Whether it is a pixel or not can be switched by setting. Hereinafter, the arrangement of filter operation output pixels is also referred to as “parallelized shape”. A region of M pixels in the vertical direction and N pixels in the horizontal direction is expressed as “M×N”.

An image processing apparatus having eight computing units will be described below as an example, but the number of computing units included in the image processing apparatus is not limited to this, and may be any number. Also, the array of output pixels (parallelized shape) can be 1×8 with 1 pixel in the vertical direction and 8 pixels in the horizontal direction, 2×4 with 2 pixels in the vertical direction and 4 pixels in the horizontal direction, and 2×4 with 2 pixels in the vertical direction and 4 pixels in the horizontal direction. An example of switching between 4×2 with 4 pixels in the direction and 2 pixels in the horizontal direction is shown. However, the arrangement of switchable output pixels (parallelized shape) is not limited to this, and can be appropriately set according to the number of arithmetic units.

In the image processing apparatus according to the present embodiment, a plurality of computing units are grouped according to the arrangement of output pixels (parallelized shape), and the computing units are composed of computing units for computing the same horizontal row of output pixels. Set the group as the same operation group. Then, it controls the transfer of calculation target data within each calculation group or between calculation groups. For example, when the parallelization configuration is 1×8, a single arithmetic group consisting of eight arithmetic units is set. Similarly, when the parallelization shape is 2×4, two operation groups each consisting of four operation units are set, and when the parallelization shape is 4×2, two operation groups are set. We set up four operation groups consisting of units. Therefore, setting an operation group is equivalent to selecting a parallelization shape.

FIG. 1 is a block diagram showing a configuration example of an image processing apparatus 101 according to the first embodiment. The image processing apparatus 101 has a shape control section 102 , a transfer control section 103 , a filter coefficient storage section 104 , an image data storage section 105 and an arithmetic processing section 106 .

The shape control unit 102 sets groups of calculators (calculation groups) based on the input setting information S1, and determines the arrangement of output pixels (parallelized shape) output by parallel calculation. The setting information S1 includes information about the size of the calculation target image, the size of the filter kernel, and the number of calculators (parallel number) of the calculation processing unit 106 . In the present embodiment, it is important to reduce the number of computing units that do not contribute to the filter computation as much as possible so as not to reduce the computation efficiency of the parallel computation even if the shape of the computation target area of the filter computation changes. Therefore, the shape control unit 102 determines, from the input setting information S1, the arrangement of output pixels (parallelized shape) that allows parallel computation with good computational efficiency.

How the shape control unit 102 determines the parallelized shape from the setting information S1 will be described with reference to FIG. FIG. 2 is a diagram for explaining an example of parallelized shape control in this embodiment. In FIG. 2, (a), (b), and (c) show images 201, 202, and 203 to be filtered. In FIG. 2, (d) shows the output pixel 204 when the parallelization shape is 1×8, and (e) shows the output pixel 205 when the parallelization shape is 2×4. and (f) shows the output pixel 206 when the parallelized shape is 4×2.

The horizontal size of the calculation target image 201 shown in FIG. 2(a) is 10, the horizontal size of the calculation target image 202 shown in FIG. 2(b) is 6, and the calculation target image shown in FIG. The horizontal size of 203 is four. Here, in this embodiment, the size of the filter kernel is 3×3, which is the same as the filter kernel 1402 shown in FIG. ) is set to 8.

When the target image of the filter computation is the target image 201, there is no computation unit that does not contribute to the filter computation regardless of which parallelized shape that outputs the output pixels 204, 205, and 206 is selected. When the target image for filter computation is the target image 202 for computation, if the parallelized shape outputs the output pixels 205 and 206, there is no calculator that does not contribute to the filter computation. Further, when the target image of the filter computation is the target image 203, if the parallelized shape outputs the output pixel 206, there is no computing unit that does not contribute to the filter computation.

Therefore, when the target image for filter computation is the target image 203, the shape control unit 102 selects 4×2, which outputs the output pixels 206, as the parallelization shape. Note that when the target images of the filter computation are the target images 201 and 202, there are a plurality of parallelized shape options. When a plurality of parallelized shapes can be selected in this manner, the shape control unit 102 selects a parallelized shape that shortens the computation processing time of the filter computation. The computation processing time of the filter computation can be obtained based on the parallelized shape, the size of the computation target image, the size of the filter kernel, and the like.

The transfer control unit 103 reads out the filter coefficient S2 and the pixel data S3 from the filter coefficient storage unit 104 and the image data storage unit 105, respectively, and controls the supply to the arithmetic processing unit . Further, the transfer control unit 103 controls the transfer of pixel data in the arithmetic processing unit 106 by outputting a transfer control signal S4 to the arithmetic processing unit 106 . Data readout from the filter coefficient storage unit 104 and the image data storage unit 105 and control of pixel data transfer in the arithmetic processing unit 106 performed by the transfer control unit 103 are parallelized as determined by the shape control unit 102. It is done based on the shape.

The filter coefficient storage unit 104 stores values of filter kernels (filter coefficients) used in filter calculation. The image data storage unit 105 stores pixel data of an image to be filtered. The image data storage unit 105 is an example of image storage means. The calculation processing unit 106 has a plurality of calculators capable of executing calculations in parallel, and performs filter calculation using the input filter coefficient S2 and pixel data S3.

FIG. 3 is a block diagram showing a configuration example of the arithmetic processing unit 106. As shown in FIG. The internal configuration of the arithmetic processing unit 106 and the connection control in the arithmetic processing unit 106 based on the shape control signal S5 input from the shape control unit 102 will be described below with reference to FIG. The arithmetic processing unit 106 has registers 301-308, 311-318, sum-of-products calculators 321-328, and selectors 331-333. Registers 301 to 308 and 311 to 318 are examples of storage means, and sum-of-products calculators 321 to 328 are examples of calculation means.

Registers 301 to 308 and 311 to 318 temporarily store pixel data to be subjected to filter calculation read from the image data storage unit 105 . Also, the registers 301 to 308 and 311 to 318 output pixel data to the connected sum-of-products calculators and the connected registers. For example, the register 308 outputs the stored pixel data to the sum-of-products calculator 328 and the register 307 . Of the registers 301 to 308 and 311 to 318, the registers 301 to 308 that are directly connected to the sum-of-products arithmetic unit are also called directly connected registers. For example, the directly connected register for multiply-accumulator 327 is register 307 . Conversely, the registers 311 to 318 that are not directly connected to the sum-of-products calculator are also called indirectly connected registers.

The sum-of-products calculators 321 to 328 perform arithmetic processing for cumulatively adding the results of multiplication of the pixel data supplied from the directly connected registers 301 to 308 and the filter coefficients supplied from the filter coefficient storage unit 104 in parallel. For example, the sum-of-products calculator 328 performs a sum-of-products operation using the pixel data supplied from the register 308 and the filter coefficients supplied from the filter coefficient storage unit 104 .

Selectors 331 to 333 select and output one from a plurality of connected inputs. A shape control signal from the shape control unit 102 is connected to the selectors 331 to 333, and the selectors 331 to 333 set which input to select and output from the plurality of inputs according to the shape control signal. It is possible. In other words, the input/output relationship of the selectors 331 to 333 can be set by the shape control signal. For example, the selector 331 selects one of the input outputs of the register 303 and the register 311 and outputs it to the register 302 . Whether the selector 331 outputs the output of the register 303 to the register 302 or outputs the output of the register 311 to the register 302 can be controlled by the shape control signal.

Here, the operation grouping of the sum-of-products operators 321 to 328 in this embodiment will be described. This grouping of operations is realized by setting the input/output relationship of the selectors 331-333. In this embodiment, the sum-of-products calculators 321 to 328 are grouped as follows.

・When the parallelization shape is 1×8

・When the parallelization shape is 2×4

・When the parallelization shape is 4×2

The shape control unit 102 groups the sum-of-products calculators 321 to 328 according to the determined parallelized shape, as shown in the table above, to perform calculation grouping. Further, the shape control unit 102 directly connects the directly connected registers of the sum-of-products arithmetic units belonging to the same arithmetic group, and interposes the indirect-connected registers between the directly connected registers of the sum-of-products arithmetic units belonging to different arithmetic groups. Set and output the shape control signal to connect with Therefore, in this embodiment, the shape control section 102 outputs shape control signals to the selectors 331 to 333 so as to select and output the outputs of the register 303 shown in the table below.

For example, when the parallelization shape is 1×8, the selector 331 selects the output of the register 303 and outputs it to the register 302 . Also when the parallelization shape is 2×4, the selector 331 selects the output of the register 303 and outputs it to the register 302 . On the other hand, when the parallelization shape is 4×2, the selector 331 selects the output of the register 311 and outputs it to the register 302 .

By controlling the input/output relationship of the selectors 331 to 333 by the shape control signal in this way, the connection relationship of the directly connected registers and the indirectly connected registers within the same operation group or between the operation groups becomes desired. For example, when the parallelization configuration is 2×4, the selector 331 is set so that the directly connected registers 301 to 304 of the sum-of-products operators 321 to 324 belonging to group B1 are directly connected to each other. There is a selector 331 between the registers 302 and 303, but since the selector simply determines the connection relationship, it is ignored as the connection relationship between the registers. A selector 332 is set so that the indirect connection registers 313 and 314 are connected between the directly connected register (register 304) belonging to group B1 and the directly connected register (register 305) belonging to group B2.

Here, when the parallelization shape is 2×4, the pixel data stored in the registers 311, 312, 315, and 316 are supplied to the sum-of-products calculator because the selectors 331 and 333 are disconnected. never Therefore, when the parallelization shape is 2×4, the operation result is not affected regardless of what data is stored in the registers 311, 312, 315, and 316. FIG. Similarly, when the parallelization shape is 1×8, the pixel data stored in the registers 311 to 316 are not supplied to the sum-of-products calculator because the selectors 331 to 333 are disconnected. . Therefore, when the parallelization shape is 1×8, the operation result is not affected regardless of what data is stored in the registers 311 to 316 .

As described above, the connection states of the registers 301 to 308 and 311 to 318 inside the arithmetic processing unit 106 are controlled by the shape control signal S5 supplied from the shape control unit 102 . For example, when the parallelization shape is 1×8, the pixel data are transferred in the order of registers 318→317→308→307→306→305→304→303→302→301. Connection state is controlled. Also, for example, when the parallelized shape is 2×4, pixel data are transferred in the order of registers 318→317→308→307→306→305→314→313→304→303→302→301. Connection state is controlled. Further, for example, when the parallelization shape is 2×4, the order is register 318→317→308→307→316→315→306→305→314→313→304→303→312→311→302→301. The connection state is controlled so that pixel data is transferred.

Next, operation of the arithmetic processing unit 106 will be described. The operation of the arithmetic processing unit 106 described below is controlled by a transfer control signal S4 input from the transfer control unit 103. FIG. It is assumed that before this operation is performed, the connection states of the registers 301 to 308 and 311 to 318 in the arithmetic processing unit 106 are controlled according to the desired parallelization configuration as described above. In other words, it is assumed that the connection states (input/output relationships) of the selectors 331 to 333 are determined.

The arithmetic processing unit 106 controls how the pixel data stored in the registers 301 to 308 and 311 to 318 are transferred between the registers according to the transfer control signal from the transfer control unit 103 . First, an outline of transfer control will be described here, and details of filter calculation by transfer control will be described later.

Transfer control of pixel data at the time of filter calculation by a transfer control signal has two transfer modes, a first transfer mode and a second transfer mode. In the first transfer mode, pixel data is transferred between registers so that the transferred pixel data is used only for calculations in the sum-of-products calculator within the calculation group. That is, in the first transfer mode, the pixel data used for calculation by the sum-of-products calculator belonging to one calculation group is not used for calculations by the sum-of-products calculator belonging to another calculation group. . When pixel data transfer in the arithmetic processing unit 106 is controlled in this first transfer mode, the arithmetic processing unit 106 performs arithmetic processing for one row of the filter kernel.

Further, in the second transfer mode, pixel data is transferred between registers so that the transferred pixel data is also used for calculation in the sum-of-products calculator of another calculation group. When the arithmetic processing unit 106 completes arithmetic processing for one row of the filter kernel, pixel data transfer is executed in the second transfer mode. Since a filter kernel generally consists of a plurality of rows, the filter operation is executed by repeating the transfer of pixel data in the first transfer mode and the transfer of pixel data in the second transfer mode.

For example, in the case of filter calculation using the filter kernel 1402 shown in FIG. 14, pixel data is first transferred in the first transfer mode, and the calculation for the first row is performed by the sum-of-products calculator. , the following equation is obtained.

Subsequently, the pixel data is transferred in the second transfer mode, and then the pixel data is transferred again in the first transfer mode, and the calculation for the next row is performed by the sum-of-products calculator, A calculation result shown in the following equation is obtained.

After that, the pixel data is transferred again in the second transfer mode, and the pixel data is transferred in the first transfer mode. A filter operation result shown in is obtained.

This series of operations will be described with reference to the flowchart shown in FIG. 4 and the time charts shown in FIGS. FIG. 4 is a flowchart illustrating an example of arithmetic processing of the image processing apparatus according to the first embodiment. FIG. 5 is a time chart showing an example of filter operation processing when the parallelization shape is 2×4, and FIG. 6 is an example of filter operation processing when the parallelization shape is 4×2. It is a time chart showing. Also, FIG. 7 is a time chart showing an example of filter calculation processing when the parallelization shape is 1×8.

First, with reference to FIGS. 4 and 5, the operation when the parallelized shape is 2×4 will be described. As described above, the parallelized shape is determined by the shape control unit 102 prior to the start of the parallel arithmetic processing shown in the flowchart of FIG. It shall be The selector 331 selects the output of the register 303 and outputs it to the register 302 , the selector 332 selects the output of the register 313 and outputs it to the register 304 , the selector 333 selects the output of the register 307 and outputs it to the register 306 . is controlled to output to Moreover, when the parallelization shape is 2×4 as described above, the data stored in the registers 311, 312, 315, and 316 are not used for the calculation. Therefore, in the time chart shown in FIG. 5, the output columns of these registers 311, 312, 315 and 316 are blank.

In step S401 shown in FIG. 4, the image processing apparatus 101 prepares (prepares pixel data) for parallel computation (filter computation) in the sum-of-products calculator of the computation processing unit 106, and prepares pixel data for use in the parallel computation. Pixel data is supplied from the image data storage unit 105 to a group of registers to be used. In step S401, the transfer control unit 103 reads pixel data of six pixels (D00, D01, . (time t0 to t5 in FIG. 5). When the transfer is completed, the transfer control unit 103 reads the pixel data of 6 pixels (D10, D11, . , instructs to transfer to the arithmetic processing unit 106 (time t6 to t11 in FIG. 5). As a result, in the next cycle (time t12 in FIG. 5), the pixel data to be computed is stored in the registers used for computation in the computation processing unit 106, and from time t12 in FIG. Sum-of-products calculations are performed by sum-of-products calculators 321-328.

After time t12 in FIG. 5, the transfer control unit 103 sequentially shifts rows, reads pixel data from the image data storage unit 105 in units of six pixels in the horizontal direction of the calculation target image, and transfers the pixel data to the calculation processing unit 106. to do so. As a result, for example, pixel data (D20, D21, . supplied to Further, for example, the pixel data (D30, D31, . supplied.

Subsequently, in step S402, the arithmetic processing unit 106 performs a sum-of-products operation of the filter coefficients (W00, W01, W02) of the first row in the horizontal direction in the filter kernel and the pixel data. Therefore, the transfer control unit 103 instructs the registers in the arithmetic processing unit 106 to perform shift processing (transfer of pixel data between registers). This transfer control is the first transfer mode, during which the pixel data supplied to the sum-of-products calculator belonging to one calculation group is not supplied to the sum-of-products calculator belonging to another calculation group. do not have. For example, the pixel data D10-D15 supplied to the sum-of-products calculators 325-328 belonging to group B2 are not supplied to the sum-of-products calculators 321-324 belonging to group B1.

In step S402, the transfer control unit 103 reads the filter coefficients (one horizontal line of the filter kernel) from the filter coefficient storage unit 104 in synchronization with the timing of the first transfer mode, and Instruct to transfer to the sum-of-products calculator. In this manner, in the arithmetic processing unit 106, the pixel data to be operated is transferred in the first transfer mode, and the sum-of-products operation for one row in the horizontal direction of the filter kernel is executed (time t12 in FIG. 5). ~ t14).

Subsequently, in step S403, the transfer control unit 103 confirms conditions for ending the parallel computation (filter computation), and determines whether or not the parallel computation (filter computation) has ended. At this point, the transfer control unit 103 determines that the parallel operation (filter operation) has not ended (NO ) and proceed to step S404.

In step S404, the transfer control unit 103 instructs the arithmetic processing unit 106 so that the pixel data supplied to the sum-of-products arithmetic unit belonging to one arithmetic group is supplied to the sum-of-products arithmetic unit belonging to another arithmetic group. (Times t15 to t17 in FIG. 5). Specifically, the transfer control unit 103 controls the operation processing unit 106 so that pixel data supplied to a sum-of-products calculator belonging to a certain calculation group is supplied to a sum-of-products calculator belonging to another calculation group. Instructs the register to shift. This transfer control is the second transfer mode, and the sum-of-products arithmetic unit stops performing the sum-of-products operation during the transfer.

By transferring the pixel data in this second transfer mode, in the next cycle (time t18 in FIG. 5), the pixel data D10-D15 can be supplied to the sum-of-products calculators 321-324 belonging to the calculation group (group B1). state. The pixel data D10 to D15 are the pixel data supplied to the sum-of-products calculators 325 to 328 belonging to another calculation group (group B2) before the previous start of the first transfer mode. Further, by reading and transferring the image data at times t12 to t17, in the next cycle (time t18), the pixel data D20 to D25 of the third row are transferred to the sum-of-products calculators 325 to 328 belonging to the calculation group (group B2). become available for supply.

After that, the process of step S402 is performed again, and the arithmetic processing unit 106 performs a sum-of-products operation of the filter coefficients (W10, W11, W12) of the second row in the filter kernel in the horizontal direction and the pixel data (at the time shown in FIG. 5). t18-t20). Thereafter, the image processing apparatus 101 repeatedly executes the processes of steps S402 and S404 until the transfer control unit 103 determines in step S403 that the parallel computation (filter computation) has ended.

In step S403 after the product-sum operation of the filter coefficients (W20, W21, W22) on the third horizontal line in the filter kernel and the pixel data, the transfer control unit 103 performs parallel operation (filter operation). It is determined to be finished (YES), and the process ends. At the end of the parallel computation (filter computation), the sum-of-products calculators 321 to 328 in the computation processing unit 106 output computation results expressed by the following equations.

When the parallel operation (filter operation) is completed by the processing at time t24 in FIG. The calculator 324 outputs the calculation result when i=0 and j=3. Further, for example, the product-sum calculator 325 outputs the calculation result when i=1 and j=0, and the product-sum calculator 328 outputs the calculation result when i=1 and j=3. be done.

Further, after time t24 in FIG. 5, reading from the image data storage unit 105 and register shift processing are performed prior to the next filter calculation in parallel with the above-described calculation processing in a pipeline manner. It is The above-described operation is performed by sequentially shifting the rows to be computed in the image to be computed, and then returning to the first row when the last row is reached and shifting the column to be computed to the next column, thereby performing the operation shown in FIG. is repeated to obtain a computation output image obtained by filtering the entire computation target image.

Next, with reference to FIGS. 4 and 6, the operation when the parallelization shape is 4×2 will be described. Similar to the case where the parallelized shape is 2×4, the determination of the parallelized shape by the shape control unit 102 is performed prior to the start of the parallel arithmetic processing shown in the flowchart of FIG. It is assumed that the connection states of the selectors 331 to 333 have been determined. The selector 331 selects the output of the register 311 and outputs it to the register 302 , the selector 332 selects the output of the register 313 and outputs it to the register 304 , the selector 333 selects the output of the register 315 and outputs it to the register 308 . is controlled to output to When the parallelization configuration is 4×2, data stored in all registers 301 to 308 and 311 to 318 are used for operations.

In step S401 shown in FIG. 4, the image processing apparatus 101 prepares (prepares pixel data) for parallel computation (filter computation) in the sum-of-products calculator of the computation processing unit 106, and prepares pixel data for use in the parallel computation. Pixel data is supplied from the image data storage unit 105 to a group of registers to be used. In this step S401, the transfer control unit 103 reads pixel data of four pixels (D00, D01, D02, D03) in the horizontal direction of the calculation target image from the image data storage unit 105, and transfers the pixel data to the calculation processing unit 106. instruction (time t0 to t3 in FIG. 6). When the transfer is completed, the transfer control unit 103 reads the pixel data of four pixels (D10, D11, D12, D13) from the next row (second row) of the calculation target image from the image data storage unit 105, and performs the calculation. The processor 106 is instructed to transfer (time t4 to t7 in FIG. 6).

Further, the transfer control unit 103 reads out pixel data of four pixels (D20, D21, D22, D23) from the next row (third row) of the calculation target image from the image data storage unit 105, and sends the data to the calculation processing unit 106. A transfer is instructed (time t8 to t11 in FIG. 6). Furthermore, the transfer control unit 103 reads pixel data of four pixels (D30, D31, D32, D33) from the next row (fourth row) of the calculation target image from the image data storage unit 105, and 106 (time t12 to t15 in FIG. 6). As a result, in the next cycle (time t16 in FIG. 6), the register used for calculation in the arithmetic processing unit 106 stores the pixel data to be calculated, and from time t16 shown in FIG. Sum-of-products calculations are performed by sum-of-products calculators 321-328.

After time t16 in FIG. 6, the transfer control unit 103 sequentially shifts the rows, reads pixel data from the image data storage unit 105 in units of four pixels in the horizontal direction of the calculation target image, and transfers the pixel data to the calculation processing unit 106. to do so. As a result, for example, pixel data (D40, D41, D42, D43) to be calculated in the fifth row are supplied from the image data storage unit 105 to the registers of the arithmetic processing unit 106 as shown at times t16 to t19 in FIG. be done. Further, for example, the pixel data (D50, D51, D52, D53) to be operated on the sixth row are supplied from the image data storage unit 105 to the registers of the arithmetic processing unit 106 as indicated at times t20 to t23. be.

Subsequently, in step S402, the arithmetic processing unit 106 performs a sum-of-products operation of the filter coefficients (W00, W01, W02) of the first row in the horizontal direction in the filter kernel and the pixel data. Therefore, the transfer control unit 103 instructs the register in the arithmetic processing unit 106 to perform shift processing. This transfer control is the first transfer mode. In step S402, the transfer control unit 103 reads the filter coefficients (one horizontal line of the filter kernel) from the filter coefficient storage unit 104 in synchronization with the timing of the first transfer mode, and Instruct to transfer to the sum-of-products calculator. In this manner, in the arithmetic processing unit 106, the pixel data to be operated is transferred in the first transfer mode, and the sum-of-products operation for one row in the horizontal direction of the filter kernel is executed (time t16 in FIG. 6). ~ t18).

Subsequently, in step S403, the transfer control unit 103 confirms conditions for ending the parallel computation (filter computation), and determines whether or not the parallel computation (filter computation) has ended. At this point, the sum-of-products operation has only been performed for the first row in the horizontal direction of the filter kernel, so the transfer control unit 103 determines that the parallel operation (filter operation) has not ended (NO), and step S404. proceed to

In step S404, the transfer control unit 103 instructs the arithmetic processing unit 106 so that the pixel data supplied to the sum-of-products arithmetic unit belonging to one arithmetic group is supplied to the sum-of-products arithmetic unit belonging to another arithmetic group. (Time t19 in FIG. 6). Specifically, the transfer control unit 103 controls the operation processing unit 106 so that pixel data supplied to a sum-of-products calculator belonging to a certain calculation group is supplied to a sum-of-products calculator belonging to another calculation group. Instructs the register to shift. This transfer control is the second transfer mode, and the sum-of-products arithmetic unit stops performing the sum-of-products operation during the transfer.

By transferring the pixel data in this second transfer mode, in the next cycle (time t20 in FIG. 6), the pixel data previously supplied to the sum-of-products arithmetic unit belonging to a different arithmetic group is transferred to the product-accumulator. It becomes ready to be supplied to the sum calculators 321-326. Further, by reading and transferring the image data at times t16 to t19, in the next cycle (time t20), the pixel data D40 to D43 of the fifth row are transferred to the sum-of-products calculators 327 and 328 belonging to the calculation group (group C4). become available for supply.

After that, the process of step S402 is performed again, and the arithmetic processing unit 106 performs the sum-of-products operation of the filter coefficients (W10, W11, W12) of the second row in the filter kernel in the horizontal direction and the pixel data (the time shown in FIG. 6). t20-t22). Thereafter, the image processing apparatus 101 repeatedly executes the processes of steps S402 and S404 until the transfer control unit 103 determines in step S403 that the parallel computation (filter computation) has ended.

In step S403 after the product-sum operation of the filter coefficients (W20, W21, W22) on the third horizontal line in the filter kernel and the pixel data, the transfer control unit 103 performs parallel operation (filter operation). It is determined to be finished (YES), and the process ends. At the end of the parallel computation (filter computation), the sum-of-products calculators 321 to 328 in the computation processing unit 106 output computation results expressed by the following equations.

When the parallel operation (filter operation) is completed by the processing at time t24 in FIG. The calculator 322 outputs the calculation result when i=0 and j=1. Further, for example, the product-sum calculator 323 outputs the calculation result when i=1 and j=0, and the product-sum calculator 328 outputs the calculation result when i=3 and j=1. be done.

Further, after time t24 in FIG. 6, reading from the image data storage unit 105 and register shift processing are performed prior to the next filter calculation in parallel with the above-described calculation processing in a pipeline manner. It is The above-described operation is performed by sequentially shifting the rows to be computed in the image to be computed, and then returning to the first row when the last row is reached and shifting the column to be computed to the next column, thereby performing the operation shown in FIG. is repeated to obtain a computation output image obtained by filtering the entire computation target image.

Next, with reference to FIGS. 4 and 7, the operation when the parallelized shape is 1×8 will be described. As described above, the parallelized shape is determined by the shape control unit 102 prior to the start of the parallel arithmetic processing shown in the flowchart of FIG. It shall be The selector 331 selects the output of the register 303 and outputs it to the register 302 , the selector 332 selects the output of the register 305 and outputs it to the register 304 , the selector 333 selects the output of the register 307 and outputs it to the register 306 . is controlled to output to Further, when the parallelization shape is 1×8 as described above, the data stored in the registers 311 to 316 are not used for the calculation. Therefore, in the time chart shown in FIG. 7, the output columns of these registers 311 to 316 are blank.

In step S401 shown in FIG. 4, the image processing apparatus 101 prepares (prepares pixel data) for parallel computation (filter computation) in the sum-of-products calculator of the computation processing unit 106, and prepares pixel data for use in the parallel computation. Pixel data is supplied from the image data storage unit 105 to a group of registers to be used. In step S401, the transfer control unit 103 reads pixel data of 10 pixels (D00, D01, . (time t0 to t9 in FIG. 7). As a result, in the next cycle (time t10 in FIG. 7), the registers used for calculation in the arithmetic processing unit 106 store the pixel data to be calculated, and from time t10 shown in FIG. Sum-of-products calculations are performed by sum-of-products calculators 321-328.

After time t10 in FIG. 7, the transfer control unit 103 sequentially shifts the rows, reads pixel data from the image data storage unit 105 in units of 10 pixels in the horizontal direction of the calculation target image, and transfers the pixel data to the calculation processing unit 106. to do so. As a result, for example, pixel data (D10, D11, . supplied to Further, for example, the pixel data (D20, D21, . supplied.

Subsequently, in step S402, the arithmetic processing unit 106 performs a sum-of-products operation of the filter coefficients (W00, W01, W02) of the first row in the horizontal direction in the filter kernel and the pixel data. Therefore, the transfer control unit 103 instructs the register in the arithmetic processing unit 106 to perform shift processing. This transfer control is the first transfer mode. In step S402, the transfer control unit 103 reads the filter coefficients (one horizontal line of the filter kernel) from the filter coefficient storage unit 104 in synchronization with the timing of the first transfer mode, and Instruct to transfer to the sum-of-products calculator. In this manner, in the arithmetic processing unit 106, the pixel data to be operated is transferred in the first transfer mode, and the sum-of-products operation for one row in the horizontal direction of the filter kernel is executed (time t10 in FIG. 7). ~ t12).

Subsequently, in step S403, the transfer control unit 103 confirms conditions for ending the parallel computation (filter computation), and determines whether or not the parallel computation (filter computation) has ended. At this point, the sum-of-products operation has only been performed for the first row in the horizontal direction of the filter kernel, so the transfer control unit 103 determines that the parallel operation (filter operation) has not ended (NO), and step S404. proceed to However, when the parallelized shape is 1×8, there is one operation group, so no processing is performed in step S404 and the process proceeds to step S402. Further, by reading and transferring the image data at times t10 to t19, the second row pixel data D10 to D19 can be supplied to the sum-of-products calculators 321 to 328 in the next cycle (time t20).

Then, the process of step S402 is performed again, and the arithmetic processing unit 106 performs a product-sum operation of the filter coefficients (W10, W11, W12) in the second row in the horizontal direction of the filter kernel and the pixel data (at the time shown in FIG. 7). t20-t22). Thereafter, the image processing apparatus 101 repeatedly executes the processes of steps S402 and S404 until the transfer control unit 103 determines in step S403 that the parallel computation (filter computation) has ended.

In step S403 after the product-sum operation of the filter coefficients (W20, W21, W22) on the third horizontal line in the filter kernel and the pixel data, the transfer control unit 103 performs parallel operation (filter operation). It is determined to be finished (YES), and the process ends. At the end of the parallel computation (filter computation), the sum-of-products calculators 321 to 328 in the computation processing unit 106 output computation results expressed by the following equations.

When the parallel operation (filter operation) is completed by the processing at time t32 in FIG. The calculator 322 outputs the calculation result when i=0 and j=1. Further, for example, the sum-of-products calculator 328 outputs a calculation result when i=0 and j=7. The above-described operation is performed by sequentially shifting the rows to be computed in the image to be computed, and then returning to the first row when the last row is reached and shifting the column to be computed to the next column, thereby performing the operation shown in FIG. is repeated to obtain a computation output image obtained by filtering the entire computation target image.

According to the first embodiment, the shape control unit 102 groups a plurality of sum-of-products arithmetic units into a plurality of operation groups to determine a parallelization shape, and sets connections between registers in the arithmetic processing unit 106. . Furthermore, the transfer control unit 103 controls the transfer of pixel data to be calculated, so that the output pixels of filter calculations that are calculated in parallel can be calculated in two dimensions on the calculation output image. Furthermore, it is possible to control its two-dimensional shape.

As a result, it is possible to suppress the occurrence of sum-of-products calculators that do not contribute to the filter calculation even for images that are small in size compared to the degree of parallelism (the number of sum-of-products calculators). can be effectively used for filter operation. Therefore, the filter operation can be performed more efficiently as the degree of parallelism is increased, and the filter operation can be performed with good operation efficiency without lowering the operation efficiency regardless of the size of the image to be operated. becomes.

As shown using the time charts (FIGS. 5, 6, and 7), the time required for the filter operation may differ depending on the parallelization shape. In such a case, the shape control unit 102 may select a parallelized shape that shortens the filter computation processing time. For example, in the case of the above-described calculation target image 201, even if any one of 1×8, 2×4, and 4×2 is adopted as the parallelization shape, there is no calculator that does not contribute to the filter calculation. However, as shown in the time charts (FIGS. 5, 6, and 7), the filter operation processing time is short when the parallelized shape is 2×4 or 4×2. It is preferable to choose 2x4 or 4x2 as the parallelization geometry.

(Second embodiment)
Next, a second embodiment of the invention will be described. In the image processing apparatus according to the first embodiment, once the connection state between the registers in the arithmetic processing unit 106 is determined by the shape control signal, pixel data is transferred between adjacent registers in the determined connection state. data transfer. Therefore, in order to transfer pixel data across operation groups in the second transfer mode, it is necessary to transfer data between adjacent registers over a plurality of clock cycles.

In the image processing apparatus according to the second embodiment, the connection state between registers is controlled according to the transfer mode by the transfer control signal, in addition to setting the connection state between registers by the shape control signal. As a result, in the second embodiment, the time required to transfer pixel data in the second transfer mode can be significantly reduced as compared with the first embodiment, and the filter operation can be performed at a higher speed than in the first embodiment. can be done.

FIG. 8 is a block diagram showing a configuration example of an image processing apparatus 801 according to the second embodiment. In FIG. 8, constituent elements having the same functions as the constituent elements shown in FIG. 1 are denoted by the same reference numerals, and overlapping descriptions are omitted. The image processing device 801 has a shape control unit 102 , a transfer control unit 803 , a filter coefficient storage unit 104 , an image data storage unit 805 and an arithmetic processing unit 806 .

In addition to the functions of the transfer control unit 103 shown in FIG. 1, the transfer control unit 803 has a function of controlling the connection state between registers in the arithmetic processing unit 806 according to the transfer mode. The image data storage unit 805 stores the pixel data of the image to be subjected to the filter operation, similarly to the image data storage unit 105 shown in FIG. Here, in this embodiment, the image data storage unit 805 has a memory width capable of reading out a plurality of pixel data in one clock cycle in order to perform filter calculation at high speed. The calculation processing unit 806 has a plurality of calculators capable of executing calculations in parallel, and performs filter calculation using the input filter coefficient S2 and pixel data S3.

FIG. 9 is a block diagram showing a configuration example of the arithmetic processing unit 806. As shown in FIG. In FIG. 9, constituent elements having the same functions as the constituent elements shown in FIG. 3 are given the same reference numerals, and redundant explanations are omitted. The arithmetic processing unit 806 has registers 301-308, 311-318, sum-of-products calculators 321-328, and selectors 331-333, 941-946, and 951-956.

Selectors 941 to 946 and 951 to 956 select and output one from a plurality of connected inputs. A transfer control signal from the transfer control unit 803 is connected to the selectors 941 to 946 and 951 to 956, and the selectors 941 to 946 and 951 to 956 select which input among a plurality of inputs according to this transfer control signal. It is possible to set whether to select and output. In this embodiment, the transfer control signal switches the input/output relationship of the selectors 941 to 946 and 951 to 956 depending on whether the first transfer mode is set or the second transfer mode is set. Is possible.

In this embodiment, the transfer control unit 803 outputs transfer control signals to the selectors 941 to 946 and 951 to 956 so that the input and output are connected as shown in the table below. The table below shows how the selectors 941 to 946 and 951 to 956 become input/output connection states in accordance with the transfer control signals for three different parallelization configurations. The operations of the selectors 331 to 333 controlled by the shape control section 102 are the same as in the first embodiment.

・When the parallelization shape is 1×8

This table shows that the selector 941 selects and outputs the output of the register 302 in the first transfer mode when the parallelization shape is 1×8, for example. Similarly, the selector 942 selects and outputs the output of the register 303 . Note that the selector 331 for selecting and outputting the output of the register 303 when the parallelization shape is 1×8 is directly connected to the selector 942, so the selector 331 is described in parentheses. there is Also, hatched lines indicate that any input may be used as an output. Furthermore, when the parallelization configuration is 1.times.8, there is one operation group and there is no second transfer mode, so x is shown in the table.

・When the parallelization shape is 2×4

In this table, the output of the selectors 945 and 946 being pixel data 1 and 2 indicates that the output of the image data storage unit 805 is selected.

・When the parallelization shape is 4×2

Next, operation of the arithmetic processing unit 806 will be described. The operation of the arithmetic processing unit 806 described below is controlled by a transfer control signal S4 input from the transfer control unit 803. FIG. Before this operation is performed, as in the first embodiment, the connection states of the registers 301 to 308 and 311 to 318 in the arithmetic processing unit 806 are determined according to the desired parallelization configuration.

In the second embodiment, similarly to the first embodiment, the filter operation is executed by repeating the transfer of pixel data in the first transfer mode and the transfer of pixel data in the second transfer mode. be. In the first transfer mode, the connection states of the registers 301-308 and 311-318 are the same as in the first embodiment, and the operations are also the same. That is, in the first transfer mode, pixel data is transferred between registers so that the transferred pixel data is used only for calculations in the sum-of-products calculator within the calculation group.

The second transfer mode differs from the first embodiment. In the first embodiment, the connection state between registers does not change between the first transfer mode and the second transfer mode. Therefore, in order to transfer pixel data across operation groups in the second transfer mode, it is necessary to transfer pixel data over a plurality of clock cycles. In contrast, in the second embodiment, by changing the connection state between registers between the first transfer mode and the second transfer mode, the transfer time of pixel data performed in the second transfer mode is shortened. do.

A series of operations in the second embodiment will be described with reference to the flowchart shown in FIG. 10 and the time charts shown in FIGS. FIG. 10 is a flow chart showing an example of arithmetic processing of the image processing apparatus according to the second embodiment. FIG. 11 is a time chart showing an example of filter operation processing when the parallelization shape is 2×4, and FIG. 12 is an example of filter operation processing when the parallelization shape is 4×2. It is a time chart showing.

First, with reference to FIGS. 10 and 11, the operation when the parallelized shape is 2×4 will be described. As in the first embodiment, determination of the parallelized shape by the shape control unit 102 is performed prior to the start of the parallel arithmetic processing shown in FIG. In steps S1001 to S1003, the image processing apparatus 801 makes preparations (preparing pixel data) for parallel computations (filter computations) in the sum-of-products calculator of the computation processing unit 806, and registers to be used in the parallel computations. Pixel data is supplied from the image data storage unit 805 to the group.

First, in step S<b>1001 , the transfer control unit 803 sets the second transfer mode, reads pixel data of six pixels in the horizontal direction of the calculation target image from the image data storage unit 805 , and transfers the data to the calculation processing unit 806 . command (time t0 in FIG. 11). In FIG. 11, pixel data D00 to D05 are output in parallel from the image data storage unit 805 at time t0 (denoted as D0* in FIG. 11). Since the registers are connected in the second transfer mode, in the next cycle (time t1 in FIG. 11), the registers 305 to 308, 317, and 318 in the arithmetic processing unit 806 have Pixel data D00 to D05 are stored.

Subsequently, in step S1002, the transfer control unit 803 determines whether pixel data to be processed is stored in a group of registers used for parallel computation in the computation processing unit 806, that is, whether preparation for filter computation is complete. determine whether or not At this time, the registers 301 to 304, 313, and 314 in the arithmetic processing unit 806 do not store pixel data, so the transfer control unit 803 determines (NO) that the preparation for the filter operation is not completed. The process proceeds to step S1003.

In step S1003, the transfer control unit 803 instructs the registers in the arithmetic processing unit 806 to perform shift processing (pixel data transfer between registers) in the first transfer mode (from time t2 in FIG. 11). t3).

Further, returning to step S1001, the transfer control unit 803 transfers the pixel data D10 to D15 read from the image data storage unit 805 to the registers 305 to 308, 317 and 318 in the second transfer mode. At the same time, the transfer control unit 803 instructs to transfer pixel data between registers in the arithmetic processing unit 806 (time t4 in FIG. 11). As a result, in this cycle (time t4 in FIG. 11), the registers used for calculation in the arithmetic processing unit 806 store the pixel data to be calculated, and from time t4 shown in FIG. A sum-of-products operation is performed by computing units 321-328.

At this time, in step S1002, the transfer control unit 803 determines that preparation for filter calculation is completed (YES), and the process proceeds to step S1004. In step S1004, the arithmetic processing unit 806 performs a sum-of-products operation between the filter coefficients (W00, W01, W02) of the first row in the horizontal direction in the filter kernel and the pixel data (time t4 to t6 in FIG. 11). In parallel with the sum-of-products operation, pixel data transfer between registers is performed in the first transfer mode (time t5 to t6 in FIG. 11).

Subsequently, in step S1005, the transfer control unit 803 confirms conditions for ending the parallel computation (filter computation), and determines whether or not the parallel computation (filter computation) has ended. At this point, the transfer control unit 803 determines that the parallel calculation (filter calculation) has not ended because the sum-of-products calculation of the first horizontal row of the filter kernel and the pixel data has only been performed ( NO), the process proceeds to step S1006.

In step S1006, the transfer control unit 803 instructs the arithmetic processing unit 806 to transfer pixel data between registers in the second transfer mode (time t7 in FIG. 11). From this cycle (time t7 in FIG. 11), sum-of-products calculations are performed by sum-of-products calculators 321 to 328 for the second row of the filter kernel.

After that, the process of step S1004 is performed again, and the arithmetic processing unit 806 performs a sum-of-products operation of the filter coefficients (W10, W11, W12) in the second row in the horizontal direction of the filter kernel and the pixel data (at the time shown in FIG. 11). t7-t9). In parallel with the sum-of-products operation, pixel data transfer between registers is performed in the first transfer mode (time t8 to t9 in FIG. 11). Thereafter, the image processing apparatus 801 repeatedly executes the processing of steps S1004 and S1006 until the transfer control unit 803 determines in step S1005 that the parallel computation (filter computation) has ended.

In step S1005 after the product-sum operation of the filter coefficients (W20, W21, W22) on the third horizontal line in the filter kernel and the pixel data, the transfer control unit 803 performs parallel operation (filter operation). It is determined to be finished (YES), and the process ends. At the end of the parallel computation (filter computation), the sum-of-products calculators 321 to 328 in the computation processing unit 806 provide computation results similar to those of the first embodiment.

Next, with reference to FIGS. 10 and 12, the operation when the parallelized shape is 4×2 will be described. Similar to the case where the parallelization shape is 2×4, the parallelization shape is determined by the shape control unit 102 prior to the start of the parallel arithmetic processing shown in the flowchart of FIG.

First, in step S<b>1001 , the transfer control unit 803 sets the second transfer mode, reads pixel data of four pixels in the horizontal direction of the calculation target image from the image data storage unit 805 , and transfers the pixel data to the calculation processing unit 806 . command (time t0 in FIG. 12). In FIG. 12, pixel data D00 to D03 are output in parallel from the image data storage unit at time t0 (indicated as D0* in FIG. 12). As a result, in the next cycle (time t1 in FIG. 12), the registers 307, 308, 317, and 318 in the arithmetic processing unit 806 store the pixel data D00 to D03 to be calculated.

Subsequently, in step S1002, the transfer control unit 803 determines whether or not preparations for filter calculation have been completed. At this time, the registers 301 to 306 and 311 to 316 in the arithmetic processing unit 806 do not store pixel data, so the transfer control unit 803 determines that the preparation for the filter operation is not completed (NO). The process proceeds to step S1003. In step S1003, the transfer control unit 803 instructs the registers in the arithmetic processing unit 806 to perform shift processing (pixel data transfer between registers) in the first transfer mode (from time t2 in FIG. 12). t3).

The image processing apparatus 801 repeatedly executes the processing of steps S1001 and S1003 until the transfer control unit 803 determines in step S1002 that preparation for the filter operation is complete. The process of step S1001 is performed at times t4, t7 and t10 in FIG. 12, and the process of step S1003 is performed at times t5 to t6 and t8 to t9 in FIG. Then, at time t10 in FIG. 12, the registers used for calculation in the arithmetic processing unit 806 are in a state of storing the pixel data to be calculated, and from time t10 shown in FIG. A sum-of-products operation is performed.

In step S1004, the arithmetic processing unit 806 performs a sum-of-products operation between the filter coefficients (W00, W01, W02) of the first row in the horizontal direction in the filter kernel and the pixel data (time t10 to t12 in FIG. 12). In parallel with the sum-of-products operation, pixel data transfer between registers is performed in the first transfer mode (time t11 to t12 in FIG. 12).

Subsequently, in step S1005, the transfer control unit 803 confirms conditions for ending the parallel computation (filter computation), and determines whether or not the parallel computation (filter computation) has ended. At this point, the transfer control unit 803 determines that the parallel calculation (filter calculation) has not ended because the sum-of-products calculation of the first horizontal row of the filter kernel and the pixel data has only been performed ( NO), the process proceeds to step S1006.

In step S1006, the transfer control unit 803 instructs the arithmetic processing unit 806 to transfer pixel data between registers in the second transfer mode (time t13 in FIG. 12). From this cycle (time t13 in FIG. 12), sum-of-products calculations are performed by sum-of-products calculators 321 to 328 for the second row of the filter kernel.

After that, the process of step S1004 is performed again, and the arithmetic processing unit 806 performs a sum-of-products operation of the filter coefficients (W10, W11, W12) in the second row in the horizontal direction in the filter kernel and the pixel data (at the time shown in FIG. 12). t13-t15). In parallel with the sum-of-products operation, pixel data transfer between registers is performed in the first transfer mode (time t14 to t15 in FIG. 12). Thereafter, the image processing apparatus 801 repeatedly executes the processing of steps S1004 and S1006 until the transfer control unit 803 determines in step S1005 that the parallel computation (filter computation) has ended.

In step S1005 after the product-sum operation of the filter coefficients (W20, W21, W22) on the third horizontal line in the filter kernel and the pixel data, the transfer control unit 803 performs parallel operation (filter operation). It is determined to be finished (YES), and the process ends. At the end of the parallel computation (filter computation), the sum-of-products calculators 321 to 328 in the computation processing unit 806 provide computation results similar to those of the first embodiment.

According to the second embodiment, similarly to the first embodiment, it is possible to perform filter calculation with good calculation efficiency without lowering the calculation efficiency regardless of the size of the calculation target image. Become. Further, in the second embodiment, the connection state between registers is changed between the first transfer mode and the second transfer mode, thereby shortening the transfer time of pixel data performed in the second transfer mode. can be done. Therefore, in the image processing apparatus according to the second embodiment, it becomes possible to perform filter calculations at a higher speed than in the first embodiment, and the calculation efficiency of filter calculations can be improved.

(Third Embodiment)
Next, a third embodiment of the invention will be described. In the first and second embodiments, examples have been shown in which the shape control unit 102 changes the parallelized shape according to the shape of the computation target area of the filter computation. As shown in the first embodiment, there are cases where the filter operation processing can be performed faster with a two-dimensional parallelized shape than with a one-dimensional parallelized shape. In such a case, even without changing the parallelization geometry, it is possible to perform filter operation processing at a higher speed than in the case of a one-dimensional parallelization geometry simply by realizing a two-dimensional parallelization geometry. is.

In the third embodiment, a case in which the present invention is applied to simply implement a two-dimensional parallelized shape will be described. In other words, in the third embodiment, an example in which a parallelized shape is a two-dimensional fixed shape and a filter operation is performed will be described. Therefore, the image processing apparatus according to the third embodiment does not require a shape control section that is necessary for switching between a plurality of parallelized shapes.

FIG. 13 is a block diagram showing a configuration example of an image processing apparatus 1301 according to the third embodiment. As described above, in this embodiment, parallelized shapes are fixed, so the image processing apparatus 1301 does not have a shape control unit for switching between a plurality of parallelized shapes. In FIG. 13, constituent elements having the same functions as the constituent elements shown in FIG. 1 are denoted by the same reference numerals, and overlapping descriptions are omitted. The image processing device 1301 has a transfer control unit 1303 , a filter coefficient storage unit 104 , an image data storage unit 105 and an arithmetic processing unit 1306 .

The transfer control unit 1303 reads the filter coefficient S2 and the pixel data S3 from the filter coefficient storage unit 104 and the image data storage unit 105, respectively, and controls the supply to the arithmetic processing unit 1306. FIG. Further, the transfer control unit 1303 outputs a transfer control signal S4 to the arithmetic processing unit 1306 to control the transfer of pixel data within the arithmetic processing unit 1306 . Here, since the parallelization shape is fixed in this embodiment, the transfer control unit 1303 also performs data transfer control suitable for a predetermined parallelization shape. For example, if the parallelization shape is 2×4, the same operation as that in the case of the parallelization shape of 2×4 among the operations of the transfer control unit 103 or 803 described in the first or second embodiment is performed. conduct.

The arithmetic processing unit 1306 has a plurality of arithmetic units capable of executing arithmetic operations in parallel, and performs filter arithmetic using the input filter coefficient S2 and pixel data S3. Here, since the parallelization shape is fixed in this embodiment, the arithmetic processing unit 1306 also performs the operation when the predetermined parallelization shape is used. Therefore, the selectors 331 to 333 shown in FIGS. 3 and 9 are not required, and instead, the registers may be connected so as to realize the input/output connection of the selectors in a predetermined parallel configuration.

By configuring the image processing apparatus 1301 in this way, it becomes possible to realize filter operation processing having a two-dimensional parallelized shape.

(Another embodiment of the present invention)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

It should be noted that the above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed to be limited by these. That is, the present invention can be embodied in various forms without departing from its technical concept or main features.

101, 801, 1301: image processing device 102: shape control unit 103, 803, 1303: transfer control unit 104: filter coefficient storage unit 105, 805: image data storage unit 106, 806, 1306: arithmetic processing unit 301 to 308, 311 to 318: registers 321 to 328: sum-of-product calculators 331 to 333: selectors 941 to 946, 951 to 956: selectors

Claims (8)

An image processing device for performing filter arithmetic processing by scanning a filter kernel with respect to pixel data of an image stored in an image storage means,
a plurality of temporary storage means for temporarily storing the plurality of pixel data read out from the image storage means, the connection state of which is controlled according to the arrangement of pixels for which the filter operation processing is performed in parallel;
performing parallel filter computation using the filter coefficients in the filter kernel and the plurality of pieces of pixel data stored in the plurality of temporary storage means; a plurality of computing means grouped into one or more computing groups by
Controlling the transfer of the pixel data used for the filter calculation process between the plurality of temporary storage means , and when the plurality of calculation means are grouped into a plurality of calculation groups, the same transfer mode is used in the first transfer mode. Transfer of transferring the pixel data so as to be used by the arithmetic means belonging to the arithmetic group, and controlling transfer of the pixel data so as to be used by the arithmetic means belonging to another arithmetic group in a second transfer mode. and a control means. 2. The image processing apparatus according to claim 1 , wherein said transfer control means changes a connection state between said plurality of temporary storage means between said first transfer mode and said second transfer mode. The transfer control means is
controlling the transfer of the pixel data in the first transfer mode while the filter operation processing for one row in the filter kernel is being performed;
3. The image processing apparatus according to claim 1 , wherein the transfer of the pixel data is controlled in the second transfer mode when the filtering operation processing for one row in the filter kernel is completed. In the second transfer mode, the transfer control means transfers the plurality of pixel data used for performing filter calculation processing for one row in the filter kernel in the first calculation group to the first calculation group. 4. The plurality of pixel data are transferred so as to be used for performing another row of filter operation processing in the filter kernel in a second operation group different from the The image processing device according to any one of 1. 5. The apparatus according to any one of claims 1 to 4 , further comprising connection control means for controlling a connection state of said plurality of temporary storage means in accordance with the arrangement of pixels on which said filtering operations are performed in parallel. The described image processing device. 6. The apparatus according to any one of claims 1 to 5 , further comprising grouping means for grouping said plurality of arithmetic means into one or a plurality of arithmetic groups according to the arrangement of pixels on which said filter arithmetic processing is performed in parallel. The image processing apparatus according to any one of items 1 to 3. 7. The image processing apparatus according to claim 6 , wherein each of said operation groups is composed of said operation means for performing said filter operation processing in parallel on pixels in the same row. An image processing method for performing filter arithmetic processing by scanning a filter kernel with respect to pixel data of an image stored in an image storage means,
a storage step of storing the plurality of pixel data read from the image storage means in a plurality of temporary storage means whose connection state is controlled according to the arrangement of pixels in which the filter operation processing is performed in parallel;
one or more filtering operations using the filter coefficients in the filter kernel and the plurality of pieces of pixel data stored in the plurality of temporary storage means according to the arrangement of pixels in which the filtering operations are performed in parallel; A calculation step performed in parallel by a plurality of calculation means grouped into a plurality of calculation groups ;
Controlling the transfer of the pixel data used for the filter operation processing between the plurality of temporary storage means , and when the plurality of operation means are grouped into a plurality of operation groups, the same operation is performed in a first transfer mode. Transfer of transferring the pixel data so as to be used by the arithmetic means belonging to the arithmetic group, and controlling transfer of the pixel data so as to be used by the arithmetic means belonging to another arithmetic group in a second transfer mode. and a control step.

Images