JPS6326912B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a parallel image processing processor that performs local neighborhood image processing such as spatial product-sum operations, and particularly relates to
This invention relates to a parallel image processing processor with an architecture suitable for LSI.

The image processing processor is part of the Ministry of International Trade and Industry's large-scale project ``Pattern Information Processing System'' (a collection of research and development results was published in October 1980).
Many of them are trying to speed up the processing of image data in parallel, as has been developed in . Since image data has a two-dimensional spread, it is difficult to process all image data in parallel. However, since there are many calculations between neighboring image data, such as spatial product-sum calculations that realize noise removal and contour extraction functions, for example, it is difficult to process local data in m rows by n columns of an image in parallel. many. This type of locally parallel image processing is described in the above-mentioned document or Masatsugu Kido: Trends in Image Processing Hardware: Information Processing Computer Vision Study Group Material 8-6.
(September 1980), which is comprehensively explained.
There are no LSI versions other than the CCD analog processing type. In order to convert a processor with a conventional architecture into an LSI, the following steps must be taken: Integration degree Number of pins The purpose of the present invention is to provide a parallel image processing processor with an architecture suitable for LSI conversion and also suitable for high-speed processing. There is something to do.

The present invention is characterized by an image data input port, a plurality of shift registers that sequentially take in image data from the image data input port, and a plurality of shift registers that input the contents of each of the shift registers and perform image processing operations. a processor element, a first arithmetic circuit that adds the arithmetic results of each of the processor elements, and an arithmetic result data input port that inputs arithmetic result data of the basic module at the previous stage;
a second arithmetic circuit that adds the arithmetic result data and the arithmetic result of the first arithmetic circuit;
between an operation result data output port that outputs operation result data of an operation circuit, between the shift register and the processor element, between the processor element and the first operation circuit, and between the first operation A pipeline register disposed between the circuit and the second arithmetic circuit is basically modularized, and further, a second pipe is provided between the arithmetic result data input port and the second arithmetic circuit. This is where line registers are inserted to create a basic module.

FIGS. 1 to 3 are diagrams of an embodiment that forms the premise of the present invention, which has been recently considered.

FIG. 1 shows the configuration of a typical image processing system, in which an industrial television camera 5 is used as an image input device, an image memory 3 is used as an image storage device, and an image memory 3 is used as an image storage device.
A CRT monitor 4 for displaying this content is also provided. The image information in the image memory 3 is processed by the image processing processor 2, and the results are also stored in the image memory 3 or provided to the management processor 1 which controls the entire system.

A typical image processing function is spatial product-sum operation. As shown in FIG. 2, for example, local image data of 4×4 pixels f ₁₁ to f ₄₄ are multiplied by predetermined loads w ₁₁ to w ₄₄ and the sum is calculated.

This results in noise removal Contour enhancement Image processing such as

As an image processing processor that processes local image data of, for example, 4×4 pixels, there are four processor elements (PE) as shown in Figure 3.
A parallel image processing processor (referred to as a type) 2- is a combination of four image processing processor basic modules 10 having image processing processors #1 to #4) 12. From the image memory 3, the local image data is 1
Columns (f ₁₄ to f ₄₄ in FIG. 3) are applied in parallel, and the calculation result (g in FIG. 3) is stored in the image memory 3.

The basic module 10 includes an image data input port 24 that takes in image data of a row to be processed, and a calculation result data output port 35 that outputs internal processing results.
have. When image data f ₁₄ is input, each adjacent pixel is transferred through the shift register 11.
f ₁₃ , f ₁₂ , and f ₁₁ are also input to corresponding PE#4-1. For pixel f ₁₁ , the size of the spatial product-sum operation is 4×4
In case of expansion above, the image data is outputted from the image data output port 25. The PE 12 contains image data f to be processed from the shift register 11,
Load data w from the load storage memory 15 is given, and multiplication is performed. This result is 4 PE1
A partial sum is calculated by the arithmetic circuit 13 which adds the results of the two results. The partial sums input from the calculation result input port 30 are accumulated one after another by the calculation circuit 14,
The calculation result output port 35 outputs the result to the basic module 10 at the next stage.

In this way, by stacking the basic modules 10 in four stages, g= _4,4 〓 ^i,j=1,1 f _i,j *w _ij is output from the final basic module 10D.

This time chart is shown in FIG. The above calculation is executed within the basic clock time Δt1 and the result g
is output, and in the next Δt1, the result g for the input image of 4×4 picture elements shifted by one pixel is output. Therefore, the spatial product-sum calculation results of all 4×4 picture elements for image data that are input one after another are output one after another.

FIG. 5 shows an embodiment of the parallel image processing processor according to the present invention, in which the basic clock time Δt1 of the type I image processing processor 2- of the above-mentioned embodiment is shown.
This figure shows a configuration that is shortened by pipeline processing. This is called a pipeline version parallel image processing processor 2-P. That is, in the type, the basic clock time Δt1 is the input processing of image data f _i,j to the shift register 11, and the sum-of-products load by the processor element 12.
The multiplication process of w _i,j and the image f _i,j, the partial sum processing by the arithmetic circuit 13, and the partial sum accumulation process by the arithmetic circuit 14 had to be longer than the sum of all processing times.
On the other hand, by interposing the pipeline register 16 between and, as in the example of FIG. (not the sum of all). This time chart is shown in FIG.
The process is executed at time 1, and at time 2, 3, and 4. At time 2, the processing for the next input image, at time 3, at time 4, and at time 5 are executed, and by operating each component one after another in a pipeline manner, it is possible to improve the processing speed.

FIG. 7 shows a second embodiment of the present invention, which shows a configuration in which the basic clock Δt2 of the parallel image processing processor 2-P described above can be further shortened. It is called the processing processor 2-PS. The basic clock time Δt2 in the P type shown in FIG. 5 is highly likely to be constrained by the partial sum accumulation time of processing. This is because when the basic module 10 has n stages, Δt2 requires n times the sum of the processing time in the arithmetic circuit 14 and the input/output time of the arithmetic results 30 and 35. Especially basic module 10
When converting into LSI, input/output delay time cannot be ignored. For this reason, a pipeline register 16 is further added to the type IP shown in FIG.
The time regulation is set to 1/n. PS of this figure 7
In the type, as shown in the time chart of FIG. 8, the partial sums of the basic modules 10A to 10D are calculated at the same time 3, and the timing in the cumulative part does not match. In the PS shown in FIG. 7, a variable stage skew correction shift register 17 for timing adjustment is installed immediately after the image data input port 24. Since the number of pipeline stages in the cumulative path of each basic module 10A to D is one stage, the number of stages of the variable stage skew correction shift register 17 is as follows: Basic module 10A...0 stage B...1 stage C ......2 stages D......Set to 3 stages. In this way, the inconsistency (... section) in the time chart of Fig. 8 is corrected,
Pipeline operation is possible for continuous Δt3 time.

In addition, as can be easily understood, the skew register 1
7 can be installed immediately after the arithmetic circuit 13 for calculating the partial sum, or even if it is installed immediately before or after each PE 12, the timing mismatch will be solved in the same way.

FIG. 9 shows configurations of different types of processing, and even in such types, pipeline processing can be applied as in the above embodiment. In the configuration described above, image data is input to each PE 12#1 to 4 via the shift register 11.
It distributed pixels adjacent to . On the other hand, in this embodiment, the input image data is
are given in common, and the multiplication results are accumulated via an arithmetic circuit 18 and a register 19 to output a partial sum ^Σ1 . This operation will be explained with reference to the time chart in FIG.

Image from image data input port 20 at time 1
f ₁₁ is input, load storage memory 1 is input at PE12#1
The product f ₁₁ *w ₁₁ with the load w ₁₁ read from 5 is set in register 19 #2.

Image data _f12 is input at time 2, and PE12
In #2, _{the product f 12 *} w _{12 with the load w 12} is taken, and the sum of this and the value f ₁₁ *w ₁₁ of register 19 #2 is f ₁₁ *
w ₁₁ +f ₁₂ *w ₁₂ is taken by the arithmetic circuit 18 and set in register 19#3.

Image data _f13 is input at time 3, and PE12
At #3, the product f ₁₃ *w _{13 with the load w 13} is taken, and _the sum of this and the value f ₁₁ *w ₁₁ +f ₁₂ *w ₁₂ of register 19 #3 is f ₁₁ *w ₁₁ +f ₁₂ *w ₁₂ +f ₁₃ *w ₁₃ is the calculation circuit 18
and set in register 19#4.

Image data _f14 is input at time 4, and PE12
At #4, _{the product f 14 *} w ₁₄ with the load w 14 is taken, and this and the value of register 19 #4 f ₁₁ *w ₁₁ +f ₁₂ *w ₁₂ +f ₁₃
The sum ^{Σ 1} _{11 =} f ₁₁ *w ₁₁ +~+f ₁₄ *w ₁₄ with *w 13 is taken by the arithmetic circuit 18 . This partial sum Σ ₁ is accumulated in the arithmetic circuit 14 of each basic module 10A to D,
g= _4,4 〓 ^i,j=1,1 f _i,j *w _i,j is output from the final stage.

Thereafter, the spatial product-sum calculation result g is output at intervals of each basic clock time Δt4.

This type of parallel image processing processor 2-
Also, types P and PS as well as types
can be considered, and it is possible to reduce the basic clock time Δt4. Since these can be easily inferred, they are omitted here.

FIG. 11 shows another embodiment with a further different processing form. In contrast to the method described above in which a product-sum load (memory) 15 was applied independently to each PE
In the configuration shown in the figure, the product-sum load (memory) is common to all PE12.
25, and is called a type of parallel image processing processor 2-. This operation will be explained with reference to the time chart in FIG.

First, it is assumed that the image _f14 has already been input from the image data input port 20 at time 1. At this time, f ₁₁ , f ₁₂ , f ₁₃ , and f ₁₄ are given to PEs 12 #1 to #4 via the shift register 11, respectively.
The load w ₁₁ is then read out from the load storage memory 15 and multiplied by each input image. In the arithmetic circuit 20, the value held at the beginning of time 1 is cleared to "0", and the products of the aforementioned _f11 to _f14 and _w11 are held respectively.

At time 2, image _f15 is input, and PE12#1~
#4 is given f ₁₂ ~ ₁₅ respectively and the following loads
The product is taken with w ₁₂ . Thereafter, the arithmetic circuit 20 performs an accumulation process with the previous value. For example, in #1
f ₁₁ *w ₁₁ +f ₁₂ *w ₁₂ , f ₁₂ *w ₁₁ +f ₁₃ *w ₁₂ in #2
is retained as a result.

The same processing as above is executed at times 3 and 4, and the arithmetic circuits 20 #1 to #4 have #1: Σ ¹ ₁₁ = f ₁₁ * w ₁₁ + f ₁₂ * w ₁₂ + f ₁₃ * w ₁₃ + f ₁₄ *
w ₁₄ #2:Σ ¹ ₁₂ =f ₁₂ *w ₁₁ +f ₁₃ *w ₁₂ +f ₁₄ *w ₁₃ +f ₁₅ *
w ₁₄ #3:Σ ¹ ₁₃ =f ₁₃ *w ₁₁ +f ₁₄ *w ₁₂ +f ₁₅ *w ₁₃ +f ₁₆ *
w ₁₄ #4:Σ ¹ ₁₄ =f ₁₄ *w ₁₁ +f ₁₅ *w ₁₂ +f ₁₆ *w ₁₃ +f ₁₇ *
w ₁₄ and the respective first partial sums are obtained, and this is obtained at time Δ
It is set in the shift register 21 at the end of the process.

At times 5-8, each basic module 10A-D
From the shift register 21, Σ ¹ ₁₁ ~ Σ ⁴ ₁₁ , Σ ¹ ₁₂ ~
Σ ⁴ ₁₂ , Σ ¹ ₁₃ to Σ ⁴ ₁₃ , Σ ¹ ₁₄ to Σ ⁴ ₁₄ are sequentially accumulated by the arithmetic circuit 14, and the results g ₁₁ to _{g 14} are output. At the same time, image data f ₁₅ to _{f 18} in PE #1 and image data f 15 to f 18 in PE #2
f ₁₆ ~ f ₁₉ , f ₁₇ ~ f ₂₀ for PE#3, f ₁₈ ~ for PE #4
The same processing as times 1 to 4 is executed for f ₂₁ ,
Find the partial sums Σ ¹ ₁₅ , Σ ¹ ₁₆ , Σ ¹ ₁₇ , Σ ¹ ₁₈ , and calculate from time 9 to 1.
2
These are accumulated and results g ₁₅ to g ₁₈ are obtained.
In this way, spatial product-sum calculation results are continuously output.

This type of parallel image processing processor 2-
Also, types P and PS as well as types
can be considered, and it is possible to reduce the basic clock time Δt5.

According to the present invention, since it is excellent in expandability,
It is possible to use an architecture that is suitable for LSI implementation and can increase processing speed.

[Brief explanation of the drawing]

Figure 1 is a diagram showing the configuration of an image processing system, Figure 2 is a diagram explaining an example of local parallel processing, Figure 3 is a diagram showing an example of local parallel processing,
9 and 11 are block diagrams of a parallel image processing processor to which the present invention is applied, FIGS. 5 and 7 are diagrams of an embodiment of the parallel image processing processor according to the present invention, and FIGS. Figure, Figure 8, Figure 10
12 are diagrams showing time charts of each parallel image processing processor. 2... Parallel image processing processor, 3... Image memory, 10... Image processing processor basic module, 11... Input image shift register, 12...
Processor element, 13...Partial sum calculation circuit, 14...Partial sum accumulation calculation circuit, 15...Load storage memory, 16...Pipeline register, 17
...(variable number of stages) skew correction shift register,
18... Propagation/accumulation calculation circuit, 19... Propagation register, 20... Accumulation calculation circuit, 21... Partial sum output shift register, 24... Image data input port, 25... Image data output port, 30... Calculation result data input port, 35... Calculation result data output port.

Claims (1)

[Scope of Claims] 1. A parallel image processing processor that takes in image data from an image data supply source and performs local parallel image data processing, including an image data input port and a plurality of processors that sequentially take in image data from the image data input ports. a plurality of processor elements that input the contents of each of the shift registers and perform image processing operations; a first arithmetic circuit that adds the operation results of each of the processor elements; a calculation result data input port for inputting calculation result data in the module;
a second arithmetic circuit that adds the arithmetic result data and the arithmetic result of the first arithmetic circuit;
between an operation result data output port that outputs operation result data of an operation circuit, between the shift register and the processor element, between the processor element and the first operation circuit, and between the first operation 1. A parallel image processing processor characterized in that a plurality of sets of image processing processor basic modules each consisting of a circuit and a pipeline register arranged between the circuit and the second arithmetic circuit are arranged in parallel. 2. A parallel image processing processor that takes in image data from an image data supply source and performs local parallel image data processing, including an image data input port, a shift register that sequentially takes in image data from the image data input port, and each of the shift registers. a plurality of processor elements that input the contents of and perform image processing operations; a first arithmetic circuit that adds the operation results of each of the processor elements;
an arithmetic result data input port for inputting arithmetic result data in the preceding basic module; a second arithmetic circuit for adding the arithmetic result data and the arithmetic result of the first arithmetic circuit; between an arithmetic result data output port that outputs arithmetic result data, the shift register and the processor element, between the processor element and the first arithmetic circuit, and between the first arithmetic circuit and the first arithmetic circuit. image processing comprising a first pipeline register disposed between the second arithmetic circuit and the second arithmetic circuit; and a second pipeline register disposed between the arithmetic result data input port and the second arithmetic circuit. A parallel image processing processor characterized in that a plurality of processor basic modules are installed in parallel.