JPS61251972A - Image processing device - Google Patents

Abstract

PURPOSE:To shorten a arithmetic time and to reduce the number of bits of an adder and its cost by adding the multiplied results of the prescribed number of picture elements and executing the inter-image product sum arithmetic. CONSTITUTION:The product sum arithmetic between the prescribed data group corresponding to digitized image information and picture element data corresponding to image information in another frame memory, for instance, is applied to a frame memory where the digitized image information is stored, and the image arithmetic is executed. In this case a frame memory 1 is scanned by a raster scan system, and an adder 4 such as an accumulator adds a multiplied result by a multiplier 3. When the addition by the prescribed number of picture elements is terminated, data is transferred to a master processor 6 from a buffer memory 5 to execute the inter-image product sum arithmetic. As a result functions of hardware for exclusive use of the arithmetic can be reduced.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to an image processing apparatus that performs image calculation processing by performing a product-sum operation on digitized image information.

(Prior Art) In order to recognize or identify an object included in an image captured by an imaging device, an accumulation method is used to correlate with other reference image data and evaluation data. A sum operation (summed product) is performed.

” The sum of products operation is N=ΣΣS1 [i, j]XS2 [i, j]j*
D 1-”.

Here, 31 [i, jl are image data to be processed arranged in a 256×256 matrix, and S2 [
i, jl are other reference image data or evaluation data arranged in the same arrangement as the above-mentioned SL[i, jl.

Such a product-sum operation can be performed by reading corresponding data from the frame memory 2 and the storage unit 3 and sequentially executing the product-sum operation using the CPU shown in FIG.

FIG. 5 shows an example in which a large-capacity adder 5 and a multiplier 6 are used to provide dedicated hardware for executing product-sum operations at high speed.

(Problem to be Solved by the Invention) However, although the device shown in FIG. 4 has the advantage of having fewer components, it is not suitable for calculating a large amount of data from the frame memory 2 because the calculation time in the CPU is long. do not have.

Further, according to the apparatus shown in FIG. 5, special hardware such as the adder 5 and the multiplier 6 makes it possible to perform high-speed multiply-accumulate operations. In order to perform a product-sum operation for pixels, the scale of dedicated hardware becomes large, making it difficult to implement.

The present invention has been made to solve these problems, and an object of the present invention is to provide an image processing device that can shorten the calculation time and at the same time reduce the number of bits of the adder, thereby reducing costs.

(Means for Solving the Problems) The present invention provides an image processing device that extracts image information from a frame memory, performs a product-sum operation, and performs image arithmetic processing. a storage unit that stores a data group; a multiplier that multiplies the pixel data by data corresponding to the pixel data;
an adder that adds the outputs of the multiplier; a buffer memory that holds the output of the adder until the addition of a predetermined number of pixels of the image information is completed; and an arithmetic device that performs a sum-of-products operation.

(Function) The present invention performs a sum-of-products operation on a memory that stores digitized image information and a predetermined data group corresponding thereto, for example, pixel data corresponding to image information in another frame memory. When performing image calculations, the frame memory is scanned using the rask scan method, the multiplication results are added using an adder such as an accumulator, and when the addition for a predetermined number of pixels is completed, the upper part is transferred from the buffer memory. The data is transferred to a processor (CPU) to perform inter-image product-sum calculations, thereby reducing the functionality of hardware dedicated to calculations.

(Example) Hereinafter, an example of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of the present invention, which is a main part of an image processing apparatus that extracts image information from a frame memory 1 in parallel and performs M product-sum calculations to perform image calculation processing. a storage unit 2 that stores a predetermined data group corresponding to the pixel data stored in the frame memory 1; a multiplier 3 that multiplies the pixel data by data corresponding thereto;
an adder 4 that adds the output of this multiplier 3;
A buffer memory 5 that holds the output of the above image information for a predetermined number of pixels until the addition is completed;
and an arithmetic unit 6 which performs spatial product-sum calculations on each of the above-mentioned image information based on the outputs of , and a timing control section 7 which controls the reading of each data and controls the timing of the adder 4 and the buffer memory 5. The timing control section 7 is
It is connected to a higher-level processor 6 via a data input/output circuit 8, and transfers the contents of the buffer memory 5 to the higher-level processor 6 during calculation time in the multiplier 3 and adder 4.

Reference numeral 9 denotes a serial access memory that reads and stores pixel data one row at a time from the frame memory l, and after the adder 4 outputs the addition result of one row of pixel data to the buffer memory 5, the upper processor 6 outputs the data. Forward. Then, the upper processor 6 performs an inter-image product-sum calculation from the addition results transferred for each row, and also judges the image processing results.

FIG. 2 is a block diagram of main parts showing a specific circuit configuration of this embodiment device. In the figure, lO,11 is a first and second frame memory of N rows and N columns that stores, for example, 256 (N) x 256 (N) pixel data, and a data transmitter/register/<25. 26 and a data input/output terminal PI10 connected to the system data bus 15, and a data input/output terminal PI10 connected to the shift register 12.13. an address terminal ADH and a chip select terminal CE connected to the output of the multiplexer 24;
It has a write enable terminal WE and a transfer terminal TR connected to the frame memory read/write control circuit 22. The contents and functions of the main terminals are as follows.

Ilo is a terminal for inputting and outputting one pixel data to and from the main CPU 14 via the bus transmitter/receiver 25 and 26, and is connected to, for example, an 8-bit data line.

PIlo is a terminal for sending and receiving one row of pixel data in parallel with the shift register 12.13. If one pixel is 8 bits, it is connected to the shift register 12.13 by an 8x256 tree signal line. Ru.

ADR: An address specifying one pixel and an address specifying all pixels of one row at the same time are selectively added.

CE, when accessing frame memory 10.11, “
0°'.

TR, pixel data for one row of the frame memory 10.11 (for example, specified by the signal line of the upper half of the address) to or from the shift register 12.13 to the frame memory 10.11 signal for transfer to.

WE, frame memory to, 11 read/write distinction, shift register 12, 13 and frame memory i
A terminal for distinguishing the direction of data transfer for one line between o and li as follows.

When CE is O°', Light when WE is “0”°, WE leads by 1°°.

If TR is “0゛9”, WE is O°゛ and shift register 12° 13 to frame memory 10.11 transfer, WE is “”i” and frame memory io.

11 to shift register 12.13 transfer.

Further, in this embodiment, the shift registers 12 and 13 have a number of stages sufficient to store pixel data for one row of the frame memory 10, and have terminals connected to the frame memories io, tt.
There are a terminal for parallel transfer of pixel data for one row, a terminal to which one pixel data from the multiplexer 23 is added, and a terminal for sending one pixel data to the arithmetic circuit 27.

In the shift registers 12 and 13, each time the shift clock 5CLK sent from the control circuit 22 is applied, the contents are sequentially shifted to the right by one pixel data.

The arithmetic circuit 27 performs, for example, an AND operation between the two pixel data 0UTA and 0UTB added from the shift register 12.13, selected by the function selection signal g from the microprogram controller 19.

The timing of the accumulator 4 and buffer register 5 is similarly controlled by the controller 19, and when the multiplication function is selected by the signal g, the multiplication result MULO between the corresponding pixel data is sent to the accumulator 4 and sequentially added. The addition result is added for one row of pixel data and then held in the buffer register 5.

The video digitizer 18 externally controls the camera by sending vertical and horizontal synchronization signals to a camera such as an ITV camera (not shown), and also uses pixel data obtained by sampling the video signal from the camera at a predetermined cycle. It is sent to bus 28. Such a sampling operation is performed when a video signal capture command is sent from the microprogram controller 19, and the controller 19 is notified of this fact during the sampling period and at the end of sampling as a signal a.

The microprogram controller 19 takes pixel data into the frame memory 10.11, controls inter-pixel calculations, etc., and sends signals with the following contents to the peripheral circuits.

Signal b; Command signal C for video signal capture by the video digitizer 18; Signal signal d indicating the end of processing instructed by the main CPU; Command signal f for incrementing and clearing the address counter 21 by the counter control circuit 20; Control circuit 22, e.g. lead/
Write specification, controller 19 is frame memory 10
．． 11 is used, and a control signal for the shift clock 5CLK.

Signal f': Command to synchronize and clear buffer register 5 and accumulator 4.

The main CPU 14 also has a system data bus 15 and a system address bus 16, and the system data bus 15 includes commands, a start/stop control circuit 17,
A data transmitter/receiver 25, 26 and a buffer register 5 are connected to the system address bus 16.
A multiplexer 24 is connected to. The control circuit 17 receives macro commands such as image capture commands, image calculation commands, stop commands, etc. to the microprogram controller 19, and controls the controller 19 in accordance with these commands.

Address counter 21 is composed of a row counter and a column counter. The row and column counters are cleared or counted up by a signal from the counter control circuit 20, and the contents i of the row and column counters are sent to the multiplexer 24. In addition, each frame memory 10
．． When the number of rows and columns is counted up to 11, an overflow signal is sent to the controller 19 as a signal e.

The frame memory code/write control circuit 22 outputs a switching signal j from multiplexers 23 and 24 in response to a command f from the controller 19.

k, level control signals 1. to terminals WE, TR of frame memories to, ti; m. Send shift clock 5CLK.

Next, the operation of the above-mentioned embodiment apparatus will be explained in each case.

[Capturing pixel data to be processed into the frame memory 10] For example, after placing an object to be processed within the field of view of the camera and setting it to an imaging state, the pixel data to be processed is transferred from the main CPU 14 to the microprogram controller 19 via the control circuit 17. When commanded to import data, the controller 19
is the address counter 2 via the counter control circuit 20.
Clear the row counter and column counter of 1 to zero,
The control circuit 22 controls the multiplexers 23 and 2.
4 to video digitizer bus 28, address counter 21
to instruct the video digitizer 18 to capture the video signal. In response, the video digitizer 18 detects the starting point of the effective area of the video signal (usually the starting point of the first horizontal scanning line), samples the effective area at a predetermined period, and transfers the digital pixel data to the video digitizer bus. Output to 28. It also notifies the microprogram controller 19 that sampling is in progress.

Upon receiving this notification, the controller 19 increments the column counter of the address counter 21 via the counter control circuit 20 at a predetermined period, and causes the control circuit 22 to send the shift clock 5CLK to the shift registers 12 and 13. As a result, the pixel data output to the video digitizer bus 28 is transferred to the multiplexer 2.
3, pixel data is input to shift registers 12 and 13 one by one.

When one row of pixel data is input to the shift registers 12 and 13, the column counter of the address counter 21 overflows. When the controller 19 detects this using the signal e, the column counter is cleared to zero and the control circuit 22 The contents of the first shift register 12 are transferred to the frame memory 10. This transfer is performed by the control circuit 22 between the TR terminal of the first frame memory 10 and the WE terminal.
This is achieved by setting both terminals to "O". At this time, the contents of the row counter of the address counter 21 are 0''
Therefore, one row of pixel data of the first shift register 12 is stored at the address of the first row of the frame memory 10. When the transfer of the contents of the shift register 12 is completed, the contents of the destination counter are counted up.

When the sweep period of the first horizontal scanning line ends, the signal being sampled again is transferred from the video digitizer 18 to the controller 19.
, the same processing as described above is performed again, and the second row pixel data input to the shift register 12 is stored in the second row address of the frame memory 10.

This operation is continued until the last row, and at the end of the last row, when the video digitizer 18 sends a capture end signal to the controller 19, the controller 19 sends the pixel data to be processed to the frame memory 10 using the signal C. The main CPU I 4 is notified that the transfer has been completed.

[Capturing reference pixel data into frame memory 11] For example, when a reference object is placed in the field of view of the camera to set the imaging state, and the main CPU 14 instructs the controller 19 via the control circuit 17 to capture the reference pixel data, the above-mentioned Frame memory 11
Reference pixel data is stored in . However, in order to transfer the contents of the shift register 13 to the frame memory 11, the control circuit 22 sets both the WE terminal and the TR terminal of the frame memory 11 to "0", and sets the WE terminal of the frame memory IO to "0".
Both the terminal and the TR terminal are set to "1".

[Image Arithmetic Processing] When the main CPU 14 issues, for example, an AND operation command to the colon controller 19 via the control circuit 17, the controller 19 switches the arithmetic circuit 27 to multiplication mode using the signal g, and the control circuit 22 switches the multiplexer 24 to the multiplication mode. Switching to the address counter 21 side, the row counter and column counter of the address counter 21 are respectively cleared via the counter control circuit 20. And the third
As shown in the timing chart in the figure, by setting the TR terminal of the frame memory 10911 to "°0" and the WE terminal to "1", the frame memories 10 and 11 indicated by the row counter of the address counter 21 at time t1 The pixel data of the first row is transferred to the shift registers 12 and 13.

When this transfer is completed, the controller 19 causes the control circuit 22 to generate a shift clock 5CLK at a predetermined period, and sequentially shifts the shift registers 12 and 13 to the right by one pixel data. As a result, two pixel data (1) to (256) in the same row and column are sequentially added to the two inputs of the arithmetic circuit 27, and the multiplication result (
0p1 to op256) are repeatedly accumulated in the accumulator 4.

When the controller 19 detects that the inter-pixel calculation for one row has been completed by the overflow signal of the column counter of the address counter 21, the controller 19 causes the control circuit 22 to start the accumulator 4 at time t2 shown in FIG. The contents are transferred to the buffer register 5 and then cleared, and the addition result held in the buffer register 5 is sent to the main CPU via the system bus 15.
Transferred to 14.

When the controller 19 completes the multiplication between pixels for one row as described above, it counts up the destination counter of the address counter 21 and clears the column count.
Also, by setting the TR terminal to "O" and the WE terminal to "'1" at time t3 after setting the TR terminal to "1", the pixel data of the second row of the frame memory 1O111 is transferred to the shift register 12.13. The pixel-to-pixel calculation for moving to the second row is continued using the same processing as described above. In addition, (
1') to (3') are the pixel data of the second row, OPI', O
P2' indicates the result of the multiplication.

The completion of the inter-pixel calculation on the last row can be determined by the contents of the destination counter of the address callan 21, and when the controller 19 determines this, it notifies the main CPU 14 by signal C that the inter-pixel calculation has been completed. .

[Transfer of calculation results from main CPU 14 to frame memory IO] When a stop command is input from main CPU 14 to controller 19 via control circuit 17, controller 19
The control circuit 22 switches the multiplexer 24 to the system address bus 16 side. This allows the main CPU 14 to access the frame memory 10.11, and the data transmitter/receiver 2
5, the calculation results of each row added by the main CPU 14 can be transferred to a predetermined address.

Next, a description will be given of how much hardware logic is reduced by the above embodiment device.

If the memory size of frame memory 10.11 is set to 25
Assuming that 6×256, one pixel data is 8 bits, a 32-bit accumulator is required to perform the inter-image product-sum operation for the entire screen using the conventional device shown in FIG. However, in the device of the above embodiment, the results are transferred to the main CPU for each line, and the vertical integration is carried out by this CPU.
When executed, j=1 to 25 of the total sum of equation (1)
Since it is only necessary to hold 6 operation results, the number can be reduced to 24 bits.

In the above embodiment, the capacity of the shift registers 12 and 13 is set to one row of the frame memories io and it, and the pixel data of one row of the frame memories io and it is processed all at once. The capacity of .13 can be used for the plurality of frame memories 10.11 to process multiple pixel data at once, and the shift register 12.13 can be used for half of one row of frame memories io and 11. It may be set as a capacitor, and the number of bits of the accumulator 4 can be reduced depending on the number of inter-image product-sum calculation circuits.

(Effects of the Invention) As described above, according to the present invention, it is possible to provide an image processing device that can shorten calculation time and at the same time reduce the number of bits of an adder to reduce costs.

[Brief explanation of drawings]

Fig. 1 is a block diagram showing an embodiment of the present invention, Fig. 2 is a diagram showing the circuit configuration of the main part of the embodiment, and Fig. 3 is a diagram showing the operation timing of the embodiment. 5 are block diagrams of conventional devices, respectively. l...Frame memory, 2...Storage unit, 3...Multiplier, 4...Adder, 5...Buffer memory, 6
...Arithmetic device.

Claims (3)

[Claims] (1) In an image processing device that performs image calculation processing by extracting image information from a frame memory and performing a product-sum operation,
a storage unit that stores a predetermined data group corresponding to the pixel data stored in the frame memory; a multiplier that multiplies the pixel data and its corresponding data; and an adder that adds the outputs of the multipliers. and a buffer memory that holds the output of the adder until the addition of a predetermined number of pixels of the image information is completed, and an arithmetic unit that performs a product-sum operation of each of the image information from the output of the buffer memory. An image processing device characterized by: (2) The image processing device according to claim 1, wherein the storage unit is a frame memory other than the frame memory that stores image information different from the frame memory. (3) A serial access memory is provided that reads and stores pixel data one row at a time from the frame memory, and the adder outputs the addition result of one row of pixel data to the buffer memory before the data is sent to the arithmetic unit. The image processing apparatus according to claim 1 or 2, wherein the image processing apparatus is configured to transfer images.