JPH04291681A - Filtering processing system for very high speed image processing system - Google Patents

Abstract

PURPOSE:To drastically shorten a filtering processing time by attaining memory access by a hardware configuration. CONSTITUTION:In an image processing system for executing parallel processing in each picture element based upon row parallel type local operation, the terminal address is connected to the leading address. The addressed of a finite length endless type slit memory 4 capable of storing the necessary number of data in a column to be processed are pointed out by a base pointer 1 and a reference pointer 2, the contents of the base pointer 1 are changed one by one by an external controller in each change of a row to be processed and the leading address is cleared following the terminal address. Simultaneously the contents of the pointer 1 are written in the pointer 2, and simultaneously with the specification of the address of the slit memory 4 by the pointer 2, the contents of the pointer 2 are changed only by '1' and the area of the memory 4 corresponding to the mask size of a filter is successively accessed by the pointer 2.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention is an ultra-high-speed image processing system RIP that can perform preprocessing of large-scale images in real time.
E (Real-time Image Process)
This relates to the filtering processing method of the Sssing Engine.

0002

2. Description of the Related Art The need for image processing using computers is increasing in fields such as medical care and industrial production. A near future goal in these application fields is real-time processing of two-dimensional or more multidimensional large-scale images (high-quality images). In order to improve the recognition ability of the system, it is essential to increase the resolution of the image itself, but to do so, it is necessary to increase the number of pixels and the mask size of the filter in preprocessing. be.

The following three items can be considered as examples of specific demands for image processing in recent years.

(2) It is possible to process multivalued and color images having 2048×2048 or more pixels per image.

[0005] ■ Local parallel processing such as filtering can be performed at high speed on the above image, and ■ 50 × 5
It is possible to perform filtering processing using a mask with a size of about 0.

[0006] In image processing, preprocessing centered on filtering is often performed, and as the size of the image and the scale of the filter increase, the calculation time required for preprocessing increases rapidly.

[0007] Image processing systems that have been put to practical use in medical care, industrial production, etc. have been designed to handle images with pixels of about 512 x 512, but images with pixels of about 2048 x 2048
In order to perform preprocessing of complex grayscale images such as those of medical X-ray film, which have a resolution on the order of pixels, it is difficult to perform real-time processing even using a general-purpose large-scale computer such as a supercomputer. In order to realize a dedicated system that efficiently preprocesses such large-scale images, parallelization and pipeline processing are considered effective. In particular, since many filtering processing algorithms frequently use parallel product-sum operations, spatial parallel processing on a pixel-by-pixel basis is considered effective.

[0008]

[Problems to be Solved by the Invention] By the way, data storage in so-called row-parallel local operation, in which parallel processing is performed pixel by pixel by preparing the same number of arithmetic elements as the number of pixels in one row of an image, is as follows: Until now, the use of finite-length memories with linear addresses has mainly been considered. However, when a program attempts to execute row-parallel local operations using such memory, it is necessary to sequentially change (move) the storage location of the original data and intermediate results in memory, and there are no effective methods for doing so. One is the origin (base pointer) of the storage location of original data and intermediate results in memory.
However, in this method, access to the original data and intermediate results must always be done via the base pointer, and operations that change the contents of the base pointer and operations specified by the base pointer There is a problem in that the memory needs to be accessed twice for the operation of reading the contents of the memory, and processing overhead is also generated due to pointer operations at memory boundaries, resulting in longer processing time.

The present invention is intended to solve the above problems, and provides a filtering processing method for an ultra-high-speed image processing system that can significantly shorten filtering processing time by realizing memory access using a hardware configuration. The purpose is to provide.

0010

[Means for Solving the Problems] The present invention provides an input unit consisting of a plurality of input elements that captures all or part of one line of image data in raster scan order, and the image data from each input element is transferred simultaneously. , a processing unit consisting of a plurality of processing elements that perform image processing operations in parallel on a pixel basis, an output unit consisting of a plurality of output elements to which processing data from each processing element is transferred simultaneously, an input unit, a processing unit, In an ultra-high-speed image processing system that is equipped with a controller that controls the output unit and performs image processing in parallel on a pixel-by-pixel basis for each row, each processing element has a memory capacity that can store the required number of columns of data to be processed. an endless memory whose end address is connected to the start address, a base pointer whose contents are changed by 1 at each line break according to instructions from the controller, and where the start address is written next to the end address, and a base pointer where the start address is written after the end address. In addition to writing the contents of the base pointer, it also has a reference pointer that specifies the address of the memory by changing the contents by 1 each time data is read from the endless type memory. The feature is that the type memory area is accessed sequentially.

[0011]

[Operation] The present invention provides an image processing system using row-parallel local operations that performs parallel processing on a pixel-by-pixel basis by preparing the same number of arithmetic elements as the number of pixels in one row of an image or less than the number of pixels in one row of an image.
The end address is connected to the start address, and the address of a finite-length endless type memory (hereinafter referred to as slit memory) that can store the required number of column data to be processed is determined using a base pointer and a reference pointer, and the base pointer is Every time the line to be processed changes, the contents are changed by 1 by the external controller, and the starting address after the end address is reset to 0. At the same time, the contents of the base pointer are written to the reference pointer, and the reference pointer is used to read the address of the slit memory. By changing the contents of the reference pointer by 1 at the same time as the specification, and sequentially accessing the slit memory area corresponding to the filter mask size using the reference pointer, each processing element is made aware of which row it is currently processing. This makes it possible to perform ultra-high-speed processing.

0012

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a diagram for explaining memory access in the present invention, FIG. 2 is a diagram showing the hardware configuration of an image processing system according to the present invention, and FIG. 3 is a diagram showing each image processing element. In the figure, 1 is a base pointer, 2 is a reference pointer, 3 is an adder/subtractor, 4 is a slit memory, 10 is an input unit (LIU), 10-1 to 10-n are latch circuits (IE), and 20 is a processing unit ( LPU), 20-
1 to 20-n are processing elements (PE), 30 is an output unit (LOU), and 30-1 to 30-n are latch circuits (OE).
, 40 is a host computer, 50 is an external controller, 21-i is a selector, 22-i is an arithmetic logic unit (
ALU), 23-i is a register file, 24-i is a flag register, 25-i is a communication controller, 26-i
is a bus.

First, the ultra high-speed image processing system R of the present invention
IPE will be explained with reference to FIGS. 2 and 3.

In the RIPE of the present invention, 65536 gradations (
The processing of each row for grayscale image data (16 bits) is as follows:
It is divided into three stages: input, calculation, and output, and processed in a pipeline to perform input/output of image data and calculation processing in parallel.In the calculation stage, the number of pixels in one row of the image is the same as the number of pixels in one row of the image. Number of PEs (Processing)
SIMD (Singing
le Instruction stream
Since multiple data stream type parallel processing is performed, image data is processed for one row at the same time.

The hardware configuration of the RIPE of the present invention is as shown in FIG. 2, and this system operates as a back-end processor of the host system, and includes an external controller 50, an input unit 10, an arithmetic unit 20,
It consists of an output unit 30. External controller 5
0 operates in synchronization with the host computer 40, input unit 10, arithmetic unit 20, and output unit 30.
It has a RAM that stores processing programs created by the user and a ROM that stores basic processing programs in advance, and executes instructions stored in the RAM or ROM according to instructions from the host computer 40. Each step is sequentially sent to each arithmetic unit 20, and each arithmetic unit functions as a processing machine that executes only what it is instructed to do. Note that the processing program created by the user is downloaded in advance from the host computer 40 to the RAM of the controller 50 prior to processing.

[0016] There are 16 LIUs 10 in charge of the input stage.
It is composed of n bit-width latch circuits (IE) and operates as a shift register. Image data stored in other memory or image data read by a camera is input in raster scan order, and the pixel data is When the pixel data for one row of the image is completed, the pixel data for one row is simultaneously shifted to each PE2 of the LPU 20.
0-1 to 20-n simultaneously and in parallel.

The LPU 20 is composed of n PEs and is in charge of arithmetic stages, and is composed of modules as shown in FIG. FIG. 3 shows the i-th PE 20-i, which sequentially reads data from the latch circuit 10-i into the slit memory 4 and transfers this data to the bus 2.
6-i, the ALU 22-i calculates the intermediate result, stores the intermediate result in the register 23-i, and stores the result in the selector 21-i.
The signal is output to the latch circuit 30-i through the latch circuit 30-i. Each PE receives commands from the controller 50 one step at a time and performs the same processing all at once.No program is stored in its own memory, and it simply operates as a processing machine in response to commands from the outside.

The slit memory 4 has k as shown in FIG.
It is an endless type memory with a bit width address space and a finite length (2k) in which the start address and the end address are connected, and is used to store data necessary for filtering processing. In other words, in local processing for image data, in order to determine the output value of one pixel,
Pixel data in the vicinity of that pixel is also required, and in this case, if all PEs were to have their own necessary data internally, the data would be duplicated in the entire system, which would be uneconomical. Therefore, the slit memory in each PE stores the necessary number of column data that that PE is responsible for processing, that is, the number of vertical pixels of the mask, and the remaining neighboring data is stored in other PEs. communication controller 25-i between adjacent PEs.
By performing data transfer through. In this embodiment, each slit memory has 64 bits each having a width of 16 bits.
Spatial parallel processing can be realized using a mask consisting of cells with a length of 64 or less in the column direction.

The ALU 22-i performs 16-bit arithmetic and logic operations on each pixel. The processing content varies depending on the type of filter processing, but AL
U22-i executes commands given one by one from the external controller.

The register file 23-i is a file to which registers for storing data such as intermediate results are allocated.
16 16-bit wide GR (General Re
A communication register CR (Communication Regis) stores data transferred between adjacent PEs through the communication controller 25-i.
ter), etc., and the flag register 24-i is for storing the contents of flags such as sign, zero, overflow, carry, etc.

When the selector 21-i performs local processing,
In the outer periphery of the image, the calculation results are invalid because neighboring data cannot be completely obtained. Conventional image processing algorithms usually force the output value to 0, or set an appropriate constant or neighboring value depending on the processing content. This function is performed by making it possible to set the output value at the outer periphery of the image to either a constant value or a calculated value, although this is invalid.

In this manner, each PE sequentially reads the necessary data read into the slit memory 4, receives data from an adjacent processing unit through the communication controller, performs filtering processing, and outputs it to the latch circuit.

The LOU 30 in FIG. 2 is composed of n latch circuits each having a width of 16 bits, and serves as an output stage.
The data calculated with 0 is simultaneously transferred to the LOU 30 for one row, and then sequentially shifted and output pixel by pixel in raster scan order.

In such a RIPE system, taking a local averaging filter as shown in FIG. 4 as an example, when the center of the mask is (i, j), the data in the column direction of each pixel is Because it is held within
PE20-j performs addition of column j, that is, ■+■+■, and then PE20-(j-1), which handles column j-1, adds (
Receiving the addition result (■+■+■) of column j-1) and adding it to the addition result of example j, PE2, which is in charge of column (j+1),
Addition result of column (j+1) at 0-(j+1) (■+■+
(2) is received, added to the addition result of the j example and the (j-1) column, and finally divided by 9 to perform local averaging processing.

In this case, for example, the size of the mask is 5×5.
Assuming that, as shown in FIG. 5, data in two adjacent columns needs to be transferred twice. In other words, two P apart
Normally, two transfers are required to receive data from E, but if you transfer data to columns one column next to each other, for example, the data in columns (j-2) and (j+2) will be transferred to the next column (j -1)
Since the data is transferred to and held in the column (j-1) and (j+1), the data in the (j-2) and (j+2) columns is transferred from the adjacent (j-1) and (j+1) columns in the same way as the data in that column. Therefore, all PEs can acquire the necessary data by simply transferring data to the PEs in the next column next to each other.

[0026] In such filtering processing,
In the present invention, as shown in FIG. 1(a), the contents are sequentially changed by 1 each time a row changes from the external controller 50, and the base pointer 1 is cleared when the end of the slit memory is reached, which is necessary for filter processing. Specify the base point of the slit memory, and at the same time write the contents of the base pointer to the reference pointer. For reference pointer 2, write the contents of the base pointer according to the mask size of the filter. For example, if the mask is 5 × 5, when the contents of the base pointer is n, , the reference pointer 2 writes its contents into the memory address register 3 based on the base pointer value n, and also increments its own value by ``+1'' or ``-1'', and by sequentially performing this operation automatically n+4 (or n
Data can be read by specifying addresses up to -4). Then, the value of base pointer 1 is (2k
-1), the value is reset to 0 at line break and returns to the first address. Therefore, the structure of the slit memory is an endless ring, as shown in FIG. 1(b). The contents of this base pointer are updated one by one by the controller each time the line to be read changes, and each PE simply executes filtering processing without being aware of which line it is currently processing.

The image processing of the present invention is performed as shown in FIG.
Either the image data directly read in by the camera 63 is A/D converted and read directly into the RIPE system 60, or the data once read into the memory 61 is read out and then RIPE is performed.
You can either load it into the system or perform either process.
This result can be output to the monitor 64 or written into the memory 61 again in response to an instruction from the host computer 40.

[0028] In the above embodiment, a case has been described in which PEs are prepared for the pixels of one row of an image and image processing for one row is executed simultaneously in parallel, but the present invention is limited to this. Even if the PE is less than the pixels of one line of the image, it can be handled by sequentially overlapping and shifting the PEs, and in this case, change the degree of overlap as appropriate depending on the filter mask size. do it.

[0029]

As described above, according to the present invention, the base pointer whose contents are sequentially changed one by one from the outside and the value specified by the base pointer are set as the origin, and each time a memory address is specified, the base pointer is incremented or The content of the slit memory is read by the reference pointer that is decremented, and the data necessary for filter processing is read by the hardware configuration, so the processing time is several minutes compared to processing by software using normal memory. It becomes possible to reduce the value to 1 or less. In addition to the local averaging filter, the filtering process of the present invention can be applied to various filters such as a Laplacian filter, a Gaussian smoothing filter, a median filter, a pseudo-median filter, and an adaptive smoothing filter.

[Brief explanation of the drawing]

FIG. 1 is a diagram for explaining the hardware configuration of memory access in the present invention.

FIG. 2 is a diagram for explaining the hardware configuration of the image processing system of the present invention.

FIG. 3 is a diagram showing each image processing element.

FIG. 4 is a diagram for explaining filter processing.

FIG. 5 is a diagram for explaining data transfer in filter processing.

FIG. 6 is a diagram for explaining the RIPE system.

[Explanation of symbols]

1...Base pointer, 2...Reference pointer, 3...
Adder/subtractor, 4...Slit memory, 10...Input unit (
IU), 10-1 to 10-n... latch circuit, 20... processing unit (PU), 20-1 to 20-n... processing element (P
E), 30...output unit (OU), 30-1 to 30-
n...Latch circuit, 40...Host computer, 50...External controller, 21-i...Selector
r), 22-i... Arithmetic logic unit (ALU), 23-
i...Register file, 24-i...Flag register, 2
5-i...Communication controller, 26-i...Bus.

Claims (1)

[Claims] Claim 1: An input unit consisting of a plurality of input elements that captures all or part of one row of image data in raster scan order, and the image data from each input element is transferred simultaneously, and the image data is processed in parallel pixel by pixel. A processing unit consisting of a plurality of processing elements that perform processing operations, an output unit consisting of a plurality of output elements to which processing data from each processing element is transferred simultaneously, and a controller that controls the input unit, processing unit, and output unit. In an ultra-high-speed image processing system that sequentially processes images pixel by pixel in parallel for each row, each processing element has a memory capacity that can store the required number of columns of data to be processed, and the end address is the start address. An endless memory connected to the controller, a base pointer whose contents are changed by one at each line break according to commands from the controller, and a start address written next to the end address, and a base pointer whose contents are written at each line break. , and a reference pointer that specifies the address of the memory by changing the contents by 1 each time data is read from the endless memory, and sequentially accesses the area of the endless memory corresponding to the mask size of the filter using the reference pointer. A filtering processing method for an ultra-high-speed image processing system, which is characterized by: