JPH10334225A - Image processor and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To speed up the whole processing system by reducing the load on individual control systems corresponding to respective process stages and a control system regarding the whole processing in the image processor which performs pipeline processes. SOLUTION: Image processes are divided by processing stages and connected in series, and those processing stages are connected by double buffers 8 capable of storing processing results, and the switching of the double buffers 8 and the processing start of each processing stage 7 are controlled by a global controller according to the processing time of the stage having the longest processing time and the states of the input buffer and output buffer 4 before and after the pipeline processor 1. Further, processings at the respective stages are controlled by local controllers 6 provided independently of the respective stages. A host system 2 is permitted to access the double buffers 8 to obtain test easiness and flexibility to modifications of process algorithm.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates generally to pipelined image processing systems and control methods, and more particularly to processing systems employing block-oriented data streams.

[0002]

2. Description of the Related Art Generally, in a system for performing image processing,
In many cases, various calculations or processes are sequentially performed on continuous portions of an original image. In such a system, high-speed processing is often required in order to perform processing in accordance with the processing speed of a connected input device or the processing speed of a connected output device.

In such a situation, in order to achieve the target performance of the system, it is effective to sequentially connect individual processing modules for processing data blocks and perform a pipeline operation.

However, in order for modules having different processes to operate efficiently and accurately in a pipeline, a large load is concentrated on a control system that manages the entire pipeline, which eventually results in a bottleneck of the entire system. It often happens. This is, for example, one of the processing modules
Finally, if there is a module that performs run-length extension, the data volume will increase downstream and the processing cycle will increase at the boundary of the module, so the upstream data cannot be processed at the conventional speed . In such a case, the data is overwritten and erased unless the control system slows down the processing speed of the upstream pipeline. Such processing modules are particularly troublesome because the pipeline processing cycle varies depending on the data. Further, a troublesome problem may occur at a connection point between the pipeline processing device and the external system.

When the processing speed of the pipeline processing device is different from the processing speed of the external system, a buffer for speed buffering is prepared. However, even when a buffer having a sufficient capacity cannot be ensured due to cost relations or restrictions on the buffer area. Many. When such a pipeline device is implemented as an LSI, it is particularly desirable that the pipeline device can be used regardless of the connected buffer capacity in order to achieve a mass production effect. In such a case, it is necessary to accurately stop the pipeline processing unit in response to the buffer empty or buffer full.
Restart control must be possible.

One of the general solutions to such a problem is
First, if a problem occurs or is predicted to occur, part or all of the pipeline processing unit is stopped until the problem is solved. In this case, the stopped pipeline processing unit must save a state for restarting correctly when the problem is solved.
This is a very expensive process. In particular, when performing image processing including compression and decompression on a set of images defined as a block, the amount of data to be processed is large, and the compression / decompression operation itself requires many compression / decompression operations and statistical operations. Since there are many combinations, the pipeline is inevitably lengthened and the amount of state to be controlled is considerably large.

On the other hand, Japanese Patent Application Laid-Open No. 6-187434 discloses that logically or functionally different processing modules are mounted as one stage of a pipeline and connected to the next processing module via buffer storage. Discloses a device that monitors a buffer status in order to perform flow control correctly between adjacent pipeline stages, and controls the pipeline as a local control device that controls the synchronization of adjacent pipeline stages. .

According to the contents disclosed in Japanese Patent Application Laid-Open No. 6-187434, A. A buffer having a size determined by the following relational expression is arranged between the processing stage (i) for executing the process (i) and the processing stage (i + 1) for executing the subsequent process (i + 1).

Relational expression: Buffer capacity is SIZE OF
BUFFER (i), Pmax is the maximum time required to completely process a data block by process (i), RP (i) is the data rate generated by process (i), and RP (i + 1) is Assuming a data rate from the subsequent processing (i + 1), the buffer capacity is represented by the following equation.

[0010]

EQUATION 1 SIZE OF BUFFER (i) = (RP
(I) -RP (i + 1)) × Pmax

B. Further, a buffer having a size determined by the following relational expression is arranged in the interface between the first stage of the processing system and the external system.

Relational expression: The capacity of the buffer is SIZE_SR
Assuming IB and Tmax as the maximum time required to accomplish the non-calculated event, the capacity of the buffer is given by:

[0013]

SIZE_SRIB = (RP (i) −RP (i
+1)) × Tmax

C. If the buffer (i) is not empty (empty), each processing module monitors the empty flag of the buffer and actively executes the “adopt local control mechanism” to start the execution of the processing (i + 1). It operates, thus avoiding concentration of load on the control system and enabling efficient pipeline operation.

However, this is effective only in a system in which the relation of the data input speed of the external system <the processing speed of the pipeline processing system <the data consumption speed of the external system is satisfied. Because, when the data consumption system of the external system is slower than the processing speed of the pipeline processing system, it is necessary to stop the pipeline in order to prevent overwriting of data. In that case, the processing speed of the subsequent stage RP (i + 1) Becomes 0, so that A. And B. This is because data is overwritten with the buffer size defined in the above. That is, the image processing apparatus disclosed here does not support output buffer full.

Also, in the processing of block data, the data output order of the preceding stage processing module often does not match the processing order of the subsequent stage processing module. do not do. Examples of such a method include a two-dimensional DCT operation (discrete cosine transform) and a zigzag scan employed in a compression algorithm known as JPEG. Therefore, the control method disclosed in Japanese Patent Application Laid-Open No. 6-187434 is not suitable for such an application.

In addition, the active operation of each processing module can avoid concentration of load on the processing system to some extent, but this causes a problem that testing and debugging become difficult.

Japanese Unexamined Patent Publication No. Hei 5-298436 discloses a method of controlling a plurality of processing modules connected in series so that they are terminated within the same time. However, a memory is prepared as an input / output buffer. Therefore, only a uniform pipeline without buffer empty or buffer full is mentioned, and the present invention is the purpose of the invention,
It is also different from the action.

[0019]

A first object of the image processing apparatus and the image processing method according to the present invention is to provide an image processing apparatus with an individual control system corresponding to each processing stage and a control system relating to overall processing. An object of the present invention is to increase the speed of the entire processing system by reducing the load.

A second object of the image processing apparatus and the image processing method of the present invention is to ensure the testability of the image processing apparatus executed by the operation in the passive mode controlled by the host system. .

Further, a third object of the image processing apparatus and the image processing method of the present invention is to provide a configuration capable of flexibly responding to a change in a processing algorithm in a processing stage.

[0022]

According to an aspect of the present invention, there is provided an image processing apparatus for processing image data, wherein a plurality of processes are sequentially performed on the image data. A plurality of processing stages connected in series, and a buffer located between the plurality of processing stages, for storing processing results of the preceding processing stage and passing it to the next processing stage, each of which is a buffer for each processing stage. A double buffer composed of two buffers each having a capacity capable of storing the processing result of the stage, and a global control device that outputs a processing start signal (LSTART) for notifying permission to start processing in the processing stage. It is characterized by having.

In the image processing apparatus according to the present invention, a processing start signal (LSTAR) output from the global control device is provided.
T) is characterized in that it also functions as a signal for controlling the switching timing between the buffers of the double buffer.

Further, in the image processing apparatus of the present invention, a processing start signal (LSTART) is received from the global control apparatus, and provided in correspondence with each of the plurality of processing stages.
It is characterized by having a local control device for executing the processing control of each corresponding processing stage.

In the image processing apparatus of the present invention, there is provided an image processing apparatus for processing image data, comprising: a plurality of processing stages connected in series for sequentially executing a plurality of processes on an image; Located between the processing stages of
A buffer for storing the processing result of the preceding processing stage and passing it to the next processing stage, each buffer being composed of two buffers each having a capacity capable of storing the processing result of each processing stage; A global control device for notifying permission to start processing in the processing stage and outputting a processing start signal (LSTART) which also functions as a signal for controlling the switching timing of the buffers in the double buffer, and a global control device , A control signal (CONTROL) is output, and a processing start signal (L
And a host system connected to the double buffer so as to access the double buffer.

In the image processing apparatus according to the present invention, a processing start signal (LSTAR) output from the global control device is provided.
The signal interval of T) is set based on the processing time of the processing stage having the longest processing time among the plurality of processing stages.

Further, in the image processing apparatus of the present invention, an input buffer for outputting processing data to the first processing stage of the plurality of processing stages, and an output buffer for inputting processing data from the last processing stage of the plurality of processing stages. The global controller receives a signal indicating the data accumulation state of the input buffer and the output buffer from the input buffer and the output buffer, respectively, and according to the data accumulation state of each input / output buffer, the global controller It has a configuration in which output control of a processing start signal (LSTART) is performed.

In the image processing apparatus of the present invention, the sum of the processing times of two or more consecutive processing stages among the serially connected processing stages is greater than the processing time of any one of the other processing stages. It is characterized in that each processing stage is configured to be as follows.

Further, the image processing apparatus according to the present invention is characterized in that the double buffer can be selected to load an intermediate result from a preceding processing stage or data from a host. I do.

The image processing apparatus according to the present invention is an image processing apparatus for processing image data, comprising: a plurality of processing stages connected in series for sequentially executing a plurality of processes on an image; Located between the processing stages of
A buffer for storing the processing result of the preceding processing stage and passing it to the next processing stage, each buffer being composed of two buffers each having a capacity capable of storing the processing result of each processing stage; , Processing start signal (LST)
A global control device for outputting an ART), among the plurality of processing stages, a first processing stage is an image analysis stage for performing image analysis for image compression, and a second processing stage is a second processing stage. An image thinning-out stage for thinning out an image using the processing result calculated by the first processing stage, and a third processing stage is a lossless compression for performing lossless compression on the image thinned out in the second processing stage. It is characterized by being constituted by a stage.

Further, according to the image processing method of the present invention, a plurality of processing stages connected in series to sequentially execute a plurality of processes on image data, and a plurality of processing stages located between the plurality of processing stages, Memorize the processing results at the stage,
Buffers for passing to the next processing stage, each having a capacity to store the processing result of each processing stage.
In an image processing method in an image processing apparatus having a double buffer composed of a plurality of buffers, each processing is triggered by a processing start signal (LSTART) output from a global control device connected to a plurality of processing stages. Start image processing on the stage,
A step of performing buffer switching between buffers; and a step of performing image processing in the processing stage by a local control device that is provided corresponding to each of the plurality of processing stages and performs processing control of the corresponding processing stage. Performing the steps.

Further, in the image processing method of the present invention, a plurality of processing stages connected in series to sequentially execute a plurality of processings on image data, and a plurality of processing stages located between the plurality of processing stages, and a preceding processing A buffer for storing the processing result of the stage and passing it to the next processing stage, each of which has two buffers each having a capacity capable of storing the processing result of each processing stage; In an image processing method in an image processing apparatus, a control signal (C) is transmitted from a host system operatively connected to a double buffer to a global controller.
ON), a step of starting image processing in each processing stage and executing the switching between the buffers of the double buffer triggered by the control signal outputting step, and a step of outputting a double buffer from the host system. A step of executing access to the buffer; and executing image processing in the processing stage under control of a local controller provided for each of the plurality of processing stages and executing processing control of the corresponding processing stage. And a step.

[0033]

FIG. 1 shows an example of the configuration of a pipeline image processing apparatus according to the present invention. The pipeline processing device 1 includes:
A host system 2 that supplies block data to the pipeline processing device 1, receives processing results and intermediate results processed by the pipeline processing device 1, and gives instructions to a global control device 5 (described later). It is connected. Further, the pipeline processing device 1 transmits block data from the host system 2 to the pipeline processing device 1.
Are connected to an input buffer 3 for buffering a shift in the data processing speed due to a change in the processing speed of the two. Further, the pipeline processing device 1 is connected to an output buffer 4 for buffering a shift in data processing speed when the host system 2 receives a processing result from the pipeline processing device 1. The global control device 5 monitors the status flags of the input buffer 3 and the output buffer 4, and sometimes switches the double buffer 8 (described later) according to an instruction from the host system 2.
(Described later) is instructed on the operation start timing of each processing stage 7 (described later). When given the operation start timing of the corresponding processing stage 7 (described later) by the global control device 5, the local control device 6 controls the corresponding processing stage 7 up to a certain time by the global control device 5 or other control device. Is performed independently of the local control device 6. The processing stage 7 performs one or a plurality of logical or functional operations, and a unit of the functional operation is referred to as an operation module. The operation processing time of each processing stage is calculated by integrating and reconfiguring the operation modules. For example, it is adjusted so that the processing is performed in substantially the same time. Between each processing stage 7, two buffers having sufficient capacity to hold the intermediate output of the integrated and reconstructed processing stage, a so-called double buffer 8
And the buffer switching timing is instructed by the global controller 5. The contents of the double buffer 8 can be read or written by the host system 2. It has become. The host system 2, the pipeline processing device 1, the input buffer 3, and the output buffer 4
And transfer various data.

FIG. 1 is for explaining the basic configuration of the image processing apparatus according to the present invention. The input data of the block data is not limited to the data from the host system 2 but is used as a scanner. Or a digital camera. Similarly, the output result of the pipeline processing device 1 may be output not only to the host system 2 through the output buffer 4 but also to a display device or a printer. FIG. 1 shows an example in which the number of processing stages is three, but the number of processing stages in the image processing apparatus of the present invention is not limited to three.

[0035]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Three embodiments of the image processing apparatus according to the present invention will be described below. The first embodiment describes details of a processing device common to the present invention, the second embodiment describes a process when the image processing device of the present invention is used as an image data encoding system, and the third embodiment describes a process when the image processing device is used as an image data encoding system. A description will be given of a processing system in which part of a processing stage is performed by a host system in order to show testability and flexibility of the image processing apparatus of the present invention.

Embodiment 1 In FIG. 1, the pipeline processing device 1 has two operation modes. One is that after receiving an operation start instruction from the host system 2, an empty flag (EMP) indicating the empty state of the input buffer 3 is used.
TY) and a full flag (FULL) indicating that the accumulated data in the output buffer 4 is full.
This is a passive mode (or a test mode) in which the operation is performed for a fixed time every time an operation instruction is issued from the host system 2. This mode switching is controlled by the value of bit0 of the configuration register (11) in the pipeline processing device 1 shown in FIG.

In the example of the configuration register (11) in the pipeline processor 1 shown in FIG. 2, when the value of bit0 of the register is 0, the active mode in which the pipeline processor operates actively When the value of the register is 1, the passive mode is activated for a fixed time every time an operation instruction is issued from the host system 2.

The host system 2 is a general-purpose personal computer or workstation, and has a general-purpose bus for I / O connection, a DMA (Direct Memory Access) for managing data transfer between a main memory and an I / O device.
ss) One or more devices are provided, and are responsible for execution of application programs and communication / control with the pipeline processing device 1. For example, if some processing for an image is specified as an application program, and the pipeline processing apparatus 1 performs the processing, the host system 2
Activates the pipeline processing device 1 and, at the same time, transfers the block data expanded in the memory to the input buffer 3 by the DMA device in the host system 2. When the host system 2 receives the result processed by the pipeline processing device 1, another DMA device is used, which transfers the processing result stored in the output buffer 4 to the host system 2.

Further, the host system 2 can set the pipeline processing device 1 to the passive mode, and at this time, the double buffer 8 in the pipeline processing device 1 can be read / written. . It is also possible to instruct the global control device 5 to operate the pipeline processing device 1 for a certain period of time.

The input buffer 3 can receive data from the host system 2 or data from a scanner or a digital camera (not shown). It is possible to output data to the pipeline processing device 1. The output order at this time does not necessarily have to match the input order, and the number of data output in one request is arbitrary. When the input buffer 3 is full of data, the input buffer full flag is activated, and when the input buffer 3 does not hold the data necessary for the first processing stage of the pipeline processing device 1. Has the function of activating the input buffer empty flag.

The output buffer 4 stores processing results output from the pipeline processing apparatus 1 and stores the stored processing results in the host system 2 or an external DM.
The device A transfers the data to the host system 2 or a CRT or printer (not shown). At this time, the accumulation order of the processing results and the transfer order of the accumulated data do not necessarily have to match, and the number of data accumulated at one time is arbitrary. When the output buffer 4 is full of data, the output buffer full flag is activated. When the output buffer 4 becomes empty, the output buffer empty flag is output.
Has the ability to activate the flag.

As described above, the global control device 5 has two operation modes. In the active mode, after the host system 2 is started up, the input buffer empty signal and the output buffer full signal are monitored, and if neither signal is active, the double buffer 8 is switched. A signal (LSTART) notifying that the operation is OK even when the operation of each corresponding processing stage 7 is started is sent to the local control device 6. The LSTART signal is
The basic operation clock of the pipeline control device 1 becomes active only for one cycle, and the next active timing is shown in FIG. 2 unless the input buffer empty signal or the output buffer full signal becomes active during that time. It becomes active at timings spaced by the value written in the block cycle register (11). If during that time the input buffer empty signal or the output buffer full signal becomes active,
The interval until it becomes active for that period is extended. Typically, the block cycle register (1
The value written in 1) is the number of processing cycles of the processing stage having the longest processing cycle.

Each of the plurality of processing stages 7 shown in FIG. 1 executes a different process, and there is a difference in each processing time. The image processing apparatus of the present invention efficiently controls these different processing cycles by combining one global control device 5 and one local control device 6. The start timing of each processing in each processing stage 7 in the pipeline processing device 1 is as follows:
It is controlled by an LSTART signal notified from the global controller 5 to each local controller 6. Therefore, this LS
The interval between the TART signals is basically determined by the processing stage having the longest processing cycle among the plurality of processing stages 7 of the pipeline processing apparatus 1, and is set according to the processing cycle.

In response to an operation instruction from the host system 2, the passive mode (test mode
In the mode), the global controller 5 controls the host system 2
The LSTART signal is activated for one clock cycle only when an instruction is issued from the CPU, and the LSTART signal is not activated unless the next instruction comes. At this time, the input buffer empty signal and the output buffer full signal are ignored, and similarly, the value of the block cycle register shown in FIG. 2 has no meaning.

FIG. 3 shows an implementation example of the global control device 5.
Configuration register (see Fig. 2 (10))
The mode bit (bit0) is used for switching the two-input one-output multiplexer (MUX), and the above-mentioned active mode and passive mode are selected. When detecting the rising edge of the input signal, the pulse generator generates an active pulse for one cycle of the basic clock. The counter is cleared to 0 when a reset signal (not shown) becomes active and when the signal of the coincidence comparator becomes active. If the WAIT signal is not active, the counter counts up on the basis of the basic clock.
When the AIT signal is active, the current count value is held without counting up. The value of the above-described block cycle register is input to the coincidence comparator, and is compared with the count value of the counter. The WAIT signal is an AND connection of the input buffer empty signal and the output buffer full signal. Signals from the host system are connected to the pulse generator through the MUX.

There are as many local control devices 6 as the number of processing stages 7, each of which controls a corresponding processing stage. The local controller 6 receives the LSTART from the global controller 5
When the signal becomes active, a predetermined sequence control is performed. This sequence performs an independent operation that is not affected by the behavior of the global control device 5 and other local control devices 6. In particular, the local controller 6 in the middle except for the first stage connected to the input buffer and the last stage connected to the output buffer is required for the operation when receiving the LSTART signal from the global controller 5. Spaces for writing all input data and all output data are provided in the front and rear double buffers 8, respectively, so that control can be performed without being affected by input buffer empty or output buffer full. Therefore, control is simplified since only the operation sequence needs to be controlled. In contrast, local controllers corresponding to the first and last stages require control corresponding to input buffer empty and output buffer full, respectively.

The processing stage 7 performs one or a plurality of logical or functional operations, and a unit of the functional operation is referred to as an operation module. By integrating and reconfiguring the operation modules, adjustment can be made so that the processing is performed in substantially the same time. This is because the processing time allowed for one processing stage is determined by the processing stage having the longest processing time as described above, so that even if a plurality of continuous operation modules are integrated into one processing stage, the total If the processing time is shorter than the processing time of the processing stage having the longest processing time, it is convenient to combine the processing stages because the double buffer 8 can be reduced.

An operation module having a high parallelism in the processing algorithm and performing a parallel processing with a high degree of parallelism will shorten the processing time. It is also possible to process. Generally, the processing time is shortened by increasing the degree of parallelism, but if the peripheral processing speed cannot follow it, H
/ W resource (circuit scale) is wasted.
Therefore, it is most desirable that all the processing stages be controlled with substantially the same processing time. In some cases, the processing time of each processing stage cannot be made substantially the same depending on the content of the calculation, but even in such a case, the processing speed (output throughput) does not change.

The double buffer 8 is composed of two buffers having a capacity sufficient to hold the intermediate output of the integrated and reconfigured processing stage, and the buffer switching timing is controlled by a global control unit. 5, which is performed by the LSTART signal described above. That is, LS which plays the role of the processing start signal of each processing stage
The TART signal causes data to be input and output from each double buffer 8 to successive processing stages.

All storage elements in double buffer 8 are addressed and read / read from host system 2.
It is writable. The host system 2 can read / write data in the double buffer 8 and the host system 2 can control the global controller 5 in the passive mode. The buffer 8 can be read / written. This adds testability and flexibility to the pipeline processing apparatus of the present invention, which will be described in a third embodiment.

The writing source to the double buffer 8 is controlled by a multiplexer connected between each processing stage 7 and the double buffer 8 as shown in FIG. The multiplexer has a function of selecting either the preceding processing stage 7 or data from the host system 2 and outputting the data to the double buffer 8. The data selection in the multiplexer is based on the source
This is performed based on the value of the data selection register (12).
In the example shown in FIG. 2, each multiplexer (MUX1, MUX1,
2,3. . When the value assigned to n) is 0, the output data of the preceding processing stage is selected, and when the value is 1, the data of the host system is selected.

The bus 9 connects the host system to the pipeline processing device, input buffer, and output buffer, and transfers various data.

[Embodiment 2] FIG. 5 shows an embodiment in which the pipeline image processing apparatus of the present invention is applied as an image data encoding system. The original image to be encoded is
In main memory (not shown) in system 2;
The block is assumed to be an 8 × 8 square 64 pixels. First, the host system 2 sets the pipeline processing device 1 to the active mode, and the DM in the host system 2
The device A is started, and a continuous image area for eight lines is transferred to the input buffer 3.

If the output buffer 4 is not full and one or more blocks are present in the input buffer 3, the global controller 5 switches the double buffer 8 and simultaneously executes the processing stage corresponding to each local controller 6. 7 is notified by an LSTART signal. image·
In the data encoding system, the first stage is an image analysis stage for analyzing block data, in which a level that does not cause a human visual problem even when the data is decimated is calculated, and each pixel is described in a page description. Language (PDL: Pa
This is a stage for performing an operation for identifying whether the image is a PDL image drawn by “ge Descriptive Language” or a scanned image input by a scanner or a copying machine.

FIG. 6 is a diagram showing an intermediate result that each processing stage of the image data encoding system shown in FIG. 5 passes to the next processing stage. FIG. 6A illustrates intermediate data from the image analysis stage which is the first processing stage in FIG. 5 to the image thinning stage which is the next processing stage. The identification information is composed of two 64-bit buffers each of which is a 64-bit buffer corresponding to the block.
The result indicating whether each pixel of the block data is a PDL image or a scanned image is written in a corresponding area as “1” for a PDL image and “0” for a scanned image.

The simplest method for identifying whether a pixel is a PDL image or a scanned image is as follows:
The difference value between adjacent pixels is calculated, and if the difference value is larger than a certain threshold value, it is determined that the image is a PDL image, and if not, it is determined that the image is a scan image. The image quality control information is
This is a plurality of pieces of threshold information obtained by the image analysis circuit of the stage, and controls the image-thinning stage-thinning-thinning-thinning level of the second stage.

The second stage is an image thinning stage in which thinning is actually performed using the processing result of the first stage. Here, an algorithm known as a wavelet is used as the thinning algorithm. At the time of this thinning processing, the thinning level is controlled using the threshold value calculated in the first image analysis stage and passed to the large two stages as image quality control information. Also, the thinning process is not performed on the pixels determined as the PDL image.

Here, the thinning algorithm and the thinning hardware will be described with reference to FIGS. As shown in FIG. 7, first, a data block of 64 pixels is divided into 16 2 × 2 sub-blocks. Next, frequency division is performed on each of the 16 sub-blocks. Assuming that four pixels in a 2 × 2 sub-block are a, b, c, and d as shown in FIG. 7, a low-pass filter operation in the horizontal direction is performed as a first filter operation.
A high-pass filter operation and a low-pass filter operation in the vertical direction are performed. Here, L1 and L2 are horizontal low-pass filter outputs, H1 and H2 are horizontal high-pass filter outputs, and VL1 and VL2 are vertical low-pass filter outputs, each of which is determined by the following equation.

[0059]

L1 = (a + b) / 2, L2 = (c + d) / 2 H1 = (ab) / 2, H2 = (cd) / 2 VL1 = (a + c) / 2, VL2 = (b + d) ) / 2.

Next, a low-pass filter operation and a high-pass filter operation are performed on the outputs (L1, L2, H1, H2) obtained from the first filter operation in the vertical direction as a second filter operation. LL and HL are vertical low-pass filter outputs, and LH and HH are vertical high-pass filters.
Assuming a pass filter output,

[0061]

LL = (L1 + L2) / 2, HL = (H1 + H2)
/ 2 LH = (L1-L2) / 2, H2 = (H1-H2)
/ 2.

Next, pixels to be sampled are determined as shown in FIG. 8 based on the image control information (thresholds: Thl1, Tlh1, Thh1) obtained in the first stage. As a result, when all the 2 × 2 pixel sub-blocks in the 4 × 4 pixel sub-block are sampled only for the LL component, the frequency division is further continued. In the case where the information amount is compressed (decimated) to the maximum, 64 pixels are sampled into one pixel.

FIG. 9 shows a configuration diagram of a thinning circuit.
A total of 21 PEs (processing elements) are connected in a pyramid shape, one at the top, four at the middle, and 16 at the bottom. At the bottom of the pyramid, 16 PEs are placed, and each P
E is responsible for the processing of 4 Pixels, giving a total of 64
Process Pixel. Furthermore, the four higher PEs are responsible for processing the results output by the sixteen PEs. The top PE processes the results output by the four PEs.

When the start of the operation is notified from the local controller 6, the 16 PEs at the bottom first execute the algorithm described with reference to FIGS. The four PEs located in the middle part operate only when all the corresponding lower four PEs are sampled only for the LL component. For example, PE
16 operates only when the corresponding {PE0, PE1, PE4, PE5} are all sampled only as LL components. Similarly, the top PE has four Ps in the middle.
It operates only when all E are sampled only in the LL component.

The execution of the algorithm by the PE is pipelined, the first filter operation is performed in the first stage, the second filter operation is performed in the second stage, and the threshold value is calculated in the third stage. Sampling is performed by the comparison operation, and in the fourth stage, the result is written in order to transmit the sampling result to the PE belonging to the higher order. The PEs belonging to the higher ranks see this result and determine whether or not to operate. Since the operation of each stage is very simple, the operation of one stage can be completed in one basic cycle. As described above, each PE differs only in the data to be operated, and the operation itself is exactly the same.
The data to be processed is independent as long as Therefore, high-speed processing is possible by preparing a plurality of PEs for performing the operation and performing the parallel operation.

FIG. 10 is a flow chart when 16 PEs are operated in parallel. In the worst case, the processing is completed in 12 cycles. That is, first, 16 PEs (PE
0 to 16), the first filter operation (1st s
stage), the second filter operation (2nd tag)
e), further performs sampling (3rd Stage) by a comparison operation with a threshold value, and writes the result (4t
h Stage). 16P on these 4 stages
E is decimated to 4 PEs (PEs 16 to 19), and the same four stages for the 4 PEs are executed.
Similar processing is performed for the PEs (PE20).
When such processing is performed, 64 pixels are thinned out to one pixel. In such a case, the processing time is the worst (longest), but is the maximum compression rate in terms of the compression rate.

Similarly, FIG. 11 is a flow chart in the case of operating four PEs in parallel.
The process is completed in four cycles. That is, PE0-PE
3, PE4-PE7, PE8-PE11, PE12-P
E15 is set as a set to sequentially execute four stages of processing, consuming 4 × 4 16 cycles.
Four cycles are consumed from 6 to processing of PE19, and finally four cycles are consumed for processing of PE20. Thus, a total of 16 + 4 + 4 = 24, which is 24 cycles. Similarly, although not shown, when only two PEs are operated in parallel, the processing is completed in 44 cycles in the worst case.

The third stage of the pipeline processing in the image data encoding system shown in FIG. 5 is a lossless compression stage for performing lossless compression on a decimated image. As shown by the equation (2) in FIG. 6, the lossless compression stage has a lossless compression circuit. As an algorithm, prediction coding using neighboring pixels as reference pixels is used, and Huffman coding and run-length coding are performed on prediction results.

The predictive coding is performed in order from the upper left end to the lower right end of the data block. At this time, in the predictive coding of the thinned data, the predictive coding is performed on the interpolated data in advance. Therefore, in the prediction coding operation, the comparison operation with the reference pixels must be performed in order one pixel at a time. Therefore, even if the comparison operation with a plurality of reference pixels can be completed in one cycle, all the pixels are processed. It takes at least 64 cycles. On the other hand, the first and second stages have a high degree of parallelism in operation, and can perform processing in 64 cycles or less.

Thus, the processing time cycle of the third stage requiring the longest processing time is written in the block cycle register. If the WAIT signal from the input buffer or the output buffer is not active, the global control unit 5 Activates the LSTART signal in synchronization with the processing time cycle of the third stage.

Further, since the third stage requires the longest processing time, it can be understood that it is meaningless if the degree of parallelism of the operations of the first and second stages is made too high. For example, the second
In the example of the stage, it is found that the degree of parallelism of the PE is 2 which is sufficient, so that the circuit scale can be suppressed.

The processing time cycle requiring the longest processing time is set in the block cycle register, and on the condition that this time has elapsed (confirmation by the coincidence comparator in FIG. 3), the LSTART interval is set. Then, by outputting from the global control device 5 to the local control device 6, data can be input / output between the processing stages 7 (via the double buffer 8), and the smooth pipeline processing can be performed. Can be controlled.

[Embodiment 3] FIG. 12 shows a pipeline image processing system of the present invention in a test or passive mode or a test mode, that is, a mode in which the pipeline image processing system operates for a predetermined time in response to an operation instruction from the host system 2.
1 illustrates an embodiment in which an image data encoding system is used. The major difference from the embodiment of FIG. 5 is that the device operates in passive mode and the global controller 5 does not rely on the input buffer empty flag or the output buffer full flag, (CONT
ROL signal).

At the time of testing the pipeline processing apparatus, first, the host data of FIG. 2 is sent to the double buffer 8 at the stage before the processing stage in which the host system 2 performs the test so that the test data from the host system 2 can be sent. Set the selection register appropriately. Next, host system 2
Sends test data to the pipeline controller 1 and, when it is finished, activates the CONTROL signal connected to the global controller 5. In the passive mode, the global controller 5 captures a rising edge of the CONTROL signal and generates a pulse for one cycle of the basic clock. When this pulse becomes active, the local controller 6 starts the operation of the corresponding processing stage. Next, after a certain period of time, the host system sets the host data selection register of FIG. 2 so that the data from the double buffer 8 in which the processing result of the tested processing stage is stored can be read in order to confirm the processing result. Then, the test is performed by reading the double buffer 8 and comparing it with the expected value calculated in advance.

At this time, since the pipeline processing device implemented by hardware is sufficiently fast, the host system 2
After activating the CONTROL signal, the double buffer 8 where the processing result is stored without pause
You can choose and lead it.

Next, a more flexible image data encoding system to which the image processing apparatus of the present invention is applied will be described with reference to FIG. Generally, it is difficult to change a function once implemented in hardware (especially LSI) later. In the second embodiment, it has been described that the first stage of the image data encoding system is a stage for analyzing an image.

In this image analysis stage, a threshold for image thinning used in the image thinning stage of the second stage, an operation for identifying whether each pixel is drawn by a PDL image or a scan image, and the like are performed. ing. However, in such an analysis operation, generally, the more complicated the operation result is, the more complicated the operation is required. Therefore, when the processing device is implemented by hardware, a trade-off is made between the processing speed, the circuit scale for the calculation, and the certainty of the calculation result. However, new algorithms may be detected after hardware implementation, or the processing speed of the host system may be significantly improved, allowing the host system to execute more complex algorithms.

The following is such a case, that is, the host
Processing using the execution of the algorithm by the system will be described step by step.

First, the pipeline control device 1
Set to passive mode by system 2. Next, the host system 2 performs an analysis operation on the data block. Next, the host system 2 stores the data in the double buffer 8 before the image thinning stage of the second stage.
The data block itself and PDL /
Transfers the identification information of the scanned image and the image quality control information (threshold information) for thinning out the image.
Activate the signal (input to global controller 5).
Next, the CONTROL signal is deactivated and the analysis operation for the next data block is started.

After the analysis is completed, the same steps as before are repeated until all the data blocks in the original image are completed. Meanwhile, the DMA device in the host system returns the processing result transferred to the output buffer to the host system.

As described above, the CONT of the host system 2
By controlling the global control device 5 with the ROL signal, the operation of each processing stage 7 can be indirectly controlled, and access from the host system 2 to each double buffer 8 can be performed at an arbitrary timing. Can be performed. With this configuration, a new analysis algorithm can be executed by the host system, and processing using more efficient processing, for example, encoding of image data can be supported.

[0082]

As described above, according to the present invention, when a plurality of processes are sequentially performed on an original image divided into blocks, the plurality of processes are divided into a plurality of processing stages and connected in series. These processing stages are connected by a double buffer composed of two buffers large enough to store the processing results of each processing stage, and the switching of the double buffer and the start of processing of each processing stage are set in advance. A global control unit that controls the processing time of the processing stage with the longest processing time in each processing stage and the status of the input buffer and output buffer connected between the host system and the global control device. When the start of processing is announced, the global control unit and other local processing units are set to the processing time of the processing stage with the longest processing time in each of the preset processing stages. The load is distributed to the control device by performing dual control by the local control device that operates independently of the control device, so that scalable and high-speed operation is possible. In addition, global control is performed by an instruction from the host system. What can be done and the double buffer read / write
By having the function performed from the system, it has both testability and flexibility.

In consideration of the processing time of each stage, the processing time of the processing stage having the longest processing time is adjusted.
By integrating and reconfiguring other processing stages to match the timing of data processing and input / output between each stage, it is possible to configure a more efficient pipeline image processing device,・ Usage can be optimized.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration of an embodiment of an image processing apparatus according to the present invention.

FIG. 2 is a diagram showing a configuration of a control register existing in the pipeline control device for controlling the pipeline control device.

FIG. 3 is a diagram showing a configuration of a global control device in the image processing device of the present invention.

FIG. 4 is a diagram illustrating a source data selection function in the image processing apparatus according to the present invention.

FIG. 5 is a configuration diagram of an example in which a pipeline control device in the image processing device of the present invention is implemented as an image data encoding system.

FIG. 6 is a diagram illustrating intermediate data stored in a double buffer in the image data encoding system.

FIG. 7 is a diagram for explaining the details of an image thinning algorithm, and is a diagram corresponding to the first half thereof.

FIG. 8 is a diagram for explaining the details of an image thinning algorithm, and is a diagram corresponding to the latter half thereof.

FIG. 9 is a configuration diagram of an image thinning circuit.

FIG. 10 is a diagram showing a time chart of a thinning algorithm implemented at a degree of parallelism of 16 using 16 processing elements.

FIG. 11 is a diagram showing a time chart of a thinning algorithm implemented with a degree of parallelism of 4 using four processing elements.

FIG. 12 illustrates an embodiment implemented as a test or passive mode image data encoding system.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Pipeline processing unit 2 Host system 3 Input buffer 4 Output buffer 5 Global control unit 6 Local control unit 7 Processing stage 8 Double buffer 9 Bus 10 Configuration register 11 Block cycle register 12 Source data selection register

Claims (11)

[Claims] 1. An image processing apparatus for processing image data, comprising: a plurality of processing stages connected in series for sequentially executing a plurality of processes on the image data; A buffer for storing processing results of the preceding processing stage and transferring the processing results to the next processing stage, each of which has two buffers each having a capacity capable of storing the processing results of each processing stage. An image processing apparatus, comprising: a buffer; and a global control device that outputs a processing start signal (LSTART) for notifying permission of starting processing in the processing stage. 2. The processing start signal (LSTART) output from the global control device also functions as a signal for controlling a timing of switching between the buffers of the double buffer. Image processing device. 3. A local control which receives a processing start signal (LSTART) from the global control device and is provided corresponding to each of the plurality of processing stages, and executes processing control of each corresponding processing stage. The image processing device according to claim 1, further comprising a device. 4. An image processing apparatus for processing image data, comprising: a plurality of processing stages connected in series for sequentially executing a plurality of processes on the image; and a position between the plurality of processing stages. A buffer for storing the processing result of the preceding processing stage and passing it to the next processing stage, each buffer being composed of two buffers each having a capacity capable of storing the processing result of each processing stage. A global control device that outputs a processing start signal (LSTART) that functions as a buffer and a signal that controls the switching timing between the buffers of the double buffer while notifying permission to start processing in the processing stage; The control signal (CONTROL) is sent to the global controller.
L) for controlling the output of the processing start signal (LSTART), and a host system operably connected to the double buffer. 5. A signal interval of a processing start signal (LSTART) output from the global control device is set based on a processing time of a processing stage having a longest processing time among the plurality of processing stages. The image processing apparatus according to claim 1. 6. An input buffer for outputting processing data to a first processing stage of the plurality of processing stages; and an output buffer for inputting processing data from a last processing stage of the plurality of processing stages; The dynamic control device receives a signal indicating the data accumulation state of the input buffer and the output buffer from the input buffer and the output buffer, respectively, and according to the data accumulation state of each input / output buffer, the global control device The image processing apparatus according to any one of claims 1 to 5, wherein output of the processing start signal (LSTART) is controlled. 7. Each of the processing stages connected in series such that the sum of the processing times of two or more consecutive processing stages is longer than the processing time of any one of the other processing stages. The image processing apparatus according to claim 1, wherein the image processing apparatus is configured. 8. The apparatus according to claim 4, wherein said double buffer is configured to select whether to load an intermediate result from a preceding processing stage or data from a host. An image processing device according to any one of the above. 9. An image processing apparatus for processing image data, comprising: a plurality of processing stages connected in series for sequentially executing a plurality of processes on the image; and a position between the plurality of processing stages. A buffer for storing the processing result of the preceding processing stage and passing it to the next processing stage, each buffer being composed of two buffers each having a capacity capable of storing the processing result of each processing stage. A buffer, and a global controller that outputs a processing start signal (LSTART) for notifying permission to start processing in the processing stage, wherein the first processing stage includes image compression. Is an image analysis stage for performing image analysis for the image processing, and the second processing stage thins out an image using the processing result calculated by the first processing stage. An image decimation stage, the third process stage, the image processing apparatus characterized by being constituted by a lossless compression stage performing lossless compression on the decimated image by the second processing stage. 10. A plurality of processing stages connected in series for sequentially executing a plurality of processes on image data, and a processing result at a preceding processing stage, which is located between the plurality of processing stages, is stored. A buffer for passing to the next processing stage, each of which has two buffers each having a capacity capable of storing a processing result of each processing stage; In the above, image processing is started in each processing stage triggered by a processing start signal (LSTART) output from a global control device connected to the plurality of processing stages, and switching between the buffers of the double buffer is performed. And a processing step provided for each of the plurality of processing stages. Image processing method characterized by comprising the step of performing the image processing in the processing stage under the control of the local control device which executes processing control of the di- and. 11. A plurality of processing stages connected in series to sequentially execute a plurality of processes on image data, and a processing result at a preceding processing stage, which is located between the plurality of processing stages, is stored. A buffer for passing to the next processing stage, each of which has two buffers each having a capacity capable of storing a processing result of each processing stage; A control signal output step of outputting a control signal (CONTROL) to the global controller from a host system operably connected to the double buffer; and the control signal output step, Image processing is started at each processing stage, and the double
Performing a buffer switching between buffers; performing an access to the double buffer from the host system; providing the processing corresponding to each of the plurality of processing stages; Performing image processing in the processing stage under the control of a local control device that performs processing control of the stage.