JPS62245378A - Image processor - Google Patents

Abstract

PURPOSE:To attain a fast process with a compact constitution, by performing the data compression of a picture data in a form easy to be processed, decoding and processing a compressed data, and performing a compression process again. CONSTITUTION:The picture data read by a reader 1 is signal-converted by a converter 11, and is converted to a luminance signal and a color difference signal used in an NTSC signal. The signal is compressed in the form easy to apply an edit process by a compressor/decoder 2, and is stored on a picture data filing disk memory 3. The picture data in the disk is read out on an image memory 5, then being processed and edited. An edited result is returned to its original picture data from the image memory 5 through a decoder 6, and after being converted to a chrominance signal corresponding to a printer by a converter 12, it is outputted to a color printer 7. Also, the compressed data, after being decoded at the compressor/decoder 2, and processed at a CPU8, is compressed again at the compressor/decoder 2, and is stored in the image memory 5.

Description

[Detailed description of the invention] Technical fields> The present invention relates to an image processing device that processes image data.

<Conventional technology> Conventionally, high-density color image data is converted.

Alternatively, the editing processing devices include Cytex's Response 300 series and Crossfield's 5TU.
DIO-800 series printing systems are known.

However, these devices have the drawbacks of large size and long processing time.

Purpose of the present invention: Eliminating the drawbacks of the conventional systems mentioned above, the present invention provides an image processing system that is compact, inexpensive, and capable of high-speed processing. The huge amount of image data generated when reading is compressed in a format that is easy to edit, reducing the image data to a fraction of its size, decoding and processing the compressed image data, and compressing it again.

<Embodiment> (Overview) Generally, the above two editing functions are required as functions of an image processing device. The former is generally a hardware pipeline 7'.

This device is called a sensor, and in this device, performs i-image editing functions for certain items that require high speed.

The latter processing by the CPU is executed for items that are interactive with humans (although it may take some time).

That is, the former pipeline processor.

For example, affine transformation (enlargement/
reduction, movement, rotation), spatial filter processing (image enhancement, smoothing, etc.) and one-occupation table (
The latter processing by the CPU mainly performs sequential processing of images such as color conversion processing using LUT (LUT), and processing that is generally difficult to implement in complex processing hardware. These processes are generally creative processes created by the operator's imagination, and can be tolerated even if they take a certain amount of time. However, this function needs to be highly functional.

In order to implement the two editing processor sections with maximum performance, it is necessary to consider the system architecture of the editing device. In other words, in order to be able to perform both processes with sufficient functionality and high speed, it is necessary to construct a system.

It is necessary to consider how to hold (format), signal flow, function analysis, etc. of the increasing amount of image data handled.

As a result of various studies, the following conclusions were reached regarding the system architecture for a color image processing device.

(1) To perform image editing, image data must be stored as compressed data.

(2) As a compression method, block encoding or vector quantization, in which an m×m block is used as one code, is preferable.

(3) In order to perform high-definition image editing, it is necessary to rewrite the image data inside mXm.

In (1), in order to perform image editing processing with 5 high resolutions and high gradations, the image data capacity is extremely large.
When reading in color with t/PIXEL, R, G, B3
The data capacity for each color is approximately 48MB. In order to perform the above-mentioned image editing interactively and with high functionality, it is important to compress the color image data and make it easier to edit. It was concluded that block coding or vector quantization-L method (2) is optimal for this purpose.

This is a method that converts mXm pixels of an image into one fixed-length code, which provides a high compression rate and allows for easy editing because the position information of the image data is retained.

When we have image data using such a compression method, in order to perform high-definition image editing, we need to rewrite the pixel data within the compression unit mXm as the editing result, as shown in (3). ! I noticed K. That is, as a result of editing a set of code data in units of mXm using the vector quantization method, it is necessary to change the code value depending on the editing item.

Hereinafter, the present invention will be explained in detail based on examples.

(Detailed Description of Embodiments) FIG. 1 is a block diagram of an image processing apparatus showing an embodiment of the present invention. Image data read by reader l (
For example, R, G, and B (8-bit digital data each) are converted into signals by the converter 11 and converted into a luminance (Y) signal and color difference signal (1, Q) used in the NTSC signal. Such conversion can be obtained, for example, by matrix calculation using R, G, and H data. Here, the coefficients of the conversion matrix are modified as appropriate in accordance with the color separation characteristics, γ characteristics, etc. of the league. What about such Y, I, Q signals? The data is compressed by the compressor 2 mentioned above and written to the disk memory 3 for image data files. The image data on the disk is read onto an IC memory called an image memory 5 and is processed and edited. Here, basic processing is performed by a hardware pipeline processor 4 in order to perform high-speed processing.

On the other hand, the image data on the image memory 5 is subjected to various processing by the CPU 8, and processing corrections are performed. The editing process is displayed on the color CRTLO by the CRT controller 9, and the editing status can be monitored. The edited result is returned to the original image data from the image memory 5 through the decoder 6, and the converter 12 converts this image data into a color signal (yellow) compatible with the printer.

magenta, cyan, black) and output to the color printer 7.

(compression encoding) Next, a method for compressing image data will be described.

If the spatial frequency of the Y signal, which is luminance data, is well preserved by dividing it into three color signals of luminance and chrominance, such as Y, I, and Q, the spatial frequency of the I, Q signal, which is chrominance data, can be reduced to a certain extent. It is known that there is little visual deterioration in image quality (cutting of high frequency components).

Therefore, a data compression method may be considered in which, for example, the I and Q signals are represented by the average value of m×m blocks (m is an integer), thereby reducing the amount of data of a color image. For the block size of I and Q signals, a block size such as 2X2.4X4.6X6 is selected depending on the required image quality and allowable memory capacity.For example, if the block size is 4x4,
As mentioned above, the memory capacity of A41page is 348MBy.
te is Y signal 16MB'yte (uncompressed)+i, Q signal 2MByte=18MByte in total, resulting in a compression ratio of about 2.7.

On the other hand, for the Y signal, unlike the compression of the I and Q signals, a compression method that sufficiently reduces the resolution data is required. The first method is a block coding method.

This method uses the average value i of pixel data X within mXm blocks.
, the standard deviation σ can be calculated by requantizing the calculated value of 0, for example, (X−X)/σ, which represents the gray scale information for each pixel in about several bits. This compressed data format is as shown in Figure 2(a), where the average value and standard deviation are followed by the grayscale information for each pixel, and the grayscale information is arranged one-to-one according to the pixel position within the block. correspond to

This is the method of bottle protonation of 45 in the city of 45.

This method divides pixel data in an mXm block into the mean value k,
Data compression is measured by expressing the standard deviation σ and a code representing the image pattern. This compressed data format is as shown in FIG. 2(b).

Below is an example in which resolution data is taken only for the Y signal, and average data within mXm is taken for the IQ signal.
The IQ signal may also be encoded in the same format as the Y signal.

This compression form compresses one image consisting of NXNuT elements into m
The compressed image data is cut into X m small sections and converted into a fixed-length code of one bit, and has a memory configuration consisting of bits in the depth direction in an address space of ΣXΣ m. Therefore, the address information of the image is held, and the normal MH, MMH, MR.

Compared to variable code length compression methods such as MMR, it has better random access performance such as image retrieval and writing at arbitrary locations.
It can be said that it is ideal for editing work, etc.

(Image rotation processing) When attempting to scale the image data by one rotation based on the above compressed data, it is necessary to rewrite the compressed data itself.

Figure 3 is an explanatory diagram of this, and ■ in the same figure shows the delimitation of compressed data when image data of mXm pixels is compressed. The compressed pixels will be referred to as "block pixels").

◎ in the same figure is the result of rotating the image for each block pixel,
The letter T is cut into pieces like this. In order to avoid this, the block pixels are decoded and returned to the original image data as shown in ○ in the same figure, and each pixel is rotated and converted into O in the same figure.
You need to do this and then compress it again.

At this time, the m.times.m pixel unit A to be compressed, shown in FIG.

That is, in order to achieve high definition, it is necessary to convert the original image data from block pixels to block pixels.

Now, considering the case where an image is rotated, if the image is decoded and rotated back to the original image data, pixel data will be moved, and the memory capacity required to store the original image data will become extremely large. The worst is 48M, which is the same as the original image data.
An intermediate buffer with a data capacity of 1 byte is required.

Furthermore, when two-dimensional conversion and spatial filtering of one image are directly processed two-dimensionally as in the past, there is a problem in that the processing circuit is more complicated than a one-dimensional processing circuit.

A method for extremely simply realizing the operations of decoding→spatial filtering→rotation-compression encoding or decoding→rotation-spatial filtering→compression decoding will be described below.

First, the rotation method will be described. Generally any angle 0
The rotation of can be realized by the following affine transformation.

The coordinates of the rotated image, θ, is the rotation angle. When this two-dimensional transformation is performed, the coordinates of the image before rotation are 1″ image fl;
1(1')7)'vZ;"(?! a M HIM,
z is generally an irrational number. Therefore, in order to perform rotation with high precision, some non-integer coordinates [W
] data must be obtained by two-dimensional interpolation of the at point (Xil (integer) data ') of the rotated data. Also, as shown in Figure 3, the data resulting from the rotation must be neatly packed in mXm units. This makes recompression difficult unless there is an intermediate buffer of considerable capacity.

Therefore, the two-dimensional affine transformation equation (1) is decomposed into the product of two one-dimensional transformations in the χ direction and the y direction as follows.

The one-dimensional transformation T1-r in the first χ direction is as follows.

In other words, in the one-dimensional transformation T1, the value of y does not change, but the width in the χ direction is reduced by a factor of COSθ.
The rectangle in a) is transformed as shown in (b) by the one-dimensional transformation T1 in the χ direction, and the second one-dimensional transformation T2 in the y direction of zero order is as follows.

That is, in the one-dimensional transformation T2 in the y direction, (b) in FIG.
As shown in (c), the value of χ remains unchanged and the width in the y direction is 1.
/cos An image (C) is obtained which is magnified 0 times and rotated by an angle θ.

As described above, in the one-dimensional transformation T1 in the χ direction.

If one line of data in the χ direction is read out, the coordinates of the converted data are on that line, and the data at the address point of the converted image can be determined by one-dimensional interpolation. That is, processing can be performed line by line in the χ direction. Therefore, if m lines of processing results of one-dimensional transformation T1 are collected, m×
Recompression in units of m is easily possible. Similarly, 1 in the y direction
Regarding the dimensional transformation T2, it is also possible to read one line of data in the y direction and process it line by line. However, the memory for storing compressed data (5 in Figure 1) is
It is necessary to have a configuration that allows access in two directions.

Next, compressed data → decoding → rotation → spatial filtering →
Details of compression encoding and conversion of compressed data→decoding-spatial filtering→rotation→compression encoding will be explained based on the embodiments shown in FIGS. 5 to 9.

FIG. 5(A) is a block diagram of the entire circuit configuration for performing compressed data→decoding→rotation→spatial filtering→compression encoding. The compressed data is divided into two areas 511 and 512: a data area that is human data, and an area where processed data is stored. Before executing processing, the CPU 8 uses a selector 52 to select a data area that will be input data.
Select by instructing 1.

The selector 521 selects one of the compressed data memories 1 and 2 (511 and 512) according to the signal 502 from the CPU 8, and selects the input data side memory address 591 from the address line.
, and the data line is the input compressed data line 50
Connect to 3.

In the case where the other address line is connected to the output data side memory address 592 and the data line is connected to the output side compressed data line 501, the address bus and data bus of both 511 and 512 are completely disconnected (put into high impedance state) It assumes three different states.

Hereinafter, 521 will be explained assuming that the compressed data memory 1 is connected as the input data memory and the compressed data memory 2 is connected as the output data memory. The address calculation circuit 57 receives the main scanning synchronization clock and the sub-scanning synchronization clock, and outputs an input data address 591 of the input compressed data memory, an output data address 592 and an output data address 592 to the output compressed data memory to which the processed data is to be output. , intermediate buffer addresses 593 and 59 required during processing
Outputs 4. The compression memory 511 stores data corresponding to the input data address 591 on the input compressed data line 50.
3, the decoder 53 decodes this and selector 522
The 0 selector 522 outputs to the intermediate memory l or 2 (541 or 542) through the 0 selector 523, which toggles the connection between the intermediate memories l and 2 every time the value of 593 becomes a multiple of m, and operates similarly to the 0 selector 522. However, when 522 connects intermediate memory l, it connects intermediate memory 2, and 522
The data buses 522 and 523 are alternately connected to the data buses 541 and 542, such that when the intermediate memory 2 is connected, the intermediate memory 1 is connected. Further, here, we will proceed with the following discussion taking the case of m=4 as an example. 591 was an address bus, but both the main scanning address and the sub-scanning address were 3, counting from the lower bit.
It consists only of bits above the 22nd bit (that is, the 22nd bit). Further, in the input side intermediate buffer address 593, only 3 bits and 1 bit of the sub-scanning address counting from the lower bit are input to the selectors 522 and 523, and intermediate memories 1 and 2 (541 and 542) are input to the selectors 522 and 523.
) only the main scanning address is input. Intermediate memories 1 and 2 each consist of line buffers for four rasters, and the raster output from decoder 53 is outputted to one of the intermediate memories through selector 522. The same input-side intermediate buffer address is input to the other intermediate memory, and decoded data is written to the address in one intermediate memory. On the other hand, from the same main scanning address of the other intermediate memory, the decoded data of each of the four line buffers one block row before /( is outputted to the interpolation processing circuit 55 through the selector 523. The interpolation processing circuit 55 ,
Input the decimal part of the 594 output side intermediate buffer main scanning address (consisting of an integer part and a decimal part), and create data interpolated from two successive data for each of the four line buffer outputs. The 0 selectors 524 and 525 output the output to the intermediate memory 3 or 4 corresponding to the address given by the integer part of 594 through the selector 524.
is a selector pair that selects intermediate memories 3 and 4 by toggling, similar to the aforementioned selectors 522 and 523. At this time, 524 and 525 are the output side intermediate buffer address 5.
The 1st bit of the 3rd bit counting from the lower bit of the sub-scanning address of 94 is input as a selection signal. The main scanning address of the same output side intermediate buffer address is input to intermediate memories 3 and 4, data from the interpolation processing circuit is input to one memory, and data from the other memory is input. The interpolated data of the previous block row is input to the one-dimensional spatial filtering circuit 58 in parallel for four rasters through the selector 525. -dimensional spatial filtering circuit 58.

It consists of four one-dimensional spatial filters shown in FIG. 9, one corresponding to each raster. The output of 58 is output to compressor 56. The compressor 56 inputs the data, and outputs the compressed data to the selector 521 via the output compressed data bus 501 when the data for 4 pixels from each of the 4 rasters is input. The data on 501 is compressed into data memory 2 (512
) is output to the address specified by the output side compressed data address 592.

Next, the address calculation circuit 57 will be explained. As shown in FIG. 6, reference numerals 57 include a clock switching section 61, an address calculation section 62. Output switching section 63°66 and counters 64, 6
Consists of 5. As mentioned above, for rotation, if the input side address is (χ, y) and the output side address is (X, y),
The relationship expressed by equation (1) holds true. At this time, consider dividing the rotation operation into two terms as shown in equation (2), and set the address in the middle as (
X', Y) and decomposed into equations (3) and (5), they can be written as equations (4) and (6), respectively. Further, when the sub-scan clock treats the χ-th pixel of the y-th raster as the (χ, y) pixel.

Y=r'Cy)=y -------(4)-
〇x'=x' (χ, y)=χcos O-17sin
OX' (0, O) = 0 -1------
(4)-■X'(Oj"l) = U+1)・(
−+in θ)=j[phase](−sinO)+10(−
sinO)=K(0,D+l ・(-8inO)
−−−−−−−−−−(4)−■ X'(i+1.j+1) m X'(0, j+1)
+ (i+1) cos θ= X'(0,j
+1) + i a cos O+ cosθII K
(+, j+1) + cos 0−−−−−
−−−−− (4)−■.

Therefore, Y in the calculation of equation (3) can be calculated only by counting the sub-scanning synchronization clocks, and remains unchanged no matter how many main-scanning synchronization clocks are inserted between the sub-scanning cycle clocks. Therefore,
As shown in FIG. 7(A), the address calculation unit 62 counts up the sub-scanning synchronization clock with a counter 76 and sets it to Y. Further, the value -5inO is set in advance in 711 by the CPU, and O is reset in 731 as the initial value. Adder 721 adds the value held by 731 and 711 each time the sub-scanning clock is input.
Adds the values held by and outputs the sum. 731 latches the output value and holds it as a new value, (4)-
■The zero-order 732 that executes the calculation of the formula is O as the initial value.
is set by the CPU, and the adder 722
The value held by 32 and the value held by 712 are added each time the main scanning clock is input, and the 0 selector 74, which executes the calculation of 4)-■, selects and outputs the output of 731 only when the sub-scanning clock is input. , and others selectively output the output of 722. Further, when at least one of the main scanning clock or the sub-scanning clock is input, 732 latches the output of the selector 74, and takes in the calculation result of (4)-■ or (4)-■ as appropriate. .

Next, in equation (6), it is exactly the same.

X=X (X') =X' ---------(
6)-■Y'= Y'(X',Y') =X: t
anO+y'/cosOY'(0,0)=0-
−−−−−(6)−■Y' (i+1.Q)−(i+1
)tanO+mi @tan θ +tan
O-Y (i, O) + tan O----(8)-
■Y (i+1.j+1)-Y (i*1.0)+
(j+1)) -ε-6-1--7〇-1! =IY (in, 0)・j・cos O”cos O
=Y (i÷1.D◆πd1 −−−−−− (6) −(■).

In this case, the calculation of equation (5) holds exactly the same argument as in equation (3) when the main scanning synchronization clock and the sub-scanning synchronization clock are replaced with 7i. Therefore, in this case, the CPU 8 switches 61, 63, and 66 in advance.

Also, there are seven values to set in 711 and 712, respectively.
t anO and 5 blocks below. This is shown in Figure 7(B).

In addition, the compressed data address and intermediate buffer address are prepared for the input side and output side because the intermediate buffer is the main scanning side address and sub scanning side address of the compressed data memory in the case of equation (3). are associated with the addresses as they are on the main scanning side and sub-scanning side, respectively.

In the case of executing formula (5), the main scanning side address direction of the compressed data memory is made to correspond to the sub-scanning direction address of the intermediate buffer, and the sub-scanning side address direction of the compressed data memory is made to correspond to the main scanning direction address of the intermediate buffer. It is for use. As a result, even if the intermediate buffer is a raster buffer, it can be applied to both equations (3) and (5).

Next, the interpolation processing circuit 55 will be explained.

The interpolation processing circuit 55 receives the decoded data, the decimal part (2-1 bits, 2-2 bits, and 2-3 bits) of the output intermediate buffer address 594, and the 20 bits of the integer part. A total of 4 bits are input. The interpolation processing circuit 55 is a circuit 801 configured as shown in FIG.
Consisting of 4 sets of parts and 1 set of other parts, 801
Each of the parts corresponds to each of the four rasters, one for each of the four rasters, and operates in parallel. The circuit of FIG. 8 includes a latch 811 for holding two consecutive pieces of decoded data.
, 812, a part 802 that detects whether the integer part of the output intermediate buffer address has changed, and a part 803 that outputs a complement III coefficient from the decimal part of the output intermediate buffer address. 802 outputs whether the integer part of the output side intermediate buffer address has changed by detecting whether the least significant bit (20th place bit) of the integer part has changed compared to the previous timing. There is. 803
is from the fractional part of the output intermediate buffer address of two successive clocks.

(1-(X') V (n) +a"V (n-1)
(7) Here, V (i) outputs interpolation coefficients α' and 1-α' for performing interpolation represented by the decoded data of the i-th clock. 821
is a lookup table which inputs the fractional part of the output intermediate buffer address of two successive clocks and outputs α' and l-α'.

Output (1-a') V (n) in equation (7) with the 822 lookup table and α'V(n-1) with the 823 lookup table, and use the 83 adder to calculate (7). It obtains the result of the expression. The results obtained are
It is outputted to the intermediate memory 3 or 4 through the selector 524 in synchronization with the output 84 of the output signal 802 .

Next, the -dimensional spatial filtering circuit shown in FIG. 9 will be explained. Let V'(t) be the output corresponding to the first pixel of the -dimensional spatial filter.

Let the input of the first pixel be V (i).

V' (i) = β2V(i-2)◆βIV(i-1
)+βoV(i)+βI V(i÷1)+β2V(
i+2). Here, β0
゜β1. β2 is a filter coefficient. Before executing the process, the CPU 8 executes βO1β1. β2 is set. It also has four latches arranged in series, each of which latches with the main scanning synchronization clock. Latch 941 latches image data input, and latch 942 latches 941. 943 latches the value latched by 942, and 944 latches the value 943 latches. As a result, V (i-2), V (i-1)
, ----, V (i+2) is obtained. Also, LUP92
1 to 925 are the corresponding filter coefficients (β2.β1.
This is a lookup table that inputs β0) and the corresponding image data V and outputs the product. Adders 931-9
Reference numeral 34 sequentially adds the outputs of each LUT to obtain V'.

As described above, the series of operations has been explained. The intermediate memory 1.2 and the intermediate memory 3.4 are all line memories consisting of 4 raster raster buffers, but the selectors 522, 523 and 524° 525 are used for main scanning, respectively. If configured to switch every 4 clocks, the memory for 4 pixels can be
Of course, it can be used as a tree buffer.

Furthermore, the -dimensional spatial filtering circuit is not necessarily the fifth one.
It is not necessary to place it in the position shown in Figure (A), as long as it is in the middle of the data flow from the decoder to the compressor. In these cases, the configuration shown in FIG. 9 may be used and the operation may be performed in the same manner.

Furthermore, the compressed data memories 1 and 2 are physically the same, and can be used as input compressed data memories during reading and as output compressed data memories during writing. It is clear from equations (3) and (5) that the input compressed data is read into the intermediate memory when reading, and when writing, it is not output to input data areas other than the previously read data area.

As described above, compressed image data can be rotated extremely easily.

In addition, although the rotation process was explained in this embodiment,
The present invention can also be applied to ground processing such as variable magnification processing.

Effects> As described above, the present invention makes it possible to perform image processing based on compression-encoded image data at high speed, with high definition, and with simple hardware.

[Brief explanation of drawings]

FIG. 1 is a block diagram of the image processing device of this embodiment, FIG. 2 is a diagram showing the data format of block encoding and vector quantization, and FIG. 3 is an explanatory diagram of the image rotation method of this embodiment. Figure 4 is an explanatory diagram of a method for rotating an image by one-dimensional transformation, Figures 5 (A), (B), (C), and (D) are image processing block diagrams, and Figure 6 is an address calculation circuit diagram. , FIGS. 7A and 7B are circuit diagrams of the address calculation section, FIG. 8 is an interpolation processing circuit diagram, and FIG. 9 is a one-dimensional spatial filtering circuit diagram.

Claims (4)

[Claims] (1) The image editing processing device includes an image memory that compresses and encodes a block consisting of a plurality of pixels and holds image data, a decoding means that decodes the encoded data, and a one-dimensional transformation process of the decoded data. An image processing apparatus comprising a processing means, a filtering means for performing a one-dimensional spatial filtering process on decoded data, and a compressor for re-encoding the results of the conversion process and the filtering process. (2) In claim 1, the processing means is
, Y orthogonal two directions. (3) In claim 2, the processing means has a line buffer memory of a plurality of lines for storing decoded image data, and the contents of the line buffer memory are linearly processed in the line direction. An image processing device characterized by performing original processing. (4) An image processing apparatus according to claim 3, wherein the processing means performs image rotation processing by performing one-dimensional processing two or more times in the X direction and the Y direction.