JPH08205026A - Image information processor - Google Patents

Abstract

PURPOSE: To apply image processing such as color conversion, synthesis and edit or the like to the image of a wide range from a still image to a moving image, a pre-format image, an image independently of a transfer rate and an image of a scalerable format. CONSTITUTION: A decoder 1 expands a compressed image fed from an external input device so as to be decoded into an original image. An input buffer memory section 2 converts an image signal into component signals of Y, R-Y, G-Y or R, G, B and a transfer rate of an image signal is converted. An image processing section 3 applies image processing such as color conversion, modification, reduction and geometrical conversion to an image signal. An encoder 4 compresses an image processing signal. An output buffer memory section 5 converts the transfer rate of transmission data being a compressed image processing signal. A control section 6 controls each processing by the decoder 1, the input buffer memory section 2, the image processing section 3, the encoder 4 and the output buffer memory section 5.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image information processing apparatus for performing image processing such as image signal synthesis, transformation, color conversion and the like.

[0002]

2. Description of the Related Art Image information processing apparatuses perform image processing such as image composition, color conversion, and geometric conversion. As shown in FIG. 13, this image information processing apparatus has a composite (composite) signal, a Y / C signal, and a component (Comp.
Write it as onent. ) Receive signal via switcher 71. In this image information processing apparatus, the above image processing is applied to the component signal. For this reason, Y / C signals that are not component signals and composite signals are
After the switcher 71, it is converted into a component signal.

First, the composite signal is separated into Y and C signals by the Y / C separation circuit 72, and then by the decoder 73.
It is converted into Y, RY, and BY signals which are component signals. The Y / C signal is converted into a component signal by the decoder 73. The component signal output from the decoder 73 is supplied to the A / D conversion circuit 74.
The A / D conversion circuit 74 includes a write clock generation circuit 75.
The component signal is converted into a digital signal based on the clock of. This digital component signal is supplied to the frame synchronizer 76.

The frame synchronizer 76 writes the digital component signal in an internal frame memory on the basis of an external clock (referred to as EXT key in the drawing) signal generated by the switcher 71 from the image signal in the standard format. , And adjusts the frame position, the phase of the color subcarrier, and the like so as to be suitable for the digital multi-effect (hereinafter, referred to as DME) processing to be described later. The digital component signal output from the frame synchronizer 76 is supplied to the switcher 77.

Y / C separation circuit 78, decoder 79, A /
The D conversion circuit 80, the write clock generation circuit 81, and the frame synchronizer 82 also perform the same processing as described above on the image signal of the standard format of the other system via the switcher 71. And the frame synchronizer 8
The digital component signal which is the output of 2 is also supplied to the switcher 77. The switcher 77 is also supplied with Y, RY and BY digital component signals from a test pattern signal generating circuit 83 for generating test pattern signals such as color background signals, color bar signals and grid signals.

The switcher 77 switches and selects each of the digital component signals. Then, the digital component signal switched and selected by the switcher 77 is supplied to the two-dimensional variable low-pass filter (LPF) 84. The two-dimensional variable LPF 84 removes a high frequency signal so that aliasing does not occur in the digital component signal. The digital component signal from which the high frequency signal has been removed is supplied to the field memory 85. In this field memory 85, the write and read addresses from the system controller 87 are DM
It is supplied via the E processing unit 88. The system controller 87 also supplies the DME processing unit 88 with the data required for the desired DME processing.

Then, the DME processing unit 88 uses the field memory 85 for the digital component signal via the two-dimensional variable LPF 84 according to the address and data supplied from the system controller 87, for example, image synthesis, transformation, and color. DME processing such as conversion and geometric conversion is performed. Here, for example, when the digital component signal is subjected to deformation processing, the field memory 85
The pixel read from the signal is missing. Therefore, the signal read from the field memory 85 is supplied to the interpolation processing circuit 86 and subjected to predetermined interpolation processing. The interpolation output signal output from the interpolation processing circuit 86 is supplied to the data mixing circuit 89. From the system controller 87 to the data mixing circuit 89, the DME
The same addresses and data that are supplied to the processing circuit 88 are also supplied and mixed with the interpolated output signal. The mixed output signal which is the output signal of the data mixing circuit 89 is
It is supplied to the synthesis circuit 91. The digital component signal delayed by the field delay circuit 90 is also supplied to the synthesizing circuit 91 and is synthesized with the mixed output signal. The combined output of the combining circuit 91 is a so-called special effect output signal, which is converted into an analog signal by the D / A conversion circuit 92, and an analog composite signal Y / C.
It is output in the form of signals and component signals.

[0008]

By the way, in the conventional image information processing apparatus as shown in FIG. 13, input / output signals are limited to standard composite signals, Y / C signals, and component signals. Such as deviating from the standard image signal,
Images with different resolutions, images with different transfer rates, or images with different image sizes could not be handled. Further, even with the same image signal, if the systems are different, such as the HDTV system and the NTSC system, the same device cannot process. That is, in the conventional image information processing apparatus, a so-called free format image signal that does not depend on resolution, an image signal that does not depend on the transfer rate, or a so-called scalable format image signal that does not depend on the image size, It was not possible to handle image signals of different formats. Further, as a matter of course, since compressed images cannot be handled, it is not possible to smoothly switch compressed images that have been subjected to interframe compression processing.

Further, first, the composite signal and the Y / C signal are converted into component signals in the front stage of the switcher 77, and image processing is performed in the rear stage of the switcher 77 using the component signals. Therefore, in the image processing subsequent to the switcher 77, the focus is on processing the entire screen of the component signal in real time. For this reason, it is necessary to perform image processing for the entire screen even for only a part of the screen, and effective use such as activating resources for other processing cannot be performed. It was impossible to use it efficiently.

The present invention has been made in view of the above circumstances, and is a so-called free format image signal that does not depend on resolution, an image signal that does not depend on transfer rate, or an image signal that does not depend on image size. Various types of image processing can be performed on so-called scalable format image signals and image signals of different formats, and compressed images that have undergone interframe compression processing can be switched smoothly and used efficiently. It is an object of the present invention to provide an image information processing device capable of performing the above.

[0011]

An image information processing apparatus according to the present invention is an image processing apparatus which expands an input compressed image signal, and an image obtained by performing various image processing on the image signal output from the expansion apparatus. Image processing means for outputting a processed signal;
A compression unit that compresses the image processing signal output from the image processing unit and outputs a compressed image processing signal, a control that controls the decompression process of the decompression unit, the image processing of the image processing unit, and the compression process of the compression unit. Means to solve the above problems.

Here, the control means causes the expansion processing of the expansion means, the image processing of the image processing means, and the compression processing of the compression means to be executed in real time. Also,
It may be executed in non-real time. Further, the decompression means, based on the input compressed image signal, compressed / non-compressed state identification information, compression method information, image size information according to the number of pixels in the horizontal and vertical directions, and processing screen size information for determining a processing area. , Attribute information including image format information and input / output signal rate information is read out and supplied to the control means.
Note that this attribute information may be directly supplied from the outside, for example, by a keyboard.

Also, in this image information processing apparatus, the first input / output unit causes the image signal output from the expansion unit to be input / output, and the second input / output unit outputs the image signal from the compression unit. The compressed image processing signal is input / output. Then, the control means is caused to execute the input / output processing of the first input / output means, the image processing of the image processing means, and the input / output processing of the second input / output means in real time.

Here, the control means controls the decompression means, the first input / output means, the image processing means, the compression means, and the second input / output means in accordance with the attribute information. Further, the first input / output unit includes a system conversion unit that converts the system of the image signal, and a storage unit that has a sufficient capacity, regardless of the image size of the image signal. Therefore, the first input / output unit can convert the input image signal into a component signal and the transfer rate of the input image.

Further, the second input / output means has a storage section having a sufficient capacity which is independent of the image size of the image signal. Further, the compression means includes a system conversion unit that converts the system of the image signal. Also,
The decompression means of the image information processing apparatus according to the present invention is supplied with the compressed image signal which is switched and selected by the switching selection means which switches and outputs a plurality of compressed image signals. In this case, the switching selection means switches and selects the plurality of compressed image signals synchronized in a predetermined number of frames in the predetermined number of frames.

Further, the image information processing apparatus according to the present invention is
A switching selection unit that selectively switches and outputs a plurality of input compressed image signals, a decompression unit that decompresses the compressed image signal via the switching selection unit, and the image signal output from the decompression unit. A first input / output unit that inputs and outputs, an image processing unit that performs various image processing on the image signal output from the first input and output unit, and outputs an image processing signal, and an output from the image processing unit A hardware module unit including a compression unit that compresses the generated image processing signal and outputs the compressed image processing signal; and a second module that inputs and outputs the compressed image processing signal output from the compression unit; Expansion processing of the expansion unit in the module unit, input / output processing of the first input / output unit, image processing of the image processing unit, compression processing of the compression unit, input / output processing of the second input / output unit Control hardware Has a software control unit and a software unit that sets the connection of each unit of the hard module unit via the hard module control unit and performs a role assignment process.The software unit freely selects the hard module unit. The above problem is solved by performing intelligent image processing.

[0017]

In the image information processing apparatus according to the present invention, the decompressing means decompresses the input compressed image signal and outputs the resulting image signal, and the image processing means performs various image processing to output the image processed signal and compress it. Means compress the image processed signal. Here, the expansion means, the image processing means, and the compression means are controlled by the control means.

Further, in this image information processing apparatus, the image signal output from the expansion means is input to and output from the first input / output means, and the compression means is output to the second input / output means. The compressed image processing signal is input and output. For example, the first input / output unit includes a system conversion unit that converts the system of the image signal, and a storage unit that has a sufficient capacity regardless of the image size of the image signal. Therefore, the first input / output unit can convert the input image signal into a component signal and the transfer rate of the input image.

Further, the image information processing apparatus according to the present invention is
Since the software module freely selects the hardware module section to perform intelligent image processing, the flexibility and expandability of the entire processing of the device can be enhanced.

[0020]

Embodiments of an image information processing apparatus according to the present invention will be described below with reference to the drawings. An image and an audio signal associated with the image are input to the image information processing apparatus according to this embodiment. The two signals are separated, processed separately and recombined. The main focus here is on the implementation method that can handle a wide range of images from still images to moving images, so-called free format images that do not depend on resolution, images that do not depend on transfer rate, and images that are so-called scalable format that do not depend on image size. Is set and the same processing is performed for the audio signal, and the description thereof will be omitted.

Hereinafter, this image information processing apparatus will be referred to as JPEG.
(Joint Photographic Coding Experts Group, a study organization for color still image coding), images compressed by standardized coding methods and MPEG (Moving Picture Coding
The explanation will be given assuming that an image compressed by an encoding method standardized by g Experts Group, a study organization for encoding moving image for storage) is treated as an input image.

As shown in FIG. 1, this image information processing apparatus has a decoder 1 having a function of expanding the compressed image supplied from an external input device so as to restore the original image.
And an image signal output from the decoder 1 to Y, RY, G
An input buffer memory unit 2 having a function of converting to a component signal of Y or R, G, B and converting a transfer rate of the image signal; and an input buffer memory unit 2
Image processing unit 3 having a function of performing various kinds of image processing such as color conversion, deformation, reduction, and geometric conversion on an image signal which is an output signal of
An encoder 4 having a function of compressing an image processing signal which is an output signal of the image processing unit 3, and an output buffer memory unit 5 having a function of converting a transfer rate of sending data which is a compressed image processing signal of the encoder 4. , A decoder 1, an input buffer memory unit 2, an image processing unit 3, an encoder 4 and a control unit 6 for controlling each process of the output buffer memory unit 5. The image information processing apparatus also includes a keyboard 7 that is a means for externally supplying processing parameters and processing control data, and a monitor 8 that displays processing results and input images.

The decoder 1 has a function of decompressing a compressed image into an original image, a function of separating an image signal, an audio signal and their attribute information from an input signal, and a communication function with the control section 6. . Here, the attribute information is information indicating the property or characteristic of a signal. For example, in the case of an image signal, compressed / non-compressed state identification information, compression method information, image size information according to the number of pixels in the horizontal / vertical direction, a processing area There is processing screen size information for designating, for example, image system information such as NTSC, PAL, RGB, and input / output signal rate information. This attribute information can also be given to the decoder 1 via the control unit 6.

Specifically, the decoder 1 expands the compressed image signal from the external input device while changing the expansion method according to the instruction of the control unit 6. An image which is not compressed by the compressed / uncompressed state identification information of the attribute information and which does not need to be expanded is not compressed but bypassed. This decoder 1
2, is composed of a decoder 11 for performing decoding such as Huffman decoding, an IDCT circuit 12, and a deblocking circuit 13, and is compressed by the encoding method standardized by the JPEG. JPE for decoding images
G decoder 10, buffer memory 21, variable length code decoder 22, dequantizer 23, and IDCT circuit 24
And an adder 25, a forward compensating circuit 26, a forward + backward compensating circuit 27, a backward compensating circuit 28, a frame memory 29, and a frame memory 30, which are compressed by the encoding method standardized by the above MPEG. And an MPEG decoder 20 for decoding the generated image.

On the input side of the JPEG decoder 10 and the MPEG decoder 20, there is provided a decoder selector 14 for selecting which decoder the data is passed through.
An output selector 15 is provided on the output side of the JPEG decoder 10 and the MPEG decoder 20. The decoder 11 of the JPEG decoder 10 decodes Huffman-encoded data, for example. The IDCT circuit 12 performs a discrete cosine inverse transform process on the decoded data. The deblocking circuit 13 restores the blocked data to an original image.

Next, the MPEG decoder 20 will be described, but before that, an encoding method standardized by MPEG will be described. An image frame compressed by this encoding method includes an I (Intra) frame, P
(Predictive) frame, B (Bidirectional Predicti)
ve) There are three types of frames. The I frame is a frame that is compressed in the frame, the P frame is a frame that is compressed using the prediction information of the previous I frame, and the B frame is a frame that is compressed using the prediction information of the I and P frames that are the previous and next frames. is there. A compressed image is a group of pictures consisting of a set of these frames.
Pictures, hereinafter referred to as GOP. ) Is composed of compression units. GOP has one I-frame,
For example, there are 15 frames such as "IBBPBBPBBPBBPBB".

The buffer memory 21 of the MPEG decoder 20 temporarily stores the data necessary for the subsequent decoding process. The variable-length code decoder 22 decodes the variable-length coded data. The inverse quantizer 23 multiplies the output data of the variable length code decoder 22 by the quantization number and returns it to the value in the frequency domain. The IDCT circuit 24 subjects the output data of the inverse quantizer 23 to discrete cosine inverse transform processing. The forward compensation circuit 26 retrieves image information in the forward direction, which is the same direction as the flow of time, from the frame memory 29 and reconstructs an image. The backward compensation circuit 28 includes a frame memory 3
The image information is reconstructed by extracting the image information in the backward direction, which is the reverse direction of the flow of time from 0. The forward / backward compensation circuit 27 includes a frame memory 29 and a frame memory 30.
Then, the image information in both directions is taken out to reconstruct the image. The adder 25 adds the reconstructed image output from the forward compensation circuit 26, the forward + backward compensation circuit 27, and the backward compensation circuit 28 and the image of the processing result of the IDCT circuit 24. The expansion processing of this decoder 1 is, for example, 8 × 8.
It is parametric so that an image of any size such as a pixel block unit can be accepted, and an image of a size specified according to the image size information of the attribute information can be expanded. For example, in the case of an image with a fraction of 8 × 8 pixel blocks, dummy data is added to make the size non-fractional.

The input buffer memory unit 2 has a function of converting the image signal into, for example, a component signal and a transfer rate of the image signal, and also has a communication function with the control unit 6. As shown in FIG. 3, the input buffer memory unit 2 includes a system conversion encoder 31 and a buffer memory 32 with a rate conversion function. The system conversion encoder 31 converts the composite signal expanded by the decoder 1 or the Y / C signal into Y, RY, GY or R, G, B component signals handled by the image processing of the image processing unit 3. Convert. This method conversion is processed according to the image size information and the image method information of the attribute information given from the control unit 6. However, in the case of simple frame switching or the like, the composite signal or the Y / C signal may be used as it is, and conversion to the component signal is not necessary.

The buffer memory 3 with rate conversion function is also provided.
2 has a sufficient capacity regardless of the image size of the image signal. Writing to the buffer memory with rate conversion function 32 is performed at the rate of input to the buffer memory 32, and reading is performed at the internal processing rate.
The rate of input to the buffer memory 32 is the output rate of the format conversion encoder 31. If the read rate is faster than the write rate, a wait state is entered during the read. Buffer memory 32 with rate conversion function
Consists of two memories, one of which is a write memory and the other of which is a read memory. The roles of read / write alternate. That is, the buffer memory with rate conversion function 32 has a double buffer memory structure. Each of the two memories has an address generator that works independently. The input rate is generated by the address generator in each memory according to the image input / output rate information of the attribute information given from the control unit 6. By adjusting the size of the address generation block and the block address interval, inputs of various rates can be converted into the internal rate of the processing system. In this embodiment, since the image data is handled in blocks, the above rate is an average rate over time.

The image processing unit 3 has a function of performing image processing such as image generation and composition, painting and special effects, and a control unit 6
It also has a communication function with. This image processing unit 3 is shown in FIG.
, The color conversion circuit 33, the variable tap low-pass filter 34, the image memory 35, the interpolation filter 36,
It is composed of a synthesizing circuit 37, an address generator 38, and an image processing control section 39.

The color conversion circuit 33 changes the color of each pixel of the image according to the instruction of the image processing control unit 39. Generally, each color is composed of three colors of R, G, B or Y, RY, BY, and color conversion is performed by changing the mixture ratio. The variable tap low-pass filter 34 has a low-pass filter function for performing anti-aliasing processing prior to reduction processing. This variable tap low pass filter 34
Since the tap coefficient can be changed in accordance with the instruction from the image processing control unit 39, the operating low frequency range can be changed according to the degree of reduction. The low-pass filter function is also used for blurring processing called defocus, which is one of the special effects. The image memory 35 is a working memory for performing coordinate transformation called geometric transformation. The address for conversion is generated by the address generator 38. The interpolation filter 36 has an interpolation function for filling empty pixels generated by coordinate conversion with surrounding pixel values.
The synthesizing circuit 37 synthesizes a plurality of processed images. The address generator 38 generates an address for geometrically transforming the image on the image memory 35. The image processing control unit 39
A control signal is issued to the color conversion circuit 33, the variable tap low-pass filter 34, the address generator 38, the interpolation filter 36, and the synthesizing circuit 37 to instruct the processing. This image processing control unit 3
A control signal from the control unit 6 is supplied to 9.

Here, the processing range of the image and the area where the image is stored in the memory are set by the parameters according to the processing screen size information and the image size information of the attribute information transmitted from the control section 6. As a result, an image of any size can be processed with any processing screen size. The encoder 4 has a function of compressing the image signal, an image format conversion function, and a communication function with the control unit 6. Here, the encoder 4 compresses the image signal while changing the compression method according to the instruction of the control unit 6, but bypasses the image that does not need to be compressed. Further, the encoder 4 uses the image format conversion function to convert the image format by the component described above into an output image format according to the image format information of the attribute information supplied from the control unit 6. Since this conversion is also processed according to the image size information of the attribute information from the control unit 6, any image can be handled. The transmission of the attribute information is performed using the communication function with the control unit 6.

This encoder 4 is, as shown in FIG.
Similar to the decoder 1, the JPEG encoder 40 and M
It is composed of two systems of the PEG encoder 50. These two systems are separated by an encoder selector 17 and combined by an output selector 18. Further, in front of the encoder selector 17, there is provided a system conversion decoder 16 which executes the above-mentioned image system conversion function.

The JPEG encoder 40 comprises a blocking circuit 41, a DCT circuit 42, a quantizer 43, a Huffman encoder 44, a run length encoder 45, and a multiplex circuit 46. On the other hand, the MPEG encoder 50 includes a buffer memory 51, a DCT circuit 52, a quantizer 53, a Huffman encoder 54, a buffer memory 55, a motion vector detection circuit 56, a forward prediction circuit 57, and a backward direction. Prediction circuit 58, frame memory 59, frame memory 60, and inverse quantizer 6
1 and an IDCT circuit 62.

The selection of these two encoders 40 or 50 is performed by the encoder selector 17 according to the compression method information of the attribute information supplied from the control section 6. The blocking circuit 41 of the JPEG encoder 40 divides one image into small blocks, for example, blocks each including 8 × 8 pixels. The DCT circuit 42 performs a discrete cosine transform process on each block including, for example, 8 × 8 pixels. The quantizer 43 quantizes the power of 64 pixel data for each block by dividing the power by 64. Huffman encoder 44
Is a Huffman code for the DC component of, for example, 64 spectra output from the quantizer 43. The run length encoder 45 converts the remaining quantized AC component into a run length code. The multiplex circuit 46 selectively combines the Huffman coded data and the run length coded data.

The buffer memory 51 of the MPEG encoder 50 temporarily stores the data required for the encoding process. Generally, one GOP worth of data is stored.
The DCT circuit 52, like the DCT circuit 42, performs a discrete cosine transform process. The quantizer 53 performs a process of dividing each discrete cosine transform value by the quantization number. The Huffman encoder 54 converts the quantized data into Huffman code. The buffer memory 55 stores the processing result data until a predetermined aggregated result is obtained and output. The motion vector detection circuit 56 is a block of a standard image called a reference frame (generally composed of 16 × 16 pixels).
, To which position in another image, that is, a movement vector is obtained. The forward prediction circuit 57 extracts from the frame memory 60 a block corresponding to a vector obtained from a temporally previous image. The backward prediction circuit 58 is
A block corresponding to a vector obtained from a temporally subsequent image is extracted from the frame memory 59. Inverse quantizer 6
1 cancels the quantization in the quantizer 53 in order to create a coded frame corresponding to a B frame or a P frame. ID
Similarly, the CT circuit 62 performs a discrete cosine inverse transform process in order to solve the discrete cosine transform process in the DCT circuit 52. The frame memories 59 and 60 store the images reproduced by the inverse quantizer 61 and the IDCT circuit 62,
Forward prediction circuit 57 and backward prediction circuit 58, respectively
Save for the prediction process performed in. Buffer memory 5
1 and the DCT circuit 52, the subtractor 63 provided between
The selection piece a of the changeover switch 64 is connected. “0” is supplied to the selected terminal b of the changeover switch 64, the output of the forward prediction circuit 57 is supplied to the selected terminal c, and the output of the backward prediction circuit 58 is supplied to the selected terminal e. The forward prediction circuit 5 is connected to the selected terminal d.
The addition output of the adder 65 for adding the output of 7 and the output of the backward prediction circuit 58 is supplied. Therefore, the subtractor 6
3 subtracts the output of the selected terminal b, c, d or e switched by the selector switch 64 from the output of the buffer memory 51. That is, the subtractor 63 outputs "0" when it cannot predict from the frame to be encoded.
The forward prediction value (extracted block value) is subtracted in the case of forward prediction only, two combined values are subtracted when there are forward and backward predictions, and the backward prediction value is subtracted in the case of only backward prediction value. The adder 65 adds and combines the forward and backward predicted values. In the case of the method of making a prediction frame by averaging preceding and following prediction frames, the adder 66 uses the changeover switch 6 having the selected piece a and the selected terminals b and c.
Used together with 7 to perform frame addition.

The output buffer memory unit 5 includes an encoder 4
In addition to the function of converting the transfer rate of the data sent from the device, the communication function with the control unit 6 is provided. The output buffer memory unit 5 is composed of a buffer memory 69 with a rate conversion function, as shown in FIG. The buffer memory 69 with a rate conversion function temporarily stores a compressed image or an output image for rate adjustment. Like the buffer memory with rate conversion function 32 of the input buffer memory unit 2, the buffer memory with rate conversion function 69 has a sufficient capacity, regardless of the image size of the image signal.
It is a double buffer memory system, and each memory has an address generator that operates independently. The input rate is
The address generators in the respective memories generate addresses according to the image size information and the image input / output rate information of the attribute information given from the control unit 6. By adjusting the size of the address generation block and the block address interval, the internal rate of the processing system can be converted into various output rates. That is, the output buffer memory unit 5 can also perform arbitrary rate conversion.

The control unit 6 has a communication function with the decoder 1, the input buffer memory unit 2, the image processing unit 3, the encoder 4, the output buffer memory unit 5 and the keyboard 8, and the decoder 1, the input buffer memory unit 2 and the image processing. It has a function of controlling each processing of the unit 3, the encoder 4, and the output buffer memory unit 5. Specifically, the control unit 6 controls the decoder 1, the input buffer memory unit 2, the image processing unit 3, the encoder 4, and the output buffer memory unit 5 according to the attribute information and the user's instruction given on the keyboard 7.

In this image information processing apparatus, a matrix switcher 70 is provided between the decoder 1 and the external input device as shown in FIG. 7 to select a plurality of compressed image signals from the external input device. May be. In this case, the matrix switcher 70 switches and selects the plurality of compressed image signals synchronized in GOP units in GOP units.

The operation of the image information processing apparatus configured as described above will be described below. In FIG. 1, a compressed image signal input from an external input device determines compression / non-compression state identification information, compression method information, image size information according to the number of pixels in the horizontal and vertical directions, and a processing area at the beginning of the data. Processing screen size information, such as NTSC, PAL, RGB
It has header information containing attribute information such as image method information and input / output signal rate information.

The decoder 1 reads the header information and sends the attribute information to the control unit 6. The control unit 6 sends the compressed / non-compressed identification information, the compression method information, and the image size information to the decoder 1 again from the attribute information. The decoder 1
The attribute information read by the decoder 1 itself may be used as it is or changed without receiving the attribute information from the control unit 6. The attribute information can also be given from the keyboard 7. The control unit 6 supplies the image size information, the image method information, and the image input / output rate information of the attribute information to the input buffer memory unit 2. Further, the control unit 6 supplies the image size information and the processing screen size information of the attribute information to the image processing unit 3. In addition, the control unit 6
Is an encoder 4 for the compressed / non-compressed state identification information of the attribute information, compression method information, image size information, and image method information
Supply to. The control unit 6 also supplies the image size information and the image input / output rate information of the attribute information to the output buffer memory unit 5.

The input compressed image signal is supplied to the decoder 1 directly from the external input device as shown in FIG. 1 and after passing through the matrix switcher 70 as shown in FIG. It may be supplied to the decoder 1. When the matrix switcher 70 is used, a plurality of compressed image signals are selected by the instruction of the control unit 6.
Switching of input image signals is performed in frame units. However, in the case of a compressed image using inter-frame processing, switching is performed in units of a plurality of frames, for example, in MPEG, as described above, in units of GOP. Here, each compressed image using the inter-frame processing includes GOP length, I picture,
It is assumed that the P picture and the B picture have the same configuration and the GOP phases are the same. When a switching instruction is supplied from the control unit 6 to the matrix switcher 70, the matrix switcher 70 looks at the header information indicating the head of the GOP supplied from the decoder 1 and switches the input image signal in GOP units. The image switching timing is later than the instruction, and in general, G
It becomes the head of OP. This matrix switcher 70
For example, it is possible to switch input image signals in GOP units as shown in FIGS. GOP shown in FIG.
Is composed of a total of 15 frames of "IBBPBBPBBPBBPBB". In addition, the GO shown in FIG.
P is 16 in total of "IBPBPBPBPBPBPBPBP"
It is composed of frames. In addition, G shown in FIG.
The OP is composed of a total of 6 frames of "IBBBBB".

According to the attribute information from the control unit 6, the decoder 1 outputs a signal that does not need decompression processing to the input buffer memory unit 2 as it is, while performing a decompression process selected according to the selection information on a signal that requires decompression processing. Give. The decompression process determines the processing range according to the image size information. This allows the decoder 1 to expand an image of any size. The input buffer memory unit 2 uses the system conversion encoder 31 according to the image system information from the control unit 6 to system-convert the above image signal of the composite signal or the Y / C signal into a component signal suitable for internal processing. At the time of this system conversion processing, the processing range is determined according to the image size information from the control unit 6. As a result, the system conversion process in the input buffer memory unit 2 is also effective for images of arbitrary size.
Since the component signal is converted into the internal processing rate, the component signal is written into the buffer memory 32 with the rate conversion function at the input rate and read again at the internal rate. The buffer memory 32 with a rate conversion function has a double buffer structure, which is independent of the image size and has a sufficient capacity as described above, has independent address generators, and reads and writes at different block rates. . Since the address area generated by the address generator is determined according to the image size information from the control unit 6, the conversion of the transfer rate in the input buffer memory unit 2 is effective even for an image of an arbitrary size. .

According to the image size information from the control unit 6, the image processing unit 3 designates the end point P _E (x, y) in the area of the image memory 35 in which an image of an arbitrary size is stored. Secure and read / write address range. Accordingly, the image processing unit 3 can perform image processing on an image of any size. Further, the image processing range is determined according to the processing screen size information from the control unit 6. As a result, the image processing unit 3 can use the processing system resources only for the processing within the specified range, and the wasteful processing system resources used for the processing that were not reflected in the conventional result can be effectively used for other processing. Became. Here, this image processing unit 3
The image processing of will be described. Input buffer memory unit 2
The image signal which is the output of the above is supplied to the image processing unit 3.
The image signal that has entered the image processing unit 3 is subjected to various image processes in accordance with instructions from the control unit 6. As shown in FIG. 4, the image processing unit 3 includes an image processing control unit 39 that controls the image processing unit 3. Color conversion in units of pixels or blocks of a plurality of pixels is performed by the color conversion circuit 33 according to an instruction from the image processing control unit 39. When color conversion is not necessary, the color conversion circuit 33 bypasses the image signal according to an instruction from the image processing control unit 39. The image signal that has passed through the color conversion circuit 33 is supplied to the variable tap low-pass filter 34, and the high-frequency signal is removed in preparation for the deformation and reduction performed by the circuit in the later stage described later. The process of removing the high-frequency signal is necessary to prevent the high-frequency signal from adversely affecting the low-frequency signal in terms of frequency, such as aliasing due to the deformation or reduction process. How much the high frequency signal is removed is determined by the image processing control unit 39 by instructing the high frequency band to be removed depending on the degree of deformation or reduction. If you don't need to limit the bandwidth,
This processing function is bypassed by an instruction from the image processing control unit 39. The image signal that has passed through the variable tap low-pass filter 34 is supplied to the image memory 35. This image memory 35
Is a working memory for performing two-dimensional and three-dimensional geometric transformations. Address generation for geometric transformation is performed by the address generator 38 and supplied to the image memory 35.
The image processing control unit 39 instructs what kind of address is generated by the address generator 38. This instruction is generally given by modeling data and conversion rule data.
The data group read from the image memory 35 is generally incomplete as raster data and has many missing pixels. The interpolation filter 36 performs an interpolation process for filling in this omission using the surrounding pixels. The image processing control unit 39 instructs the accuracy of interpolation. As the interpolation method, there are a nearest neighbor method, a linear interpolation method, a cubic interpolation method, and the like, and the latter gives a more accurate interpolation value. The raster signal in which the voids are filled with the interpolation is supplied to the synthesizing circuit 37. The synthesizing circuit 37 synthesizes a plurality of processed images two-dimensionally or three-dimensionally. The image processing control unit 39 supplies a control signal such as depth information at the time of composition. At the final stage of composition, the image is converted into a two-dimensional image and displayed on the monitor 8.

Further, the two-dimensional image is supplied to the encoder 4, and the format is converted and compressed. The format conversion from the component signal is determined according to the image format information of the attribute information supplied from the control unit 6. Whether to compress or not and the compression method are determined according to the compression / non-compression state identification information of the attribute information supplied from the control unit 6 and the compression method information. If the format conversion processing and the compression processing are not required, they are not processed and sent to the output buffer memory unit 5. At the time of the format conversion processing and the compression processing, processing of the image range designated according to the image size information from the control unit 6 is performed. As a result, the encoder 4 can handle images of any size.

The output of the encoder 4 enters the output buffer memory section 5. The output buffer memory unit 5 is similar to the buffer memory with rate conversion function 32 of the input buffer memory unit 2 and has a sufficient capacity regardless of the image size, has a double buffer structure, and has an independent address generator. Since the buffer memory 69 with a rate conversion function for reading and writing at different block rates is provided, the rate conversion of the sending data can be performed. Since the address area generated by the address generator is determined in accordance with the image size information from the control section 6, the output buffer memory section 5 can also convert the rate of an image having an arbitrary size.

As described above, the image information processing apparatus according to the present embodiment has a wide range of images from still images to moving images, free format images that do not depend on resolution, images that do not depend on transfer rate, and image sizes that are not dependent on transfer rate. Image processing such as color conversion, composition, and editing can be performed on an independent scalable format image. Further, since the decoder 1 can be supplied with a compressed image signal from the matrix switcher 70 for switching and selecting the plurality of compressed image signals synchronized in a predetermined number of frames in the predetermined number of frames, It is possible to smoothly switch compressed images that have been subjected to inter-frame compression processing. Further, since the image processing unit 3 can determine the image processing area in accordance with the processing screen size information, it is possible to effectively use the wasteful processing system resources used for the processing that are not reflected in the result in the other processing. It became so.

Next, a modification of the embodiment of the image information processing apparatus according to the present invention will be described with reference to FIG. In this modification, the hardware processing unit 100 and the software processing unit 110 are included.
And an input / output control unit 120 and a data storage unit 130. In this modified example, the processing is divided into a hardware processing unit 100 and a software processing unit 110, so that the flexibility and expandability of the entire processing of the device are enhanced. The hardware processing unit 100 is
Mainly performs mechanical processing such as filtering and software processing, which has a heavy load. The software processing unit 110 performs intelligent processing and processing with high expandability.

The hardware processing unit 100 is the same as that shown in FIGS.
The configuration is almost the same as that of the image information processing apparatus of the above-described embodiment described using FIG. That is, for example, two external input devices 101 and 10 such as a VTR or a data recorder.
The plurality of compressed images supplied from 2 are switched by the matrix switcher 70 and supplied to the decoder 1.
It is expanded by the decoder 1. The image signal expanded by the decoder 1 is supplied to the input buffer memory unit 2. The signals passed through the input buffer memory unit 2 are the color converter 33, the variable tap low-pass filter 34, and the frame memory 3.
5, the interpolation filter 36, the synthesis circuit 37, and the address generator 38. These units are connected by a local bus 103 so that each function of hardware can be used as a hardware module that can be used by software during software processing. The image signal image-processed by the image processing unit 3 is output to the external output device 104 via the encoder 4 and the output buffer memory unit 5.
Here, each unit described above is controlled by the hardware module control unit 105.

The software processing section 110 includes a CPU 111, a cache memory 112, a main memory 113 and a CPU bus 1.
14 and a memory bus control unit 115. The memory bus control unit 115 is connected to the local bus 103 and the hardware module control unit 105.
Image data and a control signal to the hardware module controller 105. The memory bus control unit 115 is connected to the graphic monitor control unit 1 via the peripheral bus 116.
21, a media control unit 122, and a switcher control unit 1
23, a small computer system interface (hereinafter referred to as SCSI) adapter 124, and an operation panel 125. The graphic monitor control unit 121 has a built-in video memory and controls the display of the graphic monitor 126 using the video memory. The media control unit 122 includes external input devices 101 and 102 such as VTRs and data recorders, and external output devices 10 such as VTRs and data recorders.
4 controls the image information input / output timing. The switcher controller 123 controls the matrix switcher 70. The SCSI adapter 124 is a magneto-optical disk (indicated as MO in the figure) device 1 connected by a SCSI bus 127.
31, a CD-ROM 132, a hard disk device (referred to as HDD in the drawing) 133, and the like as an interface of a data storage device. The operation panel 125 is used to instruct processing and input / output of image information.

A method of realizing a hardware module that can be used from software is such that when a subroutine library is called from a program being executed by the CPU 111, if the subroutine library is a soft library, the program address corresponding to the subroutine library is determined at linking. It jumps and is executed. Also, when the subroutine library is a hard library, the address jumps to the address corresponding to the hard module determined at the time of linking. The address corresponding to the hard module stores the procedure of sending necessary data to the hard module, starting the execution, confirming the end of the execution, and returning the execution to the main program of the software. For example, when the variable tap low-pass filter 34 is called as a hardware module, the CPU 1
According to the instruction of 11, data is sent from the main memory 113 to the variable tap low-pass filter 34 via the memory bus control unit 115 and the local bus 103. Next, the CPU 111 instructs the variable tap low-pass filter 34 to execute via the memory bus control unit 115 and the local bus 103. CPU111
Confirms the end of execution and sends the processed data to the main bus 1 via the local bus 103 and the memory bus control unit 115.
Collect at 13. Then, the process returns to the main program to execute the next step.

As described above, the image information processing apparatus according to the modified example shown in FIG. 12 enhances the flexibility and expandability of the entire processing of the apparatus, and also a wide range of images from still images to moving images and resolutions. Free format images that do not depend on
Image processing such as color conversion, composition, and editing can be performed on an image that does not depend on the transfer rate or a scalable format image that does not depend on the image size. In addition, it is possible to smoothly switch compressed images that have been subjected to inter-frame compression processing. Furthermore, it has become possible to effectively use the wasteful processing system resources used for the processing that were not reflected in the conventional results for other processing.

As another modified example, the CPU 111
An apparatus is conceivable in which a plurality of units including the cache memory 112 and a plurality of units are connected to the CPU bus 114. Therefore, in this other modification, when the load is heavy,
The load can be reduced by performing parallel processing.

[0054]

As is apparent from the above description, according to the image information processing apparatus of the present invention, a wide range of images from still images to moving images, free format images independent of resolution, and transfer rates are provided. It is possible to perform image processing such as color conversion, composition, and editing on an image that does not depend on the image and a scalable format image that does not depend on the image size. Further, since the decompression means can supply the compressed image signal from the switching selection means for switching and selecting the plurality of compressed image signals synchronized in a predetermined number of frames in the predetermined number of frames, It is possible to smoothly switch compressed images that have been subjected to inter-frame compression processing. Further, since the image processing means can determine the image processing area according to the processing screen size information, it is possible to effectively use the wasteful processing system resources used for the processing which were not reflected in the result in other processing. Became.

Further, by dividing the processing into the hardware processing section and the software processing section, the image information processing apparatus according to the present invention can increase the flexibility and expandability of the entire processing of the apparatus, and can increase the flexibility of the processing of a still image to a moving image. Image processing such as color conversion, composition, and editing can be performed on a wide range of images, free format images that do not depend on resolution, images that do not depend on transfer rate, and scalable format images that do not depend on image size. In addition, it is possible to smoothly switch compressed images that have been subjected to inter-frame compression processing. Furthermore, it has become possible to effectively use the wasteful processing system resources used for the processing that were not reflected in the conventional results for other processing.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of an image information processing apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a detailed configuration of a decoder of the image information processing apparatus shown in FIG.

3 is a block diagram showing a detailed configuration of an input buffer memory of the image information processing apparatus shown in FIG.

FIG. 4 is a block diagram showing a detailed configuration of an image processing unit of the image information processing apparatus shown in FIG.

5 is a block diagram showing a detailed configuration of an encoder of the image information processing apparatus shown in FIG.

6 is a block diagram showing a detailed configuration of an output buffer memory of the image information processing apparatus shown in FIG.

7 is a block diagram showing a matrix switcher that can be provided in a stage preceding the decoder of the image information processing apparatus shown in FIG.

FIG. 8 is a diagram showing an example of a GOP.

FIG. 9 is a diagram showing an example of a GOP.

FIG. 10 is a diagram showing an example of a GOP.

FIG. 11 is a diagram for explaining range designation or image frame designation of the input buffer memory unit.

FIG. 12 is a block diagram showing a detailed configuration of an image information processing apparatus according to another embodiment of the present invention.

FIG. 13 is a block diagram showing a configuration of a conventional image information processing apparatus.

[Explanation of symbols]

 1 decoder 2 input buffer memory unit 3 image processing unit 4 encoder 5 output buffer memory unit 6 control unit

Claims (12)

[Claims] 1. A decompression means for decompressing an input compressed image signal, an image processing means for subjecting the image signal output from the decompression means to various image processing and outputting an image processing signal, and the image processing means. A compression means for compressing the image processing signal output from the compression means and outputting a compressed image processing signal; and a control means for controlling the expansion processing of the expansion means, the image processing of the image processing means, and the compression processing of the compression means. An image information processing device characterized by the above. 2. The image according to claim 1, wherein the control means causes the decompression processing of the decompression means, the image processing of the image processing means, and the compression processing of the compression means to be executed in real time. Information processing equipment. 3. The control means causes the decompression processing of the decompression means, the image processing of the image processing means, and the compression processing of the compression means to be executed in non-real time. Image information processing device. 4. The decompression means, based on the input compressed image signal, compressed / non-compressed state identification information, compression method information, image size information according to the number of pixels in the horizontal and vertical directions,
The attribute information including processing screen size information, image method information, and input / output signal rate information for determining a processing area is read out and supplied to the control means.
The image information processing device described. 5. A first input / output unit for inputting / outputting the image signal output from the expansion unit, and a second input / output unit for inputting / outputting the compressed image processing signal output from the compression unit. The control means causes the input / output processing of the first input / output means, the image processing of the image processing means, and the input / output processing of the second input / output means to be executed in real time. The image information processing apparatus according to claim 1. 6. The control means controls the decompression means, the first input / output means, the image processing means, the compression means, and the second input / output means according to the attribute information. The image information processing apparatus according to claim 5, which is characterized in that. 7. The first input / output unit includes a system conversion unit that converts the system of the image signal, and a storage unit that is independent of the image size of the image signal and has a sufficient capacity. The image information processing apparatus according to claim 5, wherein the image information processing apparatus comprises: 8. The image information according to claim 5, wherein the second input / output unit includes a storage unit having a sufficient capacity, which is independent of the image size of the image signal. Processing equipment. 9. The image information processing apparatus according to claim 1, wherein the compression means includes a system conversion unit that converts a system of the image signal. 10. The image information processing apparatus according to claim 1, wherein the decompression means is supplied with a compressed image signal which is switched and selected by a switching selection means which switches and outputs a plurality of compressed image signals. . 11. The switching selection means switches and selects the plurality of compressed image signals synchronized in a predetermined number of frames in the predetermined number of frames. Image information processing device. 12. A switching selection unit for selectively switching and outputting a plurality of input compressed image signals, an expansion unit for expanding the compressed image signals via the switching selection unit, and an output from the expansion unit. A first input / output unit for inputting / outputting the image signal; an image processing unit for performing various image processing on the image signal output from the first input / output unit to output an image processing signal; A hardware module including a compression unit that compresses an image processing signal output from the image processing unit and outputs a compressed image processing signal, and a second input / output unit that inputs / outputs the compressed image processing signal output from the compression unit. Section, the expansion processing of the expansion section of the hard module section,
A hardware module control unit for controlling the input / output processing of the first input / output unit, the image processing of the image processing unit, the compression processing of the compression unit, and the input / output processing of the second input / output unit; An image information processing apparatus comprising: a software unit that sets the connection of each unit of the unit via the hardware module control unit and performs a role sharing process.