JPH08205175A - Image information processor - Google Patents

Abstract

PURPOSE: To smoothly switch compressed images which are processed by inter- frame compression. CONSTITUTION: A matrix switcher 1 selectively inputs plural compressed images, puts P frames into one frame in a GOP, and modifies prediction data of a B frame. A decoder 2 expands the compressed images so as to restore the image to the original image. An input buffer memory 3 converts the image signal into component signals and converts a transfer rate. An image processing part 3 performs various image information processes for the image signal. An encoder 5 compresses the image processed signal. An output buffer memory part 6 converts the transfer rate of sent data as the compressed image processed signal. A control part 7 controls the respective processes of the matrix switcher 1, decoder 2, input buffer memory part 3, image processing part 3, encoder 5, and output buffer memory 6.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image information processing apparatus for performing processing such as image signal synthesis, transformation and color conversion.

[0002]

2. Description of the Related Art Image information processing apparatuses perform image processing such as image synthesis, color conversion, and geometric conversion while selecting a plurality of image signals from a large number of input image signals. For example, the image information processing apparatus synthesizes the image signals of the two systems while switching at a designated frame by using a switcher that is a switching unit.

This image information processing apparatus, as shown in FIG. 17, is a composite (hereinafter
It is called composite, and is written as Composite in the figure. ) Signal, Y / C signal, and component (indicated as Component in the figure) signal via the switcher 141. In this image information processing apparatus, the above image processing is applied to the component signal. Therefore, it is necessary to convert the Y / C signal and the composite signal, which are not the component signals, into the component signals after the switcher 141.

First, the composite signal is separated into Y and C signals by the Y / C separation circuit 142, and then the decoder 143.
Then, it is converted into Y, RY, and BY signals which are component signals. The Y / C signal is converted into a component signal by the decoder 143. The component signal output from the decoder 143 is supplied to the A / D conversion circuit 144. The A / D conversion circuit 144 converts the component signal into a digital signal based on the clock of the write clock generation circuit 145. This digital component signal is supplied to the frame synchronizer 146.

The frame synchronizer 146 writes the digital component signal in an internal frame memory on the basis of an external clock (referred to as EXT key in the drawing) signal generated by the switcher 141 from the video signal in the standard format. , Read it out, and use the Digital Multi Effec
t, hereinafter referred to as DME. ) Adjust the frame position, color subcarrier phase, etc. to obtain a signal suitable for processing. The digital component signal output from the frame synchronizer 146 is supplied to the switcher 14
7 is supplied.

A Y / C separation circuit 148, a decoder 149,
A / D conversion circuit 150, write clock generation circuit 15
The 1 and frame synchronizer 152 also perform the same processing as described above on the video signal of the standard format of the other system via the switcher 141. The digital component signal output from the frame synchronizer 152 is also supplied to the switcher 147. The switcher 147 is also supplied with Y, RY, and BY digital component signals from a test pattern signal generation circuit 153 that generates test pattern signals such as a color background signal, a color bar signal, and a grid signal.

The digital component signal passed through the switcher 147 is a two-dimensional variable low-pass filter (LP).
F) is supplied to 154. This two-dimensional variable LPF 154
Removes high frequency components of the digital component signal so as not to cause aliasing in the digital component signal. The digital component signal from which the high frequency component has been removed is supplied to the field memory 155. Write and read addresses are supplied to the field memory 155 from the system controller 157 via the DME processing unit 158. The DME processing unit 158 performs image processing such as image synthesis, color conversion, and geometric conversion on the digital component signal according to an instruction from the system controller 157. Therefore, the system controller 157 supplies the DME processing unit 158 with the data required for the desired DME processing.

Then, the DME processing unit 158 uses the field memory 155 to perform the above image information processing on the digital component signal via the two-dimensional variable LPF 154 according to the address and data supplied from the system controller 157. Here, when the digital component signal is subjected to, for example, a modification process, a pixel missing occurs in the signal read from the field memory 155. Therefore, the field memory 1
The signal read from 55 is supplied to the interpolation processing circuit 156 and subjected to predetermined interpolation processing. Interpolation processing circuit 15
The DME processed signal which is the output signal of No. 6 is supplied to the data mixing circuit 159. In this data mixing circuit 159,
From the system controller 157 to the DME processing circuit 15
The same addresses and data that are supplied to 8 are also supplied and mixed with the DME processed signal. The mixed output signal which is the output signal of the data mixing circuit 159 is supplied to the combining circuit 161. The digital component signal delayed by the field delay circuit 160 is also supplied to the synthesis circuit 161, and is synthesized with the mixed output signal. The combined output of the combining circuit 161 is a signal subjected to image information processing, converted into an analog signal by the D / A conversion circuit 162, and an analog composite signal,
It is output in the form of Y / C signals and component signals.

[0009]

By the way, in the conventional image information processing apparatus as shown in FIG. 17, input / output signals are limited to standard composite signals, Y / C signals, and component signals. Such as deviating from the standard video signal,
Images with different resolutions, images with different transfer rates, or images with different image sizes could not be handled. Further, even with the same image signal, if the systems are different, such as the HDTV system and the NTSC system, the same device cannot process. That is, in the conventional image information processing device, a so-called free format image signal that does not depend on resolution,
An image signal that does not depend on the transfer rate, a so-called scalable format image signal that does not depend on the image size, and an image signal of a different system cannot be handled. In addition, as a matter of course, since compressed images cannot be handled, it is not possible to smoothly switch between compressed images as if they were compressed between frames.

The present invention has been made in view of the above circumstances, and is a so-called free format image signal that does not depend on resolution, an image signal that does not depend on transfer rate, or an image signal that does not depend on image size. Provided is an image information processing apparatus capable of performing various kinds of image processing even on so-called scalable format image signals and image signals of different systems, and capable of smoothly switching compressed images that have been subjected to interframe compression processing. With the goal.

[0011]

An image information processing apparatus according to the present invention selectively captures a plurality of compressed images subjected to interframe compression processing, and also interframe forward prediction code in units of a predetermined number of frames. Selective means for converting the encoded image into an intra-coded image and changing the prediction data of the bidirectional predictive encoded image, decompressing means for decompressing the compressed image signal selected by the selecting means, and output from the decompressing means A first input / output unit for inputting / outputting the generated image signal;
Image processing means for subjecting the image signal output from the input / output means to various image processing to output an image processing signal, and an image processing signal output from the image processing means to output a compressed image processing signal. A compression unit, a second input / output unit for inputting / outputting the compressed image processing signal output from the compression unit, a selection process of the selection unit, a decompression process of the decompression unit, and a first I / O unit of the decompression unit. The above problem is solved by having an input / output process, an image process of the image processing unit, a compression process of the compression unit, and a control unit for controlling the input / output process of the second input / output unit.

In this case, the selecting means suppresses an increase in the amount of information due to the change of the prediction data of the bidirectional predictive-coded image under the control of the control means. Further, the decompression means, based on the input compressed image signal, compressed / non-compressed state identification information, compression method information, image size information according to the number of pixels in the horizontal and vertical directions, and processing screen size information for determining a processing area. , Attribute information including image format information and input / output signal rate information is read out and supplied to the control means.

Further, the first input / output means has a format conversion section for converting the format of the image signal, and a storage section having a sufficient capacity, regardless of the image size of the image signal. Consists of The second input / output unit has a storage unit having a sufficient capacity, which is independent of the image size of the image signal.

Further, the compression means has a system conversion section for converting the system of the image signal.

[0015]

In the image information processing apparatus according to the present invention, the selecting means selectively fetches a plurality of compressed images subjected to the interframe compression processing, and at the same time, outputs the interframe forward prediction coded image in a predetermined number of frames. Since the intra-frame coded image is formed and the prediction data of the bidirectional predictive-coded image is changed, the compressed image subjected to the inter-frame compression process can be smoothly switched. Further, the expansion means expands the input compressed image signal to output an image signal, the first input / output means inputs / outputs the image signal under the control of the control means, and the image processing means outputs the image signal. Is subjected to various image processing to output an image processing signal, the compression means compresses the image processing signal, and the second input / output means inputs and outputs the compressed image processing signal, so that it does not depend on the resolution. Applying various image processing to such so-called free format image signals, image signals that do not depend on transfer rate, image signals of so-called scalable format that does not depend on image size, and image signals of different methods. You can

[0016]

Embodiments of an image information processing apparatus according to the present invention will be described below with reference to the drawings. This embodiment is an image information processing apparatus that performs image information processing such as generation of a wide range of images from still images to moving images, color conversion, composition, editing, etc., for example, a compressed image signal subjected to interframe compression processing. Can handle. An image that has undergone interframe compression processing is an MPEG (Moving Picture C
There is an image compressed by a coding method as standardized by the oding Experts Group, a study organization for video coding for storage). In addition, the image information processing apparatus described above uses JPEG (Jo
Int PhotographicCoding Experts Group, a study organization for color still image coding) can handle images compressed by standardized coding methods.

First, before describing the structure of the present embodiment, the image compression processing standardized by MPEG will be described. The image compressed by this image compression processing includes an I (Intra) picture that is a frame encoded image and a P (Predic) that is an inter-frame forward prediction encoded image.
(Tive) picture, B (Bidi) which is a bidirectional predictive coded image.
rectional Predictive) There are three types of pictures. In the following, for simplification of description, these I picture, P picture, and B picture will be described as I frame, P frame, and B frame. That is, the I frame is a frame that has been intra-frame compressed, the P frame has been compressed using the prediction information of the previous I frame, and the B frame has been compressed using the prediction information of the previous and subsequent I and P frames. It is a frame. The compressed image is composed of a compression unit called a group of pictures (hereinafter, referred to as GOP) composed of a set of these frames. The GOP has one I frame, and is composed of 15 frames, such as "IBBPBBPBBPBBPBB".

Then, as shown in FIG. 1, the image information processing apparatus selectively fetches the plurality of compressed images supplied from a plurality of external input devices such as a VTR and a data recorder and, at the same time, the GOP. A matrix switcher 1 for converting P frames into I frames and changing prediction data of B frames, a decoder 2 for expanding the compressed image to restore the original image, and an image signal expanded by the decoder 2. Is Y, R-Y, G-Y or R,
An input buffer memory unit 3 having a function of converting the G and B component signals and converting the transfer rate of the image signal, and color conversion, transformation, reduction, or the like to an image signal which is an output signal of the input buffer memory unit 3. An image processing unit 4 having a function of performing various kinds of image information processing such as geometric transformation, an encoder 5 having a function of compressing an image processing signal which is an output signal of the image processing unit 4, and a compressed image processing signal of the encoder 5. The output buffer memory unit 6 having a function of converting the transfer rate of the output data, the matrix switcher 1, the decoder 2, the input buffer memory unit 3, the image processing unit 4, the encoder 5, and the output buffer memory unit 6 are controlled. It has a control unit 7. Further, the image information processing apparatus is connected with a keyboard 8 which is a means for externally supplying processing parameters and processing control data, and a monitor 9 for displaying processing results and input images.

As shown in FIG. 2, the matrix switcher 1 comprises a cross point switch 10, GOP reconstruction units 11 and 12, and a synthesis unit 13. The crosspoint switch 10 inputs a plurality of compressed images from the plurality of external input devices, and selects, for example, two of them from an instruction from the control unit 7. The GOP reconstruction units 11 and 12 are
For example, when one continuous image is cut in the middle of a certain GOP and is connected to the other image, the incomplete GOP of the one image cut from the middle is reconstructed into a complete GOP. Synthesis part 1
3 connects the two GOP images reconstructed by the GOP reconstruction units 11 and 12.

As shown in FIG. 3, the GOP reconstruction units 11 and 12 include an input side buffer memory 14, a variable length code decoder 15, an inverse quantizer 16, an adder 17, and a buffer memory 18. , Quantizer 19 and forward compensation circuit 20
A backward compensation circuit 21, a frame memory 22,
The frame memory 23, the work memory 24, the prediction data correction circuit 25, the encoder 26, and the GOP assembly circuit 2
7 and a work memory / output side buffer memory 28.

The input side buffer memory 14 temporarily stores the data of the compressed image necessary for the subsequent processing. The variable length code decoder 15 decodes the compressed image data which has been variable length coded. The inverse quantizer 16 multiplies the image data decoded by the variable length code decoder 15 by the reciprocal of the quantization table and returns the value in the frequency domain. The forward compensation circuit 20 is
The image is reconstructed by taking out the dequantized image data in the forward direction, which is the same direction as the flow of time, from the frame memory 22. The backward compensating circuit 21 takes out the inversely quantized image data in the backward direction, which is the reverse direction of the flow of time, from the frame memory 23 and reconstructs the image. The adder 17 adds the reconstructed image data output from the forward compensation circuit 20 and the backward compensation circuit 23 and the image data that is the processing result of the inverse quantizer 16. The buffer memory 18 is, for example, 1 GOP necessary for the subsequent processing.
Temporarily stores minute data. The quantizer 19 multiplies the added image data extracted from the buffer memory 18 by each discrete cosine transform value with the coefficient of the quantization table, and determines the final GOP total data amount in advance. It also performs processing to keep it within a certain fixed value (reference data amount). The prediction data correction circuit 25 corrects the B frame compressed image data by adding / deleting forward and backward prediction data. The work memory 24 is a work memory for performing correction processing on the B frame compressed image data predicted by the prediction data correction circuit 25. Encoder 26
Converts the quantized data into Huffman code and run length code. The GOP assembling circuit 27 determines whether the reference data amount is exceeded, decoding, requantization, Huffman table updating,
A new GOP is reconstructed and assembled from coded data that has a function such as coding and that has been modified by the prediction data modification circuit 25 and other data that does not need to be modified. The work memory / output side buffer memory 28 has a function of a work area for assembling the GOP assembling circuit 27 and a function of a memory for storing the processing result data until a certain collective result is obtained and output.

Next, the operation of the matrix switcher 1 configured as described above will be described. The cross point switch 10 shown in FIG. 2 selects, for example, two pieces of compressed image data from a plurality of pieces of compressed image data supplied from a plurality of external input devices according to an instruction from the control unit 7.
When handling compressed image data that has undergone interframe compression, the crosspoint switch 10 switches the selection at the end of the GOP including the switching frame. Accurate frame switching is performed by the GOP reconstruction units 11 and 12 in the subsequent stage. Here, when two images are switched to form one continuous image, an image already selected is called an existing image and a newly selected image is called a new image. The existing image is output through the GOP reconstructing unit 11 or 12 and the combining unit 13. Before the switching, the GOP reconstruction processing and the synthesis processing are not necessary, so that the existing image simply passes through these.

When a switching instruction comes, the new image is input to the GOP reconstructing unit 11 or 12 to which the existing image is not supplied, and the existing image ends in the designated frame so that the switching is performed in the designated frame. Na GOP
In addition, the new image is a new GOP that starts at the specified frame.
Reconfigured into. The synthesizing unit 13 GOPs each image.
Connect in units to make one continuous image.

Details of the operations of the GOP reconstruction units 11 and 12 will be described with reference to FIGS. 3 to 10.
In FIG. 3, a system including an input side buffer memory 14, a variable length code decoder 15, an inverse quantizer 16, an adder 17, a buffer memory 18 and a quantizer 19, and an adder 17 and a frame memory forming a closed loop. 23, frame memory 22, forward compensation circuit 20, and backward compensation circuit 21
The system consisting of and is a path for generating an I frame from a P frame. As will be described later, only the forward compensating circuit 20 is used when a new I frame is created, and the forward compensating circuit 20 and the backward compensating circuit 21 are used when converting a P frame into an I frame to correct the prediction information of the B frame.
Both are used.

To generate an I frame from a P frame, the previous I frame may be used to convert the P frame into an I frame. The conversion of the P frame into an I frame and the prediction information correction of the B frame described later will be described. One GOP
Has one I frame, for example "IBBPBB
PBBPBBPBB ″ is composed of 15 frames as described above. The coding sequence of the frame sequence of this GOP is as shown in FIG. 4A, and the input sequence is as shown in FIG. 4B. The input order is "BBIB
BPBBPBBPBBP ", but the encoding order is I
Shift I and P forward so that "" is at the top, and then "IBBPB
BPBBPBBPBB ″.

The GOP of the compressed image data input to the matrix switcher 1 is resolved in the variable-length code decoder 15 from the encoding order shown in FIG. 4 (A), and then shown in FIG. 4 (B). The input order is as follows. FIG. 5 shows the "IBBP" portion of the GOP that has been input in this order. Of course, each frame is a spectrum value in the frequency domain that is arranged in the input order from the left and is unencoded.

The spectrum value of the I frame F _I is added to the spectrum value of the block of the P frame F _P having the prediction vector. Here, the block consists of 8 × 8 spectral values for 64 pixels. The above-mentioned prediction vector, which is a motion vector, indicates a correspondence relationship that a certain block of the I frame F _{I is the same as} a certain block of the P frame F _P within a certain error, and brings the block from the I frame F _I. Thus, the block of the corresponding P frame F _P can be reproduced. If this operation is performed for all blocks, P frames can be converted into I frames. Variable-length code decoder 15, work memory 24, and prediction data correction circuit 25
The system consisting of B corrects the B frames F _B1 and F _B2 by embedding the forward prediction information or the backward prediction information. If there is both forward and backward prediction information, specify only one of them. This operation is performed by the prediction data correction circuit 25. The correction of the B frames F _B1 and F _B2 is performed using the information of the frame F _P ′ after the above I frame conversion. This is because the information necessary for B frame reproduction is discarded before discarding the frame F _P ′ converted into an I frame.
Required to move to frames F _B1 and F _B2 . Specifically, the spectrum value is moved from the frame F _P ′ converted into an I-frame based on the vector information, and the process of eliminating the inter-frame dependency relationship is performed. Thus, the conversion of the P frame F _{P into} the I frame has two purposes. One is to convert the discarded P frame F _P into an I frame to correct the prediction information of the B frames F _B1 and F _B2 , and the other is to discard the I frame F _I from the P frame F _P to generate a new I frame. To make the frame F _P '. Also, I frame F _I to B
It is the decompression operation that supplies the spectral values of the prediction block to the frames F _B1 and F _B2 . here,
The compression information of the frame is used as the prediction information. There are two types of prediction information for P-frames and B-frames: information that is differentiated using motion vectors and then quantized, and information that does not use motion vectors. Modification of the prediction information of the B frames F _B1 and F _B2 is necessary when discarding the I frame F _I or the P frame F _P. Since the I frame F _I and the P frame F _P have original data for which a motion vector is obtained (information that does not use a motion vector), it is necessary to embed this in the B frames F _B1 and F _B2 . This is the reason why the information quantized after the differential using the motion vector is corrected to the information not using the motion vector. Original data does not always exist in the P frame, and in the worst case, it is possible to trace back to the I frame. In this way, the GOP reconstruction units 11 and 12 convert the P frame into the I frame and correct the prediction information of the B frame.

Then, the GOP reconstruction units 11 and 12
Has an encoding order of "IBBPBBPBBPBBPBB"
The GOP frame sequence such as is reconstructed as shown in FIGS. 6 and 7 according to the difference in switching position. Here, the GOP reconstructing unit 11 reconstructs the existing image as shown in FIG. 6, and the GOP reconstructing unit 12 reconstructs the new image as shown in FIG. Also, for example, the GOP reconstruction units 11 and 12
Is an "IBBBIBBBIBBBIBBB" that is composed of only I and B frames without P frames.
The frame sequence of GOP such as B ″ is reconstructed as shown in FIGS. 8 and 9 according to the difference in switching position. Here, the GOP reconstructing unit 11 reconstructs the existing image shown in FIG.
As described above, the GOP reconstruction unit 12 reconstructs the new image as shown in FIG.

In the example of the GOP reconstruction processing shown in FIGS. 6 and 7, the P frame of the existing image is discarded, the backward prediction information is added to the B frame, and the P frame of the new image is I.
A process of adding forward prediction information to the B frame is performed. In the processing of the existing image, the P frame from the previous I frame to the next P frame is converted to an I frame, and the P frame to be a target is repeated sequentially to form an I frame.
Modify frame prediction information. In the processing of the new image, the target P frame is converted into an I frame from the previous I frame, and the prediction information of the B frame is corrected using the information of the previous I frame.

Further, in the example of the GOP reconstruction processing shown in FIGS. 8 and 9, since there is no P frame, it is not necessary to create an I frame from a P frame. Bring the I-frame to the work memory 24 in the pass that produces The processed frame data that has been subjected to these processes is supplied from the work memory 24 to the encoder 26 and converted into the Huffman code and the run length code, and then enters the work memory / output side buffer memory 28. The output of the input side buffer memory 14 is also supplied to the work memory / output side buffer memory 28. The data directly supplied from the input side buffer memory 14 is unprocessed frame data that does not need to be processed in the GOP reconstruction. The output data of the work memory / output side buffer memory 28 enters the GOP assembling circuit 27. The GOP assembling circuit 27 combines the processed frame data and the unprocessed frame data to create a new GOP. Here, the unnecessary frame information is discarded. Alternatively, it can be selectively discarded by the output of the input side buffer memory 14.

Thereafter, the GOP reconstruction units 11 and 12 carry out.
Details of the two examples of FIGS. 6 and 7 and FIGS.
I will explain it. FIG. 6A shows the GOP of the existing image.
Latter half of the input sequence frame sequence f, d, h, i, g,
k, l, j, n, o, m and switching position S₁, S₂, S
₃, S_Four, S_Five, S₆, S₇, S₈Is shown. Where d, g,
j and m are P frames, and f, h, i, k, l,
n and o are B frames. Also, each switching position
S₁, S₂, S₃, S_Four, S_Five, S₆, S₇, S₈Switching in
Is instructed by the control unit 7. 6B through 6 of FIG.
(I), the switching position S₁, S₂, S₃, S_Four,
S _Five, S₆, S₇, S₈GOP reconstructing unit when
11 shows a state of the reconstruction processing performed in 11. Where GO
The basic encoding order of P is h, i which is B frame
Is g frame which is P frame already known for encoding?
Since g must be set, g must be marked earlier than h and i.
In order to encode the P frame, g, which is a P frame, of the previous P frame
It is the order of movement to the position. Same for j and m of P frame
Is.

When the switching position S ₁ is indicated by the instruction from the control unit 7 as shown in FIG. 6A, the GOP reconstructing unit 11 does not particularly change the basic coding order and As shown in FIG. 6B, switching is performed in GOP units. When the switching position S ₂ is indicated, GO
As shown in FIG. 6C, the P reconstructing unit 11 adds backward prediction information of m that is a P frame to n and o that are B frames of the frame sequence in the above basic encoding order, Discard the m after making the correction n ', o'. The backward prediction information is added in order to delete the vector information given by m from n and o by adding the actual block image information to n and o of the B frame.
The block image information of the P frame is obtained by performing a discrete cosine transform on the difference value between the I frame and the P frame or the value of the P frame. When performing the discrete cosine transform of the difference value between the I frame and the P frame, it is necessary to reconstruct the image of m which is the target P frame from the previous I frame and the P frame in the GOP. That is, everything from the front of the GOP to the P frame required for adding the prediction information of the B frame is temporarily converted from the I frame to the I frame.

When the switching position S ₃ is indicated, GO
As shown in FIG. 6D, the P reconstructing unit 11 adds backward prediction information of m that is a P frame to n that is a B frame of a frame sequence in the above basic encoding order,
After correcting to n ', discard o'and m. When the switching position S ₄ is indicated, the GOP reconstructing unit 11 discards n, m, and o of the frame sequence in the basic encoding order, as shown in (E) of FIG.

When the switching position S ₅ is indicated, GO
As shown in FIG. 6 (F), the P reconstructing unit 11 adds backward prediction of j, which is a P frame, to k and l, which are B frames of the frame sequence in the basic coding order, and corrects them. After setting k ', l', j, n, o, m are discarded. When the switching position S ₆ is indicated, the GOP reconstructing unit 11 is, as shown in FIG. 6G, a P frame in k which is a B frame in the frame sequence in the basic encoding order. add the backward prediction of j to give a modified k ′, then l,
Discard j, n, o, and m.

When the switching position S ₇ is indicated, GO
As shown in (H) of FIG. 6, the P reconstructing unit 11 uses k, 1, j, n, and k of the frame sequence in the basic coding order.
Discard m and o. When the switching position S ₈ is shown,
As shown in (I) of FIG. 6, the GOP reconstruction unit 11 adds backward prediction information of g that is a P frame to h and i that are B frames of a frame sequence in the above basic encoding order, After modifying h ′, i ′, g, k, l, j, n,
Discard m and o.

FIG. 7A shows the GOP input of the new image.
The first half b, c, a, e, f, d of the force sequence frame sequence
h, i, g, k and switching position s₁, S₂, S₃, S_Four,
s_Five, S₆, S₇, S₈Is shown. Where a is an I frame
Yes, d, g, j are P frames, b, c, e,
f, h, i and k are B frames. Also, each switch
Position s₁, S₂, S₃, S_Four, S_Five, S₆, S₇, S₈Cut in
The replacement is instructed by the control unit 7. 7B to 7B
In FIG. 7 (I), the switching position s₁, S₂, S ₃, S_Four,
s_Five, S₆, S₇, S₈In the case of GOP reconstruction unit 12
The following is a description of the reconstruction process. Also, the basic GOP
The encoding order is such that a, which is an I frame, comes first.
The a and P frames d, g, and j into the previous I or P
The order is to move to the frame position.

In response to an instruction from the control unit 7, (A) of FIG.
When the switching position s ₁ is indicated by, the GOP reconstructing unit 12 does not particularly change the encoding order,
Switching is performed in GOP units. The switching position s ₂
Is displayed, the GOP reconstructing unit 12 discards b, which is the B frame of the frame sequence in the basic encoding order, as shown in FIG. 7C.

When the switching position s ₃ is indicated, GO
As shown in FIG. 7D, the P reconstructing unit 12 discards b and c, which are B frames of the frame sequence in the basic coding order. When the switching position s ₄ is shown,
As shown in FIG. 7E, the GOP reconstructing unit 12 converts the P frame d of the frame sequence in the basic encoding order into an I frame, and the B frames e and f into an I frame.
The forward prediction information of the frame a is added, and the correction e ′,
After setting f ', a, b, and c are discarded. The reason for adding the forward prediction information is to add the actual block image information to e and f of the B frame to delete the vector information given by a from e and f.

When the switching position s ₅ is indicated, GO
As shown in (F) of FIG. 7, the P reconstructing unit 12 converts d, which is a P frame of a frame sequence in the basic coding order, into an I frame, and f, which is a B frame, is an I frame. The forward prediction information of a is added to make a correction f ′, and then a, b, c, and e are discarded. When the switching position s ₆ is indicated, the GOP reconstruction unit 12 converts the P frame d of the frame sequence in the basic encoding order into an I frame, as shown in FIG. a, b, c,
Discard e and f.

When the switching position s ₇ is indicated, GO
As shown in (H) of FIG. 7, the P reconstructing unit 12 converts g, which is a P frame of the frame sequence in the basic coding order, into an I frame, and sets h and i to the forward prediction information of d. In addition, after modifying h ', i', a, b, c, d, e,
Discard f. When the switching position s ₈ is shown, GO
As shown in (I) of FIG. 7, the P reconstructing unit 12 converts g, which is a P frame of the frame sequence in the above basic coding order, into an I frame, adds the forward prediction information of d to i,
After the correction i ′, a, b, c, d, e, f, h are discarded.

In this way, the GOP reconstruction units 11 and 12 are
Reconstructs an existing image and a new image independently.
Then, the synthesizing unit 13 processes each image in GOP units.
Connect them into one continuous image. FIG. 8A shows the above.
The latter half h of the GOP input sequence frame sequence of the existing image,
e, j, k, l, i, o, p, q, m, and switching position
S₁, S₂, S₃, S_Four, S _Five, S₆, S₇, S₈Is shown. here
Where e, i, m are I frames, and h, j, k, l,
o, p, and q are B frames. Also, each switching position
S₁, S₂, S₃, S_Four, S_Five, S₆, S₇, S₈Switching in
Is indicated by an instruction from the control unit 7. 8B to 8B
The switching position S is shown in FIG.₁, S₂, S₃, S_Four,
S_Five, S₆, S₇, S₈In the case of GOP reconstruction unit 11
7 shows a state of GOP reconstruction performed. Here, the basics of GOP
Encoding order is such that the I frame e, i, m
It is the order of moving to the position of the I frame.

When the switching position S ₁ is indicated by the instruction from the control unit 7 as shown in FIG. 8A, the GOP reconstruction unit 11
In particular, as shown in FIG. 8B, switching is performed in GOP units without changing the basic coding order. When the switching position S ₂ is shown, the GOP
As shown in (C) of FIG. 8, the reconstructing unit 11 sets backward prediction information of m, which is an I frame, to o, p, and q which are B frames of the frame sequence in the above basic encoding order. In addition, m is discarded after the modification is made o ', p', q '.

When the switching position S ₃ is indicated, GO
As shown in (D) of FIG. 8, the P reconstructing unit 11 adds backward prediction information of m that is an I frame to o and p that are B frames of a frame sequence in the above basic encoding order. , Correction o ′, p ′, and then discard m, q. When the switching position S ₄ is indicated, the GOP reconstruction unit 11
As shown in (E) of FIG. 8, backward prediction information of m, which is an I frame, is added to o, which is a B frame of a frame sequence in the above basic encoding order, and m, p , Q are discarded.

When the switching position S ₅ is indicated, GO
As shown in (F) of FIG. 8, the P reconstructing unit 11 uses o, p, q, and m of the frame sequence in the basic coding order.
Throw away. When the switching position S ₆ is shown, the GOP
As shown in (G) of FIG. 8, the reconstructing unit 11 provides backward prediction information of i that is an I frame to j, k, and l that are B frames of a frame sequence in the above basic encoding order. In addition, after modifying j ′, k ′, l ′, i, o, p,
Discard q and m.

The switching position S₇Is indicated, GO
As shown in (H) of FIG.
It is a B frame of a frame sequence in which the main encoding order is set.
The backward prediction information of the I frame i is set to j, k
In addition, after the correction j ', k', l, i, o, p,
Discard q and m. The switching position S₈Is indicated,
The GOP reconstructing unit 11, as shown in (I) of FIG.
B-frame of frame sequence in basic encoding order
The backward prediction information of I frame i is added to j
Eh, the correction j, then k, l, i, o, p, q, m
throw away. FIG. 9A shows the GOP input order of the new image.
The first half b, c, d, a, f, g, h of the frame sequence
e, j, k and switching position s₁, S₂, S₃, S_Four,
s_Five, S₆, S₇, S₈Is shown. Where a and e are I-frames
B, c, d, f, g, h, j, k are B frames
It is. Also, each switching position s₁, S₂, S₃, S_Four,
s_Five, S₆, S₇, S₈To switch between, use the control unit 7
Is shown. 9B to 9I, the switching is performed.
Position s₁, S₂, S₃, S_Four, S_Five, S₆, S ₇, S₈in the case of
Like the GOP reconstruction processing performed by the GOP reconstruction unit 12 of
Indicates a child. Also, the basic encoding order of GOP is
Move the frames a, e, i to the position of the previous I frame
The order will be.

When the switching position s ₁ is indicated by the instruction from the control unit 7 as shown in FIG. 9A, the GOP reconstruction unit 1
In particular, in the case of 2, the GO is changed without changing the encoding order.
Switching is performed in P units. When the switching position s ₂ is indicated, the GOP reconstructing unit 12 indicates, as shown in FIG. 9C, B of the frame sequence in the basic encoding order.
Discard the frame b.

When the switching position s ₃ is indicated, GO
As shown in FIG. 9D, the P reconstructing unit 12 discards b and c which are B frames of the frame sequence in the above basic encoding order. When the switching position s ₄ is shown,
As shown in FIG. 9E, the GOP reconstructing unit 12 discards b, c, and d which are B frames of the frame sequence in the above basic encoding order.

When the switching position s ₅ is indicated, GO
As shown in (F) of FIG. 9, the P reconstructing unit 12 adds the forward prediction information of a to f, g, and h which are B frames of the frame sequence in the above basic encoding order, and modifies f. ',
After setting g'and h ', b, c, d, and a are discarded. When the switching position s ₆ is indicated, the GOP reconstructing unit 12 indicates, as shown in (G) of FIG. 9, an I frame in g and h which are B frames of the frame sequence in the basic encoding order. Then, the forward prediction information of a is added to make corrections g ′ and h ′, and then b, c, d, a, and f are discarded.

When the switching position s ₇ is shown, GO
As shown in (H) of FIG. 9, the P reconstructing unit 12 adds the forward prediction information of the I frame a to the h which is the B frame of the frame sequence in the above basic encoding order, and the modified h Then, b, c, d, a, f, g are discarded. When the switching position s ₈ is shown, the GOP reconstruction unit 12
Discards b, c, d, a, f, g, and h of the frame sequence in the above basic encoding order as shown in FIG.

In this way, the GOP reconstruction units 11 and 12 are
Reconstructs an existing image and a new image independently.
Then, the synthesizing unit 13 connects the images in GOP units to form one continuous image. Here, there is a case in which the total amount of GOP data exceeds the reference amount of data, which is a predetermined total amount of data, due to the GOP rearrangement performed by the GOP reconstruction units 11 and 12. In this case, the GOP assembling unit 27 performs a data compression process for keeping the total data amount of the processed frame data and the unprocessed frame data within the reference data amount. The data compression processing of the GOP assembling unit 27 will be described with reference to the flowchart of FIG.

First, the GOP assembling section 27 operates in step ST.
As shown in FIG. 1, the total data amount of the processing target GOP stored in the work memory / output side buffer memory 28 is calculated. Then, it is determined in step ST2 whether or not the calculated total data amount of GOP is equal to or smaller than a predetermined reference data amount. Here, the reference data amount changes depending on the number of frames forming the GOP. When the total data amount of the GOP is less than or equal to the reference data, the GOP assembly circuit 27
In step ST3, the total data of the GOP is read from the work memory / output side buffer memory 28 and supplied to the synthesizing unit 13 in the subsequent stage. When the process of step ST3 is completed, the data compression process of the GOP assembling unit 27 ends.

However, if it is determined in step ST2 that the total data amount of GOP is larger than the reference data amount, GOP
The assembling circuit 27 proceeds to step ST4 to read the total GOP data from the work memory / output side buffer memory 28, and then, as shown in step ST5, the read total GOP data is processed by the variable length code decoder. Decrypt.
Then, the GOP assembling circuit 27 quantizes again in step ST6 to correct the data amount so that the total data amount of the decoded GOP becomes equal to or smaller than the reference data amount.
After that, the GOP assembling circuit 27 encodes the GOP data whose data amount has been corrected in step ST7, returns it to the work memory / output side buffer memory 28, and then requantizes it as shown in step ST8. The GOP data is read from the work memory / output side buffer memory 28.

For the GOP data read in step ST8, the total data amount is calculated by returning to step ST1.
After that, in step ST2, it is determined whether or not the data amount is equal to or smaller than the reference data, and if YES is determined, the data is supplied to the combining unit 13 in step ST3. If NO is determined, the processes of steps ST4 to ST8 are repeated.

Here, the requantization performed in step ST6 reduces the number of allocated bits by multiplying each coefficient of the quantization table used in the quantizer 19 by a weighting coefficient.
This requantization is performed on all I frames, P frames, and B frames. As the weight coefficient, a value obtained by dividing the reference data amount by the total data amount is multiplied by a constant. The constant is set so that it does not exceed the reference data amount when encoding.

In order to keep the total GOP data amount within the reference data amount, a Huffman table updating function of the encoder 26 may be further prepared to take measures to prevent the reference data from being exceeded when encoding. . In this way,
The matrix switcher 1 can accurately switch a compressed image that has been subjected to interframe compression processing in a designated frame. Further, the compressed image after switching does not affect the decompression process. Further, it is not necessary that the GOP frame configurations after switching are the same. In addition, GO of the switching image
The phases of P need not match. It is possible to suppress an increase in the GOP information amount of the image after the switching composition. In addition, cut editing can be performed without the need for image expansion.

Next, the image information processing apparatus of this embodiment restores the compressed image data connected as described above by the matrix switcher 1 to the original image by the decoder 2 shown in FIG. Also, the decoder 2
In addition to the function of restoring the compressed image to the original image, has a function of separating the image signal, the audio signal, and their attribute information from the input signal, and a function of communicating with the control unit 7.

Here, the attribute information is information indicating the characteristics and characteristics of the signal. For example, in the case of an image signal, compressed / non-compressed state identification information, compression method information, and image size information according to the number of pixels in the horizontal and vertical directions. There is processing screen size information that specifies a processing area, for example, image system information such as NTSC, PAL, RGB, and input / output signal rate information. This attribute information can also be given to the decoder 2 via the control unit 7.

Specifically, the decoder 2 expands the compressed image signal from the external input device while changing the expansion method according to the instruction of the control unit 7. An image which is not compressed by the compressed / uncompressed state identification information of the attribute information and which does not need to be expanded is not compressed but bypassed. This decoder 2
11, is composed of a decoder 31 for performing decoding such as Huffman decoding, an IDCT circuit 32, and a deblocking circuit 33, and is compressed by the encoding method standardized by JPEG. JP for decoding images
EG decoder 30, buffer memory 41, variable length code decoder 42, inverse quantizer 43, IDCT circuit 4
4, adder 45, forward compensation circuit 46, forward +
A backward compensating circuit 47, a backward compensating circuit 48, a frame memory 49, and a frame memory 50, and an MPEG decoder 40 for decoding an image compressed by the above-described MPEG standardized encoding method. Composed.

On the input side of the JPEG decoder 30 and the MPEG decoder 40, there is provided a decoder selector 34 for selecting which decoder the data is passed through.
An output selector 35 is provided on the output side of the JPEG decoder 30 and the MPEG decoder 40. The decoder 31 of the JPEG decoder 30 decodes, for example, Huffman-encoded data. The IDCT circuit 32 performs a discrete cosine inverse transform process on the decoded data. The deblocking circuit 33 restores the blocked data to an original image.

The buffer memory 41 of the MPEG decoder 40 temporarily stores the data necessary for the subsequent decoding process. The variable length code decoder 42 decodes the data encoded with the variable length. The inverse quantizer 43 multiplies the output data of the variable length code decoder 42 by the quantization number and returns it to the value in the frequency domain. The IDCT circuit 44 subjects the output data of the inverse quantizer 43 to discrete cosine inverse transform processing. The forward compensation circuit 46 retrieves image information in the forward direction, which is the same direction as the flow of time, from the frame memory 49 and reconstructs an image. The backward compensation circuit 48 is used in the frame memory 5
The image information is reconstructed by extracting the image information in the backward direction, which is the reverse direction of the flow of time from 0. The forward / backward compensation circuit 47 includes a frame memory 49 and a frame memory 5.
The image information in both directions is taken out from 0 to reconstruct the image. The adder 45 adds the reconstructed image output from the forward compensation circuit 46, the forward + backward compensation circuit 47, and the backward compensation circuit 48 and the image of the processing result of the IDCT circuit 44. The expansion process of the decoder 2 is, for example, 8 ×
It is parametric so that an image of any size such as an 8-pixel block unit can be accepted, and an image of a size specified according to the image size information of the attribute information can be expanded. For example, in the case of an image with a fraction of 8 × 8 pixel blocks, dummy data is added to make the size non-fractional.

Next, the input buffer memory unit 3 has a function of converting the image signal into, for example, a component signal and converting the transfer rate of the image signal, and the control unit 6 as well.
It also has a communication function with. As shown in FIG. 12, the input buffer memory unit 3 includes a system conversion encoder 51 and a buffer memory 52 with a rate conversion function. The system conversion encoder 51 converts the composite signal expanded by the decoder 2 or the Y / C signal into a component signal such as Y, RY, GY or R, G, B which is handled by the internal processing of the present apparatus. To do. This system conversion is processed according to the image size information and the image system information of the attribute information given from the control unit 7. However, in the case of simple frame switching or the like, the composite signal or the Y / C signal may be used as it is, and conversion to the component signal is not necessary.

Further, the buffer memory 5 with rate conversion function
2 has a sufficient capacity regardless of the image size of the image signal. Writing to the buffer memory with rate conversion function 52 is performed at the rate of input to the buffer memory 52, and reading is performed at the internal processing rate.
The rate of input to the buffer memory 52 is the output rate of the format conversion encoder 51. If the read rate is faster than the write rate, a wait state is entered during the read. Buffer memory 52 with rate conversion function
Consists of two memories, one of which is a write memory and the other of which is a read memory. The roles of read / write alternate. That is, the buffer memory with rate conversion function 52 has a double buffer memory structure. Each of the two memories has an address generator that works independently. The input rate is generated by the address generator in each memory according to the image input / output rate information of the attribute information given from the control unit 7. By adjusting the size of the address generation block and the block address interval, inputs of various rates can be converted into the internal rate of the processing system. In this embodiment, since the image data is handled in blocks, the above rate is an average rate over time.

The image processing unit 4 has a function of performing image processing such as image generation and composition, painting, special effects, and the like, as well as the control unit 7.
It also has a communication function with. This image processing unit 4 is shown in FIG.
3, the color conversion circuit 53, the variable tap low-pass filter 54, the image memory 55, and the interpolation filter 56.
And a combination circuit 57, an address generator 58, and an image processing controller 59.

The color conversion circuit 53 changes the color of each pixel of the image according to the instruction of the image processing control section 59. Generally, each color is composed of three colors of R, G, B or Y, RY, BY, and color conversion is performed by changing the mixture ratio. The variable tap low-pass filter 54 has a low-pass filter function for performing anti-aliasing processing prior to reduction processing. This variable tap low pass filter 54
Since the tap coefficient can be changed in accordance with the instruction from the image processing control unit 59, the low frequency range to be operated can be changed according to the degree of reduction. The low-pass filter function is also used for blurring processing called defocus, which is one of the special effects. The image memory 55 is a working memory for performing coordinate transformation called geometric transformation. The address for conversion is generated by the address generator 58. The interpolation filter 56 has an interpolation function for filling empty pixels generated by coordinate conversion with surrounding pixel values.
The synthesizing circuit 57 synthesizes a plurality of processed images. The address generator 58 generates an address for geometrically transforming the image on the image memory 55. The image processing controller 59
A control signal is issued to the color conversion circuit 53, the variable tap low-pass filter 54, the address generator 58, the interpolation filter 56, and the synthesizing circuit 57 to instruct the processing. This image processing control unit 5
A control signal from the control unit 7 is supplied to 9.

Here, the processing range of the image and the area where the image is stored in the memory are set by the parameters according to the processing screen size information of the attribute information transmitted from the control unit 7 and the image size information. As a result, an image of any size can be processed with any processing screen size. The encoder 5 has an image format conversion function and a communication function with the control unit 7, in addition to the function of compressing the image signal. Here, the encoder 5 compresses the image while changing the compression method according to the instruction of the control unit 7, but bypasses the image that does not need to be compressed. In addition, the encoder 5 uses the image format conversion function to control the image format of the component described above by the control unit 7.
The output image format is converted according to the image format information of the attribute information supplied from the. Since this conversion is also processed according to the image size information of the attribute information from the control unit 7, any image can be handled. The transmission of the attribute information is performed using the communication function with the control unit 7.

This encoder 5, as shown in FIG. 14, is similar to the decoder 2 in that it has a JPEG encoder 60.
And an MPEG encoder 70. These two systems are separated by the encoder selector 36,
Together with the output selector 37. Further, in front of the encoder selector 36, there is provided a system conversion decoder 38 for executing the above-mentioned image system conversion function.

The JPEG encoder 60 comprises a blocking circuit 61, a DCT circuit 62, a quantizer 63, a Huffman encoder 64, a run length encoder 65, and a multiplex circuit 66. On the other hand, the MPEG encoder 70 includes a buffer memory 71, a DCT circuit 72, a quantizer 73, a Huffman encoder 74, a buffer memory 75, a motion vector detection circuit 76, a forward prediction circuit 77, and a backward direction. Prediction circuit 78, frame memory 79, frame memory 80, and inverse quantizer 8
1 and an IDCT circuit 82.

The selection of these two encoders 60 or 70 is performed by the encoder selector 36 according to the compression method information of the attribute information supplied from the control section 7. The blocking circuit 61 of the JPEG encoder 60 divides one image into small blocks, for example, blocks each including 8 × 8 pixels. The DCT circuit 62 performs a discrete cosine transform process on each block including, for example, 8 × 8 pixels. Quantizer 6
In step 3, the power of 64 pixel data for each block is divided by a quantization coefficient and quantized. Huffman encoder 64
Is a Huffman code for the DC component of, for example, 64 spectrums output from the quantizer 63. The run length encoder 65 converts the remaining quantized AC components into run length codes. The multiplex circuit 66 selectively combines the Huffman coded data and the run length coded data.

The buffer memory 71 of the MPEG encoder 70 temporarily stores the data required for the encoding process. Generally, one GOP worth of data is stored.
The DCT circuit 72, like the DCT circuit 62, performs a discrete cosine transform process. The quantizer 73 divides each discrete cosine transform value by the quantization number. The Huffman encoder 74 converts the quantized data into Huffman code. The buffer memory 75 stores the processing result data until a predetermined aggregated result is obtained and output. The motion vector detection circuit 76 is a block of a standard image called a reference frame (generally composed of 16 × 16 pixels).
, To which position in another image, that is, a movement vector is obtained. The forward prediction circuit 77 extracts from the frame memory 80 a block corresponding to a vector obtained from a temporally previous image. The backward prediction circuit 78
A block corresponding to a vector obtained from a temporally subsequent image is extracted from the frame memory 79. Inverse quantizer 8
1 cancels the quantization in the quantizer 73 to create a coded frame corresponding to a B frame or a P frame. ID
The CT circuit 82 also performs the discrete cosine inverse transform process in order to solve the discrete cosine transform process in the DCT circuit 72. The frame memories 79 and 80 store the images reproduced by the inverse quantizer 81 and the IDCT circuit 82,
Forward prediction circuit 77 and backward prediction circuit 78, respectively.
Save for the prediction process performed in. Buffer memory 7
1 and the DCT circuit 72, the subtractor 83 provided between
The selection piece a of the changeover switch 84 is connected. "0" is supplied to the selected terminal b of the changeover switch 84, the output of the forward prediction circuit 77 is supplied to the selected terminal c, and the output of the backward prediction circuit 78 is supplied to the selected terminal e. The forward prediction circuit 7 is connected to the selected terminal d.
The addition output of the adder 85 for adding the output of 7 and the output of the backward prediction circuit 78 is supplied. Therefore, the subtractor 8
3 subtracts the output of the selected terminal b, c, d or e switched by the selector switch 84 from the output of the buffer memory 71. That is, the subtractor 83 outputs “0” when it cannot predict from the frame to be encoded.
The forward prediction value (extracted block value) is subtracted in the case of forward prediction only, two combined values are subtracted when there are forward and backward predictions, and the backward prediction value is subtracted in the case of only backward prediction value. The adder 85 adds and combines the forward and backward predicted values. In the case where the adder 86 makes a prediction frame based on the average of preceding and following prediction frames and makes a prediction, the changeover switch 8 having the selected piece a and the selected terminals b and c
Used together with 7 to perform frame addition.

The output buffer memory unit 6 includes an encoder 5
In addition to the function of converting the transfer rate of the data sent from the device, the communication function with the control unit 7 is provided. The output buffer memory unit 6 is composed of a buffer memory 89 with a rate conversion function, as shown in FIG. The buffer memory with rate conversion function 89 temporarily stores the compressed image or output image for rate adjustment. The buffer memory with rate conversion function 89 is, like the buffer memory with rate conversion function 52 of the input buffer memory unit 3, irrelevant to the image size of the image signal and has a sufficient capacity, and is a double buffer memory system. And each memory has an address generator that operates independently. As for the input rate, the address generator in each memory generates an address according to the image size information and the image input / output rate information of the attribute information given from the control unit 7. By adjusting the size of the address generation block and the block address interval, the internal rate of the processing system can be converted into various output rates. That is, the output buffer memory unit 5 can also perform arbitrary rate conversion.

The control unit 7 includes the matrix switcher 1,
Decoder 2, input buffer memory unit 3, image processing unit 4,
It has a function of controlling each processing of the encoder 5 and the output buffer memory unit 6. The operation of the image information processing apparatus configured as above will be described below. As described above in the matrix switcher 1, the compressed image signal fetched and selected for switching includes the compressed / non-compressed state identification information, the compression method information, and the image size information according to the number of pixels in the horizontal and vertical directions at the beginning of the data It has header information including processing screen size information for determining a processing area, image system information such as NTSC, PAL, RGB, and attribute information such as input / output signal rate information.

The decoder 2 reads the header information and sends the image attribute information to the control unit 7. The control unit 7 sends the compressed / non-compressed identification information, the compression method information, and the image size information to the decoder 2 again from the attribute information. The decoder 2
The attribute information read by the decoder 2 itself may be used or changed without receiving the attribute information from the control unit 7. The attribute information can also be given from the keyboard 8. The control unit 7 supplies the image size information, the image method information, and the image input / output rate information of the attribute information to the input buffer memory unit 3. Further, the control unit 7 supplies the image size information and the processing screen size information of the attribute information to the image processing unit 4. In addition, the control unit 7
The encoder 5 encodes the compressed / non-compressed state identification information of the attribute information, compression method information, image size information, and image method information.
Supply to. Further, the control unit 7 supplies the image size information and the image input / output rate information of the attribute information to the output buffer memory unit 6.

When a switching instruction is supplied from the control section 7 to the matrix switcher 1, the matrix switcher 1 switches the compressed image data in units of GOP as described above. According to the attribute information from the control unit 7, the decoder 2 outputs the signal that does not need the decompression process to the input buffer memory unit 3 as it is, while performing the decompression process selected according to the selection information on the signal that requires the decompression process. The decompression process determines the processing range according to the image size information. This allows the decoder 2 to expand an image of any size.

The input buffer memory section 3 uses the format conversion encoder 51 in accordance with the image format information from the control section 7 to format-convert the composite image signal or Y / C signal image signal into a component signal suitable for internal processing. .
At the time of this system conversion processing, the processing range is determined according to the image size information from the control unit 7. As a result, the system conversion process in the input buffer memory unit 3 is also effective for images of arbitrary size. Since the component signal is converted to the internal processing rate, it is written in the buffer memory with rate conversion function 52 at the input rate and is read again at the internal rate. This buffer memory 52 with a rate conversion function
As described above, irrespective of the image size, has a sufficient capacity, has a double buffer structure, has independent address generators, and reads and writes at different block rates. Since the address area generated by the address generator is determined according to the image size information from the control unit 7, the conversion of the transfer rate in the input buffer memory unit 3 is effective even for an image of an arbitrary size. .

According to the image size information from the control unit 7, the image processing unit 4 designates and secures an area of the image memory 55 in which an image of an arbitrary size is stored and determines an address range for reading and writing. This allows the image processing unit 4 to perform image processing on an image of any size. Further, the image processing range is determined according to the processing screen size information from the control unit 7. As a result, the image processing unit 4 can use the processing system resources only for the processing in the specified range, and can effectively use the unnecessary processing system resources used for the processing not reflected in the result in other processing. Became.

Here, the image processing of the image processing unit 4 will be described. The image signal output from the input buffer memory unit 3 is supplied to the image processing unit 4. The image signal that has entered the image processing unit 4 is subjected to various image processes in accordance with instructions from the control unit 7. As shown in FIG. 13, the image processing unit 4 includes an image processing control unit 59 that controls the image processing unit 4. Color conversion in units of pixels or blocks of a plurality of pixels is performed by the color conversion circuit 53 according to an instruction from the image processing control unit 59. When color conversion is not necessary, the color conversion circuit 53 bypasses the image signal according to an instruction from the image processing control unit 59. The image signal that has passed through the color conversion circuit 53 is supplied to the variable tap low-pass filter 54, and the high-frequency signal is removed in preparation for the deformation or reduction performed by the circuit in the subsequent stage described later. The process of removing the high-frequency signal is necessary to prevent the high-frequency signal from adversely affecting the low-frequency signal in terms of frequency, such as aliasing due to the deformation or reduction process. How much the high frequency signal is removed is determined by the image processing control unit 59 by instructing the high frequency band to be removed depending on the degree of deformation or reduction. If it is not necessary to limit the band, this processing function is bypassed by an instruction from the image processing control unit 59. The image signal that has passed through the variable tap low-pass filter 54 is supplied to the image memory 55. The image memory 55 is a working memory for performing two-dimensional and three-dimensional geometric conversion. The address generator 5 performs the geometric conversion.
8 and is supplied to the image memory 55. The image processing control unit 59 indicates what kind of address is generated by the address generator 58. This instruction is generally given by modeling data and conversion rule data. The data group read from the image memory 55 is generally incomplete as raster data and has many missing pixels. An interpolation filter 56 performs an interpolation process of filling this gap using surrounding pixels. The image processing control unit 59 instructs the accuracy of interpolation. As the interpolation method, there are a nearest neighbor method, a linear interpolation method, a cubic interpolation method, and the like, and the latter gives a more accurate interpolation value.
The raster signal in which the gaps are filled with the interpolation is supplied to the synthesizing circuit 57. The synthesizing circuit 57 synthesizes a plurality of processed images two-dimensionally or three-dimensionally. The image processing control unit 59 supplies a control signal such as depth information at the time of composition. At the final stage of composition, the image is converted into a two-dimensional image and displayed on the monitor 9.

Further, the two-dimensional image is supplied to the encoder 5, and the format is converted and compressed. The format conversion from the component signal is determined according to the image format information of the attribute information supplied from the control unit 7. Whether to compress or not and the compression method are determined according to the compression / non-compression state identification information and the compression method information of the attribute information supplied from the control unit 7. If the format conversion processing and the compression processing are not required, they are not processed and sent to the output buffer memory unit 6. At the time of the format conversion processing and the compression processing, the processing of the image range designated according to the image size information from the control unit 7 is performed. As a result, the encoder 5 can handle an image of any size.

The output of the encoder 5 enters the output buffer memory section 6. The output buffer memory unit 6 has the same capacity as the buffer memory 52 with the rate conversion function of the input buffer memory unit 3 regardless of the image size, has a sufficient capacity, has a double buffer structure, and has an independent address generator. Since there is a buffer memory 89 with a rate conversion function for reading and writing at different block rates, the rate conversion of the sending data can be performed. Since the address area generated by the address generator is determined according to the image size information from the control unit 7, the output buffer memory unit 6 can also convert the rate of an image having an arbitrary size.

As described above, the image information processing apparatus according to the present embodiment enables input / output and processing of an image independent of the resolution, transfer rate, and image size, and at the same time, a compressed image obtained by interframe compression processing. Can be switched smoothly and accurately. In particular, by including the matrix switcher 1 described above, the compressed image after switching does not affect the decompression process. Further, it is not necessary that the GOP frame configurations after switching are the same. Further, the GOP phases of the switching images do not have to match. It is possible to suppress an increase in the GOP information amount of the image after the switching composition. In addition, cut editing can be performed without the need for image expansion.

Next, a modification of the embodiment of the image information processing apparatus according to the present invention will be described with reference to FIG. In this modification, the hardware processing unit 100 and the software processing unit 110 are included.
And an input / output control unit 120 and a data storage unit 130. In this modified example, the processing is divided into a hardware processing unit 100 and a software processing unit 110, so that the flexibility and expandability of the entire processing of the device are enhanced. The hardware processing unit 100 is
Mainly performs mechanical processing such as filtering and software processing, which has a heavy load. The software processing unit 110 performs intelligent processing and processing with high expandability.

The hardware processing unit 100 has substantially the same configuration as the image information processing apparatus of the above-described embodiment described with reference to FIG. That is, the two external input devices 101 and 1
The plurality of compressed images supplied from 02 are switched by the matrix switcher 1 and supplied to the decoder 2.
It is expanded by the decoder 2. The image signal expanded by the decoder 2 is supplied to the input buffer memory unit 3. The signals passed through the input buffer memory unit 3 are the color converter 53, the variable tap low-pass filter 54, and the frame memory 5.
5, the interpolation filter 56, the synthesizing circuit 57, and the address generator 58. These units are connected by a local bus 103 so that each function of hardware can be used as a hardware module that can be used by software during software processing. The image signal image-processed by the image processing unit 4 is output to the external output device 104 via the encoder 5 and the output buffer memory unit 6.
Here, each unit described above is controlled by the hardware module control unit 105.

The software processing section 110 includes a CPU 111, a cache memory 112, a main memory 113 and a CPU bus 1.
14 and a memory bus control unit 115. The memory bus control unit 115 is connected to the local bus 103 and the hardware module control unit 105.
Image data and a control signal to the hardware module controller 105. The memory bus control unit 115 is connected to the graphic monitor control unit 1 via the peripheral bus 116.
21, a media control unit 122, and a switcher control unit 1
23, a small computer system interface (hereinafter referred to as SCSI) adapter 124, and an operation panel 125. The graphic monitor control unit 121 has a built-in video memory and controls the display of the graphic monitor 126 using the video memory. The media control unit 122 includes external input devices 101 and 102 such as VTRs and data recorders, and external output devices 10 such as VTRs and data recorders.
4 controls the image information input / output timing. The switcher controller 123 controls the matrix switcher 70. The SCSI adapter 124 is a magneto-optical disk (indicated as MO in the figure) device 1 connected by a SCSI bus 127.
31, a CD-ROM 132, a hard disk device (referred to as HDD in the drawing) 133, and the like as an interface of a data storage device. The operation panel 125 is used to instruct processing and input / output of image information.

A method of realizing a hardware module that can be used from software is such that when a subroutine library is called from a program being executed by the CPU 111, if the subroutine library is a software library, the program address corresponding to the subroutine library is determined. It jumps and is executed. Also, when the subroutine library is a hard library, the address jumps to the address corresponding to the hard module determined at the time of linking. The address corresponding to the hard module stores the procedure of sending necessary data to the hard module, starting the execution, confirming the end of the execution, and returning the execution to the main program of the software. For example, when the variable tap low-pass filter 34 is called as a hardware module, the CPU 1
According to the instruction of 11, data is sent from the main memory 113 to the variable tap low-pass filter 34 via the memory bus control unit 115 and the local bus 103. Next, the CPU 111 instructs the variable tap low-pass filter 34 to execute via the memory bus control unit 115 and the local bus 103. CPU111
Confirms the end of execution and sends the processed data to the main bus 1 via the local bus 103 and the memory bus control unit 115.
Collect at 13. Then, the process returns to the main program to execute the next step.

As described above, the image information processing apparatus which is the modified example shown in FIG. 16 enhances the flexibility and expandability of the entire processing of the apparatus, and also a wide range of images from still images to moving images and resolutions. Free format images that do not depend on
Image processing such as generation, color conversion, composition, and editing can be performed on scalable format images having different transfer rates and image sizes. In addition, it is possible to smoothly and accurately switch compressed images that have been subjected to inter-frame compression processing.

As another modified example, the CPU 111
An apparatus is conceivable in which a plurality of units including the cache memory 112 and a plurality of units are connected to the CPU bus 114. Therefore, in this other modification, when the load is heavy,
The load can be reduced by performing parallel processing.

[0086]

As is apparent from the above description, according to the image information processing apparatus of the present invention, the selecting means selectively fetches a plurality of compressed images subjected to inter-frame compression processing, and a predetermined Since the inter-frame forward prediction coded image is converted into the intra-frame coded image in the unit of the number of frames and the prediction data of the bidirectional predictive coded image is changed, the switching of the compressed image as if the inter-frame compression process was performed Can be done smoothly. In addition, a wide range of images from still images to moving images, free format images that do not depend on resolution, images that do not depend on transfer rate, scalable format images that do not depend on image size, such as color conversion, composition, editing, etc. Image processing can be performed.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of an image information processing apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a detailed configuration of a matrix switcher of the image information processing apparatus shown in FIG.

FIG. 3 is a GOP of the matrix switcher shown in FIG.
It is a block diagram which shows the detailed structure of a reconstruction part.

FIG. 4 is a diagram showing a GOP encoding order and an input order.

5 is an input sequence "IBBP" of the GOP shown in FIG.
It is an enlarged view of a part.

[Fig. 6] An encoding order is "IBBPBBPBBPBBPB.
It is a figure for demonstrating operation | movement of the GOP reconstruction part of the matrix switcher with respect to the frame sequence of GOP which is B ".

[Fig. 7] An encoding order is "IBBPBBPBBPBBPB.
It is a figure for demonstrating operation | movement of the GOP reconstruction part of the matrix switcher with respect to the frame sequence of GOP which is B ".

[Fig. 8] An encoding order is "IBBBIBBBIBBBIBB.
It is a figure for demonstrating operation | movement of the GOP reconstruction part of the matrix switcher with respect to the frame sequence of GOP which is BB ".

[Fig. 9] Fig. 9 is an encoding order "IBBBIBBBIBBBIBB.
It is a figure for demonstrating operation | movement of the GOP reconstruction part of the matrix switcher with respect to the frame sequence of GOP which is BB ".

FIG. 10 is a diagram for explaining a data compression process of a GOP reconstruction unit.

11 is a block diagram showing a detailed configuration of a decoder of the image information processing apparatus shown in FIG.

12 is a block diagram showing a detailed configuration of an input buffer memory of the image information processing apparatus shown in FIG.

13 is a block diagram showing a detailed configuration of an image processing unit of the image information processing apparatus shown in FIG.

FIG. 14 is a block diagram showing a detailed configuration of an encoder of the image information processing apparatus shown in FIG.

15 is a block diagram showing a detailed configuration of an output buffer memory of the image information processing apparatus shown in FIG.

FIG. 16 is a block diagram showing a detailed configuration of an image information processing apparatus according to another embodiment of the present invention.

FIG. 17 is a block diagram showing a configuration of a conventional image information processing device.

[Explanation of symbols]

 1 matrix switcher 2 decoder 3 input buffer memory unit 4 image processing unit 5 encoder 6 output buffer memory unit 7 control unit

Claims (6)

[Claims] 1. A plurality of compressed images subjected to inter-frame compression processing are selectively captured, and inter-frame forward predictive encoded images are converted into intra-frame encoded images in a predetermined number of frames, and bidirectional prediction is performed. Selecting means for changing the prediction data of the coded image, decompressing means for expanding the compressed image signal selected by the selecting means, and first inputting / outputting the image signal output from the expanding means
And an image processing unit for performing various kinds of image processing on the image signal output from the first input / output unit to output an image processing signal, and an image processing signal output from the image processing unit. Compressing means for compressing and outputting a compressed image processing signal, second input / output means for inputting / outputting the compressed image processing signal output from the compressing means, selection processing of the selecting means, decompression processing of the decompressing means A control means for controlling the input / output processing of the first input / output means, the image processing of the image processing means, the compression processing of the compression means, and the input / output processing of the second input / output means. Image information processing device. 2. The image information processing apparatus according to claim 1, wherein the selecting means suppresses an increase in the amount of information due to a change in the prediction data of the bidirectional predictive encoded image. 3. The decompression means, based on the input compressed image signal, compressed / non-compressed state identification information, compression method information, image size information according to the number of pixels in the horizontal and vertical directions,
The attribute information including processing screen size information, image method information, and input / output signal rate information for determining a processing area is read out and supplied to the control means.
The image information processing device described. 4. The first input / output unit has a system conversion unit that converts the system of the image signal, and a storage unit that is independent of the image size of the image signal and has a sufficient capacity. The image information processing apparatus according to claim 1, wherein the image information processing apparatus comprises: 5. The image information according to claim 1, wherein the second input / output unit has a storage unit having a sufficient capacity, which is independent of the image size of the image signal. Processing equipment. 6. The image information processing apparatus according to claim 1, wherein the compression means includes a system conversion unit that converts a system of the image signal.