JPH02254886A - Encoder and decoder for image packet - Google Patents

Abstract

PURPOSE:To improve a reproduced image quality by detecting the mean value component of an input image signal at every area obtained by dividing a single image into plural parts, obtaining a mean value eliminated image signal obtained by eliminating the mean value component detected from the input image signal, separately encoding and the mean value component and the mean value eliminated image signal, making them into packets, and transmitting them. CONSTITUTION:A mean value detector 12 calculates a mean value component (direct current component) Sm of an input image signal S in units of a sub frame obtained by dividing one frame into plural parts, and applies the mean value component Sm to a quantizer 15. The quantizer 15 quantizes the mean value component Sm to a code Smp at (n) bits, and applies the quantized code Smp to a inverse quantizer 16 and a side information multiplexing circuit 17. The reverse quantizer 16 executes reverse quantization to be the reverse processing of the quantizer 15, obtains a local reproduced mean value component Smk corresponding to the said mean value component Sm, and applies it to a subtracter 14. The subtracter 14 subtracts the local reproduced mean value component Smk from the input image signal S, and sends a subtracted output, namely, a mean value eliminated image signal S1, from which the mean value component is eliminated, to a subtracter 18.

Description

Detailed Description of the Invention [Industrial Application Fields] The present invention relates to an image packet encoder and decoder used in a communication form in which encoded image signals are multiplexed in packet units and transferred over a packet switching network. It is related to vessels.

[Prior Art] High-speed packet switching, which is a promising candidate for the Integrated Services Digital Network (ISDN), is based on the asynchronous transfer mode (AT).
Since a variable speed transmission line can be realized by M), it is basically ideal for low speed moving image transmission where the output speed is not constant. In such image packet encoding, characteristics such as the maximum bit rate and periodicity of the output signal, which greatly affect the design of the transmission path, change depending on the unit of transmission (image portion or packet length unit). In addition, television telephones, etc.
Care must be taken in the selection of the transmission unit, since the delay time, which is important in the case of real-time intercommunication, depends on this.

In view of these points, as an image packet encoding/decoding method, hybrid encoding/decoding based on an algorithm consisting of motion compensated interframe prediction, orthogonal transformation (e.g., discrete cosine transformation), and variable length coding is proposed. Using the method,
Studies have already been carried out in consideration of transmission path efficiency regarding the packet length when packetizing codes related to orthogonal transform coefficients generated by an image packet encoder (Masahiro Wada,
Co-authored with Hirohisa Yamaguchi, [Study of transmission units in packet video coding], 1988 IEICE Spring National Conference, D-87>.

[Problems to be Solved by the Invention] However, when performing packetization, it is not sufficient to simply consider the packet length. In other words, in the hybrid encoding/decoding method described above, if a packet is discarded in a high-speed packet switching network, the internal states of the image packet encoder and image packet decoder will become inconsistent, and subsequent decoding will be interrupted. There is a problem in that the reproduced image quality on the converter side is degraded.

Various methods have been proposed to deal with packet discard by interpolation processing on the image packet decoder side, but the interpolation processing configuration is complicated, and if the reproduced image signal is to be kept to a certain level through interpolation processing, it will not be orthogonal. It is undeniable that the number of packets to be transmitted becomes large because it is necessary to reduce the unit area of conversion.

The present invention has been made in consideration of the above points, and uses new packetization to prevent deterioration of the reproduced image signal due to mismatch in the internal states of the encoder and decoder due to packet discard, without using interpolation processing. It is an object of the present invention to provide an image packet encoder and decoder that can be reduced.

[Means for solving the problem] In order to solve the problem, in the first invention,
The image packet encoder was constructed using the following means.

That is, an average value component detection means for detecting an average value component of an input image signal for each region into which one image is divided into a plurality of regions, and an average value removed image signal obtained by removing the average value component detected from the input image signal are obtained. and subtraction means. Further, the average value encoding means encodes the detected average value component, the average value removed image encoding means encodes the obtained average value removed image signal, and the encoded average value component and the encoded average value component are encoded. and a packet assembling means for assembling and outputting the average value removed image signal into separate packets.

In the second aspect of the present invention, an image packet decoder that decodes the packets output from the image packet encoder according to the first aspect of the present invention is constituted by the following means.

That is, a packet separating means receives a packet output from an image packet encoder and separates an encoded average value component and an encoded average value removed image signal from the received packet; mean value removed image decoding means for decoding the average value removed image signal that has been
and average value decoding means for decoding the encoded average value component. Further, an adding means is provided to combine the decoded mean value removed image signal and the decoded mean value component to obtain a reproduced image signal.

[Operation] In the first aspect of the present invention, the average value component detection means detects the average value component of the input image signal for each region in which one image is divided into a plurality of regions, and provides the detected average value component to the subtraction means and the average value encoding means. The subtraction means removes the average value component from the input image signal to obtain a mean value removed image signal, and supplies the obtained average value removed image signal to the mean value removed image encoding means.

The average value encoding means and the mean value removed image encoding means encode the average value component and the mean value removed image signal, respectively, and provide the encoded mean value component and the mean value removed image signal to the packet assembling means, and the packet assembling means encodes the encoded average value component and the code. and the average value removed image signal are assembled into separate packets and output.

In the second aspect of the present invention, the packet separation means receives the packet output from the image packet encoder, and extracts the encoded average value component and the encoded average value removed image signal from the received packet. and the separated coded mean value removed image signal is given to mean value removed image decoding means, and the separated coded mean value component is given to mean value decoding means.

The mean value removed image decoding means decodes the encoded mean value removed image signal and provides it to the addition means, and the average value decoding means decodes the encoded average value component and supplies it to the addition means. give. The addition means combines the decoded average value removed image signal and the decoded average value component to obtain and output a reproduced image signal.

Here, the packets that are lost to the average value removed image signal are
Packets that are discarded due to packet congestion, since there are more packets related to the average value component, are often packets related to the average value removed image signal. Further, it is possible to instruct a network that can determine packets to be discarded to preferentially discard packets that fall short of the average value removed image signal.

In any case, even if packets included in the average value removed image signal are discarded, the reproduced image signal includes the average value component that most affects the reproduced image quality.

[Example] Hereinafter, an example of the present invention will be described in detail with reference to the drawings.

First, the image packet encoder will be explained with reference to FIG.

Here, FIG. 1 is a block diagram showing the configuration of an image packet encoder according to an embodiment, and FIG. 2 is an explanatory diagram showing the contents of a packet header.

In FIG. 1, an image signal S input through an input terminal 11 is applied to an average value detector 12, a motion amount detector 13, and a subtracter 14.

The average value detector 12 calculates the average value component (DC component) Sm of the input image signal S for each subframe obtained by dividing one frame into a plurality of subframes, and provides the average value component Sm to the quantizer 15. The quantizer 15 quantizes the average value component Sm into an n-bit code Smp, and provides the quantized code Smp to the inverse quantizer 16 and the side information multiplexing circuit 17. The inverse quantizer 16 performs inverse quantization, which is the inverse processing of the quantizer 15, to obtain a locally reproduced average value component Smk corresponding to the above-mentioned average value component Sm, and supplies it to the subtracter 14. The subtracter 14 subtracts the local reproduction average value component Smk from the input image signal S,
The subtracted output, that is, the average value removed image signal S1 from which the average value component has been removed, is sent to the subtracter 18.

Note that the reason why the average value component Sm is not immediately provided to the subtractor 14 but is quantized and then the dequantized average value component Smk is provided to the subtracter 14 is due to the image packet decoding described later. This is because the processing in the equipment was taken into account.

As described above, the motion amount detector 13 to which the input image signal S is supplied also receives information from the frame memory 19 about the previous frame (more precisely, the influence of multiple past frames is included, but mainly information about the previous frame) is reflected)
A locally reproduced image signal S2 is given. Motion amount detector 1
3 detects the amount of motion from the locally reproduced image signal S2 one frame before the image signal S of the current frame, and provides the motion iMv to the motion compensation circuit 20, and the side information multiplexing circuit 1
Give to 7. The motion compensation circuit 20 also stores information from the frame memory 21 one frame before, from which the average value component has been removed (more precisely, the influence of multiple past frames has entered, but mainly information from the one frame before is reflected). The motion compensation circuit 20 motion-compensates this image signal S3 according to the detected motion and tMV to calculate the average value-removed image signal of the current frame. A predicted image signal Sp is obtained and applied to the subtracter 18.

The subtracter 18 subtracts the predicted image signal Sp from the average value removed image signal S1 of the current frame, and provides the obtained difference signal e to the orthogonal transformer 22. The orthogonal transformer 22 is, for example,
It is a discrete cosine transformer, and orthogonally transforms the difference signal e in units of blocks (areas smaller than the above-mentioned subframes) that are divided into a plurality of frames, and provides orthogonal transform coefficients Q to the quantizer 23. The quantizer 23 quantizes the orthogonal transform coefficient Q in units of blocks, and provides the obtained quantization code q to the variable length encoder 24, the block validity/invalidity determination circuit 25, and the inverse quantizer 26.

The inverse quantizer 26 performs an inverse quantization process, which is the inverse process of the quantizer 23 , to obtain locally reproduced orthogonal transform coefficients Qk and provides them to the inverse orthogonal transformer 27 . The inverse orthogonal transformer 27 performs an inverse orthogonal transform process corresponding to the orthogonal transformer 22 to obtain a locally reproduced difference signal ek and supplies it to the adder 28 . The adder 28 is also supplied with the predicted image signal Sp from the motion compensation circuit 20. The adder 28 adds the locally reproduced difference signal ek to the predicted image signal Sp to obtain a locally reproduced image signal Slk corresponding to the average value removed image signal S1, and supplies the obtained locally reproduced image signal Slk to the leak coefficient multiplier 29 and the adder 30.

The leak coefficient multiplier 29 multiplies the image signal Slk by the leak coefficient β (0 × β × 1), supplies the multiplied image signal to the frame memory 21 for storage, and forms the next predicted image signal Sp. be used for.

The reason why the leak coefficient β is multiplied here is to cope with the fact that the image packet decoder, which will be described later, is equipped with a leak coefficient multiplier to reduce the influence of packet discard through the leak effect. be.

The adder 30, which is supplied with the locally reproduced image signal Slk from which the average value component has been removed, is also supplied with the locally reproduced average value component Smk. In this way, the adder 30 obtains the locally reproduced image signal Sk corresponding to the input image signal S, and this locally reproduced image signal Sk is stored in the frame memory 1.
9 and stored, and used for the next motion amount detection.

The variable length encoder 24 to which the quantization code q from the quantizer 23 is supplied performs run-length encoding and/or Huffman encoding, for example, and supplies the variable length code to the packet assembler 31.

Further, the block validity/invalidity determination circuit 25 to which the quantization code q from the quantizer 23 is given determines whether the block is an invalid block that may use the quantization code of a past frame based on the given quantization code q. or whether it is a valid block for which the quantization code of the past frame cannot be used, and provides block validity/invalidity information BY to the side information multiplexing circuit 17.

The side information multiplexing circuit 17 multiplexes the mean value code Smp given from the quantizer 15, the motion JLMv given from the motion amount detector 13, and the block validity/invalidity information BY given from the block validity/invalidity determination circuit 25. , and provides the multiplexed information (2) to the variable length encoder 32. The variable length encoder 32 performs, for example, run length encoding and/or Huffman encoding, and provides the variable length code to the packet assembler 31.

The packet assembling unit 31 includes a code based on the orthogonal transform coefficients provided from the variable length encoder 24 and the variable length encoder 3.
The side information given from 2+the code related to the average value component are separately packetized.

Here, the side information is information necessary when an image packet decoder, which will be described later, decodes a code that differs between orthogonal transform coefficients. This applies to information BY.

The reason why these codes are separated into packets is that we focused on the fact that these codes have different degrees of influence on the reproduced image quality. That is, this is because the side information operator average value component has a higher influence on the reproduced image quality.

In this way, various types of information as shown in FIG. 2 are inserted into the header of the packet in which the side information ten mean value component code or the orthogonal transform coefficient code is inserted into the information field.

In FIG. 2, destination information and source information are inserted into the first and second header areas H1 and H1 and H2, as in the conventional case.

Priority information regarding packet discard is inserted into the third header area H3. Here, the packet assembly section 31
In this case, the priority of the packet whose code inserted in the information field is an orthogonal transform coefficient code is set high, that is, it is easily discarded during packet congestion, and the code inserted into the information field is set to be a side information + average value component code. The priority of the packet is set low, that is, it is made difficult to be discarded at the peak of packet congestion. Note that since the packet assembling unit 31 has a different code input terminal, it is possible to easily insert such packet discard priority information.

In the fourth header area H4, when the packet concerned is a packet related to side information + average value component code, information indicating whether the information inserted in the information field is side information or average value component is inserted. Ru.

Position information indicating which position on the image the code inserted in the information field relates to is inserted into the fifth header area H5. In the sixth header area H6, division presence/absence information indicating whether the code inserted in the information field spans multiple packets is inserted.
A sequence number in the case of division is inserted into the header area H7.

Image type information indicating whether the image signal is a luminance signal or a first or second color difference signal is inserted into the eighth header area H8.

Here, although FIG. 1 shows a processing system for an image signal such as a luminance signal, a first color difference signal, or a second color difference signal, the packet assembling unit 31 is common to each processing system. As described above, image type information indicating whether the code is related to a luminance signal, a first color difference signal, or a second color difference signal is inserted.

The packets assembled by the packet assembling section 31 in this manner are sent to the transmission path (network) 33.

Next, the image packet decoder will be explained with reference to FIG. 3.

FIG. 3 is a block diagram showing the configuration of an image packet decoder of an embodiment corresponding to the image packet encoder described above.

In FIG. 3, a packet via a packet transmission path 40 is provided to a packet disassembly section 41. In FIG. Packet disassembly unit 4
1 decomposes a packet based on the header contents of the packet, provides a variable-length encoded orthogonal transform coefficient code to a variable-length decoder 42, and decomposes the variable-length encoded side information + average value component. The code is provided to a variable length decoder 43.

The variable length decoder 43 executes the inverse processing of the above-mentioned variable length encoder 32 to obtain the side information person average value component code I.
s is obtained and applied to the side information demultiplexing circuit 44. The side information multiplexing/demultiplexing circuit 44 separates the side information code Is, obtains a reproduction average value code Smps, a reproduction motion amount Mvs, and reproduction block validity/invalidity information BYs, and obtains an average value code Smps.
s to the inverse quantizer 45, the motion amount Mvs to the motion compensation circuit 46, and the block validity/invalidity information BYs to the block validity/invalidity determination circuit 47.

On the other hand, the variable length decoder 42 performs the inverse processing on the variable length encoder 24 described above to reproduce the reproduced orthogonal transform coefficient code q.
s is obtained and applied to the inverse quantizer 48.

The inverse quantizer 48 performs the inverse processing of the quantizer 23 described above, converts the code qs into a reproduced orthogonal transform coefficient Qs, and supplies it to the inverse orthogonal transformer 49. The inverse orthogonal transformer 49 performs the inverse processing of the above-described orthogonal transformer 22 on the reproduced orthogonal transform coefficient Qs, obtains a reproduced difference signal es, and supplies it to the adder 50.

Here, in the inverse quantizer 48 and the inverse orthogonal transformer 49, a block validity/invalidity determination circuit 47 is provided with block validity/invalidity information B.
A result determined based on Ys is given, and the inverse quantizer 48 and the inverse orthogonal transformer 49 process valid blocks and do not output outputs to invalid blocks.

Also, when packet discard occurs and the orthogonal transform coefficient code qs is not input, the inverse quantizer 48 and the inverse orthogonal transformer 49 are in a non-output state.

Note that when a transmission path is used as the packet transmission path 40 (33), which determines which packets to discard according to the priority information (H3) inserted in the header contents of the packet at the time of packet congestion, the above-mentioned Since the discard priority of packets associated with orthogonal transform coefficient codes is increased, such packets are discarded. In addition, even on a transmission path that does not perform priority processing for packet discard, the number of packets related to orthogonal transform coefficient codes is larger than the number of packets related to side information + average value component codes (in fact, about 20 times), and almost no packets are discarded. In this case, the packet corresponds to the orthogonal transform coefficient code.

In addition to the reproduced difference signal es, the adder 50 is also supplied with a predicted image signal Sps from which the average value component has been removed.

This predicted image signal Sps stores a past average value removed image signal 83s (mainly influenced by the average value removed image signal of the previous frame), which is obtained by integrating the average value removed image signals of the past several frames that have been reproduced. An average value removed image signal S read out from a frame memory 51 to be described later.
38 is moved by the above-mentioned motion amount Mvs by passing through the motion compensation circuit 46. Thus, adder 50
Thus, a reproduced average value removed image signal S1S is obtained, which is applied to the adder 52 and the leak coefficient multiplier 53.

The inverse quantizer 45 performs the inverse processing of the above-mentioned quantizer 15 on the encoded reproduced average value component Smps provided from the side information multiplexer/demultiplexer circuit 44 to obtain the reproduced average value component Sms. give to The adder 52 adds the reproduced average value Sms to the average value removed image signal SIS to obtain a reproduced image signal Ss containing the average value component, and outputs the resulting signal to the output terminal 54.
Output via.

In addition, the leak coefficient multiplier 53 to which the reproduced average value removed #image signal Sls is given, multiplies this image signal SIS by the same leak coefficient β as the leak coefficient in the leak coefficient multiplier 29 of the image packet encoder described above. Then, the multiplication output is given to the frame memory 51 to be stored and used for forming the next predicted image signal Sps.

Here, if it is an invalid block, the inverse orthogonal transformer 4
Since there is no output from 9, the motion-compensated past average value removed image signal is output as is from the adder 50 and given to the adder 52, where it is added with the reproduced average value component and output. Ru. Since the invalid block is a block whose image quality does not deteriorate even if the past reproduction average value removed image signal is used, a reproduction image signal Ss of sufficient quality is output even in this case.

On the other hand, even if the reproduced difference signal es is not output from the inverse orthogonal transformer 49 due to packet discard, the adder 50 outputs the motion-compensated past reproduced average value removed image signal as is and provides it to the adder 52. is added to the reproduced average value component in the adder 52 and output. In this case, the block is originally a block that outputs the reproduced image signal obtained by adding the reproduced difference signal and the reproduced predicted image signal. Therefore, the quality of the reproduced image deteriorates. However, since the average value component Sms, which affects the image quality the most, is secured by the addition of the adder 52, its decrease is negligible.

Further, due to such packet discard, the reproduced average value removed image signal of the past frame stored in the frame memory 51 differs from the original value, but since the integration effect is suppressed by the leak coefficient multiplier 53. , the effect becomes smaller as the frame progresses, and packet discards do not have a negative impact for a long time.

When using the image packet encoder shown in FIG. 1 and the image packet decoder shown in FIG. 3 described above, the average value component is obtained from the input image signal, and this average value is An encoding process that combines motion compensated interframe prediction, orthogonal transformation, and variable length coding is performed on the mean-value removed image signal from which the component has been removed, and the mean-value component side information and
Since the coded mean value removed image signal is assembled into separate packets and sent, even if packets are discarded, the mean value component information that has the greatest impact on image quality can be decoded by the image packet decoder. This makes it possible to suppress deterioration of the reproduced image quality due to packet discard compared to the conventional method.

Incidentally, it is conceivable to respond to packet discard by performing interpolation processing using information on surrounding blocks in the image packet restoration device, but such an interpolation processing configuration uses information on surrounding blocks, so the average value Often more complex than calculated transmission configurations. It is also possible to ensure the image quality achieved by interpolation rather than the image quality achieved by average value transmission in the case of packet discard, but in this case, the unit block area of the orthogonal transformation should be made into a very small area. Therefore, the number of packets to be transmitted increases and the transmission efficiency decreases. Therefore, depending on the target image packet transmission system, it is preferable to respond to packet discard by transmitting the average value component.

Further, according to the above embodiment, by discarding the packet,
Although the average value removed image signal used to form the predicted image signal differs from the original value, a leak coefficient multiplier is provided to attenuate the negative effect, thereby preventing the negative effect from remaining until later. be able to.

(1) In the above embodiment, the mean value removed image signal is encoded by motion compensated interframe prediction, orthogonal transformation, and variable length encoding, but other encoding processing is also possible. may be applied. For example, instead of interframe prediction, interfield prediction or interline prediction may be applied, and
Variable length encoding may be omitted.

(2) In the above embodiment, the leak coefficient multiplier 29
．． 53 is shown as being provided on the input stage side of the frame memory, but it may be provided on the output stage side of the frame memory, or may be provided on the output stage side of the motion compensation circuit.

(3) In the above embodiment, the side information and the average value component are multiplexed and packetized, but the side information and the average value component may be packetized separately.
In short, the average value component may be a packet different from the orthogonal transform coefficient code.

(4) In the above embodiment, the packet discard priority information is inserted into the packet header, but if it is known in advance that the transmission path cannot perform priority discard processing, does not need to insert the priority information. In this case as well, it is necessary to perform packetization separately for the average value component and the orthogonal transform coefficient code. Even with this method, the number of packets associated with the orthogonal transform coefficient code is greater than the number of packets associated with the average value component, so in most cases the packets associated with the orthogonal transform coefficient code are discarded and the average value component is reproduced from the image signal. can be ensured, and the reproduced image quality can be improved.

[Effects of the Invention] As described above, according to the present invention, the average value component of the input image signal is detected for each region in which one image is divided into a plurality of regions, and the average value component detected from the input image signal is removed. Since the average value removed image signal is obtained, the average value component and the average value removed image signal are separately encoded and packetized, and then transmitted.
Even when packets are discarded, a reproduced image with a guaranteed average value component is very often obtained, and the image packet code can improve the reproduced image quality compared to conventional methods without using interpolation processing. It is possible to obtain a encoder and a decoder.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing an embodiment of an image packet encoder according to the present invention, FIG. 2 is an explanatory diagram showing packet header contents, and FIG. 3 is an embodiment of an image packet decoder according to the present invention. FIG. 12... Average value detector, 13... Motion amount detector, 1
4.18...Subtractor, 15.23...Quantizer, 1
6.26.48... Inverse quantizer, 17... Side information multiplexing circuit, 19.21.51... Frame memory,
20.46...Motion compensation circuit, 22...Orthogonal transformer, 24.32...Variable length encoder, 25.47...
Block validity/invalidity determination circuit, 27.49... Inverse orthogonal transformer, 28.30.50.52... Adder, 29.5
3... Leak coefficient multiplier, 31... Packet assembler, 41... Packet disassembler, 42.43... Variable length decoder. Procedural amendment (issued by Ei) October 30, 1989 Director General of the Japan Patent Office Yoshi 1) Moon Yi 1, Indication of the case 1999 Patent Application No. 075078 2, Title of the invention Image packet encoder and decoder 3 , Relationship with the person making the amendment Patent applicant address (〒105) 1-7-12 Toranomon, Minato-ku, Tokyo Name (029) Oki Electric Industry Co., Ltd. Representative Nobumitsu Kosugi 4 Address of agent (〒108) 4-10-3-6, Shibaura, Minato-ku, Tokyo.
Contents of the amendment (1) Delete "low speed" from the second line of page 3 of the specification. (2) Each "packet comparison" on page 8, line 1 of the specification, page 15, line 5, page 15, line 8, and page 18, lines 18 to 19, is referred to as "network convergence". I am corrected. Above 5, subject of correction

Claims (2)

[Claims] (1) Mean value component detection means for detecting an average value component of an input image signal for each region into which one image is divided into a plurality of regions; and an average value component detection means for detecting an average value component of an input image signal; an average value encoding means for encoding the detected average value component; a mean value removal image encoding means for encoding the obtained average value removed image signal; and an encoded average value component. and a packet assembling means for assembling and outputting the encoded mean value removed image signal into separate packets. (2) Receive the packet output from the image packet encoder according to claim 1, and separate the encoded average value component and the encoded average value removed image signal from the received packet. packet separation means for decoding the encoded mean value removed image signal; average value decoding means for decoding the encoded average value component; 1. An image packet decoder comprising: an addition means for combining the average value removed image signal and the decoded average value component to obtain a reproduced image signal.