JPH06105971B2 - Image packet encoder and decoder - Google Patents

Description

The present invention relates to an image packet encoder and a decoder used in a communication mode in which encoded image signals are multiplexed in packet units and transferred in a packet switching network. It is related to vessels.

[Prior Art] High-speed packet switching, which is a promising candidate for integrated broadband service digital network (ISDN), can realize variable speed transmission line by asynchronous transfer mode (ATM). Optimal for image transmission. In such image packet encoding, characteristics such as the maximum bit rate and the periodicity of the output signal, which greatly influence the design of the transmission path, change depending on the transmission unit (image portion or packet length unit). Further, since it depends on the delay time which is important in the case of real-time intercommunication such as a videophone, it is necessary to be careful in selecting a transmission unit.

In view of such a point, as an image packet encoding / decoding method, hybrid encoding / decoding based on an algorithm including motion-compensated interframe prediction, orthogonal transformation (for example, discrete cosine transformation), and variable length encoding Method,
The packet length when packetizing the code related to the orthogonal transform coefficient generated by the image packet encoder has already been considered in consideration of the transmission path efficiency (Masahiro Wada,
Hirohisa Yamaguchi, "A Study on Transmission Units in Packet Video Coding", 1988 IEICE Spring National Conference,
D-87).

[Problems to be Solved by the Invention] However, when packetizing, it is not sufficient to consider only the packet length. That is, in the above-described hybrid encoding / decoding method, when packet discard occurs in the high-speed packet switching network, the internal states of the image packet encoder and the image packet decoder do not match, and the subsequent decoding is not performed. There is a problem that the quality of the reproduced image on the rectifier side is deteriorated.

Various methods have been proposed to deal with packet discard by means of interpolation processing on the image packet decoder side, but the interpolation processing configuration becomes complicated, and if it is attempted to maintain the reproduced image quality to some extent through interpolation processing, it is orthogonal. It is undeniable that the number of packets to be transmitted will be large, requiring a small conversion unit area.

The present invention has been made in consideration of the above points, and through new packetization, deterioration of reproduced image quality due to inconsistency of internal states of the encoder and the decoder due to packet discard can be achieved without interpolating processing. It is an object of the present invention to provide an image packet encoder and a decoder that can be reduced.

[Means for Solving the Problems] In order to solve the problems, in the first aspect of the present invention,
The image packet encoder was constructed by the following means.

That is, average value component detecting means for detecting the average value component of the input image signal and average value removed image signal from which the average value component detected from the input image signal is removed are obtained for each region obtained by dividing one image into a plurality of areas. And subtraction means. Also, an average value encoding means for encoding the detected average value component, an average value removal image encoding means for encoding the obtained average value removal image signal, an encoded average value component, and an encoding And a packet assembling means for assembling and outputting the converted average value-removed image signal into separate packets.

In the second aspect of the present invention, the image packet decoder for decoding the packet output from the image packet encoder according to the first aspect of the present invention is configured by the following means.

That is, a packet separating unit that receives a packet output from the image packet encoder and separates the encoded average value component and the encoded average-value-removed image signal from the received packet; An average value removed image decoding means for decoding the averaged value removed image signal,
And an average value decoding means for decoding the encoded average value component. Further, it is provided with an adding means for synthesizing the decoded average value removed image signal and the decoded average value component to obtain a reproduced image signal.

[Operation] In the first aspect of the present invention, the average value component detecting means detects the average value component of the input image signal for each area obtained by dividing one image into a plurality of areas, and supplies the average value component to the subtracting means and the average value encoding means. The subtraction unit removes the average value component from the input image signal to obtain the average value removed image signal and supplies it to the average value removed image encoding unit.

The mean value coding means and the mean value removed image coding means respectively give the mean value component and the mean value removed image signal to the coded packet assembling means, and the packet assembling means gives the coded mean value component and the coded The converted average value-removed image signal is assembled into a separate packet and output.

In the second invention, the packet separating means receives the packet output from the image packet encoder, and from the received packet, the encoded average value component and the encoded average value removed image signal. Are separated from each other, and the separated encoded average-value-removed image signal is supplied to the average-value-removed image decoding means, and the separated encoded average-value component is supplied to the average-value decoding means.

The average value removed image decoding means decodes the encoded average value removed image signal and gives it to the addition means, and the average value decoding means decodes the encoded average value component to the addition means. give. The adding means synthesizes the decoded average-value-removed image signal and the decoded average-value component to obtain and output a reproduced image signal.

Here, the packet related to the average value removed image signal is
The packets that are discarded due to network congestion because they are more than the average value component are often the average value-removed image signal packets. In addition, it is possible to instruct the network that can determine the packet to be discarded to preferentially discard the packet related to the average-value-removed image signal.

In any case, even if the packet related to the average-value-removed image signal is discarded, the reproduced image signal contains the average value component that most affects the reproduced image quality.

Embodiment An embodiment of the present invention will be described in detail below with reference to the drawings.

Image Packet Encoder First, the image packet encoder will be described with reference to FIG.

Here, FIG. 1 is a block diagram showing the configuration of the image packet encoder of the embodiment, and FIG. 2 is an explanatory diagram showing the header contents of the packet.

In FIG. 1, the image signal S input through the input terminal 11 is an average value detector 12, a motion amount detector 13, and a subtractor 14.
Given to.

The average value detector 12 is an average value component (DC component) of the input image signal S for each subframe obtained by dividing one frame into a plurality of subframes.
Sm is calculated, and the average value component Sm is given to the quantizer 15.
The quantizer 15 quantizes the average value component Sm into an n-bit code Smp, and supplies the quantized code Smp to the dequantizer 16 and the side information multiplexing circuit 17. The inverse quantizer 16 performs inverse quantization, which is an inverse process of the quantizer 15, obtains a local reproduction average value component Smk corresponding to the above-mentioned average value component Sm, and gives it to the subtractor 14. The subtractor 14 subtracts the local reproduction average value component Smk from the input image signal S, and outputs the subtraction output, that is, the average value removed image signal S1 subtractor 18 from which the average value component has been removed.

Note that the mean value component Sm is not given to the subtractor 14 immediately, but is quantized, and then the dequantized mean value component
The reason why Smk is given to the subtractor 14 is because the processing in the image packet decoder described later is taken into consideration.

As described above, the motion amount detector 13 to which the input image signal S is given is also one frame before from the frame memory 19 (more accurately, the influence of a plurality of past frames is included, but the information about one frame before is mainly used). Is reflected) is applied to the local reproduction image signal S2. The motion amount detector 13 is
The motion compensation circuit detects the motion amount from the locally reproduced image signal S2 one frame before the image signal S of the current frame and calculates the motion amount Mv.
20 and the side information multiplex circuit 17. The motion compensation circuit 20 also includes one frame before the average value component is removed from the frame memory 21 (to be precise, the influence of a plurality of past frames is included, but the information of one frame before is mainly reflected. Local reproduction average value-removed image signal S3 is given, and the motion compensation circuit 20 performs motion compensation on this image signal S in accordance with the detected motion amount Mv to obtain the average value-removed image signal of the current frame. The predicted image signal Sp is obtained and given to the subtractor 18.

Subtractor 18 from the average value removal image signal S1 of the current frame,
The predicted image signal Sp is subtracted, and the obtained difference signal e is given to the orthogonal transformer 22. The orthogonal transformer 22 is, for example, a discrete cosine transformer, and is a block obtained by dividing the differential signal e into a plurality of frames (a region smaller than the subframe described above).
The orthogonal transformation is performed in units, and the orthogonal transformation coefficient Q is given to the quantizer 23. The quantizer 23 quantizes the orthogonal transformer Q in block units, and supplies the obtained quantized code q to the variable length coder 24, the block validity / invalidity decision circuit 25, and the inverse quantizer 26.

The inverse quantizer 26 performs an inverse quantization process which is an inverse process of the quantizer 23 to obtain a locally reconstructed orthogonal transform coefficient Qk and supplies it to the inverse orthogonal transformer 27. The inverse orthogonal transformer 27 executes the inverse orthogonal transform process corresponding to the orthogonal transformer 22, obtains the local reproduction difference signal ek, and gives it to the adder 28. The predictor image signal Sp is also supplied from the motion compensation circuit 20 to the adder 28. The adder 28 adds the local reproduction difference signal ek to the predicted image signal Sp to obtain the local reproduction average value removed image signal SIk corresponding to the average value removed image signal S1 to the leak coefficient multiplier 29 and the adder 30. give.

The leak coefficient multiplier 29 uses the local reproduction average value removed image signal SI.
The k is multiplied by the leak coefficient β (0 <β <1), and the image signal after the multiplication is given to the frame memory 21 to be stored therein and used for forming the next predicted image signal Sp. Here, the reason for multiplying the leak coefficient β is that it corresponds to the fact that the image packet decoder described later is provided with a leak coefficient multiplier so as to reduce the effect of packet discard by the leak effect. is there.

Adder 30 to which the local reproduction average value removed image signal SIk is given
Is also given the local reproduction average value component Smk. Thus, the adder 30 obtains the locally reproduced image signal Sk corresponding to the input image signal S, the locally reproduced image signal Sk is given to the frame memory 19 and stored therein, and is used for the next motion amount detection.

The variable-length encoder 24 to which the quantized code q from the quantizer 23 is given performs, for example, run-length coding and / or Huffman coding, and the variable-length code is processed by the packet assembling unit.
Give to 31.

Further, the block valid / invalid determination circuit 25 to which the quantized code q from the quantizer 23 is given, the invalid block that may use the quantized code of the past frame based on the given quantized code q. Or it is a valid block for which the quantization code of the past frame cannot be used, and the block valid / invalid information BY is given to the side information multiplexing circuit 17.

The side information multiplexing circuit 17 includes an average value code Smp given from the quantizer 15 and a motion amount Mv given from the motion amount detector 13.
And the block valid / invalid information BY given from the block valid / invalid judgment circuit 25, and the multiplexed information I is given to the variable length encoder 32. The variable length encoder 32 performs, for example, run length coding and / or Huffman coding, and supplies the variable length code to the packet assembling unit 31.

The packet assembling unit 31 separately packetizes the code concerning the orthogonal transform coefficient given from the variable length encoder 24 and the code concerning the side information + average value component given from the variable length encoder 32. Here, the side information is information required when the image packet decoder described later decodes the code related to the orthogonal transform coefficient, and in the case of this embodiment, the motion amount Mv, the block valid / invalid information BY And are applicable.

The reason why the packets are separately packetized in this way is that the degree to which the reproduced image quality is affected differs depending on the codes. That is, the side information + average value component has a higher influence on the reproduced image quality.

In this way, various information as shown in FIG. 2 is inserted in the header of the packet in which the side information + average value component code or orthogonal transform coefficient code is inserted in the information field.

In FIG. 2, the first and second header areas H1 and H2
In the same manner as in the conventional art, the destination information and the source information are inserted in.

Priority information for packet discard is inserted in the third header area H3. Here, the packet assembling unit 31
Packets in which the code inserted in the information field is an orthogonal transform coefficient code have a high priority, that is, are easily discarded when network congestion occurs, and the code inserted in the information field is side information + average value component code Has a low priority, that is, it is difficult to be discarded when network congestion occurs. In addition,
Since the packet assembling unit 31 has a different input terminal for the code,
Such packet discard priority information can be easily inserted.

In the fourth header area H4, information indicating whether the information inserted in the information field is side information or an average value component is inserted when the packet is a packet having side information + average value component code. To be done.

Position information indicating which position on the image the code inserted in the information field relates to is inserted in the fifth header area H5. In the sixth header area H6,
The division presence / absence information indicating whether the code inserted in the information field is the one for a plurality of packets is inserted, and the sequence number in the case of division is inserted in the seventh header area H7.

In the eighth header area H8, the image signal is the luminance signal, the first
And image type information indicating which one of the second color difference signals is involved. Here, FIG. 1 shows the processing system for the image signal of either the luminance signal or the first or second color difference signal.
Is common to each processing system, and as described above, image type information indicating whether the code is a code related to the luminance signal or the first or second color difference signal is inserted.

The packet thus assembled by the packet assembling unit 31 is sent to the transmission line (network) 33.

Image Packet Decoder Next, the image packet decoder will be described with reference to FIG.

FIG. 3 is a block diagram showing a configuration of an image packet decoder of an embodiment corresponding to the above-mentioned image packet encoder.

In FIG. 3, the packet via the packet transmission line 40 is given to the packet disassembling unit 41. The packet disassembling unit 41
The packet is decomposed based on the header content of the packet, and the variable length coded orthogonal transform coefficient code is given to the variable length decoder 42, and the variable length coded side information + average value component code is changed. It is given to the long decoder 43.

The variable length decoder 43 performs the reverse processing of the above-mentioned variable length encoder 32 to obtain side information + average value component code Is and supplies it to the side information demultiplexing circuit 44. The side information demultiplexing circuit 44 separates the side information code Is, obtains the reproduction average value code Smps, the reproduction motion amount Mvs, and the reproduction block valid / invalid information BYs, and gives the average value code Smps to the dequantizer 45.
The motion amount Mvs is given to the motion compensation circuit 46, and the block valid / invalid information BYs is given to the block valid / invalid judgment circuit 47.

On the other hand, the variable length decoder 42 includes the variable length encoder 24 described above.
Then, the inverse process of the process is executed to obtain the reproduced orthogonal transform coefficient code qs, which is given to the inverse quantizer 48. The inverse quantizer 48 performs the inverse process of the quantizer 23 described above, and reproduces the code qs as a reproduction orthogonal transform coefficient.
It is converted into Qs and given to the inverse orthogonal transformer 49. Inverse orthogonal transformer 49
Is the orthogonal transformer 22 described above with respect to the reproduction orthogonal transform coefficient Qs.
Then, the reproduction difference signal es is obtained and given to the adder 50.

Here, the inverse quantizer 48 and the inverse orthogonal transformer 49 are given the result determined by the block valid / invalid decision circuit 47 based on the block valid / invalid information BYs, and the inverse quantizer 48 and the inverse orthogonal transform are given. The device 49 processes the valid blocks and does not output the invalid blocks.

Further, even when the packet is discarded and the orthogonal transform coefficient code qs is not input, the inverse quantizer 48 and the inverse orthogonal transformer 49 are in a non-output state.

When the transmission line that determines the discarded packet according to the priority information (H3) inserted in the header content of the packet at the time of network congestion is used as the packet transmission line 40 (33), as described above. Since the discard priority of the packet related to the orthogonal transform coefficient code is increased, the packet is discarded. In addition, even on a transmission line that does not perform priority processing for discarding packets, the number of packets for orthogonal transform coefficient coding is larger than that for side information + average value component coding (actually, about 20 times), and almost all discarded packets are discarded. In the case of, the packet is related to the orthogonal transform coefficient code.

In addition to the reproduction difference signal es, the adder 50 is also supplied with the predicted image signal Sps from which the average value component has been removed. This predicted image signal Sps stores a past average-value-removed image signal (mainly having a large influence of the average-value-removed image signal of one frame before) S3s obtained by integrating the reproduced average-value-removed image signals of the past several frames. Frame memory 5 which will be described later
The average value-removed image signal S3s read from 1 is formed by moving the above-described motion amount Mvs by passing through the motion compensation circuit 46. Thus, the reproduction average value-removed image signal SIs is obtained by the adder 50, and this is provided to the adder 52 and the leak coefficient multiplier 53.

The inverse quantizer 45 performs the inverse process of the quantizer 15 described above on the encoded reproduction average value component Smps given from the side information demultiplexing circuit 44 to obtain the reproduction average value component Sms and adder 52 Give to. The adder 52 outputs the average value removed image signal SI
The reproduction average value Sms is added to s to obtain the reproduction image signal Ss including the average value component and output through the output terminal 54.

Further, the leak coefficient multiplier 53 to which the reproduction average value removed image signal SIs is given, multiplies this image signal SIs by the same leak coefficient β as the leak coefficient in the leak coefficient multiplier 29 of the image encoder described above. The multiplied output is given to the frame memory 51 to be stored and used for forming the next predicted image signal Sps.

Here, when the block is an invalid block, the inverse orthogonal transformer 49
Since there is no output from the adder 52, the motion-compensated past predicted image signal Sps is directly output from the adder 52.
Is added to the reproduction average value component in the adder 52 and output. The invalid block is a block in which the image quality does not deteriorate even when the past reproduction average value-removed image signal is used.
Ss is output.

On the other hand, even when the reproduction difference signal es from the inverse orthogonal transformer 49 is not output due to packet discard, the motion compensated past reproduction average value removed image signal is directly output from the adder 50 and given to the adder 52. And is added to the reproduction average value component in the adder 52 to be output. In this case,
Originally, this is a block that outputs a reproduced image signal obtained by adding the reproduced differential signal and the reproduced predicted image signal, and therefore the reproduced image quality is degraded. However, since the average value component Sms that most affects the image quality is secured by the addition of the adder 52, the decrease is slight.

Further, due to such packet discarding, the reproduction average value-removed image signal for the past frame stored in the frame memory 51 differs from the original value, but the leak coefficient multiplier 53
Since the integration effect is suppressed by this, the effect becomes smaller as the frame progresses, and the adverse effect of packet discard does not extend for a long time.

Effects of Embodiments When the image packet encoder shown in FIG. 1 and the image packet decoder shown in FIG. 3 are used, an average value component is obtained from the input image signal, and this average value component is obtained from the input image signal. The removed average value-removed image signal is subjected to encoding processing that combines motion-compensated inter-frame prediction, orthogonal transformation, and variable-length encoding to obtain an average value component + side information,
Since the encoded average-value-removed image signal is assembled and sent in a separate packet, the image packet decoder decodes the average-value component information that most affects the image quality even if packet loss occurs. As a result, it is possible to suppress deterioration of image quality due to packet discard of reproduced image quality as compared with the related art.

It is conceivable that, in response to packet discard, the image packet decoder performs an interpolation process using the information of the surrounding blocks, but such an interpolation process configuration uses the information of the surrounding blocks, so the average value It is often more complex than the calculated transmission configuration. It is also possible to secure the image quality secured by interpolation rather than the image quality secured by average value transmission for packet discard, but in this case, the unit block area of orthogonal transformation is set to a very small area. However, the number of packets to be transmitted increases and the transmission efficiency decreases. Therefore, depending on the intended image packet transmission system, it is preferable to respond to packet discard by processing by transmitting the average value component.

Further, according to the above-described embodiment, packet discarding causes
The average-value-removed image signal used to form the predicted image signal is different from the original value, but since a leak coefficient multiplier is provided to attenuate the adverse effect, the adverse effect is prevented from remaining until later. be able to.

Other Embodiments (1) In the above embodiments, the average value removed image signal is coded by motion compensation inter-frame prediction, orthogonal transformation, and variable length coding. However, other coding processing is performed. You may apply. For example, inter-field prediction or inter-line prediction may be applied instead of inter-frame prediction, and variable length coding may be omitted.

(2) In the above embodiment, the leak coefficient multiplier 29,
Although 53 is provided on the input stage side of the frame memory, it may be provided on the output stage side of the frame memory.
It may be provided on the output stage side of the motion compensation circuit.

(3) Although the side information and the average value component are multiplexed and packetized in the above embodiment, the side information and the average value component may be separately packetized.
The point is that the average value component may be a packet different from the orthogonal transform coefficient code.

(4) In the above embodiment, the packet discard priority information is inserted in the header of the packet, but in the case where it is known in advance that the transmission path cannot execute the priority discard processing. Need not perform the insertion of the priority information. Also in this case, packetization needs to be performed separately for the average value component and the orthogonal transform coefficient code. Even in this case, since the number of packets for the orthogonal transform coefficient code is larger than the number of packets for the average value component, in most cases, the packet for the orthogonal transform coefficient code is discarded and the average value component is secured in the reproduced image signal. Therefore, the quality of the reproduced image can be improved.

[Effects of the Invention] As described above, according to the present invention, the average value component of the input image signal is detected for each area obtained by dividing one image into a plurality of areas, and the average value component detected from the input image signal is removed. Since the average-value-removed image signal is obtained and the average-value component and the average-value-removed image signal are separately encoded, packetized, and transmitted,
Even when packet discard occurs, a reproduced image in which the average value component is secured is very often obtained, and the reproduced image quality can be improved compared to the conventional case without using the interpolation processing. A coder and a decoder can be obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of an image packet encoder according to the present invention, FIG. 2 is an explanatory view showing a header content of a packet, and FIG. 3 is an embodiment of an image packet decoder according to the present invention. It is a block diagram showing. 12 …… Average value detector, 13 …… Motion amount detector, 14, 18 ……
Subtractor, 15, 23 ... Quantizer, 16, 26, 48 ... Inverse quantizer, 17 ... Side information multiplexing circuit, 19, 21, 51 ... Frame memory, 20, 46 ... Motion compensation circuit , 22 …… Orthogonal transformer, 24, 32 …… Variable length encoder, 25, 47 …… Block valid / invalid judgment circuit, 27, 49 …… Inverse orthogonal transformer, 28, 30, 5
0, 52 ... Adder, 29, 53 ... Leakage coefficient multiplier, 31 ...
... Packet assembly section, 41 ... Packet disassembly section, 42,43 ...
... a variable length decoder.

Claims (2)

[Claims] 1. An average value component detecting means for detecting an average value component of an input image signal, and an average value removed image obtained by removing the average value component detected from the input image signal, for each area obtained by dividing one image into a plurality of areas. A subtraction means for obtaining a signal, an average value encoding means for encoding the detected average value component, and an average value for encoding the obtained average value removed image signal by combining orthogonal transformation and interframe encoding. Removed image encoding means, first leak coefficient multiplication means for multiplying the locally reproduced average removed image signal in the inter-frame encoding by a leak coefficient, encoded average value component, and encoded average value An image packet encoder comprising: a packet assembling unit that assembles and outputs the removed image signal into separate packets. 2. A packet output from the image packet encoder according to claim 1, and a coded average value component and a coded average value-removed image signal from the received packet. A packet separating means for separating the encoded mean value-removed image signal, a mean value-removed image decoding means for decoding the encoded mean value-removed image signal by a combination of inverse orthogonal transform and interframe decoding, and an encoded mean value Average value decoding means for decoding the value component, second leakage coefficient multiplication means for multiplying the reproduction average value removed image signal in the interframe decoding by a leak coefficient, decoded average value removed image signal and decoding An image packet decoder comprising: an addition unit that combines the generated average value component to obtain a reproduced image signal.