JP2001251621A - Moving compression coding transmitter, receiver and transmitter-receiver - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving picture compression coding transmitter, a receiver and a transmitter-receiver that can attain stable moving picture communication even among terminals different in throughput. SOLUTION: A maximum decoding processing frame rate calculation section 5 calculates a maximum frame rate FRmax capable of real time decode processing from a frame coding processing time TE obtained from an image coding section 1 and a frame decoding processing time TD obtained from an image decoding section 11. A comparator section 3 compares the maximum frame rate FRmax with a maximum processing frame rate FRmax' at a receiver side received by a demultiplexer section 13 and inputs the frame rate which is smaller one to the image coding section 1. The image coding section 1 uses this received frame rate for a maximum frame rate to skip frames for coding control. As a result, even when the throughputs of both terminals are different from each other, the communication between both terminals can stably be conducted without exceeding the throughput.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture compression-encoding transmission apparatus, a reception apparatus, and a transmission / reception apparatus.
The present invention relates to a compression encoding device that compresses and encodes a moving image and transmits a compressed bit stream in real time, a receiving device that decodes and displays a received compressed bit stream in real time, and a transmission and reception device that has both of these functions.

[0002]

2. Description of the Related Art In general, a conventional moving picture compression-encoding transmission apparatus on the transmission side compresses a video signal to be transmitted at a predetermined bit rate and transmits a compressed bit stream. ISO MPEG-1, 2, 4 and ITU-T H.264. 261, H .; In motion compensation coding represented by H.263, motion vector detection processing that captures the motion of a local region in a screen, motion compensation that applies a motion vector to a previous frame to predict the current frame, and motion compensation that are performed at the time of encoding. DCT, DC applied to the difference between the image and the current frame
Quantization applied to the T signal, variable length coding for outputting the quantized signal value, motion vector and other control information, buffer for temporarily storing the variable length code, rate control for keeping the bit rate constant And so on.

At this time, the quality of the image is controlled by changing the precision of the quantization based on the amount of information generated during compression and the properties of the input image.
Alternatively, by changing the number of frame skips (frame rate), the code amount is controlled so as to be within a predetermined bit rate. When reducing the amount of generated codes, the quantization accuracy is reduced at the expense of image quality, or the number of frame skips is increased (the frame rate is reduced) at the expense of motion. Conversely, when increasing the generated code amount, the image quality is improved by increasing the quantization accuracy, or the number of frame skips is reduced (the frame rate is increased).

[0004] On the receiving side, on the other hand, the transmitted bit stream is sequentially decoded, an image frame is formed by performing the reverse process of encoding, and the image frame is displayed.

[0005]

By the way, in the conventional apparatus, a transmitting apparatus and a receiving apparatus having greatly different processing capacities,
Alternatively, when the transmitting and receiving apparatuses communicate with each other, the following problem occurs.

[0006] In the case of one-way communication, when the processing capacity of the transmitting side is much higher than that of the receiving side, the transmitting side can perform encoding processing of many frames. For this reason,
When the transmitting side transmits a coded bit stream having a high frame rate, the receiving side has a low processing capability and cannot perform decoding processing at the frame rate of the received coded bit stream. There is a risk that the reception function will fail.

In the case of software that is executed on an unspecified number of computers, the processing capability of the computers may be lower than the set encoding frame rate. Is the computer CPU processing 10
0% is occupied, so that not only real-time processing cannot be performed, but also other processing for maintaining the system cannot be performed at all, which causes a problem that the operation of the apparatus becomes unstable.

On the other hand, in the case of two-way communication, it is necessary that both perform the encoding process and the decoding process at the same time. Become.

When the processing amount is too large, in order to reduce the processing amount so as not to break down the real-time processing,
In order to reduce the number of processed frames (frame rate), it is conceivable to reduce the number of frames from the received bit stream on the slower side in the conventional apparatus because, in general, inter-frame prediction is performed. Impossible. On the other hand, thinning out transmission frames is effective,
For example, in two-way communication, if the slower processing side unilaterally thins out transmission frames, a large difference in the frame rate between the mutual terminals may occur, and there is a problem that the performance balance is largely lacked.

An object of the present invention is to provide a moving picture compression-encoding transmitting apparatus, a receiving apparatus and a transmitting / receiving apparatus which eliminate the problems of the prior art and enable stable moving picture communication between terminals having different processing capabilities. Is to do.

[0011]

In order to achieve the above object, the present invention provides a means for calculating a maximum frame rate at which encoding can be performed from the encoding processing time of a moving image frame, and a decoding process on a receiving side. Means for receiving the maximum possible frame rate, means for comparing the maximum frame rate that can be encoded and the maximum frame rate that can be decoded on the receiving side, and obtaining the smaller one, and the obtained frame rate And a means for performing coding control with a maximum frame rate.

According to this feature, the maximum frame rate that can be encoded is calculated, the calculated value is compared with the maximum frame rate that can be decoded notified from the receiving side, and the smaller frame rate is set to the maximum frame rate. By performing coding control as the frame rate, stable communication can be performed without exceeding the processing capability even when the processing capability differs between transmission and reception.

According to the present invention, there is provided means for calculating a maximum frame rate at which decoding is possible from a decoding processing time of a frame of a moving image, and means for notifying the calculated maximum frame rate at which decoding is possible to a transmitting side. The second feature is that
There is a feature.

According to this feature, the moving picture compression encoding receiving apparatus can notify its transmitting apparatus of its frame decoding capability and receive a bit stream having a frame rate corresponding to its own frame decoding capability. There is no danger of running out of its own processing power in decoding the received coded bit stream.

The present invention also provides means for calculating the maximum frame rate at which decoding can be performed from the encoding processing time and the decoding processing time of a moving image frame, Means for demultiplexing and receiving from the stream, means for comparing the calculated maximum frame rate capable of decoding and the received maximum frame rate capable of decoding of the communication partner, and obtaining the smaller one, The third feature is that a means for performing coding control with the set frame rate as the maximum frame rate is provided.

According to this feature, the maximum decodable frame rate is calculated from the time required for the encoding process and the time required for the decoding process, and the calculated value and the maximum frame rate notified from the communication partner are calculated. Can be compared with the decodable frame rate, and encoding control can be performed with the smaller frame rate as the maximum frame rate. For this reason, even when the processing capability differs between the two terminals, stable communication can be performed without exceeding the processing capability.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of an embodiment of the moving picture compression encoding transmission apparatus of the present invention.

The compression-encoding transmission apparatus according to the present embodiment includes a known image encoding unit 1, a maximum encoding processing frame rate calculation unit 2, a comparison unit 3, and a multiplexing unit 4.

The image encoding unit 1 uses the ISO M
PEG-4 and H.264 of ITU-T. 261, H .; For example, a motion compensation coding device represented by H.263 can be used.
The configuration and operation of one specific example will be described with reference to FIG.

An input image signal is supplied to a frame skip unit 10.
In step 1, the frames are thinned out according to the control signal obtained from the buffer memory 113, and are stored in the current frame memory 102. The motion detection unit 103 includes the current frame memory 102
From the image signal read from the frame memory 104 and the image signal of the previous frame read from the frame memory 104, and performs motion detection of the motion vector mv.
5 and output to the variable length coding unit 106. Motion compensation unit 10
5 performs motion compensation on the image signal of the previous frame using the motion vector mv, and outputs the motion-compensated image signal to the subtractor 107 and the adder 108.

The coding mode determination unit 109 determines the coding mode from the current frame image signal and the difference error signal output from the subtractor 107. If the coding mode is the intra mode, the switch unit 110 is set to the Intra side. On the other hand, if it is in the inter mode, connect to the Inter side. The switching means 114 is also switched in synchronization with the switching means 110. The DCT unit 111 converts an image signal or a difference error signal input in order to obtain high coding efficiency into a DCT signal.
Then, the quantization unit 112 quantizes the DCT-processed signal. The quantized coefficients output from the quantization unit 112 and the motion vector are converted into a variable length code such as a Huffman code by the variable length coding unit 106, temporarily stored in the buffer memory 113, and then at a predetermined transmission rate. Output as a bit stream. The bit stream is multiplexed by the multiplexing unit 4 in FIG. 1 with the maximum frame rate FRmax that can be encoded by the maximum encoding processing frame rate calculation unit 2 and output to a network or the like.

In addition, the inverse quantization unit 115 includes a quantization unit 112
Are inversely quantized, and the DCT coefficient obtained by the inverse quantization is restored to an image signal by the inverse DCT unit 116. The restored image signal is added to the image signal motion-compensated by the adder 108 as it is when the encoding mode is Intra, and is accumulated in the frame memory 104 when the encoding mode is Inter. The frame encoding time measuring unit 117 obtains a measurement start trigger signal a from the current frame memory 102 and a measurement end trigger signal b from the frame memory 104, measures the frame encoding time TE, and performs the maximum encoding process shown in FIG. Output to the frame rate calculation unit 2.

Next, the operation of the main part of this embodiment will be described with reference to FIG.

Maximum encoding processing frame rate calculation unit 2
Obtains the frame encoding time TE from the image encoding unit 1 and calculates the maximum frame rate F at which real-time encoding can be performed.
Calculate Rmax. The maximum frame rate FRmax is
It can be obtained from any of the following equations (1) to (3). FRmax = b / TE (1) FRmax = b / ave TE (2) FRmax = b / max TE (3)

Here, b is a margin coefficient (= 0.0 to
1.0), and ave TE and max TE indicate the average and maximum time of the frame encoding time TE, respectively.

The maximum frame rate FRmax obtained by the maximum encoding processing frame rate calculation unit 2 is calculated by a comparison unit 3.
It is compared with the maximum decoding processing frame rate FRmax 'sent from the receiving apparatus, and the smaller frame rate c
Is sent to the frame skip unit 101 of the image encoding unit 1. The frame skip unit 101 skips frames of the input image signal according to the frame rate c.
Also, the maximum frame rate FRmax is
And multiplexed with the bit stream.

As is clear from the above description, according to the present embodiment, the maximum encodeable frame rate is calculated from the time required for the encoding process, and the calculated value and the maximum frame rate notified from the receiving side are calculated. Comparing with the decodable frame rate and performing coding control with the smaller frame rate as the maximum frame rate, stable communication can be performed without exceeding the processing capacity even when the processing capacity differs between transmission and reception. Become like Therefore, for example, even when the processing capacity of the transmission side is significantly higher than that of the reception side, the encoding processing of the transmission side is suppressed by the maximum decoding processing capacity of the reception side. There is no danger of using up its own processing power for decoding the received coded bitstream and accumulating display delays or causing other reception functions to fail.

Next, a second embodiment of the present invention will be described with reference to FIG. This embodiment shows a moving image compression-encoding receiving apparatus for one-way communication, and an image decoding unit 1
1 and a maximum decoding processing frame rate calculation unit 12.

FIG. 4 is a block diagram showing an example of the image decoding unit 11. The variable length decoding unit 201 performs variable length decoding on the input bit stream. The image information p obtained by the variable length decoding is sent to the inverse quantization unit 202, the control information q is sent to the switch unit 204, and the motion vector r is sent to the motion compensation unit 206. The switch means 204 is connected to the Intra side when the control signal q indicates Intra, and is connected to the Inter side when the control signal q indicates Inter. The inverse quantization unit 202 outputs DCT coefficients by inverse quantization, and the DCT coefficients are decoded into image signals by the inverse DCT unit.

The decoded image signal is output as a screen output and stored in the frame memory 207. The image signal read from the frame memory 207 is motion-compensated by the motion compensator 206 using the motion vector r and sent to the adder 205. The adder 205 provides the motion-compensated image signal and the inverse DC
A decoded image signal is generated by adding the difference error signal output from the T unit and the decoded signal. The frame decoding time measuring section 208 starts measuring the frame decoding time by the measurement start trigger signal s from the variable length decoding section 201, ends the measurement by the measurement end trigger signal t, and calculates the frame decoding time TD. Output.

Next, the operation of the main part of this embodiment will be described with reference to FIG.

Maximum decoding processing frame rate calculator 12
Is the frame decoding time measuring unit 20 of the image decoding unit 11
8, the frame decoding time TD is obtained from the following (4) to
The maximum decoding processing frame rate FRmax 'is obtained from one of the equations (6). FRmax '= b / TD (1) FRmax' = b / ave TD (2) FRmax '= b / max TD (3)

Here, b is a margin coefficient (= 0.0 to
1.0), and ave TD and maxTD indicate the average and maximum time of the frame decoding time TD, respectively. The maximum decoding processing frame rate FRmax ′ obtained by the maximum decoding processing frame rate calculation unit 12 is sent to the compression-encoding transmission device described in the first embodiment.

As described above, according to the present embodiment, the moving picture compression-encoding receiving apparatus informs the transmitting apparatus of its own frame decoding capability, and sets the frame rate according to its own frame decoding capability. Since the bit stream can be received, there is no danger of using up its own processing power for decoding the received coded bit stream.

Next, a third embodiment of the present invention will be described with reference to FIG. This embodiment shows a moving image compression encoding transmission / reception apparatus for bidirectional communication, and includes an image encoding unit 1, a maximum decoding processing frame rate calculating unit 5, a comparing unit 3, a multiplexing unit 4, and an image decoding unit. It comprises a unit 11 and a demultiplexing unit 13. 1 and 3 in FIG. 5 indicate the same or equivalent components.

The frame decoding processing time TE obtained from the image coding unit 1 and the frame decoding processing time TD obtained from the image decoding unit 11 are input to the maximum decoding processing frame rate calculation unit 5. The maximum decoding frame rate calculator 5 calculates the maximum frame rate FRmax at which real-time decoding can be performed from these values. Assuming that a is a coding priority coefficient (0.0 to 1.0, usually 0.5) and b is a margin coefficient (0.0 to 1.0), the maximum decoding processing frame rate FRmax is as follows: 7) It is calculated by any one of the expressions (9) to (9). FRmax = b / ｛a · TE + (1-a) · TD｝ (7) FRmax = b / ｛a · aveTE + (1-a) · aveTD｝ (8) FRmax = b / ｛a・ Max TE + (1-a) ・ max TD｝ (9)

It is preferable that the encoding priority coefficient a in the equations (7) to (9) is normally set to 0.5, and the maximum number of the encoding processing frame rate and the maximum number of the decoding processing frame rate are set to the same number. If a is larger than 0.5, the number of frames to be coded is prioritized, and if smaller than 0.5, the number of frames to be decoded is prioritized. On the other hand, the margin coefficient b is set to a maximum of 1.0, and when there is a need for remaining processing power in addition to the encoding / decoding processing, the value is set to a value smaller than 1 according to the required processing power.

The maximum decoding processing frame rate FRmax obtained here is multiplexed and transmitted to the transmission line by the multiplexing unit 4. On the other hand, FRmax ′ received by the demultiplexing unit 13 is
The value is compared with the above-mentioned FRmax in the comparison unit 3, and the smaller value is input to the image encoding unit 1. The image encoding unit 1 performs frame skipping by using this input value as the maximum frame rate, and performs encoding control.

Next, in addition to the above formulas (7) to (9),
Any of equations (10) to (12) can also be used. That is, the total number of clocks per second of the system is ClkCPU
The number of clocks required for encoding processing of one frame is ClkE
Assuming that the number of clocks required for the decoding process is ClkD and the desired encoding frame rate is FRE, the maximum frame rate FRmax can be calculated from any of the following equations (10) to (12). FRmax = (b · ClkCPU−FRE · ClkE) / ClkD (10) FRmax = (b · ClkCPU−FRE · aveClkE) / aveClkD (11) FRmax = (b · ClkCPU−FRE · maxClkE) / max ClkD (12) where aveClkE and aveClkD represent the average of ClkE and ClkD, respectively, and max Clk
E and max ClkD are ClkE and CkD, respectively.
Indicates the maximum value of lkD.

In equations (10) to (12), the maximum decoding processing frame rate FRmax is normally calculated as FRmax = FRE (corresponding to a = 0.5). In the case of one-way communication, the one-way receiving side sets ClkE = 0, calculates the maximum decoding processing frame rate FRmax ', and sets the one-way transmitting side to FRmax = 0, and solves FRE. Let the desired encoding frame rate. The margin coefficient b is set to a maximum of 1.0, and when it is desired to leave processing margin in addition to the encoding / decoding processing, the value of b is set according to the processing margin to be retained.

According to this embodiment, the maximum decodable frame rate is calculated from the time required for the encoding process and the time required for the decoding process, and the calculated value is notified from the communication partner. Compares with the maximum decodable frame rate and performs encoding control with the smaller frame rate as the maximum frame rate, so that even if the processing capacity differs between both terminals, it is stable without exceeding the processing capacity Communication can be performed.

[0042]

As is apparent from the above description, according to the present invention, the moving picture compression-encoding / transmission apparatus has its own maximum frame rate that can be processed and the maximum decoding rate notified from the receiving side. Compared with the frame rate, the coding control is performed with the smaller frame rate as the maximum frame rate, so even if the processing capacity differs between transmission and reception, stable communication can be performed without exceeding the processing capacity. Will be able to do it.

Further, according to the present invention, the moving picture compression encoding receiving apparatus informs the transmitting side apparatus of its own frame decoding capability and receives a bit stream of a frame rate corresponding to its own frame decoding capability. Since it is possible, there is no possibility that its own processing power will be used up in the decoding process of the received encoded bit stream.

Further, according to the present invention, the moving image compression / encoding / transmission / reception apparatus calculates the maximum decodable frame rate from the time required for the encoding process and the time required for the decoding process, and By comparing the value with the maximum decodable frame rate notified from the communication partner, encoding control can be performed with the smaller frame rate as the maximum frame rate. Therefore, even when the processing capability differs between the two terminals, stable communication can be performed between the two terminals without exceeding the processing capability.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a first exemplary embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a specific example of an image encoding unit in FIG.

FIG. 3 is a block diagram showing a configuration of a second embodiment of the present invention.

FIG. 4 is a block diagram illustrating a configuration of a specific example of an image decoding unit in FIG. 3;

FIG. 5 is a block diagram showing a configuration of a third embodiment of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Image encoding part, 2 ... Maximum encoding processing frame rate calculation part, 3 ... Comparison part, 4 ... Multiplexing part, 5 ... Maximum decoding processing frame rate calculation part, 11 ... Image decoding part, 12 ...
Maximum decoding processing frame rate calculation unit, 13: demultiplexing unit.

Continued on the front page F-term (reference) 5C059 KK32 LB07 MA00 MA01 MA23 MC11 ME02 NN01 PP04 RC12 TA06 TB20 TC37 TC45 TD02 TD11 UA02 UA05 UA09 5J064 AA01 BA09 BA16 BB05 BC01 BD02

Claims (4)

[Claims] 1. A means for calculating a maximum frame rate at which encoding can be performed from an encoding processing time of a moving image frame; a means for receiving a maximum frame rate at which decoding can be performed on a receiving side; Means for comparing the rate with the maximum frame rate that can be decoded on the receiving side to obtain the smaller one, and means for performing coding control using the obtained frame rate as the maximum frame rate. A moving picture compression encoding transmission apparatus characterized by the above-mentioned. 2. A system comprising: means for calculating a maximum frame rate at which decoding can be performed from a decoding processing time of a frame of a moving image; and means for notifying the calculated maximum frame rate at which decoding can be performed to a transmission side. A video compression-encoding receiving device, characterized in that: 3. A means for calculating a maximum frame rate at which decoding can be performed from a coding processing time and a decoding processing time of a moving image frame, and demultiplexing the maximum frame rate at which communication can be performed from a received bit stream. Means for receiving, means for comparing the calculated maximum frame rate that can be processed for decoding with the maximum frame rate that can be processed for decoding on the communication partner side that has been received, and obtains the smaller one, Means for performing coding control at the maximum frame rate. 4. The video compression encoding transmission / reception device according to claim 3, further comprising: a unit that multiplexes the calculated maximum frame rate that can be decoded and processed into a transmission bit stream and notifies a communication partner. A video compression encoding transmission / reception device characterized by the above-mentioned.