JPH06197332A - Picture coder and picture transmitter - Google Patents

Abstract

PURPOSE:To reduce the detection time of a motion vector for motion compensation inter-frame prediction coding. CONSTITUTION:A control circuit 60 estimates a motion, vector of an object on an image pickup screen based on focal length information and object distance information from a TV camera 10 and moving information of a moving base 12. The estimated value is fed from the control circuit 60 to a motion compensation inter-frame prediction circuit 34. The circuit 34 detects a motion vector by using the estimate value from the control circuit 60 as an initial value and outputs a preceding frame through lateral movement so as to cancel the detected motion vector.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image coding device and an image transmitting device, and more specifically, to an image coding device for compressing a motion compensated image of a captured image of an image pickup means, and an adaptively adapted captured image. In addition, the present invention relates to an image transmitting apparatus that transmits by inter-frame coding or intra-frame coding.

[0002]

2. Description of the Related Art A videophone system and a videoconference system are known in which remote places are connected by a dedicated or public digital line to transmit images and voices for a conversation or a conference. In such a system, since the available communication line capacity is limited, moving images are highly efficiently compressed and encoded for transmission. There are two types of image information compression methods: intraframe compression and interframe compression. In intra-frame compression, the amount of information is compressed by linear transformation such as discrete cosine transformation (DCT) by utilizing the correlation between neighboring pixels in one frame. In inter-frame compression, the fact that there is a strong correlation between pixels at the same position on the screen in successive frames is used, and the information is obtained by substituting the pixels of past frames that have already been encoded or taking the difference. Compress the amount.

Further, a motion-compensated interframe predictive coding system is used for interframe compression. This is to perform predictive coding in consideration of the motion from the neighboring frame (for example, the previous frame) that is the predicted frame, and search for the most similar block from the neighboring frames for each block of a certain size, The block is used as a prediction block. The positional deviation of the block between the two frames is expressed by a motion vector and detected by a so-called pattern matching method.

Motion vector detection by the pattern matching method will be briefly described with reference to FIG. For the block A (for example, 8 pixels × 8 lines) of the current frame, a predetermined motion compensation range or motion vector detection range (for example, 15 pixels in the horizontal direction, vertical direction) on the previous frame (prediction frame). (15 lines in the direction), a trial vector Vk is set to evaluate the degree of power approximation, and a block that is the most approximate is searched. The evaluation function C (k) for the trial vector Vk is given by the following expression, for example. That is, C (k) = ΣF (ai-bi (k)), where ai is the pixel value of the current block A, bi is the pixel value of the block B (k) moved by the trial vector Vk on the previous frame, and F ( ) Is a non-linear function that approximately expresses the power of ai-bi (k). Σ represents the accumulation of all pixels belonging to the block to be evaluated.

The trial vector Vk that minimizes C (k) is used as the motion vector, and B of the block of the previous frame is used.
(K) is used as the prediction signal of the current frame A, and the prediction error signal and the motion vector based on the inter-frame difference are encoded and transmitted.

[0006]

However, in the conventional example,
A large number of operations for detecting a motion vector must be performed for one block, and if the above evaluation function is calculated for all trial vectors within the motion detection range, large-scale parallel processing is required, and Integrated circuit technology results in a very large scale device.

On the other hand, the amount of calculation can be reduced by narrowing the motion vector detection range or setting the trial vector roughly, but the former cannot follow the fast motion.
In the latter case, there is a problem that the coding efficiency decreases due to the decrease in motion vector detection accuracy.

It is an object of the present invention to provide an image coding apparatus and an image transmitting apparatus that eliminate such inconvenience.

[0009]

An image coding apparatus according to the present invention is an image coding apparatus for compressing an image picked up by an image pickup means on a moving table between motion compensation screens, and optical control information of the image pickup means. And an estimation means for estimating the movement of the subject on the photographing screen by referring to the movement information, and a motion vector detection means for detecting the motion vector between the screens based on the estimated value of the estimation means. Is characterized by.

The image transmitting apparatus according to the present invention is an image transmitting apparatus for adaptively compression-encoding an image picked up by the image pickup means on the movable table by one of the intra-frame compression method and the intra-frame compression method and transmitting the compressed image. The intra-screen compression method includes a motion-compensated inter-screen compression method, and an estimation means for estimating the motion of the subject on the shooting screen is provided by referring to the optical control information and the movement information of the imaging means. It is characterized in that motion information between screens necessary for the compensation screen compression method is detected based on the estimated value of the estimation means.

[0011]

The estimating means estimates in which direction and how much the subject moves on the photographing screen according to the zoom, pan and tilt of the image pickup means. Since the motion vector is detected based on this estimated value, the number of calculations or the calculation time required for detection can be reduced. As a result, the motion vector can be accurately detected in a short time, and thus highly compressed image information can be output or transmitted.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a terminal device according to an embodiment of the present invention. However, only the transmission system is shown.

In FIG. 1, 10 is a TV camera for photographing the participants of the video conference, 12 is a moving base for changing the position and direction of the TV camera 10, and 14 is an A / D for digitizing the output image of the TV camera 10. Converter, 16 is A / D
The output of the converter 14 is converted from raster format to CIF (Commo
n Intermediate Format) and QC
It is a format conversion circuit for converting to an intermediate format for transmission such as IF (Quarter CIF).

Reference numeral 20 denotes a subtracter for subtracting the prediction value in the prediction differential encoding from the output of the format conversion circuit 16,
A switch 22 selects the output of the subtractor 20 (a contact) or the output of the format conversion circuit 16 (b contact). The switch 22 is normally connected to the a contact, and is used when the interframe compression is difficult, such as when the TV camera 10 moves.
Switch to contact. A DCT 24 performs DCT conversion on the output of the switch 22 in block units of 8 × 8 pixels.
A transform circuit 26 is a quantizer circuit that quantizes the transform coefficient of the DCT transform circuit 24.

Reference numeral 28 is an inverse quantization circuit for inversely quantizing the output of the quantization circuit 26, and reference numeral 30 is an inverse D of the output of the inverse quantization circuit 28.
An inverse DCT conversion circuit for performing CT conversion, 32 is a prediction value at the output of the inverse DCT conversion circuit 30 in a normal time, and the TV camera 10
Is added to the output of the format conversion circuit 16 (current frame) and the adder 32.
The motion compensation inter-frame prediction circuit that detects a motion vector from the output (past frame) and calculates a prediction value by motion-compensation inter-frame prediction. It is a motion adaptive filter consisting of a low-pass filter for removal. The output of the motion adaptive filter 36 is used as the prediction value, and the subtracter 20 and
It is applied to the adder 32 via the switch 38. The switch 38 is normally closed and is opened when the TV camera is moved.

With the switch 22 connected to the a contact and the switch 38 closed, the circuits 20, 22, 24 and 2 are connected.
6, 28, 30, 32, 34, and 36 function as a prediction difference encoding circuit that uses the previous frame value as a prediction value.
The output of No. 6 is compression-coded between frames. On the other hand, when the switch 22 is connected to the b contact and the switch 38 is open, the circuits 20, 22, 24, 26, 28, 30, 3 are connected.
2, 34 and 36 function as an intraframe coding circuit by DCT and quantization, and perform intraframe compression coding of the output of the circuit 16.

Reference numeral 40 is a variable length coding circuit for variable length coding the output of the quantization circuit 26 and the motion vector supplied from the motion compensation inter-frame prediction circuit 34, and 42 is the circuit 4
A transmission buffer for rate-adjusting the output of 0 and thinning out frames when necessary, and an error correction coding circuit 44 for adding an error correction code to the output of the transmission buffer 42.

Reference numeral 48 is a microphone for voice input, 50 is an A / D converter for digitizing the output of the microphone 48, and 52 is an A / D converter.
An audio encoding circuit that compresses and encodes the output of the / D converter 50, and a delay circuit 54 that delays the output of the audio encoding circuit 54 for a time corresponding to the processing time of the output video of the TV camera 10.

Reference numeral 56 denotes a multiplexing circuit for multiplexing the output of the error correction coding circuit 44 and the output of the delay circuit 54, and 58 denotes I.
A line connection circuit such as a terminal adapter for connecting to a communication line such as SDN. In this embodiment, the output of the multiplexing circuit 56 is output to the communication line. Reference numeral 60 is a control circuit for controlling the diaphragm of the TV camera 10 and zooming of the photographing lens, and also for controlling the moving base 12 and the switches 22, 38, and 62 is an operating device for inputting various instructions to the control circuit 60.

The TV camera 10 sequentially supplies the focal length information of the taking lens and the subject distance information to the control circuit 60, and the control circuit 60 controls the TV camera 10 (focus adjustment, zoom, tilt). , Panning, etc.) to the motion-compensated inter-frame prediction circuit 34, the details of which will be described later.

First, the operation when the TV camera 10 is stationary or almost stationary will be described. At this time, the control circuit 6
0 connects the switch 22 to the a contact and closes the switch 38. The output video signal of the TV camera 10 is A / D
The digital signal is converted by the converter 14, and the format conversion circuit 16 converts the output of the A / D converter 14 from the raster format to a predetermined intermediate format for transmission. The output of the format conversion circuit 16 is applied to the b contact of the switch 22, the subtractor 20, and the motion compensation interframe prediction circuit 36.

The subtractor 20 subtracts the prediction value (output of the motion adaptive filter 38) to calculate an error signal for predictive differential encoding. The output of the subtractor 20 is applied to the DCT conversion circuit 24 via the switch 22, and the DCT conversion circuit 2
4 performs DCT conversion for each block of 8 × 8 pixels and outputs a transform coefficient, and the quantization circuit 26 quantizes the transform coefficient.

The inverse quantization circuit 28 inversely quantizes the output of the quantization circuit 26, and the inverse DCT conversion circuit 30 inversely quantizes the inverse quantization circuit 2.
The output of 8 is subjected to inverse DCT conversion. The adder 32 adds the predicted value to the output of the inverse DCT conversion circuit 30. The output of the adder 32 is a locally decoded value obtained by performing predictive differential encoding and then decoding the output of the format conversion circuit 16. The motion-compensated inter-frame prediction circuit 34 uses the format conversion circuit 16
From the current value (current frame) and the locally decoded value (past frame) from the adder 32, a motion vector is calculated, and the prediction value of the current frame is calculated by motion compensation inter-frame prediction. The calculated motion vector is applied to the variable length coding circuit 40, and the predicted value is applied to the motion adaptive filter 36. The motion adaptive filter 36 removes a predetermined high frequency component from the predicted value and applies it to the adder 32 via the subtracter 20 and the switch 38.

With the switch 22 connected to the a-contact and the switch 38 closed, the circuits 20 to 38 form an inter-frame predictive differential encoding circuit, whereby the image information is compressed between frames. The image data compressed by the circuits 20 to 38 is transferred from the quantization circuit 26 to the variable length coding circuit 4
Applied to zero.

The variable length coding circuit 40 includes a quantization circuit 26.
And the motion vector from the motion-compensated inter-frame prediction circuit 34 are variable-length coded. The output of the variable length coding circuit 40 is applied to the error correction coding circuit 44 via the transmission buffer 42. The error correction coding circuit 44 generates and adds an error correction code and applies it to the multiplexing circuit 56.

On the other hand, the audio signal from the microphone 48 is A / D.
It is converted into a digital signal by the converter 50 and encoded by the voice encoding circuit 52. The delay circuit 54 delays the output of the audio encoding circuit 52 by a time corresponding to the delay due to the above-described video signal processing, and supplies it to the multiplexing circuit 56.

The multiplexing circuit 56 is an error correction coding circuit 44.
And the data from the delay circuit 54 are multiplexed, and the output is output to the communication line via the line connection circuit 58.

Next, the operation when the TV camera 10 is moved while shooting and the moving speed is not enough to switch to intraframe compression will be described.

A conference participant or an operator operates the operation device 62 to instruct the control circuit 60 to move the TV camera (including zooming and focus adjustment), and the control circuit 60 responds to the instruction. And its moving base 12 are controlled. The TV camera 10 sequentially supplies the focal length information f of the taking lens and the subject distance information m to the control circuit 60. The control circuit 60 uses the information f and m and the moving table 1
From the drive angle of 2, the amount of movement of the subject on the shooting screen can be calculated.

For example, assuming that the number of effective pixels of the image sensor is V × H and the effective photosensitive surface size is X × Y (square millimeter), the movable table 12 is horizontally rotated by φ (degrees). At this time, the movement amount x of the subject image on the photographing screen is x = 2πfφ / 360, and when converted into the number of pixels, n = (X / x) × H. Similarly, the amount of movement in pixel units can be estimated for rotation in the vertical plane.

The control circuit 60 operates in this way in the TV camera 1
The focal length information f of 0, the subject distance information m, and the moving table 12
The estimated value of the image movement amount calculated from the driving amount and the inter-frame prediction circuit 34 is supplied to the motion compensation inter-frame prediction circuit 34. The motion-compensated inter-frame prediction circuit 34 uses the estimated value from the control circuit 34 as an initial value of the motion vector to be detected.
For example, by setting the estimated value as the center of the motion vector detection range, the target motion vector can be detected early. In addition, the motion vector can be accurately detected even if the motion vector detection range is set narrower than the conventional one. As a result, the motion vector can be accurately detected with a smaller number of calculations as a whole.

Although not characteristic of this embodiment, T
The operation when the amount of movement of the V camera 10 is equal to or more than a certain amount and the captured image is frame-compressed will be described. At this time, the control circuit 60 connects the switch 22 to the b contact, and the switch 38
To open. As a result, the circuits 24 to 36 compress the output image of the circuit 16 in the frame by DCT conversion.

That is, the output video signal of the TV camera 10 is A
The signal is converted into a digital signal by the / D converter 14, and the format conversion circuit 16 converts the output of the A / D converter 14 from the raster format to a predetermined intermediate format for transmission. The output of the format conversion circuit 16 is the switch 22.
B contact point, the subtractor 20, and the motion compensation inter-frame prediction circuit 36. Since the switch 22 is connected to the b contact, the output of the circuit 16 is applied to the DCT conversion circuit 24 and DCT converted. The quantization circuit 26 quantizes the output of the DCT conversion circuit 24. Since the switch 38 is opened, the motion compensation interframe prediction circuit 34 is not used.

The variable length coding circuit 40 includes a quantization circuit 26.
Variable-length code the output of. The output of the variable-length coding circuit 40 is sent to the error correction coding circuit 4 via the transmission buffer 42.
4 is applied. When the amount of data to be transmitted is too large as compared with the communication line, the transmission buffer 42 thins out the frames. The error correction coding circuit 44 generates and adds an error correction code and applies it to the multiplexing circuit 56.

The voice signal from the microphone 48 is processed by the A / D converter 50 and the voice encoding circuit 52, time-adjusted by the delay circuit 54, and supplied to the multiplexing circuit 56, as in the case where the TV camera 10 is stationary. To be done.

The multiplexing circuit 56 is an error correction coding circuit 44.
And the data from the delay circuit 54 are multiplexed, and the output is output to the communication line via the line connection circuit 58.

[0038]

As can be easily understood from the above description, according to the present invention, the motion of the subject image on the screen is estimated from the control information of the image pickup means, and the motion vector is detected based on the estimation. Therefore, the number of calculations required to detect the motion vector can be significantly reduced, and the calculation time can be shortened.

[Brief description of drawings]

FIG. 1 is a configuration block diagram of an embodiment of the present invention.

FIG. 2 is an explanatory diagram of motion vector detection in motion-compensated interframe prediction.

[Explanation of symbols]

10: TV camera 12: Mobile stand 14: A / D converter 16: Format conversion circuit 20: Subtractor 22:
Switch 24: DCT conversion circuit 26: Quantization circuit
28: Inverse quantization circuit 30: Inverse DCT conversion circuit 32:
Adder 34: Motion compensation inter-frame prediction circuit 36: Motion adaptive filter 38: Switch 40: Variable length coding circuit 42: Transmission buffer 44: Error correction coding circuit 48: Microphone 50: A / D converter 52: Voice Encoding circuit 54: Delay circuit 56: Multiplexing circuit 58:
Line connection circuit 60: Control circuit 62: Operating device

Claims (2)

[Claims] 1. An image coding apparatus for compressing an image picked up by an image pickup means on a moving table between motion compensation screens, wherein the image pickup means refers to optical control information and movement information of the image pickup means, An image coding apparatus, comprising: an estimating unit that estimates the motion of the motion vector; and a motion vector detecting unit that detects a motion vector between screens based on an estimated value of the estimating unit. 2. An image transmitting apparatus for adaptively compression-encoding and transmitting an image captured by an image pickup means on a movable table by one of an in-screen compression method and an in-screen compression method, wherein the in-screen compression method is used. Includes a motion-compensated inter-picture compression method, and is provided with an estimation means for estimating the motion of the subject on the photographing screen by referring to the optical control information and movement information of the imaging means. An image transmitting apparatus characterized in that motion information between different screens is detected based on an estimated value of the estimating means.