JP3346448B2 - Video encoding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for digitally compressing and encoding an image signal used for image communication, image recording, etc. The present invention relates to a moving picture coding device used.

[0002]

2. Description of the Related Art ITU-T (formerly CCITT) recommendation H.264. 2
61 is entitled “p × 64 kb / s video coding system for audiovisual service”, and 64 kb / s (p =
A video coding standard for communications using bit rates from 1) to 2 Mb / s (p = 30). The standardization work started in December 1984 and the recommendation was approved in December 1990. Applications include videophones, videoconferencing, and the like. H. Reference numeral 261 suppresses temporal redundancy of a moving image signal by motion compensation prediction, and suppresses spatial redundancy of each frame by discrete cosine transform (DCT) coding.

[0003] Referring to FIG. The outline of an image communication apparatus using H.261 will be described with reference to FIG. 261 will be briefly described.

The image communication apparatus shown in FIG. 4 comprises a camera 40, an A / D converter 41, a format converter 42, a video encoder 43, and a multiplexer 44. The video signal 45 is a video signal output from the camera 40 and has a signal format of NT
SC (National Television System Comitiee), PAL
(Phase Alternate Line), SECAM (Sequential Coul
eur a Memore). The digital signal 36 is an input signal to the format converter 42,
It is a digital video signal obtained by converting the signal of NTSC, PAL, SECAM or the like by the A / D converter 41. The signal 47 is a common intermediate format (Common Intermedeate Format:
CIF or Quarter CIF: QCIF). Signal 4
Reference numeral 8 denotes a video signal compressed by the video encoding unit 43, which is an input signal to the multiplexing unit 44.

[0005] A camera 40 captures an image of a person, a landscape, a calligraphy, etc.
Output in analog signal format such as AL, SECAM, etc.
The signal is converted into a digital signal 46 by the / D converter 41 and input to the format converter 42. At this time, in order to use the video encoding unit 43 in accordance with the ITU-T international standardization recommendation, the input digital signal 46 is converted into a CIF or QCIF signal 47 by the format conversion unit 42, and the ITU-T recommendation H . 261 is compressed by the video encoding unit 43 according to the high-performance encoding method.
The compressed digital video signal 48 is further multiplexed with the digitized audio signal and data signal by the multiplexing unit 44 in the ITU-T Recommendation H.264. H.221 is multiplexed into a frame structure defined by ISDN (Integrated
Services Digital Network) or a dedicated line such as a high-speed digital line and sent to the partner device.

In determining the format of the CIF or QCIF, a format predetermined by a user of the device is used by negotiation between the devices.

FIG. 5 is a block diagram of the video encoding unit 43 shown in FIG. 261 is a diagram illustrating a configuration example of a video encoding device that conforms to an encoding method defined by H.261.

First, an image to be encoded 1 is input to a motion detecting section 3 'together with a square pattern 2', and is 16 pixels × 16 pixels.
It is divided into square blocks called macroblocks of lines. The motion detection unit 3 'detects the amount of motion between the macroblock in the encoding target image 1 and the reference image, and sends the obtained motion vector 18 to the block motion compensation unit 4'. Here, the motion vector 1 of each macroblock
Reference numeral 8 denotes a displacement between the coordinates of the block having the highest matching degree with the macroblock of interest and the coordinates of the macroblock of interest in the reference image. The search range of the motion vector 18 is determined by the coordinates of the macroblock of interest and ±
It is limited to 15 pixels × ± 15 lines.

Next, the block motion compensator 4 'generates a motion compensated prediction image 7 from the motion vector 18 of each macroblock and the local decoded image 6 of the immediately preceding frame stored in the frame memory 5. The motion-compensated predicted image 7 obtained here is input to the subtracter 8 together with the image 1 to be encoded. The difference between them, that is, the motion compensation prediction error 9 is DC
The DC / DCT transform is performed in the T / quantization unit 10 ′, and the data is further quantized to become compressed difference data 11. Where DCT
Is 8 × 8. Compressed difference data 11
The (quantization index) is subjected to data compression in the differential data encoding unit 12 to become encoded differential image data 13. On the other hand, the motion vector 18 is encoded by the motion vector encoding unit 19, and the obtained motion vector encoded data 2
0 is multiplexed with the coded difference image data 13 in the multiplexing unit 21 ′ and transmitted as multiplexed data 22 ′.

Note that, in order to obtain the same decoded image as the decoding device in the encoding device, the compressed difference data 11 (quantization index) is returned to the quantized representative value by the inverse quantization / inverse DCT unit 14 ', and further inversely. After the DCT transform, the image becomes an expanded difference image 15. The expanded difference image 15 and the motion-compensated predicted image 7 are added by the adder 16 to form a locally decoded image 17. This locally decoded image 17 is stored in the frame memory 5 and used as a reference image when encoding the next frame.

[0011]

In the above prior art, a CIF or QCIF of a video signal format determined between terminals is used.
For example, when encoding a video signal of a subject requiring a high resolution such as a document or a photograph, the encoding is performed using an input image signal having a certain resolution determined by a device to be used. There was a disadvantage that it became insufficient. When encoding a video signal such as a document or a photograph, H.264 is used. It is also conceivable to switch the coding method to an image coding method other than H.261, but the user such as a terminal switches the coding method according to the type of the subject, and there is a problem that the usability is poor.

Further, in the above-described conventional technique, in a portion where the movement of an encoded image signal caused by panning, tilting, zooming, or the like of a camera is intense, although fine details cannot be visually identified, QCI
Encoding can be performed only up to a resolution of about F, which makes it difficult to encode motion information that is originally important.

Further, the above-mentioned prior art uses an input image having a constant resolution regardless of the amount of movement of the camera. Therefore, in an image in which the camera pans or tilts violently, the bandwidth of the transmission path is limited. However, there is a disadvantage in that the image cannot be transmitted in a sufficient number of frames, and conversely, if an attempt is made to encode / transmit a sufficient number of frames, the image quality is significantly reduced.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problem. When the camera does not move and the movement of the image signal is not so intense, a high-resolution input image is usually used to perform panning, tilting, zooming and the like of the camera. When included in the input signal, the user automatically
F, QCIF etc., to lower the resolution of an input image to be coded is to provide a moving picture coding instrumentation <br/> location capable quality better coding a moving picture signal.

[0015]

[0016]

[0017]

[0018]

[0019]

[0020]

[0021]

[0022]

[0023]

According to the present invention, there is provided a moving picture coding apparatus comprising: a motion detecting section for inputting a picture to be coded or a picture to be coded after spatial resolution conversion and a polygon pattern to obtain a motion vector; From the motion vector of the image to be encoded,
A camera parameter detection unit that detects panning, tilt, zoom-in or zoom-out of the encoding target image, and an input image spatial resolution that selects a spatial resolution of the encoding target image from the detected camera parameters and outputs the selected spatial resolution as an input image spatial resolution A selection unit, a spatial resolution conversion unit that converts the encoding target image into an image having a spatial resolution indicated by the input image spatial resolution, and outputs the image as the encoding target image after the spatial resolution conversion, and a frame that stores a locally decoded image A memory and the input image spatial resolution are input, and the locally decoded image of the immediately preceding frame stored in the frame memory is converted into an image having a resolution indicated by the input image spatial resolution, and is converted into a spatially decoded local decoded image. A motion compensation spatial resolution conversion unit to be output, and the spatial solution output from the motion detection unit. A block motion compensator that generates a motion-compensated predicted image from a motion vector of the degree-converted encoding target image and the local resolution-converted local decoded image; and A subtractor that takes a difference and outputs a motion compensation prediction error;
The perform suppression of the spatial redundancy of relative motion compensated prediction error, and the spatial redundancy compression unit for outputting a compressed difference data, the difference data <br/> extension length which decodes the compressed difference data in the expanded difference image An adder that adds the motion-compensated predicted image and the decompressed difference image and accumulates the local decoded image in the frame memory; and a difference that compresses and compresses the compressed difference data and outputs the resultant as difference image encoded data. A data encoding unit, a data compression encoding of a motion vector of the image to be encoded after the spatial resolution conversion, a motion vector encoding unit for outputting as a motion vector encoded data, and a data compression encoding of the input image spatial resolution, A selected spatial resolution identifier encoding unit that outputs as selected spatial resolution identifier encoded data, the differential image encoded data and the motion vector encoded data The selected spatial resolution identifier encoded data and a multiplexer for multiplexing.

By changing the resolution of the input image for each frame according to the detected movement of the camera,
It is possible to transmit a sufficient number of frames even when the camera moves violently, and when the camera is stationary, it is possible to display even details, so it is more visual than conventional coding methods. Image quality can be greatly improved.

[0025]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram of a moving picture coding apparatus according to an embodiment of the present invention. The same components as those in FIG. 5 indicate the same components.

[0027] This moving picture coding apparatus, output from the motion detection unit 3 through the switch 23, the camera parameters detector which detects camera parameters 25 of the encoding target image 1 from the motion vector 18 ₁ of the encoding target image 1 An input image resolution selecting unit 26 for selecting the resolution of the encoding target image 1 from the camera parameters 25 and outputting the selected resolution as an input image resolution 27;
The resolution conversion unit 28 converts the image into the image having the resolution indicated by the following formula, and outputs the image as the encoding target image 29 after the resolution conversion. conversion, and motion compensation resolution conversion unit 30 for outputting the resolution-converted local decoded image 31, output from the motion detector 3, a motion vector 18 _second resolution conversion after encoding target image 29 and data compression encoding, a motion vector coding unit 19 for outputting the motion vector coding data 20, from the resolution-converted local decoded image 31 motion vector 18 ₂ which block the motion compensation unit 4 which generates a motion compensated predicted image 7, an input image resolution 27 Is compressed and encoded, and the selected resolution identifier encoded data 3
4; a multiplexing unit 21 that multiplexes the difference image coded data 13, the motion vector coded data 20, and the selected resolution identifier coded data 34, and transmits the multiplexed data 22. , Switch 2
3, 32, 35 are newly provided.

Next, the operation of this embodiment will be described.

First, the image to be encoded 1 is input to the motion detecting section 3 together with the polygon pattern 2, and a motion vector 18 ₁ (motion amount) of each batch is obtained. Here, a square is often used as the polygon pattern 2. Also, the motion vector 18 ₁ may be determined by a full search in the vicinity of the same position as the macro block of interest in the reference image using the mean square error or error absolute value sum or the like as the error evaluation value, and the macro block of interest Is represented as a displacement between the coordinates of the block having the highest matching degree and the coordinates of the macroblock of interest.

Next, the motion vector 18 ₁ is input to the camera parameter detection section 24, panning, tilting, zooming, camera parameters 25 such as a zoom-out is detected.

Here, the method of detecting the camera parameters 25 is performed as follows.

In FIG. 2, the center of the drawing is defined as the origin (0,
0), the motion vector due to panning or tilting at the point (i, j) is a constant determined by the panning / tilting speed in the horizontal / vertical directions regardless of the position. This is (H, V). Further, the motion vector by zooming increases as the distance from the center increases, and if Z is a constant determined by the zoom magnification, (Z × i, Z ×
j). Therefore, the motion vector 18 ₁ due to the camera motion due to both effects is (Z × i +
H, Z × j + V). By obtaining these three constants Z, H, and V, the camera parameters 25 of the moving image can be obtained.

The center coordinates (i, j) and (-i, -j) of two blocks located symmetrically with respect to the center of the screen,
The motion vectors (V1 _x , V1 _y ), (V2 _x , V2
_y ), constants Z, H, and V are determined by the following equation.

[0034]

(Equation 1) Similarly, Z, H, and V are obtained for all symmetric blocks in the screen, and for each of Z, H, and V,
The one with the highest frequency is finally set to the values of Z, H, and V.

The obtained camera parameters 25 are input to the input image resolution selection section 26, and according to the zoom, panning, and tilt degrees, that is, (Z, H, V) values, the ITU-R. B. T. 601, a resolution of an input image such as CIF or QCIF is selected. Generally, an input image having a lower resolution is used as the camera moves more rapidly. The selected input image resolution 27 is input to the resolution converter 28 and the motion compensation resolution converter 30.

The encoded symmetric image 1 is converted by the resolution conversion unit 28 to the resolution of the image indicated by the input image resolution 27, and an image 29 to be encoded after resolution conversion is obtained. The encoding target image 29 after the resolution conversion is input again to the motion detecting unit 3 together with the polygon pattern 2 via the switch 32, and the motion amount between the reference image and each macro block at the resolution is detected and obtained. Send the motion vectors 18 ₂ to block motion compensation unit 4.

Similarly, the image 29 to be encoded after resolution conversion
The input image resolution 27 is input to the motion-compensation resolution conversion unit 30 so as to match with the resolution of the frame memory 5.
Is converted to the resolution of the image indicated by the input image resolution 27, and the resolution-converted local decoded image 31 is obtained.

FIG. 3 shows an example of the resolution conversion method. For example, CIF has a resolution of 352 pixels × 288 lines,
QCIF has a resolution of 176 pixels × 144 lines.
Therefore, between these two image formats, as shown in FIG. 3A, conversion from CIF to QCIF can be performed by thinning out one pixel and one line. Conversely, conversion from QCIF to CIF becomes possible by replacing one pixel with four pixels as shown in FIG. If more resolution is required, conversion between CIF and an image format with four times the resolution can be used. These resolution conversions can be realized by simple thinning and enlargement. Also, in encoding, since the resolution of a frame used for motion prediction matches the resolution of an input frame, prediction can be performed easily.

In the above description, QCIF and CIF have been mainly described as examples. However, it is needless to say that other types may be used, for example, １ ／ or 例 え ば of QCIF, or １ ／ of QCIF.
May be combined with conversion to an image such as 3 ×, 4 ×, or even 4 × of CIF.

Next, at block motion compensation unit 4, a local decoded image of the immediately preceding frame that has been converted to the resolution of the image shown in the motion vector 18 ₂ and the input image resolution 27 of each macro block, i.e. the resolution-converted local decoded image The motion compensation prediction image 7 is generated from the motion compensation prediction image 31 and the motion compensation prediction image 31. The motion-compensated prediction image 7 obtained here is input to the subtractor 8 together with the encoding target image 29 after resolution conversion. The difference between the two, that is, the motion compensation prediction error 9, is subjected to spatial redundancy suppression in the spatial redundancy compressor 10. In order to obtain a local decoded image 17 of the current image 29 to be subjected to the resolution conversion after encoding, the compressed difference data 11 output from the spatial redundancy compressing unit 10 is decoded by the difference data expanding unit 14 into an expanded difference image 15. The expanded difference image 15 is a motion-compensated prediction error signal in which spatial redundancy is suppressed. The expanded difference image 15 is added to the motion-compensated predicted image 7 by the adder 16 to become the local decoded image 17 of the current resolution-converted encoding target image 29. The local decoded image 17 is stored in the frame memory 5 and is referred to in encoding of subsequent frames.

On the other hand, the compressed differential compressed data 11 corresponding to the motion compensation prediction error 9 is subjected to data compression encoding by the differential data encoding section 12 to become differential image encoded data 13. The motion vector 18 is subjected to data compression encoding in a motion vector encoding unit 19 to become encoded motion vector data 20. The input image resolution 27 is data compression-encoded by the selected resolution identifier encoding unit 33, and the selected resolution identifier encoded data 3
It becomes 4. The difference image encoded data 13, the motion vector encoded data 20, and the selected resolution identifier encoded data 34 are multiplexed in the multiplexing unit 21 and transmitted or stored as multiplexed data 22.

Note that the encoding apparatus shown in the present embodiment
In the case where the resolution of the input image is not known and the camera parameters of the input image signal are known as in the case where the camera parameters are recorded at the same time when the image is captured, the camera parameter detection unit 24 is unnecessary. Yes, this apparatus can be realized by directly inputting the recorded camera parameters to the input image resolution selection unit 26.

[0043]

As described above, according to the present invention,
When the camera is stationary, it automatically encodes the high-resolution image signal, and when the camera starts moving, the resolution of the input image signal is automatically reduced to better express the motion in the moving image. That is, there is an effect that the image signal can be efficiently encoded without the user being conscious.

This is because the target image requiring resolution is
The fact that the image is often a still image such as a photograph or a document, and that the human visual characteristics are such that the resolution with respect to the resolution is degraded when the motion is large and the resolution with respect to the resolution is increased when the motion is small. Match.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a video encoding device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of an operation of detecting a camera parameter from a moving image.

FIG. 3 is a diagram illustrating an example of resolution conversion.

FIG. 4 is a diagram showing an outline of a conventional image communication device.

FIG. 5 is a diagram illustrating a configuration of an encoding device according to a conventional motion compensation prediction encoding method in block units.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Encoding target image 2 Polygon pattern 3 Motion detection part 4 Block motion compensation part 5 Frame memory 6 Local decoded image 7 Motion compensation prediction image 8 Subtractor 9 Motion compensation prediction error 10 Spatial redundancy compression part 11 Compressed difference data 12 Difference data encoding unit 13 coded differential picture data 14 difference data extension length section 15 expanded difference image 16 adder 17 local decoded image 18 _1, 18 ₂ motion vectors 19 motion vector coding unit 20 motion vector coding data 21 multiplexing unit Reference Signs List 22 multiplexed data 23 switch 24 camera parameter detection unit 25 camera parameter 26 input image resolution selection unit 27 input image resolution 28 resolution conversion unit 29 image to be encoded after resolution conversion 30 resolution conversion unit for motion compensation 31 locally decoded image after resolution conversion 32 switch 33 selected resolution identifier encoding unit 34 Selected resolution identifier encoded data 35 Switch

──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Hisashi Ibaraki 3-19-2 Nishishinjuku, Shinjuku-ku, Tokyo Within Nippon Telegraph and Telephone Corporation (56) References JP-A-7-250322 (JP, A) 6-197333 (JP, A) JP-A-9-116802 (JP, A) Kazuto Uekura, Yutaka Watanabe, Global Motion Compensation Method in Video Coding, IEICE Transactions BI, Japan, The Institute of Electronics, Information and Communication Engineers, December 25, 1993, Vol. J76-BI, No. 12, p. 944-952 (58) Field surveyed (Int. Cl. ⁷ , DB name) H04N ^7/ 24-7/68

Claims (1)

(57) [Claims] 1. A moving image encoding apparatus for inputting and encoding an image signal, comprising the steps of: inputting an encoding target image or an encoding target image after spatial resolution conversion and a polygon pattern; Unit, a camera parameter detection unit that detects panning, tilting, zoom-in or zoom-out of the encoding target image from the motion vector of the encoding target image, and selects a spatial resolution of the encoding target image from the detected camera parameters. An input image spatial resolution selection unit that outputs the input image spatial resolution, and converts the encoding target image into an image having a spatial resolution indicated by the input image spatial resolution, and outputs the image as the spatial resolution converted encoding target image. A spatial resolution converting unit, a frame memory for storing a locally decoded image, and the input image spatial resolution The local decoded image of the immediately preceding frame stored in the memory is converted into an image having a spatial resolution indicated by the input image spatial resolution,
A spatial resolution conversion unit for motion compensation to be output as a locally decoded image after the spatial resolution conversion, and a motion vector of the image to be encoded after the spatial resolution conversion and the local decoded image after the spatial resolution conversion output from the motion detection unit. A motion compensation prediction unit that generates a motion compensation prediction image from the motion compensation prediction image; a subtracter that calculates a difference between the coding target image after the spatial resolution conversion and the motion compensation prediction image and outputs a motion compensation prediction error; perform suppression of the spatial redundancy degree with respect to a spatial redundancy degree compression unit for outputting a compressed difference data, and the difference data extension length which decodes the compressed difference data in the extension length difference image, the motion compensated prediction picture And an adder for adding the decompressed difference image and storing the compressed difference data in the frame memory as the locally decoded image. A differential data encoding unit that outputs data as a data, a motion vector encoding unit that performs data compression encoding on a motion vector of the encoding target image after the spatial resolution conversion, and outputs the motion vector encoded data, and the input image spatial resolution. A selected spatial resolution identifier encoding unit that performs data compression encoding and outputs the selected spatial resolution identifier encoded data, and multiplexes the differential image encoded data, the motion vector encoded data, and the selected spatial resolution identifier encoded data. And a multiplexing unit.