JPH09214965A - Method and device for encoding moving image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To encode with high quality a section of an image signal where motion caused by panning, tilting or zooming of a camera is violent. SOLUTION: An encoding object image is inputted to a motion detection part 3 together with a polygonal pattern 2, and a motion vector 18 is found. The motion vector 18 is inputted to a camera parameter detection part 23, and a camera parameter 24 is detected. The camera parameter 24 is inputted to a resolution converting section deciding part 25, it is decided acceding to the kind and degree of that parameter which section in the encoding object image 1 is to increase the resolution and which section is to decrease the resolution, and the result is outputted as a resolution converting section identifier 26. At a spatial redundancy compressing part 10, encoding is performed with a fine quantizing characteristic value at a macro block corresponding to the section shown 'important' by the resolution converting section identifier 26 but encoding is performed with a coarse quantizing characteristic value at a macro block corresponding to the section shown 'important'.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital compression encoding method for image signals used for image communication, image recording, etc., and more specifically, for transmitting and receiving image signals between terminals and storing them in files. The present invention relates to an image coding method and apparatus used.

[0002]

ITU-T (formerly CCITT) Recommendation H.264. 2
61 is entitled “p × 64 kb / s video coding method for audiovisual service”, and is 64 kb / s (p
= 1) to 2 Mb / s (p = 30), which is a video coding standard for communication. The standardization work started in December 1984, and the recommendation was established in December 1990.
It is the moon. Examples of the application include a videophone and a video conference. H. 261 suppresses the temporal redundancy of the moving image signal by motion compensation prediction, and suppresses the spatial redundancy of each frame by discrete cosine transform (DCT) coding.

Hereinafter, with reference to FIG. The outline of the image communication apparatus using the H.261 is described in FIG. The encoding algorithm of H.261 will be briefly described.

The image communication apparatus shown in FIG. 3 includes a camera 30 and an A
The D / D converter 31, the format converter 32, the video encoder 33, and the multiplexer 34 are included. The video signal 35 is a video signal output from the camera 30, and the signal format is NTS.
C (National Television System Comitee), PAL
(Phase Alternate Line), SECAM (Sequential C)
ouleur a Memore) and other analog signals. The digital signal 36 is an input signal to the format conversion section 32, and is a digital video signal obtained by converting the signals of NTSC, PAL, SECAM, etc. by the A / D conversion section 31. Signal 3
7 is Common Intermedeate Form
at: CIF, or Quarter CIF: QCIF). Signal 3
Reference numeral 8 denotes a video signal compressed by the video encoding unit 33, which is an input signal to the multiplexing unit 34.

A camera 30, an image of a person, a landscape, a calligraphy or the like is picked up, and the video signal 35 is sent from the camera 30 to NTSC or P.
Output in analog signal format such as AL and SECAM, A
The / D conversion unit 31 produces a digital signal 36, which is input to the format conversion unit 32. At that time, since the video encoding unit 33 according to the ITU-T international standardization recommendation is used,
In the format conversion section 32, the input digital signal 36 is converted into a CIF or QCIF signal 37, and the ITU-T recommendation H.264 is provided. It is compressed by the video encoding unit 33 according to the high-performance encoding method defined by H.261. The compressed digital video signal 38 is further multiplexed with the digitized audio signal and data signal in the multiplexing unit 34.
In ITU-T Recommendation H. The ISDN (Integrated Service)
s Digital Network) and other dedicated lines such as high-speed digital lines.

FIG. 4 shows the video coding unit 33 of FIG. 3, that is, ITU-T Recommendation H.264. [Fig. 26] Fig. 26 is a diagram illustrating a configuration example of a video encoding device according to an encoding method defined by H.261.

First, the image to be coded 1 is input to the motion detecting section 3 together with the square pattern 2'and divided into square blocks called macroblocks of 16 pixels × 16 lines. The motion detection unit 3 detects the amount of motion with respect to the reference image for each macroblock in the encoding target image 1, and sends the obtained motion vector 18 to the block motion compensation unit 4. Here, the motion vector 18 of each macroblock is
In the reference image, it is represented as a displacement between the coordinates of the block having the highest degree of matching with the macroblock of interest and the coordinates of the macroblock of interest. The motion vector search range is limited to the coordinates of the macroblock of interest and ± 15 pixels × ± 15 lines around it.

Next, the block motion compensation unit 4 generates a motion compensation predicted image 7 from the motion vector 18 of each macroblock and the locally decoded image 6 of the immediately preceding frame stored in the frame memory 5. The motion-compensated predicted image 7 obtained here is input to the subtracter 8 together with the encoding target image 1. The difference between the two, that is, the motion-compensated prediction error 9 is DCT-transformed by the DCT / quantization unit 10 ′ and further quantized to be compressed difference data 11. Here, the block size of the DCT is 8 × 8.

H. In H.261, the encoding bit rate is in the range of about 40 kb / s to 2 Mb / s, but the amount of information generated in the input image signal varies depending on the complexity of the image and the intensity of motion. In order to absorb this fluctuation and transmit at a constant transmission rate, code generation amount control (rate control) is performed. H. In conventional coding methods such as H.261, MPEG1, MPEG2, this control method is free to design, but generally, a transmission buffer is provided and MQUANT (quantization characteristic) is set according to the occupancy rate of this buffer. A method of controlling a value called is often used. That is, except for the buffer occupancy, it is not used as a factor for changing the fineness of quantization.

The compressed difference data 11 (quantization index) is data-compressed in the difference data encoding unit 12 to become difference image encoded data 13. On the other hand, the motion vector 18 is coded by the motion vector coding unit 19,
The obtained motion vector coded data 20 is multiplexed with the differential image coded data 13 by the multiplexing unit 21 and transmitted as multiplexed data 22.

Since the same decoded image as that of the decoding device is obtained in the coding device, the compressed difference data 11 (quantization index) is returned to the quantized representative value by the inverse quantization / inverse DCT unit 14 ', and the inverse After the DCT conversion, the decompression difference image 15 is obtained. The decompression difference image 15 and the motion compensation prediction image 7 are added by the adder 16 to form a locally decoded image 17. The locally decoded image 17 is stored in the frame memory 5 and used as a reference image when the next frame is encoded.

[0012]

In the above-mentioned prior art, since the fineness of the quantization is determined only by the occupation ratio of the transmission buffer, regardless of the complexity of the input image signal, the panning of the camera, tilting, zooming, etc. It is impossible to maintain a visually sufficient image quality in a portion where the motion of the coded image signal caused by the above is intense.

For example, when the human eye is focused on the center point of the zoom, such as during zooming, the same quantization fineness is applied to a portion such as an edge of the screen that does not actually affect the human eye visually. Since the encoding is performed with the above, there is a problem that the center point of the zoom, which is most visually distorted, is distorted on average as in other places.

Also, for example, the camera constantly pans,
In the case of a moving image signal that tilts, it is extremely inefficient to allocate a large amount of code to a portion that obviously goes out of the screen in the next frame.

That is, in the conventional image coding method, the input image signal is coded with a constant quantization characteristic over the entire screen regardless of the amount of movement of the camera, so that the camera vigorously pans, tilts, and zooms. In an image with intense movement, the entire screen is greatly distorted, or the center point of the zoom that many people see when zooming in is blurred like other places, and the image quality is significantly degraded,
On the contrary, if the quantization characteristic value is made finer over the entire screen in order to forcibly improve the image quality, there is a problem that it is impossible to visually encode / transmit a sufficient number of frames.

An object of the present invention is to solve the above problems, and when the panning, tilting, zooming, etc. of the camera is included in the input signal, the distortion is generated without the user being aware of the movement of the camera. It is an object of the present invention to provide a moving picture coding method and apparatus capable of automatically increasing the resolution of a portion where distortion is visually noticeable when present, and coding an image signal with high quality.

[0017]

According to the moving picture coding method of the present invention, when a camera parameter of an input image signal is known, the camera parameter is used as it is, and when it is not known, the camera parameter is detected. The encoding is performed by increasing or decreasing the resolution of a specific portion of the input image signal according to the camera parameter.

According to the embodiment of the present invention, in detecting the camera parameter of the image signal, the image to be encoded is divided into a plurality of small areas, and the amount of motion between the image to be encoded and the prediction reference image is calculated. The camera parameter is obtained for each small area, and the camera parameter is detected from the obtained amount of movement for each small area.

According to the embodiment of the present invention, when the resolution of a specific part of an input image is increased or decreased according to the camera parameter, zoom-in of the camera parameter is performed as an evaluation value for selecting which part of the resolution is changed, Use the amount of zoom out.

According to the embodiment of the present invention, when the detected camera parameter includes zoom-in and zoom-out, the quantization near the center point where the camera zooms is finely quantized to increase the resolution. And encode. As a result, the visual image quality can be improved.

According to the embodiment of the present invention, when the resolution of a specific part of the input image is increased or decreased according to the camera parameter, panning of the camera parameter is performed as an evaluation value for selecting which part of the resolution is changed. Use the amount of tilt.

According to the embodiment of the present invention, when the detected camera parameters include panning and tilt,
When encoding a part that newly appears from the outside of the screen by panning or tilting the camera, it is finely quantized and the resolution is increased. As a result, the visual image quality can be improved.

According to the embodiment of the present invention, when the detected camera parameters include panning and tilt,
When a part newly appearing from the outside of the screen is encoded by panning or tilting of the camera, it is roughly quantized and the resolution is lowered to perform encoding. As a result, a large amount of code can be assigned to encode other parts.

According to the embodiment of the present invention, when the detected camera parameters include panning and tilt,
When encoding the part that goes out of the screen in the next frame due to the panning and tilting of the camera, it is coarsely quantized and the resolution is reduced. As a result, a large amount of code can be assigned to encode other parts.

The moving picture coding apparatus of the present invention determines the resolution of which part of the input image signal according to the camera parameter detecting section for detecting the camera parameter of the input image signal and the parameter detected by the camera parameter detecting section. A resolution conversion region determining unit that determines whether the image is changed, and an image encoding unit that encodes an image signal according to the determination of the resolution conversion region determining unit.

According to the detected movement of the camera, the resolution of the portion where the distortion is visually noticeable in the image signal to be encoded is increased, while the distortion is not visually noticeable in the image signal to be encoded. By lowering the resolution of the part, it is possible to greatly improve the visual image quality when the camera moves violently.

[0027]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing the structure of a moving picture coding apparatus according to an embodiment of the present invention. The same symbols as those in FIG. 4 indicate the same things.

The moving picture coding apparatus according to the present embodiment receives a motion vector 18 (motion amount) and detects a camera parameter 24 such as panning, tilting, zooming in, and zooming out, and a camera parameter detecting unit 23. 24 is input, and a resolution conversion site identifier is output that outputs a resolution conversion site identifier 26 that indicates which part of the encoding target image 1 has a higher resolution and which part has a lower resolution according to the type and degree of zoom, panning, and tilt. It is characterized in that it has a section 25.

Next, the operation of this embodiment will be described.

First, the image to be coded 1 is input to the motion detecting section 3 together with the polygonal pattern 2, and the motion vector 18 of each patch is obtained. Here, a square is often used as the polygon pattern 2. Also, the motion vector 18
Can perform a full search in the vicinity of the same position as the macroblock of interest in the reference image using the mean square error or the sum of absolute error values as the error evaluation value, and has the highest degree of matching with the macroblock of interest. It is expressed as a displacement between the coordinates of the block and the coordinates of the macroblock of interest.

Next, the motion vector 18 is input to the camera parameter detection unit 23, and the camera parameters 24 such as panning, tilting, zooming in and zooming out are detected.

Here, the camera parameter 24 is detected as follows.

In FIG. 2, the center of the screen is the origin (0,
0), the motion vector due to panning or tilting at the point (i, j) is independent of the position and is a constant determined by the panning and tilting speeds in the horizontal / vertical directions, respectively. Let this be (H, V). Also, the motion vector due to zooming increases as it moves away from the center, and can be expressed as (Z × i, Z × j), where Z is a constant determined by the zoom magnification. Therefore, the motion vector due to the camera movement due to both influences can be expressed as (Z × i + H, Z × j + V). By obtaining these three constants Z, H, and V, the camera parameter 24 of the moving image can be obtained.

The center coordinates (i, j) and (-i, -j) of the two blocks at the target position with respect to the center of the screen,
The motion vectors (V1 _x , V1 _y ), (V2 _x , V2
_y ) is used to determine the three constants Z, H, and V by the following equation.

[0036]

(Equation 1) Similarly, Z, H, and V are obtained for blocks at all symmetrical positions in the screen, and the one with the highest frequency is finally set as the value of Z, H, and V for each of Z, H, and V. To do.

The obtained camera parameter 24 is input to the resolution conversion region determination unit 25, and which part of the image 1 to be coded is selected according to the type and degree of zoom, panning, and tilt, that is, the value of (Z, H, V). Increase the resolution of
Decide which part to lower.

For example, when the zoom is detected, the photographer is gazing at the object at the end of the zoom, and naturally the viewer of the image should also gaze at that part. At this time, since the center of the screen is the focus of zooming, a flag indicating "important" is set near the center of the screen.

When the panning and tilt of the camera are detected, the quantization characteristic values of the part newly entering the frame and the part which will be outside the screen in the next frame are changed. Specifically, when the camera is panning from left to right: -The left edge of the screen is the part that disappears in the next frame, so
Flag the left edge of the screen as "not important".

-The right edge of the screen is a newly appearing part, which is an object that will continue to enter the frame for some time in the future. Therefore, when there is room in the transmission buffer, a flag indicating "important" is set. It's On the contrary, when there is no space in the transmission buffer, the image quality of other parts is improved by intentionally setting a flag indicating "not important". A flag indicating "important" or "not important" is set for each macroblock as shown in. The resolution conversion site identifier 26, which is information indicating the selected site, is input to the spatial redundancy compression unit 10.

The motion vector 18 is also sent to the block motion compensating unit 4, and the block motion compensating unit 4 generates a motion compensation prediction image 7 from the motion vector 18 of each macroblock and the locally decoded image 6 of the immediately preceding frame. To do. The motion-compensated predicted image 7 obtained here is input to the subtracter 8 together with the encoding target image 1. The difference between the two, that is, the motion compensation prediction error 9 is input to the spatial redundancy compression unit 10 together with the resolution conversion part identifier 26, and the spatial redundancy is suppressed. At this time, in the macro block corresponding to the part indicated as “important” in the resolution conversion part identifier 26, the quantization characteristic value is finely coded to increase the visually noticeable resolution. In a macroblock corresponding to a portion indicated as "unimportant", by encoding with a rough quantization characteristic value, it is possible to allocate a large amount of code in a visually noticeable place.

Local decoded image 1 of the current encoding target image 1
In order to obtain 7, the compressed differential data 11 output from the spatial redundancy compression unit 10 is decoded by the differential data expansion unit 14 into an expanded differential image 15. The decompression difference image 15 is a motion compensation prediction error signal whose spatial redundancy is suppressed. The decompressed difference image 15 is added to the motion compensation predicted image 7 by the adder 16 and becomes the locally decoded image 17 of the current image to be encoded.
The locally decoded image 17 is stored in the frame memory 5 and is referred to in the coding of subsequent frames.

On the other hand, the compressed differential compressed data 11 for the motion compensation prediction error 9 is data compressed and coded by the differential data coding unit 12 to become differential image coded data 13. The motion vector 18 is data-compressed and coded by the motion vector coding unit 19 and becomes motion vector coded data 20. The differential image coded data 13 and the motion vector coded data 20 are multiplexed in the multiplexing unit 21 and transmitted or accumulated as multiplexed data 22.

Further, in the device shown in this embodiment, the resolution of the input image is not known, and as in the case where the camera parameters are recorded at the same time when the image is taken,
If the camera parameter of the input image signal is known, the camera parameter detection unit 23 is not necessary, and the present apparatus can be realized by directly inputting the recorded camera parameter 24 into the resolution conversion region determination unit 25. Becomes

[0045]

As described above, according to the present invention,
When there is camera movement, for example when zoom is detected, the resolution in the center of the screen is automatically increased, and when panning or tilt is detected, the resolution at the edge of the screen is changed to high or low. , Without the user being aware
There is an effect that the image signal can be efficiently encoded.

This is because when the camera moves, there should be an object that the cameraman wants to photograph, and when zooming, for example, there should be something the cameraman wants to photograph near the center of the object, and to the right. When I panned,
There should be something you want to shoot on the right. It is possible to obtain a decoded image with good visual quality by increasing the resolution of the part where the photographer and the person watching the image are gazing from the camera parameters using this method.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a moving image encoding device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of an operation of detecting a camera parameter from a moving image.

FIG. 3 is a diagram showing an outline of a conventional image communication device.

[Fig. 4] Fig. 4 is a diagram illustrating a configuration of an encoding device according to a conventional block-based motion compensation predictive encoding method.

[Explanation of symbols]

 1 image to be encoded 2 polygonal pattern 3 motion detection unit 4 block motion compensation unit 5 frame memory 6 locally decoded image 7 motion compensated prediction image 8 subtractor 9 motion compensation prediction error 10 spatial redundancy compression unit 11 compression difference data 12 difference Data coding unit 13 Difference image coded data 14 Difference data decompression unit 15 Decompression difference image 16 Adder 17 Locally decoded image 18 Motion vector 19 Motion vector coding unit 20 Motion vector coded data 21 Multiplexing unit 22 Multiplexed data 23 Camera parameter detection unit 24 Camera parameter 25 Resolution conversion region detection unit 26 Resolution conversion region identifier

Claims (9)

[Claims] 1. In a moving image coding method, when a camera parameter of an input image signal is known, the camera parameter is used as it is, and when it is not known, the camera parameter is detected and input according to the camera parameter. A moving picture coding method, characterized in that the coding is performed by increasing or decreasing the resolution of a specific portion of the generated image signal. 2. When detecting a camera parameter of an image signal, an encoding target image is divided into a plurality of small regions, and a motion amount between the encoding target image and a prediction reference image is obtained for each of the small regions. 2. The moving picture coding method according to claim 1, wherein the camera parameter is detected from the calculated amount of motion for each small area. 3. The amount of zoom-in and zoom-out of the camera parameter is used as an evaluation value for selecting which part of the resolution is changed when the resolution of a specific part of the input image is increased or decreased according to the camera parameter. Item 2. The moving image encoding method according to Item 1. 4. When the detected camera parameters include zoom-in and zoom-out, fine quantization and high resolution encoding are performed when encoding the vicinity of the center point where the camera zooms. 3. The moving picture coding method described in 3. 5. The amount of panning or tilt of the camera parameter is used as an evaluation value for selecting which part of the resolution is changed when the resolution of a specific part of the input image is increased or decreased according to the camera parameter. 1. The moving picture coding method described in 1. 6. When the detected camera parameter includes panning or tilting, when the portion newly appearing from the outside of the screen is encoded by panning or tilting of the camera, fine quantization is performed to increase the resolution. The moving picture coding method according to claim 5, wherein the moving picture is coded. 7. When the detected camera parameter includes panning or tilting, coarse encoding is performed to reduce the resolution when encoding a part that newly appears from outside the screen due to panning or tilting of the camera. Encoding 5
The described moving picture coding method. 8. When a detected camera parameter includes panning or tilting, coarse panning or tilting of the camera causes a rough quantization when coding a part that goes out of the screen in the next frame. 6. The moving picture coding method according to claim 5, wherein the coding is performed by lowering the resolution. 9. A moving picture coding apparatus for inputting and coding an image signal, wherein a camera parameter detecting section for detecting a camera parameter of the input image signal, and an input image according to the parameter detected by the camera parameter detecting section. A resolution conversion part determination unit that determines which part of the signal changes the resolution and how; and an image encoding unit that encodes an image signal according to the determination of the resolution conversion part determination unit. Video encoding device.