JP2007104372A - Video encoding method, video encoding apparatus, video encoding program, and recording medium therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve subjective quality while suppressing the amount of code to be generated in a video with a camera work. <P>SOLUTION: On the basis of a camera work estimated by a camera work estimation section 14 and a motion for each small block estimated by a block unit motion search section 13, a block determination section 15 classifies small blocks into a moving object block wherein the change of the video includes an object motion other than the camera work, a frame-out background block which frames out even partially in a frame of the next time in other background blocks, and a background block that is not the frame-out background block. An image encoding section 16 sets a quantization step size so as to allocate a relatively large amount of code in the order of the moving object block, the background block that is not the frame-out background block, and the frame-out background block, and encodes the video. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a video encoding technique for efficiently encoding in consideration of human subjective quality.

  The main purpose of encoding video is to reduce the amount of data without degrading video quality as much as possible. At that time, from the viewpoint of actually viewing the video, it is important to improve the subjective quality of the person viewing the video, not the signal quality (PSNR: Peak to Signal Noise Ratio).

  As a method for improving the subjective quality, there is a method in which a region that is observed by the observer is encoded faithfully with respect to the original video, while a region that is not observed by the observer is not encoded very faithfully. In video encoding, the video quality can be controlled by the amount of code used to encode the area. Therefore, the subjective quality can be improved even with the same code amount as a whole by assigning more code amount to the region of interest of the observer than assigning the code amount uniformly in the entire video.

  For example, Non-Patent Document 1 corresponds to this. In this method, an area with temporal movement (hereinafter referred to as a moving area or a moving object) is considered as a gaze area, and the subjective quality is improved by assigning a larger amount of code to that area than other areas. Is realized. This uses the human characteristic of paying attention to moving objects, so even with the same amount of encoded video, the subjective quality is improved compared to the encoded video that improves the overall image quality. Will do.

  However, when coding a video with a small amount of motion area, the method of changing the code amount allocation between the motion region and other regions as in Non-Patent Document 1 sufficiently improves the subjective quality while maintaining the code amount. I can't let you. For example, in the case of an image of a landscape, most of the change in the image depends on camera work (the movement of the camera when the image is captured), so most of the regions are regions other than the motion region (hereinafter, It is determined to be a background region), and code amount allocation for improving subjective quality cannot be performed.

Non-Patent Document 2 below describes an example of a method for estimating camera work used in an embodiment of the present invention. Non-Patent Document 3 below describes an example of a method for obtaining an optical flow for each pixel used in an embodiment of the present invention. Non-Patent Document 4 includes H.C. Details of the H.264 video coding scheme are shown.
Teru Ando, Seiichiro Hanya, Kenji Sugiyama, “Application to Motion Object Detection and Coding from General-Purpose Video”, Image Coding Symposium 2004 (PCSJ2004), pp.41-42. M. Pollefeys, R. Koch and L. Van Gool, “Self-Calibration and Metric Reconstruction in spite of Varying and Unknown Internal Camera Parameters”, International Journal of Computer Vision, 32 (1), 7-25, 1999. Jean-Yves Bouguet, “Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the algorithm”, Innel Corporation, Microprocess or Research Labs, Open CV Documents, 1999. ITU-T H.264 ITU-T Rec. H.264, “Advanced video coding for generic audiovisual services”, 2003.

  With the conventional technique as described in Non-Patent Document 1, it is not possible to assign an appropriate code amount for improving the subjective quality to the background area. For this reason, it is necessary to be able to change the code amount allocation so that the subjective quality can be improved even in the background area.

  As a method of changing the code amount assignment in consideration of the influence on subjective quality in the background area, a method of assigning a large amount of code to the center of the screen and assigning a small amount of code to the edge of the screen can be considered. This method uses the property that many people are paying attention to the center of the image when viewing the image.

  However, such a method considering only spatial information may cause fatal image quality degradation. For example, when a characteristic landscape appears at the edge of the video, or when such a landscape moves slowly from the center to the edge of the video, this method deteriorates rather than improving the subjective quality. Will end up.

  Also, if the camera work is relatively large, many things that did not exist in the previous frame will appear in the video. This area is called an uncovered area in Non-Patent Document 1, and is an area that needs to be allocated a larger amount of code. For this reason, a large amount of code is required for most of the video, and a sufficient amount of code cannot be controlled.

  The present invention has been made in view of such circumstances, and design of a video encoding method capable of improving the subjective quality of an observer while suppressing the amount of codes in video with camerawork. The purpose is to establish the law.

  In order to solve the above problems, in the first invention, the image is divided into small blocks and encoded, and at least the degree of quantization for each block (hereinafter, an example of the quantization step size will be described). Is a video encoding method for selecting and encoding freely, a camera motion estimation step for estimating camera motion (hereinafter referred to as camera work) when a video is shot, and a small block Next, a motion search step for estimating the motion of the video shown in the block, and each small block from the estimated camera work and motion for each small block, and a background block whose video change of the small block depends on the camera work And a background block determination step for classifying the moving object blocks including movements other than camerawork, and small blocks included in the background blocks A relatively large code amount in the order of a frame-out background block determination step that extracts a block that is partially out of the frame at the next time, and a background block that is not a moving object block, a frame-out background block, or a frame-out background block A video encoding method having a quantization step size setting step for setting a quantization step size for encoding the small block is devised.

  According to this, in a video with a small number of motion areas, even if it is an area included in the same background area, by assigning the code amount according to the importance set considering the temporal change of the video, Can be improved.

  In this method, first, camera work is obtained between a frame to be encoded and a frame immediately after that frame. The camera work referred to here indicates a pixel unit mapping from the encoding target frame to the immediately following frame. In other words, it includes not only simple pan (horizontal movement), tilde (vertical movement), roll (rotation), zoom (enlargement / reduction), but also the difference in movement distance depending on the structure of the subject, which is the same as the optical flow for each pixel. It is.

  Next, the encoding target frame is divided into small blocks, and inter-frame motion prediction is performed for each block to obtain a motion vector for each block. From the optical flow for each pixel and the motion vector of each block, the block contains a change in the image caused by the movement of the subject other than the camera work (hereinafter referred to as a moving object block), and the image produced mainly by the camera work. Are classified into blocks having only the change of (hereinafter referred to as background blocks).

  Further, a block in which the background block is partially out in the next frame (hereinafter referred to as a frame-out background block) and a block in which all of the background block remains in the frame (hereinafter referred to as a frame stay background block). Classify into: Then, encoding is performed using a quantization step size set according to the block type.

  Of the background block, the area that disappears from the screen in the next frame is less likely to contribute to the subjective quality because the time seen by the observer is shorter than the area that remains in the screen in the next frame. . In other words, it is necessary for the entire video while maintaining subjective quality by improving the subjective quality by assigning more code amount to the frame stay background block than the frame out background block or reducing the code amount assigned to the frame out background block The amount of codes can be reduced.

  In the second invention, the quantization step size setting step obtains the distance to be framed out in the next frame for each pixel of the block included in the frame-out background block, and the total value of the distances of all the pixels in the block for each block The video encoding method is characterized by having a process of assigning a larger quantization step size as the value is larger.

  If the quantization step size differs greatly between adjacent blocks, the difference in signal image quality becomes clear at that part, and the subjective quality is degraded. In other words, in order to reduce the code amount without reducing the subjective quality, it is necessary to reduce the difference in quantization step size between adjacent blocks. Therefore, as the quantization step size of the frame-out background block, a value that is not significantly different from the step size is used only for the amount used in other blocks, and it can be reduced even if the number of frame-out background blocks increases. The code amount is not so large.

  On the other hand, according to the second aspect of the invention, the quantization step size is gradually changed and set so as to allocate a smaller code amount as the image is separated from the area where the image existing in the next frame is displayed. Therefore, it is possible to reduce a larger amount of codes than using a uniform quantization step size.

  According to a third aspect of the present invention, when a video is encoded using an inter-frame motion compensation technique, a quantization step size setting step is performed for a frame that may be referred to when another frame is encoded. The video encoding method is characterized in that the same quantization step size is set for the frame-out background block and the frame-stay background block.

  Even in a frame that is referenced when encoding another frame, if a frame-out background block is encoded using a large quantization step size, the prediction residual when the block is referred to increases, so that The amount of code for encoding a frame increases, and the amount of code required for encoding the entire video cannot be reduced sufficiently.

  On the other hand, according to the third invention, when encoding a frame that may be referred to, a prediction residual is large even if a frame-out background block is referred to in order not to increase the quantization step size. In other words, the amount of code can be reduced for the entire video.

  According to a fourth aspect of the present invention, when an image is encoded by using an inter-frame motion compensation technique, a frame out background is first applied to a frame that may be referred to when another frame is encoded. Set the same quantization step size for the block and the framestay background block, perform encoding, and then determine that the frame will not be referenced by other frames before Re-encoding a small block that is included and that has never been referenced from another frame by setting a new relatively larger quantization step size than the framestay background block A video encoding method characterized by the above.

  Since a frame-out background block that is never used for reference does not affect the amount of prediction residual due to motion compensation, the amount of code can be reduced by encoding with a large quantization step size. Therefore, according to the fourth aspect of the present invention, since the amount of code can be controlled in consideration of the efficiency of inter-frame motion prediction, the amount of code required to encode the entire video is further reduced while maintaining the subjective quality. It becomes possible to do.

  In the fifth aspect of the invention, encoding is performed using an inter-frame motion compensation technique, and when a frame-out background block is encoded using a quantization step size greater than a certain threshold, an estimation is performed in a camera work estimation step. The video encoding method is characterized in that encoding is performed by performing inter-frame motion compensation using a motion vector based on the camerawork.

  The motion vector for interframe motion compensation is generally the sum of the amount of code required to express the vector and the amount of code required to encode the residual signal after motion compensation. Is determined to be small. At this time, if the quantization step size when encoding the residual signal is large, the code amount necessary for encoding the residual signal after motion compensation does not change greatly for any motion vector. Therefore, a motion vector that requires a small amount of code to express is selected. However, even if the amount of code required to encode the residual signal does not change significantly, the video of the block using a motion vector that differs greatly from the actual motion becomes a video that differs greatly from the original video, and the subjective quality deteriorates significantly. I will let you.

  On the other hand, according to the fifth invention, since the motion corresponding to the actual video is used in the frame-out background block in which the quantization step size is set to be relatively larger than that of the other blocks, the accuracy of the fine portion is determined. Although it is lost, it can maintain the rough change of the video, reproduce the natural change, and improve the subjective quality even with the same code amount.

  According to the present invention, an area that does not remain on the screen in the immediately following frame is specified as a part that is relatively unaffected when a human subjectively evaluates an image, and the amount of code assigned to that part is assigned to another part. By making the code amount relatively smaller than the allocated code amount, even in a video where there are no moving objects such as landscapes, the code amount of the entire video can be reduced while maintaining the subjective quality, or the same for the entire video. It is possible to realize video coding that improves subjective quality even with a code amount.

  Specific examples of the present invention are shown below. FIG. 1 shows the configuration of a video encoding apparatus used in this embodiment.

  The video encoding device 10 includes an image input unit 11 that inputs video for each frame in the order along the time axis, and an input image memory that stores the frames until all frames at times before the own time are encoded. 12, a block unit motion search unit 13 for searching for a motion between two temporally continuous frames for each block, a camera work estimation unit 14 for estimating a camera work between two temporally continuous frames, A block determination unit 15 that determines whether each small block is a moving object block, a frame stay background block, or a frame-out background block, and an image that is divided into small blocks and encoded using inter-frame motion prediction The encoding unit 16 and a code output unit 17 that outputs a video code as the encoding result are provided.

  FIG. 2 shows an example of the input video. In the following description of the embodiment, a case where the video shown in FIG. 2 is encoded will be described as an example. It is assumed that this video is encoded in the order of frame 1, frame 3, and frame 2 while performing inter-frame motion prediction using the previously encoded frame.

  FIG. 3 is a diagram showing details of the configuration of the image encoding unit 16 in the video encoding device 10. The image encoding unit 16 includes a camera work memory 161 for storing camera work information, a block property map memory 162 for storing block type information, camera work and block types, and presence / absence of references from other frames. The image is divided into small blocks using a quantization step setting unit 163 that determines a quantization step for encoding each block from information, and a quantization step set by the quantization step setting unit 163. A video encoding unit 164 that performs video encoding using an encoding method that can use inter-frame motion prediction, a referenced map memory 165 that stores information indicating whether or not a block has been referenced by inter-frame motion prediction, And an output code memory 166 for buffering the code of the video as the encoding result.

  FIG. 4 shows a processing flow of the video encoding device 10. In this embodiment, the frames shown in FIG. 2 are sequentially input to the image input unit 11 in the order of frame 1, frame 2, and frame 3 [step A1]. The input image is stored in the input image memory 12 until the encoding target frame and the immediately subsequent frame are input, or until the last frame is input [steps A2 and A3]. That is, in this embodiment, the encoding of frame 1 is started after the input of frame 2 is completed, and the encoding of frame 2 is started after the input of frame 3 is completed. The encoding of frame 3 is started after the input of frame 3 is completed. The condition for deleting an image from the image memory is when all the frames at the time before itself and the time frame are already encoded. That is, in this embodiment, the input image memory 12 has a memory for at least two frames.

  Next, if the encoding target frame is the final frame, the encoding target frame is encoded by the image encoding unit 16 [steps A4 and A5]. On the other hand, if the encoding target frame is not the final frame, the block unit motion search unit 13 uses the encoding target frame and the immediately following frame for each small block obtained by dividing the encoding target frame in a certain unit. The position where the block exists in the immediately following frame is estimated [steps A4 and A6].

In addition, the camera work estimation unit 14 estimates the camera work in the encoding target frame and the immediately following frame [step A7]. As a method for estimating the camera work, for example, a known method as described in Non-Patent Document 2 can be used. In this method, it is sufficient to be able to know to which point a point has moved by camerawork at the next time, and it is not necessary to specify the type of camerawork. That is, the process performed in this step is to obtain the projection P from the coordinate X of the encoding target frame to the coordinate X ^* of the immediately following frame as expressed by the following equation. In addition, this indicates that the camera work estimation process only needs to obtain an optical flow for each pixel. For example, a technique for obtaining an optical flow for each pixel as described in Non-Patent Document 3 may be used. it can.

X ^* = P (X)
Next, using the obtained block-by-block motion and camera work, the block determination unit 15 classifies each block as a motion object block, a frame stay background block, or a frame-out background block [step A8]. Then, using the block information and camera work, the image encoding unit 16 encodes the encoding target frame [step A5]. The encoded frames are output from the code output unit 17 in the order of encoding, that is, in the order of frame 1, frame 3, and frame 2 [step A9]. When encoding of all frames is completed, the process is terminated [Step A10].

  FIG. 5 shows a detailed flow of block determination performed by the block determination unit 15. First, the reliability of the motion vector in units of blocks obtained by the block unit motion search unit 13 is obtained for each block by the following formula [Steps B0, B1].

Reliability = −Σ _p | Y _s (p) −Y _d (p−MV _k ) |
Here, sigma _p represents the sum of the p∈S _k. p represents pixel coordinates, S _k represents a set of pixel coordinates included in the block k in the encoding target frame, and MV _k represents a motion vector of the block k obtained by the block unit motion search. The pixel value of the pixel coordinate p in the encoding target frame is expressed as Y _s (p), and the pixel value of the pixel coordinate p in the frame at the next time of the encoding target frame is expressed as Y _d (p).

  Next, it is checked whether or not the reliability is equal to or higher than a certain threshold [Step B2]. When the reliability does not exceed a certain threshold value, the camerawork projection P estimated by the camerawork estimation unit 14 is used to check whether each pixel in the block is included in the frame at the next frame. [Step B3]. Then, it is checked whether or not the number of pixels to be framed exceeds a certain number [step B4].

  When a certain number of pixels are included in the frame at the next time frame, the block is determined to be a frame stay background block [step B5]. On the other hand, when a certain number of pixels or more are out of frame, the block is determined as a frame-out background block [step B6].

In the determination in step B2, if the reliability is equal to or higher than a certain threshold value, the motion vector CMV _k by the camera work of the block is calculated from the projection P by the camera work estimated by the camera work estimation unit 14 by the following equation. [Step B7]. N represents the number of pixels included in the block.

CMV _k = Σ _q (P (q) −q) / N
Here, sigma _q represents the sum of the q∈S _k.

Thereafter, the size of the difference vector between CMV _k and MV _k is checked [step B8]. If the size is equal to or greater than a certain threshold, the block is determined as a moving object block [step B9]. On the other hand, if the size of the difference vector does not exceed the threshold, it is checked whether each pixel in the block is included in the frame at the next time frame [step B3], and a certain number of pixels are found in the next time frame. If it is included in the frame, the block is determined to be a frame stay background block [steps B4, B5], otherwise it is determined to be a frame-out background block [steps B4, B6].

  FIG. 6 shows a detailed flow of image encoding processing performed by the image encoding unit 16. First, the camera work estimated by the camera work estimation unit 14, information indicating the type of each block of the encoding target frame determined by the block determination unit 15 (hereinafter referred to as a block property map), and the input image memory 12 The image to be encoded stored in is input [Step C1]. The camera work is stored in the camera work memory 161, and the block property map is stored in the block property map memory 162 [step C2]. The encoding of the corresponding frame and the frame at the time before that frame is completed from the memory, and these codes are deleted when they are output from the image encoding unit 16. That is, in this embodiment, it is necessary to be able to store two sets of camera work and a block property map.

  When the input of the camera work and the block property map corresponding to the encoding target frame is completed, the first quantization step is determined for each block by the quantization step setting unit 163 using the camera work and the block property map [ Step C3].

Using the determined quantization step and camera work, the moving image encoding unit 164 actually encodes the image [step C4]. The moving picture coding method used here is H.264. H.264 (see, for example, Non-Patent Document 4) is a method of encoding an image in units of blocks by dividing an image into small blocks. Further, encoding is performed using inter-frame motion prediction for each block. Any method can be used as long as the code amount and the image quality can be adjusted by changing the quantization step. However, when a frame-out background block in which a quantization step equal to or greater than a certain threshold is set is encoded using motion prediction, CMV _k obtained from camera work is used as the motion vector. In addition, when inter-frame motion prediction is used at the time of encoding, which pixel is referenced for each reference frame is recorded, and the information (hereinafter referred to as a referenced map) is stored in the referenced map memory 165. [Step C5].

  The code output from the moving image encoding unit 164 is stored in the output code memory 166 [step C6]. Then, the output code memory 166 checks whether or not the code of the frame stored in the memory earliest among the codes stored in the memory is included in the reference frame candidate list [step C7]. The reference frame candidate list is a list of frames that have a possibility of being used as a reference frame at the time of inter-frame motion prediction. The management of the candidate list is determined by a reference structure given as a parameter for encoding. For example, when a parameter that enables reference back 5 frames in the order of playback time is given, the Nth referenced frame is added to the candidate list after the encoding of the frame itself is completed, and the N + 5th frame Is deleted from the candidate list at the time when encoding of is completed. If it is on the candidate list, exit without outputting anything.

  On the other hand, if it is not on the candidate list, the quantization step setting unit 163 recalculates the quantization step using the referenced map corresponding to the frame [Step C8], and the moving image encoding unit 164 adds a new quantum Encoding is performed again using the conversion step [Step C9], and the result is output [Step C10]. Thereafter, when the corresponding code is output, the code is deleted from the output code memory 166 [step C11]. In other words, in the case of the present embodiment, the output code memory 166 needs to store codes for at least the number of reference frame candidates + 1 frame. If the code is output, the process related to the output [Step C7-C11] is repeated again. If the result of recalculation of the quantization step in step C8 is the same as the quantization step previously determined in step C3, the encoding process in the next step C9 may be omitted.

  FIG. 7 shows a detailed flow of processing performed by the quantization step setting unit 163. First, it is checked whether the quantization step determination target frame is the first frame for determining the quantization step [step D1]. In the case of a frame for which the quantization step is determined for the first time, it is checked whether it is a frame that can be used as a referenced frame when other frames are encoded [step D2]. In the case of a frame that cannot be used as a referenced frame, an evaluation value is calculated for each frame-out background block using the following formula [Step D3].

Σ _q Distance (P (q)) (1)
Distance (x) represents a function that gives the shortest distance from x to the end of the frame. However, if Distance (x)> 0, x is outside the frame; if Distance (x) = 0, x is on the frame; if Distance (x) <0, x is inside the frame. And

  Then, small quantization steps are set in the order of the moving object block, the frame stay background block, and the frame out background block [step D4]. At this time, the smaller the evaluation value obtained in advance, the smaller the quantization step is set for the frame-out background block. Here, it is assumed that the smaller the quantization step, the more code amount is allocated.

  On the other hand, in the case of a frame that can be used as a referenced frame, an equivalent quantization step is set for the frame stay background block and the frame-out background block, and a smaller quantization step is set for the moving object block [Step D5]. ]. In addition, when the quantization step determination target frame is not the frame for which the quantization step is determined for the first time, it is determined from the information stored in the referenced map memory 165 that it is a frame-out background block and any of the blocks in the block. A block in which a pixel is referenced at least once is obtained, and an evaluation value obtained by the equation (1) is calculated for the block [step D6]. Then, smaller quantization steps are set in the following classification in ascending order [Step D7].

1. 1. Moving object block 2. A frame stay background block or a frame out background block and a block in which any pixel in the block is referenced at least once. A frame-out background block and a block in which all the pixels in the block have never been referred to. However, the quantization step of the block included in 3 is smaller as the value obtained by the above equation (1) is smaller. Set.

  In the above embodiment, the moving image encoding unit 164 in FIG. 3 encodes a frame-out background block using the quantization step size set by the quantization step setting unit 163, and the quantization step size. If a certain threshold is exceeded, encoding is performed by performing inter-frame motion compensation using a motion vector based on the camera work estimated by the camera work estimation unit 14. This process is also suitable for improving the subjective quality with a limited code amount.

  The example of the quantization step size has been described as a representative example for controlling the allocation of the code amount, but other encoding parameters may be used as long as the degree of quantization can be controlled. it can.

  The above video encoding processing can be realized by a computer and a software program, and the program can be provided by being recorded on a computer-readable recording medium or provided via a network.

It is a figure which shows the structure of a video coding apparatus. It is a figure which shows the specific example of the image | video input. It is a figure which shows the structure of an image coding part. It is a figure which shows the processing flow of a video coding apparatus. It is a figure which shows the detailed processing flow of the process which determines the kind of block. It is a figure which shows the detailed processing flow of the process which performs image coding. It is a figure which shows the detailed processing flow of the determination process of a quantization step.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image coding apparatus 11 Image input part 12 Input image memory 13 Block unit motion search part 14 Camera work estimation part 15 Block determination part 16 Image coding part 17 Code output part

Claims (8)

A video encoding method in which an image is divided into small blocks and encoded, and at least the degree of quantization is selected for each block and encoded.
A camera work estimation step for estimating a camera work at the time of shooting a video;
For each small block, a motion search step for estimating the motion of the video shown in the block,
From the estimated camera work and the movement of each small block, each small block is a background block whose video change of the small block depends on the camera work, or the video change includes movement of the subject other than the camera work A background block determining step for determining whether the block is an object block;
A frame-out background block determination step for extracting a small block included in the background block and which is partially out of the frame at the next time;
A quantization degree setting step for setting a quantization degree so that a relatively large amount of code is assigned in the order of a moving object block, a background block that is not a frame-out background block, and a frame-out background block;
A video encoding method comprising: The video encoding method according to claim 1, wherein
For each pixel in the small block included in the frame-out background block, the estimated position in the next frame is obtained using the camera work that has been subjected to the camera work estimation step, and the distance that the estimated position is away from the frame end is determined as the encoding target. Set the degree of quantization so that a smaller amount of code is allocated as the total value increases for each block.
And a video encoding method. The video encoding method according to claim 1 or 2, wherein the video encoding method further performs encoding using inter-frame motion compensation for each small block,
In the quantization degree setting step, when encoding a frame that can be used as a reference frame in motion compensation, a background block that is not a frame-out background block and a frame-out background block have the same degree of quantization. Set,
And a video encoding method. The video encoding method according to claim 1 or 2, wherein the video encoding method further performs encoding using inter-frame motion compensation for each small block,
When encoding a frame that can be used as a reference frame in motion compensation, first, in the quantization level setting step, the same degree of quantization between a background block that is not a frame-out background block and a frame-out background block Is a small block that is included in a frame-out background block and has never been referenced by another frame when the frame is removed from the reference frame candidate. On the other hand, a new quantization degree is set so that a relatively small amount of code is allocated to a background block that is not a frame-out background block, and encoding is performed again.
And a video encoding method. The video encoding method according to claim 1 or 2, wherein the video encoding method further performs encoding using inter-frame motion compensation for each small block,
When encoding a frame-out background block using the quantization level set in the quantization level setting step, if the quantization level exceeds a certain threshold, the camera work estimation step estimates A video encoding method comprising: encoding by performing inter-frame motion compensation using a motion vector based on the camerawork performed. A video encoding device that performs encoding by dividing an image into small blocks, and further selects and encodes the degree of quantization for each block,
An image input means for inputting an image every frame;
Input image storage means for storing the input frames;
Camera work estimation means for estimating camera work when a video is taken from the frame stored in the input image storage means;
For each small block of the frame stored in the input image storage means, a motion search means for estimating the motion of the video shown in the block;
From the estimated camera work and the movement of each small block, each small block is divided into a moving object block in which the image change includes the movement of the subject other than the camera work, and a background block that is the other block at the next time. A block determination means for classifying a frame-out background block that is partly out of the frame and a background block that is not a frame-out background block;
Encoding means for encoding a video by setting a degree of quantization so that a relatively large amount of code is assigned in the order of a moving object block, a background block that is not a frame-out background block, and a frame-out background block;
A video encoding device comprising:   A video encoding program for causing a computer to execute the video encoding method according to any one of claims 1 to 5.   A computer-readable recording medium having recorded thereon a video encoding program for causing a computer to execute the video encoding method according to any one of claims 1 to 5.

Images